Files
lead-scraper/README.md
Timo Uttenweiler facf8c9f69 Initial commit: LeadFlow lead generation platform
Full-stack Next.js 16 app with three scraping pipelines:
- AirScale CSV → Anymailfinder Bulk Decision Maker search
- LinkedIn Sales Navigator → Vayne → Anymailfinder email enrichment
- Apify Google SERP → domain extraction → Anymailfinder bulk enrichment

Includes Docker multi-stage build + docker-compose for Coolify deployment.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-17 11:21:11 +01:00

3.1 KiB

LeadFlow — Lead Generation & Email Enrichment Platform

A unified platform for three lead-scraping pipelines with email enrichment via Anymailfinder.


Tech Stack

  • Next.js 16 (App Router) + TypeScript
  • SQLite via Prisma 7 + better-sqlite3
  • shadcn/ui + Tailwind CSS
  • Anymailfinder API v5.1 — email enrichment (bulk JSON + individual search)
  • Vayne API — LinkedIn Sales Navigator scraping
  • Apify — Google SERP scraping

Setup

1. Install dependencies

cd leadflow
npm install

2. Configure environment

Copy .env.local.example to .env.local:

cp .env.local.example .env.local

Edit .env and .env.local:

APP_ENCRYPTION_SECRET=your-32-character-secret-here!!
DATABASE_URL=file:./leadflow.db

3. Run database migration

npx prisma migrate dev

4. Start the app

npm run dev

Open http://localhost:3000


API Keys — Where to Get Them

Go to Settings in the sidebar to enter and save credentials. All keys are AES-256 encrypted before storage.

Anymailfinder

  • Sign up at anymailfinder.com
  • Account → API → copy your key (format: starts with your account prefix)
  • Pricing: 2 credits/valid decision maker email, 1 credit/person email
  • Bulk API charges only when downloading results

Apify

  • Sign up at apify.com
  • Console → Account → Integrations → API tokens
  • The app uses the apify/google-search-scraper actor (pay-per-event)

Vayne

  • Sign up at vayne.io
  • Dashboard → API Settings → generate token
  • Connect LinkedIn in the Vayne dashboard — Vayne manages the LinkedIn session on their end
  • No need to manually export cookies

Pipeline Workflows

Tab 1 — AirScale → Email

  1. Export companies from AirScale as CSV
  2. Upload → map domain column
  3. Select decision maker categories
  4. Start Enrichment → bulk API runs asynchronously
  5. Export CSV/Excel

Tab 2 — LinkedIn Sales Navigator → Email

  1. Build search in Sales Navigator, copy URL
  2. Paste URL + set max results → Start Scrape
  3. Vayne scrapes profiles (polls until done)
  4. Select profiles → Enrich with Emails
  5. Export results

Tab 3 — SERP → Email

  1. Enter search term (e.g. "Solaranlage Installateur Deutschland")
  2. Set country/language, enable social filter
  3. Select decision maker categories
  4. Start → Apify scrapes Google, domains extracted, then bulk enriched
  5. Export results

How Anymailfinder Bulk Works

  1. All domains submitted as one POST to /v5.1/bulk/json
  2. Poll status every 5s until completed
  3. Download results once (credits charged at download, not submission)
  4. Speed: ~1,000 domains per 5 minutes

Database

SQLite at ./leadflow.db. Inspect with:

npx prisma studio

Troubleshooting

Issue Solution
"API key not configured" Add key in Settings
Job stuck at "running" Check server console (npm run dev terminal)
Prisma errors on build Run npx prisma generate && npm run build