Initial commit: LeadFlow lead generation platform
Full-stack Next.js 16 app with three scraping pipelines: - AirScale CSV → Anymailfinder Bulk Decision Maker search - LinkedIn Sales Navigator → Vayne → Anymailfinder email enrichment - Apify Google SERP → domain extraction → Anymailfinder bulk enrichment Includes Docker multi-stage build + docker-compose for Coolify deployment. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
Timo Uttenweiler
135
README.md
135
README.md
@@ -1,36 +1,131 @@
|
||||
This is a [Next.js](https://nextjs.org) project bootstrapped with [`create-next-app`](https://nextjs.org/docs/app/api-reference/cli/create-next-app).
|
||||
# LeadFlow — Lead Generation & Email Enrichment Platform
|
||||
|
||||
## Getting Started
|
||||
A unified platform for three lead-scraping pipelines with email enrichment via Anymailfinder.
|
||||
|
||||
First, run the development server:
|
||||
---
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Next.js 16** (App Router) + TypeScript
|
||||
- **SQLite** via Prisma 7 + better-sqlite3
|
||||
- **shadcn/ui** + Tailwind CSS
|
||||
- **Anymailfinder API v5.1** — email enrichment (bulk JSON + individual search)
|
||||
- **Vayne API** — LinkedIn Sales Navigator scraping
|
||||
- **Apify** — Google SERP scraping
|
||||
|
||||
---
|
||||
|
||||
## Setup
|
||||
|
||||
### 1. Install dependencies
|
||||
|
||||
```bash
|
||||
cd leadflow
|
||||
npm install
|
||||
```
|
||||
|
||||
### 2. Configure environment
|
||||
|
||||
Copy `.env.local.example` to `.env.local`:
|
||||
|
||||
```bash
|
||||
cp .env.local.example .env.local
|
||||
```
|
||||
|
||||
Edit `.env` and `.env.local`:
|
||||
|
||||
```env
|
||||
APP_ENCRYPTION_SECRET=your-32-character-secret-here!!
|
||||
DATABASE_URL=file:./leadflow.db
|
||||
```
|
||||
|
||||
### 3. Run database migration
|
||||
|
||||
```bash
|
||||
npx prisma migrate dev
|
||||
```
|
||||
|
||||
### 4. Start the app
|
||||
|
||||
```bash
|
||||
npm run dev
|
||||
# or
|
||||
yarn dev
|
||||
# or
|
||||
pnpm dev
|
||||
# or
|
||||
bun dev
|
||||
```
|
||||
|
||||
Open [http://localhost:3000](http://localhost:3000) with your browser to see the result.
|
||||
Open [http://localhost:3000](http://localhost:3000)
|
||||
|
||||
You can start editing the page by modifying `app/page.tsx`. The page auto-updates as you edit the file.
|
||||
---
|
||||
|
||||
This project uses [`next/font`](https://nextjs.org/docs/app/building-your-application/optimizing/fonts) to automatically optimize and load [Geist](https://vercel.com/font), a new font family for Vercel.
|
||||
## API Keys — Where to Get Them
|
||||
|
||||
## Learn More
|
||||
Go to **Settings** in the sidebar to enter and save credentials. All keys are AES-256 encrypted before storage.
|
||||
|
||||
To learn more about Next.js, take a look at the following resources:
|
||||
### Anymailfinder
|
||||
- Sign up at [anymailfinder.com](https://anymailfinder.com)
|
||||
- Account → API → copy your key (format: starts with your account prefix)
|
||||
- Pricing: 2 credits/valid decision maker email, 1 credit/person email
|
||||
- Bulk API charges only when downloading results
|
||||
|
||||
- [Next.js Documentation](https://nextjs.org/docs) - learn about Next.js features and API.
|
||||
- [Learn Next.js](https://nextjs.org/learn) - an interactive Next.js tutorial.
|
||||
### Apify
|
||||
- Sign up at [apify.com](https://apify.com)
|
||||
- Console → Account → Integrations → API tokens
|
||||
- The app uses the `apify/google-search-scraper` actor (pay-per-event)
|
||||
|
||||
You can check out [the Next.js GitHub repository](https://github.com/vercel/next.js) - your feedback and contributions are welcome!
|
||||
### Vayne
|
||||
- Sign up at [vayne.io](https://vayne.io)
|
||||
- Dashboard → API Settings → generate token
|
||||
- **Connect LinkedIn** in the Vayne dashboard — Vayne manages the LinkedIn session on their end
|
||||
- No need to manually export cookies
|
||||
|
||||
## Deploy on Vercel
|
||||
---
|
||||
|
||||
The easiest way to deploy your Next.js app is to use the [Vercel Platform](https://vercel.com/new?utm_medium=default-template&filter=next.js&utm_source=create-next-app&utm_campaign=create-next-app-readme) from the creators of Next.js.
|
||||
## Pipeline Workflows
|
||||
|
||||
Check out our [Next.js deployment documentation](https://nextjs.org/docs/app/building-your-application/deploying) for more details.
|
||||
### Tab 1 — AirScale → Email
|
||||
1. Export companies from AirScale as CSV
|
||||
2. Upload → map domain column
|
||||
3. Select decision maker categories
|
||||
4. Start Enrichment → bulk API runs asynchronously
|
||||
5. Export CSV/Excel
|
||||
|
||||
### Tab 2 — LinkedIn Sales Navigator → Email
|
||||
1. Build search in Sales Navigator, copy URL
|
||||
2. Paste URL + set max results → Start Scrape
|
||||
3. Vayne scrapes profiles (polls until done)
|
||||
4. Select profiles → Enrich with Emails
|
||||
5. Export results
|
||||
|
||||
### Tab 3 — SERP → Email
|
||||
1. Enter search term (e.g. `"Solaranlage Installateur Deutschland"`)
|
||||
2. Set country/language, enable social filter
|
||||
3. Select decision maker categories
|
||||
4. Start → Apify scrapes Google, domains extracted, then bulk enriched
|
||||
5. Export results
|
||||
|
||||
---
|
||||
|
||||
## How Anymailfinder Bulk Works
|
||||
|
||||
1. All domains submitted as one POST to `/v5.1/bulk/json`
|
||||
2. Poll status every 5s until `completed`
|
||||
3. Download results once (credits charged at download, not submission)
|
||||
4. Speed: ~1,000 domains per 5 minutes
|
||||
|
||||
---
|
||||
|
||||
## Database
|
||||
|
||||
SQLite at `./leadflow.db`. Inspect with:
|
||||
|
||||
```bash
|
||||
npx prisma studio
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
| Issue | Solution |
|
||||
|-------|----------|
|
||||
| "API key not configured" | Add key in Settings |
|
||||
| Job stuck at "running" | Check server console (`npm run dev` terminal) |
|
||||
| Prisma errors on build | Run `npx prisma generate && npm run build` |
|
||||
|
||||
Reference in New Issue
Block a user