GrowEasy CSV Importer

An AI-powered importer that takes an arbitrary CRM lead export - a Facebook Lead Ads download, a Google Ads export, a hand-built spreadsheet, whatever a sales team happens to have - and maps it into GrowEasy's CRM schema without requiring the column names to match anything in particular.

Built as a two-service app:

• backend/ - Node.js + Express + TypeScript API. Parses the CSV, batches rows off to an LLM for field extraction, validates/normalizes the model's output, and streams progress back to the client.
• frontend/ - Next.js 14 (App Router) + TypeScript + Tailwind. A four-step wizard: upload → preview → confirm → result.

Why it's built this way

The AI never sees the whole file at once. Rows are sent in configurable batches (default 25) rather than one giant prompt. This keeps prompts small and cheap, lets failures be isolated to a single batch instead of the whole import, and is what makes a live per-batch progress bar possible in the first place.

The backend re-validates everything the AI returns. The system prompt tells the model the exact CRM schema and enum values, but LLM output is never trusted blindly - backend/src/utils/validation.ts re-derives a clean record from whatever comes back, silently fixing what it safely can (stray whitespace, malformed country codes, raw newlines that would break a CSV row) and only drops a row if it violates the one hard rule: no email and no phone number.

Provider-agnostic AI layer. backend/src/services/aiProvider.ts defines one AiProvider interface with OpenAI, Anthropic, and Gemini implementations behind it, selected via AI_PROVIDER env var. A fourth "mock" provider - a deterministic heuristic mapper - is the default so the whole pipeline (batching, retries, streaming, the UI) can be run and demoed with zero API keys and zero cost. Swap in a real key and nothing else changes.

Streaming, not polling. POST /api/import streams newline-delimited JSON: progress events as each batch finishes, then one final result. The frontend reads this with response.body.getReader() - no websockets, no polling loop, no extra infrastructure.

Project layout

groweasy-csv-importer/
├── backend/
│   ├── src/
│   │   ├── index.ts              Express app entrypoint
│   │   ├── config.ts             Env-driven config
│   │   ├── types/crm.ts          CRM schema - single source of truth
│   │   ├── routes/import.ts      POST /api/import (streaming)
│   │   ├── middleware/           multer upload filter, error handler
│   │   ├── services/
│   │   │   ├── csvParser.ts      CSV → rows (BOM, ragged rows, etc.)
│   │   │   ├── aiPrompt.ts       System prompt + JSON response parsing
│   │   │   ├── aiProvider.ts     OpenAI / Anthropic / Gemini / mock
│   │   │   └── leadExtractor.ts  Batching, retry/backoff, aggregation
│   │   └── utils/validation.ts   Normalizes + validates AI output
│   └── tests/                    Vitest unit tests
└── frontend/
    └── src/
        ├── app/page.tsx          4-step wizard state machine
        ├── components/           StepRail, FileDropzone, DataTable, ...
        ├── lib/
        │   ├── csv.ts            Client-side preview parsing (no AI)
        │   ├── api.ts            Streaming fetch client
        │   └── types.ts          Mirrors backend/src/types/crm.ts
        └── hooks/useTheme.ts     Dark mode

Running locally

You need Node 18+.

1. Backend

cd backend
cp .env.example .env
npm install
npm run dev        # http://localhost:8080

By default AI_PROVIDER=mock in .env.example, so it runs with no API key using a heuristic column-matcher. To use a real model, set in .env:

AI_PROVIDER=openai            # or anthropic, or gemini
OPENAI_API_KEY=sk-...

2. Frontend

cd frontend
cp .env.local.example .env.local
npm install
npm run dev         # http://localhost:3000

3. Or both at once with Docker

AI_PROVIDER=openai OPENAI_API_KEY=sk-... docker compose up --build

Running tests

cd backend
npm test

17 unit tests cover CSV parsing edge cases (BOM, ragged rows, blank trailing rows) and the validation/normalization layer (enum enforcement, phone/email cleanup, newline escaping, invalid-date handling).

API

POST /api/import

multipart/form-data with a single file field (the CSV). Response is application/x-ndjson - one JSON object per line:

{"type":"progress","data":{"batchIndex":0,"totalBatches":4,"rowsInBatch":25,"status":"started","attempt":1}}
{"type":"progress","data":{"batchIndex":0,"totalBatches":4,"rowsInBatch":25,"status":"succeeded","attempt":1}}
...
{"type":"result","data":{"summary":{...},"leads":[...],"skipped":[...]}}

GET /health returns { status: "ok", aiProvider: "..." }.

Design notes (frontend)

The visual language leans into the subject matter instead of a generic SaaS look: a faint graph-paper grid in the background, monospace type for raw/data values (so a spreadsheet value visually reads as data, distinct from UI copy), and a literal horizontal "rail" for the 4-step pipeline instead of a generic progress bar. The field-coverage panel on the result screen is the one visualization that's actually specific to this problem - it shows, per CRM field, what fraction of imported leads got a value, which is the real signal for "did the AI map this file well."

Tables (both the raw preview and the final CRM result) are virtualized with react-window, so a 5,000-row CSV renders as fast as a 5-row one.

Known limitations / what I'd do next

• The "mock" provider is a heuristic fallback for zero-cost local demoing, not a substitute for a real model on genuinely ambiguous columns - it's regex/keyword-based rather than intelligent.
• No persistence layer - results live in memory for the duration of one import and aren't written to a database. Adding one (e.g. Postgres + Prisma) would be the natural next step for a real product.
• No auth - fine for an assignment, not fine for production.
• npm audit flags a couple of moderate/high advisories in Next.js 14.2.x and dev-only tooling (vitest's esbuild). None apply to the features this app actually uses (no image optimization, no i18n, no websockets), but a real deployment should track and apply patches as they land.