Welcome! This repo gives you a running, continuously-growing operational database for a conversational-AI product. Your job is to ingest it into an analytical warehouse and build analytics on top. The warehouse side is not included, that's what you'll build!
A single docker compose up starts three things:
postgres: the operational database. It has a wal replication on.seed: a one-shot container that creates the schema and bulk-loads a realistic dataset (~1k users, ~50 assistants, ~5k conversations, ~100k messages, plus feedback), then exits. It's idempotent and re-runnable.streamer: a long-running container that inserts 1-10 new messages into existing conversations every 2 seconds, simulating live production traffic. This is what keeps the database growing while you work.
Data is stored in a named Docker volume (pgdata) and persists across restarts
until you reset it.
docker compose up --build
# or, in the background:
make upWithin about a minute you'll have a populated database that keeps growing. You should see the streamer logging inserts:
[streamer] inserted 7 messages (prompt, answer, tool:web_search, ...), total this run: 42
| Setting | Value |
|---|---|
| Host | localhost |
| Port | 5432 |
| Database | postgres |
| User | postgres |
| Password | secret |
psql postgresql://postgres:secret@localhost:5432/postgres
# or:
make psqlThese are throwaway dev credentials, there are no secrets in this repo.
| Command | What it does |
|---|---|
make up |
Build + start everything in the background |
make logs |
Follow logs from all services |
make ps |
Show service status |
make psql |
Open a psql shell against the running DB |
make reseed |
Wipe data and reload the seed in-place (--force) |
make down |
Stop everything and delete the data volume (full reset) |
Seven tables. UUID primary keys, timestamptz everywhere, real foreign keys,
and indexes on the hot paths (messages(created_at), messages(conversation_id),
messages(type)). Full DDL is in schema.sql.
| Table | Purpose |
|---|---|
users |
End users, email, name, department, country, locale, plan (free/pro/enterprise) |
models |
Reference table of models + token pricing (input/output_cost_per_1k) |
assistants |
Configurable AI agents (name, description, system prompt, created_by) |
conversations |
A thread for a user, optionally tied to an assistant; source (web/api/slack/teams) |
messages |
The fact table, every prompt, answer, reasoning step, tool call |
feedback |
Thumbs up/down (rating = 1 / -1) on assistant messages, optional comment |
usage |
Token usage per generation, user_id, model_id, uncached_input_tokens, cached_input_tokens, output_tokens (join models for $ cost) |
Every messages row has a non-null user_id (the conversation's user) and
model_id (the conversation's model), so you can attribute any message to a
user and model directly.
Continuously ingest this live Postgres into an analytical warehouse of your choice (ClickHouse, Databricks, Snowflake, Postgres-as-warehouse, …),
The goal is also not just to have the analytical setup but to go down the modelling part as well. Think about what are the interesting dashboards the use cases needs.
- Code: your ingestion + warehouse + analytics, runnable with clear
instructions (ideally
docker composeor a short script) as git repo or zip. - A short writeup (~1 page): how it works, why you chose the architecture, how would you address scaling needs, how freshness is maintained, what you'd do with more time, and any assumptions.
Try to invest no more than 6 hours on the challenge. After the challenge please send your submission (fork or zip) in an email to niklas@langdock.com with pascal@langdock.com and omar.eladawy@langdock.com in CC.
Treat this as not only a platform challenge but a modelling one. What models did you decide for and why, how would you prepare for scaling needs etc. We care more about clear thinking and sound trade-offs than about covering every nice-to-have.
Good luck, have fun with it!