Skip to content

Repository files navigation

marketplace-intel-platform

A small, runnable AI Application Platform for a fictional travel marketplace: it ranks and explains "experiences" (tours, attractions, activities) for a traveler — and, more to the point, it shows the platform layer around the model that a Data & AI engineering role actually pays for.

Two capabilities — a classical ML ranker and a GenAI explanation overlay — register behind one typed contract, are fed by a real feature store, served by one API, rolled out through an online experiment, and gated by an evaluation harness that blocks CI.

synthetic data → feature platform (Feast) → ranking capability (LightGBM)
   → GenAI overlay (same contract) → serving API → online experiment → eval gate (CI)

This repo demonstrates the job of an AI-platform engineer — feature reuse without train/serve skew, safe model rollout, and evaluation as a release gate, applied uniformly across classical ML and GenAI — not just a model.

Honesty statement

This is a reference platform built at learning depth, authored AI-assisted (agentic workflow; the specs under docs/ are the design record and lead the code). It runs on synthetic data and makes no production or scale claims — no throughput numbers, no load tests, no "serves N req/s". The platform patterns (typed capability contract, point-in-time feature retrieval, evaluation-as-deploy-gate) are the artifact, not the NDCG number. See ADR-0002.

The owner's production-scale experience (a federated API platform at ~500M req/month, Databricks ML productionization) lives in separate work; this repo evidences the AI-platform patterns, honestly at reference size.

Why it exists

Most candidates for "Data & AI" platform roles can talk about models; far fewer can show a feature store, a model server, an experiment, and an eval gate wired together. This is that, at reviewable size, in a marketplace-ranking domain. Audience: technical screens for Principal / Staff AI-platform roles. See docs/product/00-product-intent.md.

Start here

The three things to look at

  1. One contract, two capability types — a LightGBM ranker and a foundation-model overlay behind the same Capability.predict() surface (src/mip/contracts.py, ADR-0004).
  2. Point-in-time-correct features — one Feast definition feeds both the leak-free training join and the online serving fetch (src/mip/features/, FS-0002).
  3. Evaluation as a deploy gate — breaking the model turns CI red; the gate covers the classical and the GenAI capability (src/mip/eval/, ADR-0007).

Runtime

Local-first (ADR-0001): the default path needs no external services and no API key (Feast file offline store + sqlite online store; deterministic template explainer). docker compose is optional, adding a redis online store as the prod-like option.

make install     # editable install into a venv
make demo        # data → features → train → serve (bg) → one /rank request → eval
# or step by step:
make data        # generate synthetic marketplace data
make features    # feast apply + materialize
make train       # train the LightGBM ranker, print NDCG@k
make serve       # start the FastAPI serving API
make experiment  # simulate + analyze an A/B experiment
make eval        # run the evaluation gate (exits non-zero on regression)
make test        # pytest

GenAI overlay: off the default path it uses a deterministic template explainer (keyless, offline). To use a real foundation model:

export MIP_LLM_PROVIDER=anthropic
export ANTHROPIC_API_KEY=sk-...

It targets Claude Haiku (short, high-volume explanation task); any provider error falls back to the template explainer — the overlay never fails a request (ADR-0006).

Status

M0 (platform spine) first; M1 (serving + GenAI overlay), M2 (experimentation) follow. Production cloud mapping is documented in infra/README.md, not applied (ADR-0001). See the roadmap.

License

MIT — see LICENSE.

About

A runnable reference AI Application Platform: Feast feature store + LightGBM ranker + GenAI overlay behind one capability contract, FastAPI serving, online experimentation, and an evaluation harness wired as a CI deploy gate — for a travel-marketplace ranking demo.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages