feat: surface sample datasets with a guided first-run gallery

[lucastononro]

Summary

A bundled demo dataset library exists (sample-data/, seven datasets from Titanic to license-plate detection) but nothing surfaced it — a new user landed on an empty experiments gallery with no obvious first move. This PR adds a guided first-run: the empty gallery now offers one-click "Try a sample dataset" tiles that create a project pre-loaded with the sample's files and drop the user into the studio with a suggested first prompt already in the chat input, alongside a short inline tour of the chat↔canvas split.

Changes

Backend

• backend/samples.yml — new one-source-of-truth catalog: per-sample tile copy, task type, file manifest, and suggested prompt for all seven bundled datasets (six tabular classification/regression + the license-plates CV mini-set).
• backend/routers/samples.py — new router:
  • GET /api/samples — catalog for the gallery tiles (name, task, description, suggested prompt, file count/size, availability).
  • POST /api/projects/from-sample — creates project + initial experiment + session (mirroring POST /projects) and copies the sample files through the normal dataset-upload path: S3 at the project-owned keys, one batched Modal Volume upload, best-effort dataset versioning, dataset_ref set like a real upload.
• backend/schemas.py — ProjectFromSample request schema; backend/main.py — router registration.
• backend/config.py — SAMPLE_DATA_DIR override; well-known locations (repo checkout sibling of backend/, /app/sample-data) are probed otherwise. Deployments without sample data degrade gracefully (available: false tiles, 503 on create).
• docker-compose.yml — mounts ./sample-data read-only into the backend (it lives outside the ./backend build context).

Frontend

• frontend/src/app/experiments/page.tsx — the empty gallery renders the first-run screen: sample tiles (task badge, description, size, per-tile creating spinner) + a 3-step inline tour (chat pane / canvas pane / data at /data). Falls back to the previous empty-state copy when no samples are available.
• frontend/src/lib/suggestedPrompt.ts — sessionStorage hand-off of the suggested prompt, keyed to the new session and consumed exactly once.
• frontend/src/app/page.tsx — small effect that seeds the chat input from the hand-off when the new session becomes active (never clobbers user-typed text).
• frontend/src/lib/api.ts / types.ts — listSamples(), createProjectFromSample(), SampleDataset, CreateProjectFromSampleResponse.
• backend/routers/AGENTS.md — router layout list updated.

Test plan

Existing

• Full backend suite: 301 passed, 8 skipped (e2e-gated) — existing project/experiment creation from upload and from S3 unchanged.
• Gallery with existing experiments still renders the grouped project tables (empty-state branch is the only render change); tsc --noEmit, next lint, next build all green.

New

• backend/tests/test_samples.py (5 tests): catalog lists tabular + CV samples with prompts and available: true in a repo checkout; create-from-sample lands files at the project-owned S3 keys + Modal Volume paths and the project/experiment read back through the normal endpoints; custom project name + multi-file dataset_ref; 404 on unknown sample; 503 when sample data isn't shipped.
• Click-path: open the app with no experiments → GALLERY shows the first-run tiles → click "Titanic Survival" → tile spins, project "Titanic Survival" is created pre-loaded with titanic.csv + CONTEXT.md → you land in the studio on the new chat with the suggested EDA/train prompt pre-filled → hit send.

Caveats

• The prod backend image (ghcr.io/.../trainable-backend, built from the ./backend context) doesn't contain sample-data/; without a mount or SAMPLE_DATA_DIR, tiles are hidden and the old empty-state copy shows. Baking the data into the image needs a build-context change — left for a follow-up.
• Tiles show whenever the gallery is empty (also for returning users who deleted everything) — intended, it doubles as the empty state.

Closes #110

🤖 Generated with Claude Code

https://claude.ai/code/session_01Wp66uSq9saSpRq9Bbmjxj4

Greptile Summary

This PR surfaces a bundled sample-data/ library that previously went unused: the empty experiments gallery now shows one-click "Try a sample dataset" tiles that create a full project (S3 files + Modal Volume + DB rows) and drop the user into the studio with a suggested first prompt pre-filled.

• Backend: new services/samples.py (business logic) + thin routers/samples.py (validate → call service → return), services/datasets.py (shared path helpers extracted from private router helpers), samples.yml catalog, and Pydantic response schemas in schemas.py.
• Frontend: FirstRunSamples component in the experiments gallery, suggestedPrompt.ts sessionStorage hand-off, and a useEffect in the studio page that seeds the chat input once (never clobbers typed text).

Confidence Score: 5/5

Safe to merge — the new endpoints are additive, degrade gracefully when sample data is absent, and the extraction of private router helpers into a shared service module correctly resolves the prior cross-module coupling concern.

The core data flow (S3 upload → Modal Volume batch → DB commit) mirrors the existing experiment-creation path and is well-tested. Both new endpoints carry proper Pydantic response models and the thin-router pattern is followed. The two findings are minor: a FastAPI exception type leaking into the service layer, and a handful of is_dir() calls running on the event loop before asyncio.to_thread is invoked — neither affects correctness or data integrity.

Files Needing Attention: backend/services/datasets.py — FastAPI HTTPException in the service layer is the only thing worth revisiting before the module grows more callers.

Important Files Changed

Filename	Overview
backend/services/samples.py	New service module with full project-from-sample creation flow; S3 put_object runs synchronously on the event loop (consistent with existing experiments.py behavior), response dict returned as raw dict but FastAPI serializes it correctly via response_model on the router.
backend/services/datasets.py	Shared path helpers extracted from experiments.py private functions; correctly decouples cross-module coupling, but still imports and raises HTTPException, bringing a FastAPI dependency into the service layer.
backend/routers/samples.py	Thin router with proper Pydantic response models and asyncio.to_thread for filesystem I/O; sample_data_root() is evaluated synchronously before to_thread call, causing minor FS calls on the event loop.
frontend/src/app/experiments/page.tsx	FirstRunSamples component cleanly handles loading state, no-samples fallback, per-tile spinner, and cleanup on unmount; hooks and dependencies are correct.
frontend/src/lib/suggestedPrompt.ts	sessionStorage hand-off keyed to session ID, consumed exactly once; silently no-ops on quota/private-mode failures.
backend/schemas.py	SampleDatasetEntry, SampleProjectSummary, SampleExperimentSummary, ProjectFromSampleResponse all correctly typed; TypeScript Experiment interface fields align with what SampleExperimentSummary returns.
backend/tests/test_samples.py	5 tests covering catalog, create happy path, custom name, unknown sample, and missing-files 503; S3/Volume seams correctly patched at the service level.
docker-compose.yml	Mounts ./sample-data as read-only into the backend container at /app/sample-data, matching the second candidate in sample_data_root().

Sequence Diagram

sequenceDiagram
    participant UI as Frontend (experiments/page)
    participant API as GET /api/samples
    participant PS as POST /api/projects/from-sample
    participant SVC as services/samples.py
    participant S3 as S3 (datasets bucket)
    participant MV as Modal Volume
    participant DB as Database

    UI->>API: listSamples()
    API-->>UI: SampleDatasetEntry[] (available tiles)

    UI->>PS: createProjectFromSample(sample_id)
    PS->>SVC: get_sample() / sample_files()
    SVC-->>PS: files: [(rel_path, abs_path)]
    PS->>SVC: create_project_from_sample()
    loop each file
        SVC->>S3: put_object (sync)
        SVC->>SVC: stage tmpfile
    end
    SVC->>MV: upload_many_to_volume (batch, best-effort)
    SVC->>DB: commit(project + experiment + session)
    SVC-->>PS: ProjectFromSampleResponse dict
    PS-->>UI: "{project, experiment, session_id, suggested_prompt}"

    UI->>UI: stashSuggestedPrompt(session_id, prompt)
    UI->>UI: router.push('/')
    UI->>UI: useEffect seeds chat input once

Loading

_{Reviews (2): Last reviewed commit: "fix(review): address Greptile findings o..." | Re-trigger Greptile}

[greptile-apps]

Importing underscore-private helpers across router modules

_dataset_s3_key, _dataset_volume_path, and _dataset_ref_for are all prefixed with _, signalling internal implementation details of experiments.py. Importing them into a separate router module creates implicit cross-module coupling: any refactor of those helpers breaks samples.py silently (no enforcement from Python, and the underscore convention actively signals "do not import"). These three functions compute paths/keys that are owned by the dataset-upload concern — they belong in a shared utility module (e.g. services/dataset_paths.py) so both routers depend on a stable public API, not on each other's internals.

Context Used: backend/routers/AGENTS.md (source)

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

[greptile-apps]

Synchronous file I/O blocks the asyncio event loop

abs_path.read_bytes() (and the p.stat().st_size / p.is_file() calls in list_samples) are synchronous filesystem operations running directly on the event loop thread. For the licence-plate sample (3 JPEG images + an annotations JSON) each read_bytes() call can block for several milliseconds, which stalls all concurrent SSE streams — the backend AGENTS.md calls SSE throughput a first-class concern. Wrap the blocking work in await asyncio.to_thread(abs_path.read_bytes) (or loop.run_in_executor(None, ...)) to keep it off the event loop.

Context Used: backend/AGENTS.md (source)

[lucastononro]

Merged into integration branch staging-v0.0.5 (see the STAGING-v0.0.5.md ledger on that branch for merge order, Greptile follow-ups and test results). These changes will land on main via the staging merge — closing to clear the queue.