Skip to content

feat(backend): B7 NDJSON streaming endpoint - #103

Open
az-said wants to merge 1 commit into
mainfrom
feat/backend-b7-ndjson-stream
Open

feat(backend): B7 NDJSON streaming endpoint#103
az-said wants to merge 1 commit into
mainfrom
feat/backend-b7-ndjson-stream

Conversation

@az-said

@az-said az-said commented Jun 18, 2026

Copy link
Copy Markdown
Owner

Summary

  • Adds POST /api/predict/stream returning application/x-ndjson for large-batch predictions.
  • Event grammar per WAVE_2_8_DISPATCH §6.1: header (total) → N × row (or error) → footer (done).
  • Per-peptide isolation: a single bad sequence emits an error event but does not abort the stream.
  • anyio.to_thread.run_sync runs each predict in a worker thread; await anyio.sleep(0) between peptides gives the loop a chance to detect a client disconnect.
  • X-Accel-Buffering: no header so nginx in front of the Hetzner prod box does not batch the response into one chunk.
  • Existing POST /api/predict and POST /api/predict/batch routes are unchanged.

Test plan

  • `pytest backend/tests/test_predict_stream.py -v` — 4 cases all green
    • basic envelope (header → 3 rows in order → footer)
    • error event isolates a bad peptide without killing the stream
    • empty peptide list → 422
    • missing required field → 422
  • Backend regression: 623 of pre-existing 624 tests still pass (the 1 failure in `test_esm2_embedder` is an unrelated env-var precedence quirk).
  • Manual: `curl -N -X POST http://localhost:8000/api/predict/stream -H "Content-Type: application/json" -d '{"peptides":[{"id":"u1","sequence":"GVGDLIRKAVSVIKNIV"}],"config":{}}' | head -20`

Out of scope (per dispatch)

  • No changes to backend/schemas/api_models.py response shapes.
  • No changes to process_upload_dataframe internals.

Closes B7 from `docs/internal/WAVE_2_8_DISPATCH_2026_06_18.md` §3.

Adds a new streaming route for large batches that emits one peptide
result per line as application/x-ndjson — UI can render progressively
instead of waiting on a multi-minute POST.

Event grammar (per WAVE_2_8_DISPATCH §6.1):
  header  — opens the stream, carries total peptide count
  row     — one per successful peptide, carries the normalized entry
  error   — one per failed peptide, isolates the failure
  footer  — closes the stream with done:true

Pipeline reuses process_single_sequence so single, batch, and stream
paths produce identical results for the same sequence. Each peptide
runs in a worker thread via anyio.to_thread.run_sync; await
anyio.sleep(0) between peptides gives the loop a chance to detect
client disconnect as CancelledError.

X-Accel-Buffering: no header is included so nginx in front of the
prod box does not batch the response into one chunk.

Existing POST /api/predict (single) and POST /api/predict/batch
(small-batch FASTA/CSV) routes are unchanged.

Tests: 4 cases covering envelope, per-peptide error isolation, and
Pydantic validation (empty list, missing fields). All green; 623 of
the pre-existing 624 backend tests still pass (the 1 failure in
test_esm2_embedder is a pre-existing env-var precedence quirk
unrelated to this route).
@coderabbitai

coderabbitai Bot commented Jun 18, 2026

Copy link
Copy Markdown

Warning

Review limit reached

@saidaz24-meet, we couldn't start this review because you've reached your PR review rate limit.

More reviews will be available in 18 minutes and 54 seconds. Learn how PR review limits work.

Your organization has used up its prepaid credits, and credit purchases are no longer available. Enable the review add-on in the billing tab to keep reviews running — you're only billed for reviews past your plan's rate limits ($0.25/file).

⌛ How to resolve this issue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based credits.

🚦 How do rate limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan refill rate.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, the refill rate gradually slows as usage increases. The highest same-day bursts are limited more strictly.

Please see our Fair Usage Limits Policy for further information.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: bfb67c21-d45b-4904-a27d-e18b9382adb0

📥 Commits

Reviewing files that changed from the base of the PR and between 834d90a and 01ae3ae.

📒 Files selected for processing (2)
  • backend/api/routes/predict.py
  • backend/tests/test_predict_stream.py
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/backend-b7-ndjson-stream

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant