feat(backend): B7 NDJSON streaming endpoint - #103
Conversation
Adds a new streaming route for large batches that emits one peptide result per line as application/x-ndjson — UI can render progressively instead of waiting on a multi-minute POST. Event grammar (per WAVE_2_8_DISPATCH §6.1): header — opens the stream, carries total peptide count row — one per successful peptide, carries the normalized entry error — one per failed peptide, isolates the failure footer — closes the stream with done:true Pipeline reuses process_single_sequence so single, batch, and stream paths produce identical results for the same sequence. Each peptide runs in a worker thread via anyio.to_thread.run_sync; await anyio.sleep(0) between peptides gives the loop a chance to detect client disconnect as CancelledError. X-Accel-Buffering: no header is included so nginx in front of the prod box does not batch the response into one chunk. Existing POST /api/predict (single) and POST /api/predict/batch (small-batch FASTA/CSV) routes are unchanged. Tests: 4 cases covering envelope, per-peptide error isolation, and Pydantic validation (empty list, missing fields). All green; 623 of the pre-existing 624 backend tests still pass (the 1 failure in test_esm2_embedder is a pre-existing env-var precedence quirk unrelated to this route).
|
Warning Review limit reached
More reviews will be available in 18 minutes and 54 seconds. Learn how PR review limits work. Your organization has used up its prepaid credits, and credit purchases are no longer available. Enable the review add-on in the billing tab to keep reviews running — you're only billed for reviews past your plan's rate limits ($0.25/file). ⌛ How to resolve this issue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based credits. 🚦 How do rate limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan refill rate. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, the refill rate gradually slows as usage increases. The highest same-day bursts are limited more strictly. Please see our Fair Usage Limits Policy for further information. ℹ️ Review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (2)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Summary
POST /api/predict/streamreturningapplication/x-ndjsonfor large-batch predictions.header(total) → N ×row(orerror) →footer(done).errorevent but does not abort the stream.anyio.to_thread.run_syncruns each predict in a worker thread;await anyio.sleep(0)between peptides gives the loop a chance to detect a client disconnect.X-Accel-Buffering: noheader so nginx in front of the Hetzner prod box does not batch the response into one chunk.POST /api/predictandPOST /api/predict/batchroutes are unchanged.Test plan
Out of scope (per dispatch)
backend/schemas/api_models.pyresponse shapes.process_upload_dataframeinternals.Closes B7 from `docs/internal/WAVE_2_8_DISPATCH_2026_06_18.md` §3.