Description
In enrich_chunks, a single chunk that produces a non-transient ProviderError (e.g. unparseable JSON from the model) aborts the entire concurrent batch. asyncio.gather is called without return_exceptions=True, so the first re-raised error cancels every other in flight request in that batch. Resume picks up from the last checkpoint on the next run, but the user sees a crash mid-pipeline and concurrent work is wasted.
Steps to reproduce
king-scrape https://hono.dev/docs --name hono-test --display-name "Hono Test" --yes- Let the pipeline reach the
[enrich] running... stage on a corpus large enough that at least one chunk gets a malformed JSON response from the model (~500+ chunks with a preview quality model is enough in practice).
- Wait for the first non-transient
ProviderError to surface.
Expected behavior
A bad response from one chunk should be treated as a per-chunk failure: log it, mark that chunk as unprocessed (the existing "no valid response after retries" path), let the schema fallback try, and continue the rest of the batch. Resume should not be triggered just because one chunk in 500 was malformed.
Actual behavior
The whole batch aborts. asyncio.gather propagates the first exception, every other coroutine in that batch is cancelled, and the user sees a stack trace.
Traceback (most recent call last):
File ".../king_context/scraper/cli.py", line ..., in main
asyncio.run(run_pipeline(...))
...
File ".../king_context/scraper/enrich.py", line 341, in enrich_chunks
results = await asyncio.gather(*[guarded(c) for c in batch])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File ".../king_context/scraper/enrich.py", line ..., in _enrich_one
raise
File ".../llm_providers/openrouter.py", line ..., in complete
return parse_json_object(content)
llm_providers.base.ProviderError: openrouter: LLM response was not parseable JSONEnvironment
- King Context version (or commit SHA): v0.4.0 (
upstream/main @ 9f0e5b1)
- Install method:
pip install -e '.[all]'
- OS: macOS (Darwin 25.3.0)
- Python version: 3.14.4
- Node version (if installer-related): n/a
- Claude Code / IDE / agent (if relevant): n/a
Affected component
Additional context
Suggested fix: pass return_exceptions=True to the asyncio.gather call in enrich_chunks (src/king_context/scraper/enrich.py:341), treat any returned Exception as a per chunk miss (same as the existing None path), and continue the run. Counts and checkpoints already tolerate missing chunks. The schema-fallback client at clients.schema_fallback is the right place to absorb malformed JSON, so this becomes mostly an error routing fix rather than a behavior change.
Surfaced during end-to-end testing of #46 (content hash provenance + enrichment cache); pre existing, unrelated to that PR.
Description
In
enrich_chunks, a single chunk that produces a non-transientProviderError(e.g. unparseable JSON from the model) aborts the entire concurrent batch.asyncio.gatheris called withoutreturn_exceptions=True, so the first re-raised error cancels every other in flight request in that batch. Resume picks up from the last checkpoint on the next run, but the user sees a crash mid-pipeline and concurrent work is wasted.Steps to reproduce
king-scrape https://hono.dev/docs --name hono-test --display-name "Hono Test" --yes[enrich] running...stage on a corpus large enough that at least one chunk gets a malformed JSON response from the model (~500+ chunks with a preview quality model is enough in practice).ProviderErrorto surface.Expected behavior
A bad response from one chunk should be treated as a per-chunk failure: log it, mark that chunk as unprocessed (the existing "no valid response after retries" path), let the schema fallback try, and continue the rest of the batch. Resume should not be triggered just because one chunk in 500 was malformed.
Actual behavior
The whole batch aborts.
asyncio.gatherpropagates the first exception, every other coroutine in that batch is cancelled, and the user sees a stack trace.Traceback (most recent call last): File ".../king_context/scraper/cli.py", line ..., in main asyncio.run(run_pipeline(...)) ... File ".../king_context/scraper/enrich.py", line 341, in enrich_chunks results = await asyncio.gather(*[guarded(c) for c in batch]) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File ".../king_context/scraper/enrich.py", line ..., in _enrich_one raise File ".../llm_providers/openrouter.py", line ..., in complete return parse_json_object(content) llm_providers.base.ProviderError: openrouter: LLM response was not parseable JSONEnvironment
upstream/main@9f0e5b1)pip install -e '.[all]'Affected component
king-context)kctx)king-scrape)king-research)@king-context/cli).claude/skills/)Additional context
Suggested fix: pass
return_exceptions=Trueto theasyncio.gathercall inenrich_chunks(src/king_context/scraper/enrich.py:341), treat any returnedExceptionas a per chunk miss (same as the existingNonepath), and continue the run. Counts and checkpoints already tolerate missing chunks. The schema-fallback client atclients.schema_fallbackis the right place to absorb malformed JSON, so this becomes mostly an error routing fix rather than a behavior change.Surfaced during end-to-end testing of #46 (content hash provenance + enrichment cache); pre existing, unrelated to that PR.