Fix stuck PDF extraction task that permanently blocks new scans (live E2E bugs 3 & 4) - #49
Merged
Merged
Conversation
…new scans
Live E2E testing reproduced the user's bugs 3 and 4 exactly: choosing a PDF
hit "A PDF extraction task is already in progress" with no way to clear it.
Backend logs showed the click never reached the extraction endpoint at all -
the block was purely client-side. Neither "Clear workspace" nor "Clear &
Start Over" freed it: a minimal, isolated reproduction showed the "1 active
task: PDF Extraction 20%" banner reappear immediately after a successful
DELETE /api/v1/session/all.
Root cause: pdf_extraction tasks register with poll_backend=False - they are
updated only by the Streamlit fragment on the page that started them. If that
page is navigated away from, reloaded, or the tab closed mid-extraction,
nothing ever calls update_task() again, so the entry stays "running" forever.
poll_active_tasks() explicitly skips these tasks ("Frontend-managed tasks are
updated by the page itself"), so there is no backend reconciliation path at
all. "Clear workspace" was also gated on controls/mappings/policy only, so a
workspace with nothing but a stuck task rendered the button disabled.
Fixes:
- get_active_tasks() / has_active_task_of_type() now expire a poll_backend=
False task after 30 minutes with no update (real extractions have taken
5-9 minutes in practice) - honesty over silence: the task is marked failed
with an explanatory error, not silently dropped.
- "Clear workspace" enables whenever there is an active task too, not just
controls/mappings/policy, and explicitly cancels active tasks before
resetting so each gets a real "cancelled" transition.
5 new regression tests in app/tests/test_task_staleness.py, wired into CI.
Also updates docs/BACKLOG.md: B1 (hardcoded PDF-path GUIDs) confirmed already
fixed by the merged mapping-engine-rework work; B2 (runner credential) status
updated; B7 (this fix) recorded.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: f9597648-5968-478c-a4a0-b0bd88074134
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Live end-to-end testing against the deployed app (Playwright, authenticated via Easy Auth) reproduced the originally reported bugs 3 and 4 exactly.
What was found
Choosing a PDF hit "⚠️ A PDF extraction task is already in progress" with no way to clear it. Backend logs showed the click never reached the extraction endpoint at all — the block was purely client-side. Neither "Clear workspace" (Home) nor "Clear & Start Over" (PDF page) freed it. A minimal, isolated reproduction (network capture, no test-harness cleverness) showed the
1 active task: PDF Extraction 20%banner reappear immediately after a successfulDELETE /api/v1/session/all.Root cause
pdf_extractiontasks register withpoll_backend=False— they're updated only by the Streamlit fragment on the page that started them (_render_active_pdf_extraction). If that page is navigated away from, reloaded, or the tab is closed mid-extraction, nothing ever callsupdate_task()again, so the entry stays"running"forever.poll_active_tasks()explicitly skips these ("Frontend-managed tasks are updated by the page itself"), so there is no backend reconciliation path at all."Clear workspace" was also gated on
controls or mappings or policyonly, so a workspace with nothing but a stuck task rendered the button disabled.Fix
get_active_tasks()/has_active_task_of_type()now expire apoll_backend=Falsetask after 30 minutes with no update — comfortably above the 5-9 minutes a real extraction has taken in practice. Expiry is honest, not silent: the task is markedfailedwith an explanatory error, not silently dropped."cancelled"transition rather than vanishing.Verified
app/tests/test_task_staleness.py), wired into CI: recent tasks stay active, abandoned ones expire honestly, a stale task no longer blocks a fresh scan, backend-polled tasks are untouched by the staleness check (only thepoll_backend=Falseescape hatch is affected), and a malformed timestamp doesn't crash the check.test_state_init_helpers.py,test_policy_display.py,test_column_mapping.py,test_pdf_upload_state.py) still passes — 26/26.Also in this PR
docs/BACKLOG.md: confirmed B1 (PDF-path hardcoded GUIDs) is already fixed by the merged mapping-engine-rework work (policy_mapper.pynow delegates toAIMappingService); updated B2 status; recorded this fix as B7.Co-authored-by: Copilot App 223556219+Copilot@users.noreply.github.com
Copilot-Session: f9597648-5968-478c-a4a0-b0bd88074134