fix(tasks): respect BACKGROUND_TASK_FOREGROUND_GATE in heartbeat-triggered background task stops#5538
Open
maxmilian wants to merge 1 commit into
Open
fix(tasks): respect BACKGROUND_TASK_FOREGROUND_GATE in heartbeat-triggered background task stops#5538maxmilian wants to merge 1 commit into
maxmilian wants to merge 1 commit into
Conversation
…gered background task stops stop_background_tasks_for_foreground() cancelled every executing scheduler task unconditionally, so browser heartbeats aborted scheduled tasks even with BACKGROUND_TASK_FOREGROUND_GATE=false (the /api/activity/heartbeat call site is not gated, unlike the interactive middleware). Gate the method itself so all call sites honor the opt-out. Runs cancelled by this path also recorded the misleading 'Stopped by user' and missed the 15-minute foreground defer, because only the in-task monitor set foreground_cancel['hit']. Track externally stopped task ids in _foreground_stops so the CancelledError path applies the foreground pause message and defer semantics. Also pass BACKGROUND_TASK_* through docker-compose so .env settings reach the container. Fixes odysseus-dev#5536 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
stop_background_tasks_for_foreground()cancelled every executing scheduler task unconditionally, so browser heartbeats aborted scheduled tasks even withBACKGROUND_TASK_FOREGROUND_GATE=false— the/api/activity/heartbeatcall site is not gated, unlike the interactive middleware (which already no-ops viashould_track_interactive_request). This PR gates the method itself on the env gate so every call site (and any future one) honors the opt-out. It also fixes the two secondary symptoms from the issue: runs cancelled by this path recorded the misleading "Stopped by user" instead of "Paused because Odysseus became active" and missed the 15-minute foreground defer (only the in-task monitor setforeground_cancel["hit"]) — externally stopped task ids are now tracked in_foreground_stopsso theCancelledErrorpath applies the correct message + defer, with cleanup at the task-levelfinallyso the flag can't leak from a queued-behind-the-semaphore cancel and misclassify a later genuine user stop. Finally,docker-compose.yml(and both standalone GPU variants) now passBACKGROUND_TASK_*through to the container, so.envsettings actually reach it.Target branch
dev, notmain. All PRs land indev;mainis curated by the maintainer at each release. If your PR is onmainby accident, click "Edit" on this PR and change the base.Linked Issue
Fixes #5536
Type of Change
Checklist
devdocker compose uporuvicorn app:app) and verified the change works end-to-end. Type-checks and unit tests are not enough.How to Test
Automated:
python -m pytest tests/test_foreground_gate_heartbeat_stop.py— 4 regression tests, all red on unmodifieddev(except the gate-on guard) and green here. Full suite: 4619 passed; the 5 remaining failures (docker-socket / workspace-confine) reproduce identically on unmodifieddevon the same machine — pre-existing, unrelated.Live app verification (what I ran,
uvicorn app:appwith a black-hole LLM endpoint so a task hangs mid-run):BACKGROUND_TASK_FOREGROUND_GATE=false uvicorn app:app --port 7801, create an LLM task whoseendpoint_urlpoints at a TCP socket that accepts and never responds,POST /api/tasks/{id}/run, wait for the run to reachrunning.POST /api/activity/heartbeata few times (what an open browser tab does).dev: the run flips toabortedwith"Stopped by user"— the bug.running; heartbeats no longer touch background tasks while the gate is off."Paused because Odysseus became active"(and the task is deferred ~15 min) instead of pretending the user stopped it.docker compose configrenders the newBACKGROUND_TASK_*env passthroughs; settingBACKGROUND_TASK_FOREGROUND_GATE=falsein.envnow reaches the container.User-facing stop is unchanged:
stop_task(the/api/tasks/{id}/stopendpoint) still records"Stopped by user"with normal rescheduling — covered by the existingtest_task_scheduler_cancel.pyplus a new leak-regression test for the queued-cancel path.Visual / UI changes — REQUIRED if you touched anything that renders
N/A — backend only (
src/task_scheduler.py,src/interactive_gate.py, compose files, tests). No rendering surface touched.Screenshots / clips
N/A — backend only.