fix: render-error hang — busy-state, stale-running, and cancel fixes (#34) - #35
Merged
Conversation
SceneEventBroker previously only paired enterBusy(Rendering) with DzScene::renderFinished(DzRenderer*). When a render errors, DAZ Studio pops a blocking modal dialog and renderFinished never fires, leaving the plugin permanently busy and every subsequent /execute returning 503 until Studio is restarted. DzRenderMgr::renderFinished(bool succeeded) is now connected as a second, guaranteed exit path -- it fires even on render failure. A m_renderBusy guard flag ensures the two "finished" signals don't double-decrement the busy depth counter when both fire for the same render. Closes daz-script-server-3l5o.
Records the GH #34 investigation, fix, and live DS4/DS6 verification notes for daz-script-server-3l5o, plus creation of the follow-on beads (eaub, q0l4, j1pg).
…34) AsyncRequestManager had no watchdog for a request stuck in RUNNING forever -- if the underlying DazScript call blocks behind a DAZ Studio modal (e.g. a render that succeeds but fails to save), GET /requests/:id would report "running" indefinitely with no way for a client to detect failure short of an unbounded wait. Adds AsyncRequestManager::failStaleRunning(), invoked from the existing 5-minute cleanup timer, which marks any request RUNNING for longer than ServerConfig::ASYNC_STALE_RUNNING_TIMEOUT_MIN (30 min default) as FAILED with a diagnostic message. Since the underlying main-thread call may still be blocked and complete later once a human dismisses the dialog, markCompleted() now no-ops if the request is already in a terminal state, so a late real completion can't flip a timed-out request's status back to completed/cancelled. Live-verified on DAZ Studio 4 (temporarily lowered timeout/interval to 1 min for a fast test, reverted before this commit): a render stuck behind an "Error Saving" modal correctly flipped to "failed" with the timeout message while the modal was still up, and stayed failed after the modal was later dismissed. Closes daz-script-server-eaub.
…is a no-op (GH #34) cancelJson()/cancelRenderJson() only transitioned a request to CANCELLED when it was still QUEUED. For a RUNNING request, they set cancelRequested and fired killRenderOnMainThread() (best-effort, render-only), but left the tracker's status as 'running' while the HTTP response unconditionally claimed {"status":"cancelled"}. If killRender() had nothing to kill (e.g. the renderer already wedged behind a modal) or the request was a plain non-render script (no kill mechanism at all), GET /requests/:id would report 'running' forever, contradicting the cancel response. Both paths now mark the request CANCELLED immediately, the same way the QUEUED branch already did, while still attempting killRenderOnMainThread() as a best-effort interrupt for render requests. markCompleted() already no-ops for requests not in RUNNING state (added for the stale-running watchdog, daz-script-server-eaub), so a late real completion from an underlying call that couldn't actually be killed can't flip the status back. Closes daz-script-server-q0l4.
#34) Live-verified against DAZ Studio 4: 2 billion loop iterations took 60-120s to actually finish on the main thread, which was fine for the test itself (which only waits for "running" then cancels) but starved later tests' main-thread-dependent calls while it drained in the background. 500 million iterations (~8s) still reliably lands in "running" before cancel, with much less residual blocking afterward.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes the render-error hang reported in #34: an Iray render that fails and pops a blocking modal dialog left the plugin permanently wedged, with no way for an HTTP client to detect or recover from it short of restarting DAZ Studio.
SceneEventBrokeronly wiredenterBusy()/exitBusy()toaboutToRender/renderFinished, but a render that errors pops a blocking modal without ever callingrenderFinishedin some paths — leaving/executepermanently 503ing. VerifiedrenderFinished(bool)fires as the guaranteed exit signal even on non-happy-path outcomes, and wired busy-state clearing off of it.RUNNINGbehind a modal. AddedAsyncRequestManager::failStaleRunning(), run from the existing 5-minute cleanup timer, marking any requestRUNNINGlonger thanASYNC_STALE_RUNNING_TIMEOUT_MIN(30 min default) asFAILEDwith a diagnostic message, soGET /requests/:idno longer polls"running"forever.markCompleted()now no-ops for requests already in a terminal state, so a late real completion can't flip a timed-out request back.cancelJson()/cancelRenderJson()previously only transitionedQUEUEDrequests toCANCELLED; aRUNNINGrequest stayedrunningin the tracker even though the HTTP response unconditionally claimed{"status":"cancelled"}, particularly whenkillRender()had nothing to kill (renderer already wedged behind a modal) or the request was a plain non-render script (no kill mechanism at all). Both cancel paths now mark the requestCANCELLEDimmediately, matching the response, while still attempting a best-effortkillRenderOnMainThread()for render requests.Explicitly out of scope (see #34 comment for full findings): suppressing the modal itself, or a headless mode. No hook exists anywhere in the public DAZ Studio SDK to intercept the Iray render-error modal (raised by the closed-source
dzirayrenderer.dll), and a generic dialog-auto-click watchdog was evaluated and rejected as too risky — it could misfire on a legitimate interactive dialog a human is looking at. The fixes above bound the API-visible symptom instead:/executefails fast with 503 immediately, and any in-flight async request (including cancels) resolves to a terminal status within a bounded window rather than hanging indefinitely.Closes #34.
Test plan
./build.sh build— clean build, no errors/executepermanently busyfailedand staysfailedafter the modal is later dismissed"running", is cancelled, and both the cancel response and a follow-up status poll agree on"cancelled"instead of the tracker staying stuck at"running"pytest tests/test_api.py -k "cancel or async"passing against the live plugin