fix(core): re-read the fence before the queue-empty assert in Device::maintain - #9958
fix(core): re-read the fence before the queue-empty assert in Device::maintain#9958AdrianEddy wants to merge 4 commits into
Conversation
…:maintain `Device::maintain` samples `current_finished_submission` from the fence before calling `Queue::maintain`, but observes `queue_empty` inside it. `Device::poll` takes `&self` and `Device` is `Send + Sync`, so another thread can retire and triage submissions between those two observations, leaving the sampled value stale while the queue is genuinely empty. The defensive assert then fires with the two values off by one. Re-read the fence for the assert. This is sound because an empty tracker implies some thread triaged past `wait_submission_index`, which requires a fence read at least that high, and the fence is monotonic.
andyleiserson
left a comment
There was a problem hiding this comment.
It's possible this is a regression introduced by #9475, but I haven't gone back and studied the old code to assess that, and it doesn't really matter. Mentioning it so the link is here for archeological purposes.
| let finished_submission = | ||
| match unsafe { self.raw().get_fence_value(self.fence.as_ref()) } { | ||
| Ok(fence_value) => fence_value.max(current_finished_submission), | ||
| Err(e) => { | ||
| let hal_error: WaitIdleError = self.handle_hal_error(e).into(); | ||
| return (user_closures, Err(hal_error)); | ||
| } | ||
| }; |
There was a problem hiding this comment.
I guess reading the fence value is fairly cheap, but even so, I'm not sure we want to be doing extra bookkeeping for asserts without a compelling reason, which I'm not sure I see here.
I thought about removing the assert entirely, but I think another possibility is:
- Change
if queue_emptyon line 1074 toif queue_empty && wait_result == Some(true)(plus changes to savewait_result) - Change the assertion to:
assert!(wait_submission_index.is_some_and(|w| finished_submission >= w)
The idea being, if the poll was successful, then:
wait_submission_indexwas notNone- On a successful (not timed out) wait, any fence value read after that really should exceed
wait_submission_index - If the poll was not successful but we happened to empty the queue after that, give up on this assertion.
I guess a consequence of the change to the queue_empty condition on line 1074, is that some racy cases will return Err(WaitIdleError::Timeout) instead of Ok(PollStats::QueueEmpty). That seems like it should be okay? Anything that is sensitive to the difference is probably racy and broken anyways. And it gives back the protection the assert would have offered: if the caller really wants to see empty, they can call again, and see that. We'll only return QueueEmpty in cases where we're sure.
Or an entirely different approach, declare that QueueEmpty is a footgun (since the queue can immediately go non-empty again) and stop reporting it entirely, but that would be a breaking change.
There was a problem hiding this comment.
Good call — done in a981c28, with one adjustment: the gate is wait_succeeded != Some(false) rather than == Some(true), i.e. it only excludes the timed-out case. Requiring a successful wait would make PollType::Poll (where the wait outcome is None) unable to report QueueEmpty — poll_all(force_wait: false) computes all_queue_empty from exactly that status, and non-blocking is_queue_empty() drain loops would never terminate.
With that gate the assert keeps its current shape and is sound with the existing sample: when it runs with a wait index, the wait must have succeeded, current_finished_submission was read after that wait, and the fence is monotonic — so no re-read, no extra bookkeeping. It is admittedly close to tautological now (it only catches a backend whose wait lies about success or whose fence values regress), so I'm equally happy to drop it entirely if you'd prefer.
On the consequence: agreed it's fine, and arguably overdue — PollStatus::QueueEmpty is documented as "this implies that the given Wait was satisfied", which the racy path violated, and Wait::timeout already promises PollError::Timeout on a timed-out wait. The changelog entry now spells out the behavior change.
On dropping QueueEmpty entirely: I'd push back a little. For a non-blocking Poll there is no other signal for "all in-flight work has retired", and poll_all / drain loops depend on it. The "queue can immediately go non-empty again" caveat is inherent to any point-in-time status and the docs already state it.
Re-verified with the same stress harness, with the drain loop now asserting that Poll observes QueueEmpty: unmodified trunk still panics in under 1 s on Vulkan and DX12; this branch runs clean for 30 s on each (814 k / 515 k submissions, 4.4 M / 9.5 M wait polls, 12,720 / 8,050 full drains). cargo xtask test poll is 72/72 across all GPU entries.
The re-read is taken from the same fence, later on the same thread, so under the monotonicity this fix already relies on it can never be below `current_finished_submission`. The `max` was therefore dead — and in the one case where it would not be, a backend whose fence regressed, it would hide that from the assert rather than let it fire.
…g the fence
Per review feedback, the extra fence read existed only to service the
defensive assert. Instead, capture the previously ignored `Ok(bool)` wait
outcome and only report `QueueEmpty` when the wait did not time out. A
timed-out wait racing a concurrent poller resolves to `WaitSucceeded` or
`Timeout` instead, and the next poll observes the empty queue. This makes
the documented `PollStatus::QueueEmpty` contract ("implies that the given
Wait was satisfied") actually hold.
The gate excludes only the timed-out case (`!= Some(false)`) rather than
requiring a successful wait, so `PollType::Poll` keeps its ability to
report `QueueEmpty`, which `poll_all(force_wait: false)` and non-blocking
drain loops rely on. With the gate in place, the existing fence sample is
provably at or above the wait index whenever the assert runs: it was read
after the successful wait, and the fence is monotonic.
Connections
None.
Description
Device::maintaincan panic on the defensive queue-empty assert when more than one thread polls the same device:current_finished_submissionis read from the fence beforeQueue::maintain, butqueue_emptyis observed inside it.Device::polltakes&self, so another thread can retire and triage submissions between those two observations — leaving the sampled fence value stale while the queue is genuinely empty.Nothing spans the two observations: the lifetime-tracker mutex is only held inside
Queue::maintain, and thecommand_indicesread guard is dropped before it. That guard closes a different window — a poller racing a submitter betweenlast_successful_submission_indexadvancing andtrack_submissionlanding — but it does nothing against a concurrent triager, which drains underlock_life.The window is easy to hit with a zero-timeout
Wait, the idiomatic "has this submission retired yet?" probe: the wait returnsOk(false), thatboolis deliberately ignored, socurrent_finished_submissionstays belowwait_submission_index— and any concurrent poller that then drains the lifetime tracker makesqueue_emptytrue. The racing thread need not be anotherWait: a plainpoll(Poll)drain loop triages too.The fix (per review) captures the previously ignored
Ok(bool)wait outcome and reportsQueueEmptyonly when the poll itself proved it: the branch is gated onwait_succeeded != Some(false), so a timed-out wait falls through to the existing fence-value check and resolves toWaitSucceededorTimeoutinstead; the next poll observes the empty queue. This also makes two documented contracts actually hold:PollStatus::QueueEmpty("this implies that the given Wait was satisfied") andPollType::Wait::timeout("the poll will returnPollError::Timeout").The gate excludes only the timed-out case rather than requiring a successful wait, so
PollType::Pollkeeps its ability to reportQueueEmpty—poll_all(force_wait: false)computesall_queue_emptyfrom exactly that status, and non-blockingis_queue_empty()drain loops rely on it. With the gate in place the assert is sound with the existing fence sample: when it runs with a wait index, the wait must have succeeded, the sample was taken after that wait, and the fence is monotonic. No extra fence read, no new fallible call.Testing
Reproduced on unmodified
trunkwith a small stress harness — one thread submitting copies and draining withpoll(Poll), four threads spammingpoll(Wait { submission_index: None, timeout: Some(ZERO) }). The drain loop additionally asserts that non-blockingPollstill observesQueueEmpty(guarding thepoll_all(false)semantics):QueueEmpty, 470 kWaitSucceeded, 3.52 MTimeout); DX12, 30 s — 515,200 submissions, 8,050 drains, 9.52 M wait polls. No panic, every drain loop terminated.cargo xtask test poll— 72/72 pass across all GPU entries.cargo test -p wgpu-core --all-features— 65 pass. Clippy clean.I have not added this as a
#[gpu_test]; both racing roles are plain public API so it could ship as one, skipping the noop backend (which executes submissions synchronously, so the fence never lags and the race is unreachable there). Happy to add it if wanted.Squash or Rebase?
Squash (the history reflects the review iteration).
Checklist
wgpumay be affected behaviorally.CHANGELOG.mdentries for the user-facing effects of this change are present.