Repository navigation
Conversation
In Hyrule Field Ocarina of Time reads GPU-written memory about 80 times a second (the "CPU read of GPU-written memory" count), and every such read that GetFlushArea can't serve makes the reading thread wait for a flush on the GPU thread. This measures how much of each frame those waits take, to tell whether they hold the frame rate down. One line, "GPU waits in 10.0 s:", always on: - frames presented, and the time between presents (average, longest); - reactive flushes: count, total and longest wait, per frame, by the cache that held the data (texture or buffer) and by guest core, split into queue (until the GPU thread starts the flush), flush (applying the collected CPU writes, then the downloads: the time spent in Scheduler::Finish waiting for the host GPU, how many flushes had nothing to download and how many bytes they copied back) and wake; - the four areas that waited longest and, for buffer pages, the GPU writes that marked them in that window (what wrote them, the range, the largest one); - the other sync operations guest threads wait for (CPU writes over GPU-written buffers, WaitForIdle); - guest reads and writes that took the rasterizer path, by core, with the time of their cache lookups (GetFlushArea, OnCPUWrite), which take the buffer and texture cache locks the GPU thread holds while it draws; - how busy the GPU thread and the split's back thread were, every Scheduler::Finish with its time, and every wait of FlushQueryResults for pending query results (macro DMA reads of report regions, zortos293#215). Cost: a counter only its own core writes per rasterizer-path read or write; two clock reads more per GetFlushArea or OnCPUWrite call; one load per buffer write mark (four compares while pages are watched); two clock reads and an atomic add per GPU thread idle period, per Scheduler::Finish and per query result wait. The clock is the wall clock's time stamp counter (steady_clock where it is not invariant), the one EDEN_NX2_GPU_PROFILE uses. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…current entry A guest thread that reads GPU-written memory queues a sync operation and waits for it. The operation ran at the next draw on the back, or when the tick sent through the GPU thread's queue got there: behind every command list the front still had, and then behind everything it had handed the back. In Ocarina's Hyrule Field the front was busy 99.9% of the time and these waits spent 1201.6 ms of 10 s in that queue, 1.17 ms a flush, before the flush itself started (test38). The guest synchronized with the GPU work it reads before it read, and with the split every signal it can see (syncpoints, semaphore releases) runs on the back after the work before it. So the back can run the flush at its next entry boundary. GPU::OnCPURead asks it to (GpuSplit::RequestTick): the back runs GPU::TickWork after the entry it is on, or wakes and runs it at once when it waits for work. Without the split nothing changes, the tick through the queue still runs it there. Both renderers also tick after compute dispatches, as they do after draws, so compute work doesn't hold a flush until the next draw. The split test checks that a request wakes an idle back, runs between two entries, and does nothing before the back runs. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…t sync Before the split's front fetches a command list on pages no GPU work has written yet, it waited until its back had run every compute dispatch and copy sent so far (DmaPusher::Step, dma_pusher.cpp:205; GpuSplit:: WaitForWriters), in case one of them writes the list first: the GPU-written page bits (nx2.h:36) learn of a write only when the back records it. In Ocarina of Time's Hyrule Field (EDEN_NX2_GPU_PROFILE, about 50 fps) these were about 7,500 waits a second at 19 us each, 142 ms of every second of the front, which runs in near lockstep with the back. Nearly all of them were for lists the CPU wrote: the lists Ocarina's culling writes every frame are on pages already marked, which take the copy path (ReadGpuWrittenCommands, dma_pusher.cpp:616) and still wait for their writers there (line 642). Only about 220 lists a second take it. The hardware gives a fetch no such ordering. The PBDMA fetches a GP entry's commands ahead of the engines that run the work before it; only a host-level wait in between (a GP entry's sync bit, a host WFI, a syncpoint acquire) makes that work complete before the fetch. NoteSync marks those (dma_pusher.cpp:138, puller.cpp:201 and 395), and semaphore acquires drain the split anyway (puller.cpp:258). Lists on pages that draws write already wait only for the work before the last of them (BufferCache::Nx2ReadListCopy, buffer_cache.h:2208: NeedsSync, then WaitForSync), and draws were never waited for here (IsMemoryWriter, dma_pusher.cpp:37: compute and copy methods only). So wait only for the compute and copy work sent before the last host-level wait: GpuSplit::WaitForSyncedWriters waits for the earlier of the last writer and the last sync. A list on fresh pages that work sent after the last sync writes is read as the CPU submitted it, as the hardware could. The split test checks that a list fetch waits for a writer before the last sync, not for a writer after it, and not for other work before it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…_NX2_DRAIN_CHECK) Opt-in: with EDEN_NX2_DRAIN_CHECK unset, each site only tests a constant. With it set, the GPU thread logs one line every 10 s: Drain checks in 10.0 s: command lists that did not wait for writers sent after the last sync N, GPU-written by those writers M | acquires at addresses the GPU never released A (met before and after the drain ..., only after ..., only before ..., neither ...), at ones it released B (...) | polls of suspended acquires at addresses never released C (...), at released ones D (...) - Command lists on pages no GPU work had written that did not wait for the compute and copy work sent after the last host-level wait (the previous commit): at the list's place, the back checks whether that work made the list's pages GPU-written (DmaPusher::CheckUnwaitedWriters). M must stay 0: a list it counts was read as the CPU submitted it although work before it rewrote it. - Semaphore acquires and the polls of suspended ones (Puller): whether the value met the acquire before the front drained the split as well as after, by whether the GPU had run a release at that address before (any engine: the rasterizers' Query with a payload). Skipping the drain only at addresses the GPU never released, "only before" there is what an acquire would get wrong, "only after" what would wait longer, and "met before and after" the drains it would save. The existing diagnostics are unchanged. Rebased onto upstream: EDEN_NX2_DRAIN_CHECK is a plain flag read again by VideoCore::CreateGPU, as upstream reads its diagnostic switches since zortos293#714, so it works from nx2_switches.txt too, not only from the environment. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…lit when they can Every Switch 2 semaphore acquire drained the split before it read the word (Puller::ProcessSemaphoreTriggerMethod), and so did every poll of a suspended one (CanResumeAcquire): the front waited until the back had run everything queued, in case a release among it changes the word. In Ocarina of Time's Hyrule Field (EDEN_NX2_GPU_PROFILE, about 50 fps) that was about 7,100 drains a second at 34 us, 239 ms of every second of the front. Counted without changing anything (EDEN_NX2_DRAIN_CHECK, per 10 s): of about 48,000 acquires, 25,000 met their value before the drain as well as after it and none before but not after; of 145,000 polls, the drain changed the outcome of about 10. Now a greater-or-equal acquire is checked first without the drain. The words Switch 2 syncs on only advance (NVN2 even releases QMD completions as maxima), so a value that meets one now still does once the back ran the queued work. Only a release the front sent to the word that the back may not have run yet can move it: the front now records those as it sends them (GpuSplit::NoteSemaphoreWrite, from the puller's releases, the 3D and compute engines' report semaphores and the copy engine's), and the last one decides. A counter report or a reduction, whose value the front can't tell, sends the acquire down the drain path as before. - Met: the acquire passes without waiting for the back. When it is met by a release still queued, NoteSync makes what the front reads after it (GPU-written command lists) wait for the work sent before the release. - Not met: an acquire drains as before. A suspended one polls again later without draining: until the back runs a release, nothing sent to the word meets it, and the back's release latches the waiter or reaches memory on its own. After 2 s a poll drains once to report the wait. Words the buffer cache may hold newer than memory (Nx2MayBeGpuWritten), equal and mask acquires, High GPU accuracy and delayed fences (where releases reach memory later) and macros run on the back keep the drain. The split test checks the record of queued writes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
With EDEN_NX2_DRAIN_CHECK set, every acquire or poll of a suspended one that the previous commit decides without draining the split is checked against the drain path anyway: the front drains, reads the word as before (and, for a poll, asks whether a release latched the waiter). The 10 s "Drain checks" line gains: without the drain: acquires passed N (by a release still queued Q), not met after the drain W; polls resumed N (by a release still queued Q), not met after the drain W, kept waiting K (met after the drain M) W must stay 0: a pass the drained word does not meet. Such a decision is corrected in this mode (it takes the drain path), so the run stays right while it counts. M are polls that resume one poll later than a drain would have resumed them. Unset, each site tests a constant. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
When a macro on the split's front reads memory (MME_DMA_READ and MME_DMA_READ_FIFOED, FrontDmaRead), the front drained the whole split (maxwell_3d.cpp:1431 before this change), had the back download what the buffer cache holds of the words (line 1449) and write out the reports the query cache holds there (line 1456), then read them. In Ocarina of Time's Hyrule Field (EDEN_NX2_GPU_PROFILE, about 50 fps) that was 25,500 reads a second, all by macro 0xe2c (24 bytes, a ReportSemaphore::Compare: the render enable condition of two reports), at 9.6 us each: 246 ms of every second of the front, which runs in near lockstep with the back. The words were written long before almost every time: occlusion reports a frame earlier, status reports some objects back. EDEN_NX2_CMDPAGE_DIAG found 0.3% of the reads changed by the drain. What a drained read returns is guest memory once the back ran every entry before it, plus what those two caches write out. Only entries that write guest memory there, make it GPU-written or leave reports for the query cache can change that. The 3D engine's front sends nearly all entries and knows where its own writes go, so it notes them as it sends them (FrontNoteWrites, maxwell_3d.cpp:1429): - constant buffer data at the constant buffer address and offset, a report or semaphore release at the report address (CounterReport writes its value and timestamp, now or once the host result is in), and a pitch inline upload over its lines are placed at their addresses; - a draw, or any entry the back runs while one may be pending in the draw manager (an instance loop's deferred draw is issued by the next register write, maxwell_3d.cpp:361; FrontState::draw_pending, line 1045), a block linear upload, a render enable condition read from memory (ProcessQueryCondition reads it through every cache, line 864) and a syncpoint increment (macros read syncpoints through memory too, memory_manager.cpp:306) count as writes anywhere; - the other executable methods change registers and host state only, and state writes the back sinks run nothing yet. Every other entry, from the other engines, the puller or deferred calls, counts as a write anywhere unless its sender says otherwise (GpuSplit::Writes). GpuSplit::WaitForWrites (gpu_split.cpp:301) waits for the last entry that may write anywhere and the last one placed in the words read. It places writes by the device memory the GPU addresses map to, so the same memory under another GPU address is found: GPU maps drain the split before later commands (GPU::MapInOrder, gpu_thread.cpp:128), and the unmaps it is sent are deferred calls. Past those entries no queued work can make the words GPU-written or add reports there, so the caches hold now what they would after a drain. The front checks them where it can without the back: the buffer cache as before (IsMemoryDirty), and the query cache under its lock for a report whose value or timestamp reaches into the words (HoldsQueries, query_cache_base.h:96). When either holds them, it drains and has them written out as before; otherwise it reads the words at once. The fencing thread writes only reports the cache holds, and those reads drain. The split test checks that a macro's DMA read on the front goes ahead of other queued work and of writes elsewhere, waits for a write of its words under another GPU address and sees it, waits for work that does not note its writes, and where writes are placed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…MA_READ_VERIFY) A macro DMA read on the split's front now waits only for the queued work that can write its words and drains only when the caches hold them (FrontDmaRead). EDEN_NX2_DMA_READ_VERIFY=1 checks that against what every read did before: after reading the words without a drain, the front drains anyway, has the back write out what the buffer and query caches hold of them, reads them again and gives the macro those. It counts the reads whose words differ (CheckDmaRead) and logs every 10 s how many reads it made, how many without a drain, how many of those waited for queued writers and how many differed, with the first 16 differing reads and their words. The differences must stay 0; a guest CPU write that lands between the two reads would count too. Off, it reads one constant flag per read. Rebased onto upstream: EDEN_NX2_DMA_READ_VERIFY is a Maxwell3D member read when the engine is made, as zortos293#714 made EDEN_NX2_3D_METHODS, so it works from nx2_switches.txt too. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A guest CPU read of GPU-written buffer memory (reactive flushing) had the GPU thread download it with Scheduler::Finish. The GPU thread waited for the host GPU with the buffer cache locked, and the reading thread waited for the GPU thread. In Ocarina of Time that was about 1,500 flushes every 10 seconds and 80-90 ms of each second on the split's back thread. Meanwhile every guest core that looked up a flush area spun on the buffer cache lock. Now the GPU thread only records the copies and submits them (RasterizerInterface::FlushRegionForRead, BufferCache::DownloadMemoryForRead). The reading thread waits for that submission's tick itself. It then writes the bytes to guest memory before it reads (FinishRead), and holds the lock only while it copies them out. Until the bytes land: - The ranges stay GPU-modified, so nothing treats guest memory there as current. Another read of them waits for the same copies instead of flushing again (RasterizerDownloadArea::in_flight). - CPU writes keep their bytes. The copy-out leaves out the writes that are still collected, which now includes those InvalidateGPUCache has gathered but not applied yet. Applying a write takes its bytes out of the download (ClearDownload). - A newer GPU write keeps its range GPU-modified. The older download still writes what a synchronous flush would have left there. - Downloads are written oldest first, and before a later fence's downloads (PopAsyncBuffers). None lands over newer bytes. - A write to guest memory from the GPU's side first writes the downloads under it. This covers semaphore releases, inline and constant buffer uploads (MemoryManager) and DMA clears (DMAClear). While no download is pending, the check costs one relaxed load. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…CHECK)
EDEN_NX2_READ_CHECK=n checks 1 in n read downloads (1 checks them all). The
GPU thread copies a second time, in the same submission, what a synchronous
flush of the read would have copied. Once the download is written, guest
memory is compared with that copy, except where a newer value has replaced it
since (CPU writes, ClearDownload). Each byte a synchronous flush would copy
must also be in this download or an earlier one.
Every 10 seconds the log has a line "Read download check
(EDEN_NX2_READ_CHECK): ... bytes differ, ... GPU-modified bytes no download
brought (both must stay 0)". Each difference also gets an error line, up to
16 of them.
The "GPU waits" line now shows separately from the wake-up how long reading
threads spent waiting for their downloads and copying them to guest memory
("then copies on the reading thread"). It also shows how many reads only
waited for an earlier read's downloads. "downloaded" counts the read
downloads again.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… query copies Two writers went around the read downloads of 6f8c97dd86. The query cache writes report values and semaphore payloads into guest memory through device memory pointers, not through the memory manager: the completion operation, the immediate payload path and SemiFlushQueryDirty. A read's download still on its way over those bytes landed the GPU's older copy over them later. FlushRegionForRead even wrote them right after taking the download, where a synchronous flush had written them after its copy. They now land the downloads first, as MemoryManager::Write does. SyncValues copies query results into the buffer cache's buffers on the GPU without recording a write. A read download taken before the copy kept its bytes as the newest, so a later read skipped them. OutdateReadDownloads, which MarkWrittenBuffer now calls as well, takes them out of the earlier downloads' clean ranges, and the later read downloads them again. Both ask ReadDownloadsMayCover first: whether a read download is pending, and on Switch 2 whether the bytes' pages were ever GPU-written. The memory manager's writes ask it too, so semaphore words and constant buffer data on other pages no longer take the buffer cache's lock while a read is pending. SemiFlushQueryDirty runs with the query cache's cache_mutex held. Nothing that holds the buffer cache's lock takes cache_mutex, so the new lock order cannot deadlock. Tests: the next read downloads a query copy made after an earlier read's download again. On Switch 2, a write to a page no GPU write reached doesn't wait for a pending download, and a write to a GPU-written page lands it first. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The check from a06257737b compared every GPU-modified byte of the read
with the GPU's copy at the read, guest memory included where an earlier
read's download brought the bytes. After that download lands, the GPU's
side can write newer bytes there before the later download lands:
constant buffer data or an inline-to-memory write through the memory
manager, which lands the earlier download first and tells the buffer
cache only at the next FlushCaching, or a query result. Synchronous
flushes keep those bytes too: the earlier read's flush had cleaned
them, so the later one doesn't copy them. The check reported them as
differences. In the harness, constant buffer data written that way
gives exactly the test47 line, "guest memory ...+0x30 differs from a
synchronous flush", with the newer bytes in guest memory.
Each check now notes the bytes earlier downloads bring (kept), and
what guest memory holds there right after each of them lands. At its
own landing:
- Bytes rewritten since are left out and counted ("got newer values
after an earlier read's download brought them"). The first 16 are
logged with their writer: the memory manager's writes and the query
cache's results note themselves while read downloads are pending.
- Kept bytes are compared as they landed: a difference there means the
GPU wrote them after the earlier download without taking them out of
it, and is logged as such.
- This download's own bytes are compared as before.
The summary line adds the rewritten count. Tests: constant buffer data
over an earlier read's download is left out, and a GPU write that
skipped OutdateReadDownloads is found.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Ocarina of Time reads back two compute results every frame, one page each. Each read waited 2.6-2.8 ms for its download. The read-trace (test47d) shows that the dispatch writing a page ended on the host GPU 6.8 ms before the read came. The download was recorded only at the read, behind about 770 draws, 95 render passes and 10 submissions recorded since the write. The host GPU had to run all of them before the copy. The buffer cache now learns which pages the guest reads after each GPU write. A page qualifies once two of its writes in a row were read. Four writes in a row without a read stop it, and 64 pages are kept, the one read longest ago making room. When such a page is written, its download is recorded right behind the writing work (RecordEarlyDownloads): - After a dispatch, at once: the scheduler is outside any render pass. Pages written by a draw wait for the next dispatch, a read's flush or the frame's end. - It doesn't submit on its own. The copies go with the GPU thread's next submission, as a fence's do (QueueForHostRead). A thread that must wait for one before it is submitted asks the split's back to submit it. The back submits it itself. An early download is an ordinary read download, recorded before any read asks for it. The read finds it the way it finds an earlier read's download. GetFlushArea now writes such downloads to guest memory itself once the host GPU has run them. The read then waits for nothing. Finished early downloads also land after each dispatch, every 32 draws and at the frame's end, so most reads find the bytes already in guest memory. Everything that keeps read downloads correct applies unchanged: - A newer GPU write before the read leaves its bytes to a download of their own (OutdateReadDownloads). For a learned page, that is the next early download. - CPU writes keep their values (ClearDownload, pending writes at landing). - Writes to guest memory from the GPU's side land the downloads under them first. - Downloads land oldest first, and before a later fence's downloads. Early downloads are bounded to 1 MiB a frame and 32 not yet in guest memory. Past either bound, reads flush on demand as before. EDEN_NX2_EARLY_READ_DOWNLOADS=0 turns them off for comparisons. Tests: a page read after each write gets its download behind the next write. A read before the host GPU ran it waits for that copy alone; after, it waits for nothing. A newer dispatch or draw write gets its own download. CPU writes keep their values. A page no longer read stops after four writes. A semaphore release over a pending early download lands it first. Finished downloads land within 32 draws or at the next dispatch. The 33rd download still on its way, and the frame's downloads past 1 MiB, are left to their reads' flushes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The read trace (EDEN_NX2_READ_TRACE), carried onto early read downloads, and the read download check (EDEN_NX2_READ_CHECK) extended to the reads they serve. The read trace is new in this series. It places a guest read of GPU-written memory on the host GPU's timeline: Vulkan writes device timestamps at the start and end of every submission, after each draw or dispatch that writes pages the guest read back before, and around the read's copies; the reading thread reads the device's clock while the GPU thread runs the flush (calibrated timestamps, an extension the variable enables). Once the bytes are in guest memory, the wait splits into: until the write's work ended on the GPU, until the copy could start, the copy, the wake-up, the buffer cache lock and the copy into guest memory. Every 10 s the log has a "Read trace" line for all reads and one for each of the four areas that waited longest. With the variable unset, nothing changes but a render pass count. Read trace: the copy marks now go around every read download's copies (RecordReadDownload), early ones included. The work marks come from NoteWorkRecorded. For a read that waits for an early download, "the write's end to the copy's start" is the GPU time between the dispatch and its copy. Each line also counts the reads that "waited for an early download". Every 10 s a new line follows the "all" line, reported even in frames where no read waits (TickFrame): Read trace early downloads: N recorded (K KiB) | reads they served: a found the bytes in guest memory, b found them done on the host GPU (landed at the read), c waited for them (in the lines above) | writes from the GPU's side that waited for one: d (t ms) The reads counted under a and b don't wait at all, so the per-read lines above leave them out. Read check: before this commit the check never saw these reads. A landed early download leaves the page clean, and one still on its way covers the read, which then records nothing. Now an early download that lands records which bytes it wrote to guest memory and what they were. In check mode, a read that an early download serves flushes. Its check copies the read's bytes from the GPU at the read, as a synchronous flush would. The bytes still on their way are compared as kept bytes are. The landed ones are compared the same way, against their record: - If guest memory changed since the landing, the bytes are counted as rewritten and not compared. - Otherwise they must equal the GPU's copy. A difference means the GPU changed them after the early download without anything downloading them again. That is an error, logged like other differences. Newer downloaded bytes or guest values drop the record (WriteDownloadedMemory, ClearDownload), and each read takes what it compared. A check whose read gets all its bytes from other downloads becomes a download of its own. The read waits for it, as does a write from the GPU's side over those bytes. With neither variable set, nothing changes. Tests: early-downloaded bytes, landed or on their way, compare equal (0 differ, 0 uncovered). A GPU change nobody recorded after the landing is found. A guest write there is left out as rewritten. Rebased onto upstream: EDEN_NX2_READ_TRACE is a plain flag read again by VideoCore::CreateGPU, as upstream reads its diagnostic switches since zortos293#714, so it works from nx2_switches.txt too, not only from the environment. The test build also had the pipeline library work (zortos293#579) here in vulkan_device.cpp; this commit adds only the calibrated timestamps extension. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
LsfgBindings holds u32 binding counts and LSFG_MIP_LEVELS is a size_t, so the pair built from it narrows implicitly. MSVC reported that as C4267, a warning, until the read trace header that vk_scheduler.h includes in this test build turned the same instantiation into C4244, which the build treats as an error (/we4244). Make the conversion explicit. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Ocarina of Time's render enable macro (0xe2c, code at instruction 0x142) reads the six words of a report pair with MME_DMA_READ_FIFOED and writes the render enable mode, about 25,000 times a second in Hyrule Field. On the split's front each read waits for the queued work that can write the words (FrontDmaRead): 25,060 waits a second, 252.8 ms of every second of a front that is 99.9% busy (test46b, No sharpening; the back is 77-81% busy). The back holds what the read needs at the call's point of the method stream, so such macros now run there. A static pass over the macro code (TuringMacroInterpreter::AnalyzeEffects, macro_turing.h:266) follows a call from its start along every path, each branch either way and each loop any number of times, with the register, carry and method values the code fixes: it finds the methods the call can write (as long as each is a constant of the code) and whether it uses the data RAM. A macro whose code reads memory into its FIFO, uses no data RAM and writes only registers the front can do without the value of (BackWritable, maxwell_3d.cpp:1200) goes to the back (FrontRunOnBack, line 1562): its call is one entry with its parameters (GpuSplit::DeferWords), and the back runs it with a macro engine of its own, its code copied from the front's, where ProcessMacro would. It reads the words as a drained front did. Not back-writable: macro code and bindings, the shadow RAM mode, draws, constant buffer data, reports, uploads and syncpoints (the front tracks what they do), the registers the front reads to note their memory writes, registers the back changes itself (the bind pointer, shadow_scratch[0], logic_op.enable on AMD) and MME_DMA_READ, whose words land in the front's data RAM. The registers such a macro can write, and in a tracked shadow RAM mode their entries, are then the back's until the front knows them again: a front write to one is sent even if the front's copies hold the value, a replay first takes the entry from the back (FrontTakeOver, line 1699), and a front macro or DMA read that reads one waits for the back to run the macro and takes the back's value (FrontRegister, line 1685). No entry after that macro writes the register, so the value can be read while the back runs on. The waits count as "back register", and the macro code changing under queued back calls as "back code". The back counts writes outside the predicted set; the front then runs that macro itself from then on and takes the back's registers as SPLIT_VERIFY does. Macros that don't qualify keep the narrower wait (MKW's read into data RAM word 0x27). EDEN_NX2_BACK_MACROS=0 turns this off. turing_macro_effects.cpp checks the analysis on hand-made code and against the interpreter on random code. The split DMA read test now also runs a macro on the back: the front goes on past queued work and the back reads the word written behind it, a write of the value the front's stale copy holds is sent, a front macro that reads the register waits and gets the back's value, and a replay takes the shadow RAM entry from the back. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
EDEN_NX2_BACK_MACRO_VERIFY=<n> (64 without a number): the first n calls
of each macro the back runs (FrontRunOnBack) also run the old way. The
front drains the split, runs the call against its copies of the
registers (the back's for those back macros may have written, exact
once drained), its DMA reads made once the caches wrote the words out,
and writes nothing (FrontRunTheOldWay). It then sends the call, whose
back run records its writes (MacroEngine::split_record), drains again
and compares the (method, value) lists. The first 16 differences are
logged with both lists; there must be none.
Every 10 s it logs the calls on the back, how many were checked and
differed, and the front's reads and replays of registers back macros
wrote, with how many of those waited for the back. Off, it costs a test
of a static flag where it counts. The waits themselves, and the back's
macro time, are in EDEN_NX2_GPU_PROFILE ("back register" and "back
code", the back thread's "macro" zone).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…s GPU work wrote In Ocarina of Time's Hyrule Field (EDEN_NX2_GPU_PROFILE, test46b and test47) the split's front waits for the host GPU 169-204 times a second, 550-765 us each (93-156 ms of every second), for the copy of a GPU-written command list page (BufferCache::Nx2ReadListCopy, the "list GPU wait" zone). The back has nothing queued meanwhile: across two boots its idle time moved one for one with these waits. Only pages whose last noted write is GPU work take that copy: a compute dispatch's storage write, or a copy engine copy of GPU-written bytes (DMACopy notes only the source ranges that are GPU-modified). The CPU, inline uploads and copies of bytes guest memory holds write guest memory and retire the copy instead. So the words GPU work wrote exist only on the host GPU, and those must wait. But the copy served the whole 4 KiB page: a list elsewhere on the page waited for it as well, and read from the copy the words the host had written, which the buffer it came from need not have had uploaded yet. - BufferCache::Nx2ListPage keeps the page's words GPU work wrote that guest memory lacks: set by every GPU write noted to the page (Nx2NoteListWrite, the whole range a dispatch or copy may write), cleared for the words the host writes whole (Nx2NoteListHostWrite) and for all of them when the front flushes the page (Nx2NoteListSynced, Nx2FlushListPages). - Once the front has flushed the page (before, GPU work may have written it unnoted, before it became a command list page), a list on none of those words reads guest memory and waits for no copy. A list on some of them reads those words from the copy and the others from guest memory. - The copy engine tells the command list pages what it writes to guest memory: DMACopy notes the copied bytes as a host write before it notes the GPU-written source ranges as GPU writes, and DMAClear notes the cleared words as an inline write does. Both left the page's copy current, so the next list there read the bytes from before. DMAClear also checked only a quarter of its range for registered buffers: it passed the word count as bytes. - A list on a GPU-written page waited for every dispatch and copy sent so far (GpuSplit::WaitForWriters). As for lists on other pages (Step, since dab34bfd8c), the work that rewrites it is ordered before the fetch only by a host-level wait between them: wait for the work before the last one (WaitForSyncedWriters). The profiler counts this wait as its own site, "list copy writers". - While the front waits for the GPU to finish a copy (BufferCacheRuntime::WaitSubmittedTick), the back runs only what the front published. When the front had not waited for the back before (the writers were done), the entries sent since its last publish, fewer than 256 ring words, waited out the copy as well: it publishes them first. Tests (buffer_cache_ports): lists beside the words a dispatch wrote read guest memory, lists on them read the copy and spanning ones both; pages the front has not flushed read every word from the copy; host writes give the words they cover whole back to guest memory and retire the copy; a flushed page reads guest memory until GPU work writes it again; copy engine clears give their words back. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…_NX2_LIST_STATS, EDEN_NX2_LIST_VERIFY) EDEN_NX2_LIST_STATS=1 logs every 10 s from the front, for the command lists on GPU-written pages (DmaPusher::ReadGpuWrittenCommands): - "list stats": the lists and their sizes; for each page read, how Nx2ReadListCopy served it (page synced, words clean, copy ready, copy waited, unknown); the copy waits' time and spread; the reads and waits by the page's last writer (dispatch, copy, draw); how many copies held the bytes guest memory already had (the GPU rewrote the same words); and at copy waits: whether the back had work and how much of it the front had not published yet, whether other channels had lists queued for the GPU thread (which could run while the channel waits for the copy), and how long before the wait the back had recorded the copy. - "list stats writers": the dispatches (by shader hash) and copy engine copies (by source GPU address) that last wrote the pages read, with their reads, waits and writes to command list pages. - "list stats pages": the pages read, the bytes the lists read on them, and the bytes and writer of their last write. EDEN_NX2_LIST_VERIFY=1 reads each such list again as the commit before this one did: after every dispatch and copy sent so far (WaitForWriters), from the pages' whole copies (Nx2ListStats::t_reference), or drained and flushed. When the two differ it also drains and flushes, which tells the stale read. "list verify" counts lists and differences every 10 s, the first 32 differences are logged with the list's pages, and the front parses the words read last. The writers are tagged in RasterizerVulkan::DispatchCompute and AccelerateDMA::BufferCopy, the queued lists counted in ThreadManager. Test (buffer_cache_ports): the read as before takes the whole copy. Rebased onto upstream: EDEN_NX2_LIST_STATS and EDEN_NX2_LIST_VERIFY are plain flags read again by VideoCore::CreateGPU, as upstream reads its diagnostic switches since zortos293#714, so each works from nx2_switches.txt too, not only from the environment. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Semaphore acquires and the polls of suspended ones no longer drain the split. Fix 2 only checked greater-or-equal acquires without the drain, and drained for the rest, for words shaders write, for releases it couldn't value, and whenever the value wasn't met yet. In Ocarina of Time that was about 4,800 drains a second at the acquire site (182 ms/s on the GPU thread), most of them at words QMD releases advance. Now the front decides each acquire (equal, greater-or-equal, mask) from what it can see: - The last release sent to the word, while the back hasn't run it: met passes (with NoteSync, as before), not met suspends. - A release the front can't value (a 3D counter report, a reduction it can't compute): a positioned wait for that release only, then memory. - Otherwise memory. Greater-or-equal holds once met, since the words only advance. Another operation holds only for the value the word has when the channel reaches it: compute and copy work sent before the acquire may release it unseen (QMD releases), so a met value waits for that work first (a positioned wait, not a drain). - Words on GPU-written pages that the buffer cache holds GPU data for: memory still decides greater-or-equal acquires it meets. Otherwise the back flushes the word, the channel suspends, and the polls read memory once the flush ran. This keeps the MustFlushRegion check for semaphores shaders write. An unmet acquire registers its waiter and suspends the channel. A release the back runs latches it (Nx2SemaphoreWaiters::OnWrite). The polls decide the same way without draining, so other releases resume it from memory or the view. The fence work is a plain Defer. It is sent once the back has run everything before the acquire and the acquire still waits, which is where the drain path sent it. Acquires that the queued work releases send none. SEM_EXECUTE acquires never blocked; they drop their drain too, keep the NoteSync it implied, and are checked for their warning only. High GPU accuracy, delayed fences, macros on the back and unmapped semaphores keep the drain path. Compute report reductions are valued on the front when the word's value before them is known: the last release sent to it, or memory when no compute or copy work is queued. That holds only for reports sent directly, since a macro's own launches count as work only once the macro ran. GpuSplit gains a position API: Position, Ran, LastWriterPosition, WaitForPosition, and the position of a queued semaphore write. The GPU thread's poll loop counts a resumed channel as progress, as its comment says, so it keeps polling fast while channels move. Tests: a Catch2 case and the standalone split harness (real back thread) cover the positioned wait for a queued release, which leaves the work after it running. Rebased: the back macros commit before this one added the same GpuSplit::Position(), so this commit keeps that one instead of adding a second (two definitions did not compile, here and in the next commit, until the commit after them merged the position waits). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
EDEN_NX2_DRAIN_CHECK now checks every decision the semaphore view makes against the drain path: the front drains and reads the word as before anyway. This replaces fix 2's checks, whose code the previous commit replaced. In this mode a pass that the drained word doesn't meet is corrected (the channel waits), and a wait that the drained word meets ends. The 10 s "Drain checks" line ends with: semaphore view: acquires passed N (by memory M, by a release still queued Q, after a positioned wait W), not met after the drain X, suspended S (met after the drain D); polls resumed R (...), not met after the drain Y, kept waiting K (met after the drain Z) X and Y must stay 0. D and Z count acquires and polls that a drain would have passed or resumed sooner. EDEN_NX2_ACQUIRE_STATS counts the decisions without changing what runs (use it without the drain check, whose drains change the timing). Every 10 s it logs: - equal, greater-or-equal and mask acquires: passed by memory, by a queued release, after a positioned wait, suspended, or drained; - the suspended ones by word: on GPU-written pages (and held newer by the buffer cache), and by the releases the front sent there (puller, 3D reports, compute reports, copy engine, or none: QMD releases or the CPU); - what resumed them: a latch by its writer (payload releases, query results, QMD reductions, QMD maxima), memory, a queued release, or the drain check; also how many after a flush; - how long they waited: total, maximum, and how many over 1 ms. Nx2SemaphoreWaiters records which writer latched a waiter for this. EDEN_NX2_GPU_PROFILE gets an "acquire poll" zone around the poll in Scheduler::ResumePending. "resume pending" then holds the resumed channels' own command processing, which is most of it: a channel suspended on an acquire parses the lists it was sent meanwhile inside ResumePending. In test47 that was 165 ms/s at 50,000 calls/s, while "push" was 3.6 ms/s. Rebased onto upstream: EDEN_NX2_ACQUIRE_STATS is a plain flag read again by VideoCore::CreateGPU, as upstream reads its diagnostic switches since zortos293#714, so it works from nx2_switches.txt too, not only from the environment. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ires The back macros and the semaphore view each added GpuSplit::Position(), and a positioned wait with the same body (WaitUntil, WaitForPosition). Keep the semaphore view's pair and wait for back-owned registers and macro code through WaitForPosition. Rebased: the semaphore view commit no longer adds a second Position(), so this one removes only WaitUntil. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…_GPU_COST) The host GPU is busy about 10 ms of a 20 ms frame in Ocarina of Time, and only about 3.4 ms of that is timed draws and dispatches. EDEN_NX2_GPU_COST=1 shows where the rest goes. It needs EDEN_DKB_FRAME_TRACE=1, whose frames it times. In a sampled submission both command buffers record through a copy of the dispatch table whose work entries (draws, dispatches, render pass begins and ends, barriers, copies, clears, fills, query commands, conditional rendering, transform feedback) write a bottom-of-pipe timestamp after the real call. Binds and dynamic state write none. The timestamps of a command buffer, from a top-of-pipe one at the submission's start to one at its end, split its GPU time into intervals. Each interval ends with one Vulkan command: the time from the end of the work before it to the end of that command's own. Each interval goes to the command's kind and to the source line that recorded it: - Scheduler::Record and RecordWithUploadBuffer take the caller's std::source_location, kept per command while the variable is set; - a render pass's end carries the line that ended it; - its begin carries the line that ended the pass before; - a submission's commands carry the line that asked for it; - BufferCacheRuntime::CopyBuffer passes its caller's line on, so its render pass ends and copies count at the upload, download or copy that asked. After each render pass end, the trace also notes what ran before the next begin (only barriers, copies, dispatches, queries, clears) and whether that begin resumed the same framebuffer. Each upload that can't go to the upload command buffer is also classified by where its pages were last used: done on the GPU, submitted but not done, or in the command buffer being recorded. Every 10 seconds, over the frames sampled in that time, per frame: - "GPU cost": busy time, then each category with its kinds; - "GPU cost by site": the 28 source lines and kinds with the most time; - "Render pass breaks at <line>": for the 14 lines that ended the most passes, the draws in each pass, how many had storage writes, the next begin, what ran between, and the GPU time of the end, the work between and the next begin; - "Uploads": uploads a second by where they ran and why. The "Render pass ends" lines of EDEN_NX2_GPU_PROFILE also name the function now. The trace only reads: the commands, their order and their synchronization stay as they are. With the variable unset nothing changes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…fter waits In test47f most suspended acquires wait for a compute dispatch: of about 20,000 resumes per 10 s, about 15,000 were latched by a QMD reduction or maximum. The front could not tell what a launch releases, so it suspended the channel until the back ran it. The front now runs the QMD scheduling of every channel as the back will (FrontQmdView, video_core/front_qmd.h): the same NativeQmdScheduler code, on a table of its own, fed the same signals in ring order (PCAS methods and inline QMD launches, sent directly or by macros). When a launch runs there, the front notes what its releases write: - plain and four-word releases: the payload; - maxima: the maximum with the word when the front knows the word (the last release sent there, or memory when nothing queued writes it), else the payload as a lower bound; - reductions: the result when the word is known, else unknown. The semaphore view takes a lower bound for greater-or-equal acquires of it or less (passed by the queued release), and waits for its position otherwise. This is exact only if the front reads the descriptors the back will read. The back reads them later, from guest memory without the GPU caches. They hold the same where no map still queued can change, and where nothing queued writes the QMD except through the caches, on pages GPU work never wrote. An inline QMD launch reads the words the back writes right before. Elsewhere the front's table may no longer match the back's: it stops valuing, counts the launches as writing anywhere, and takes the back's table over at the next QMD method after the back has run everything sent. To tell the writes apart, GpuSplit::Writes gets Caches: entries that write the caches anywhere and guest memory only through them, where GPU work wrote. WaitForWrites still waits for them. The new MemoryWritesTarget does not. Changes: - Caches: 3D draws (and the deferred draw after a macro), conditional rendering, syncpoint increments, back macros, fence work, list-end submits and CPU flush requests. - Noted, with their writes placed: puller releases, compute reports (16 bytes) and inline QMDs (256 bytes). - Noted, writing nothing: channel binds, the dirty checks of macro payloads, command signals, invalidations, methods an engine only keeps in its method sink, and compute register and macro code writes. - Unchanged (anywhere): uploads, copies, maps and the rest. Write slots go from 1,024 to 16,384, so a QMD rarely shares one with the constant buffer words of queued draws. QMD table entries no longer keep a copy of the descriptor (it was read again before each use), which keeps the table copy small. Churn. In 47e every acquire drained the split, and a drain resets the publish interval to 8 words, so the back restarted at once. In 47f a resume or a positioned wait kept it at 256. The back then idled until the front had parsed 256 more words: 3,377 idle periods a second at 90 us, against 9,520 at 30 us. Now the end of a positioned wait, a resumed channel and each command from the CPU publish after 8 words again, doubling to 256. A met suspended acquire also no longer polls twice per resume (the scheduler asked, then the channel's Step asked again). Tests: the QMD scheduler with its own table, the split's write classes and lower bounds, and the front valuing inline, deferred, chained and reduced launches against a back run on the same thread. A harness ran FrontQmdView against a model of the back lagging behind. It covered 200 random runs with patches, unplaced writes and releases into QMDs. In 589,000 valued signals and 233,000 checked releases, the note always matched what the back wrote. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
EDEN_NX2_DRAIN_CHECK now also checks the QMD view. The back records what
it releases for QMD launches. After the QMD methods the front sent (a
signal, an inline launch, a run of them, or a macro's write of one), it
compares that list with what the front noted, in order. Same words, and
values equal, or at least as large for a lower bound. The "Drain checks"
line ends with:
QMD view: methods valued N with R releases, released otherwise than
noted W; not valued U
W must stay 0. The first 16 that differ are logged with both lists. The
acquire checks already cover the decisions the new notes make.
EDEN_NX2_ACQUIRE_STATS adds:
- "passed by a queued QMD launch's release": passes by a queued release
at words the QMD view noted releases to;
- QMD launches the front valued, among the releasers of words that
suspended acquires waited at ("none" is now other QMD releases and the
CPU);
- a "QMD view" line: signals valued, and why the others were not (table
not synced yet, work that may write anywhere still queued, a QMD on a
GPU-written page, a write into the QMD still queued, a release of the
same signal into it); releases noted exact, at least or unknown; how
often the front took the back's table over, and how long that took.
The resume counts are no longer doubled: the clean commit polls a
suspended acquire once per resume.
EDEN_NX2_GPU_PROFILE adds to the profile line:
- "publishes to an idle back late by T ms (N/s)": time the back waited
for work while the front held words it could have run. It counts from
when the back ran out, or from when the first of those words was sent,
if that was later. This is the churn cost the clean commit's earlier
publishes cut.
- "resnapshot words": words copied again when a channel resumes, with a
"resnapshot" zone for the time the copies take.
DeferFunction takes a write class, so the checks' entries write nothing
for the view.
Checked with a harness (FrontQmdView against a model of the back, 200
random runs): all 519,764 check lists matched, as did the clean commit's
233,000 notes. A second program prints the new report lines.
Rebased onto upstream: GpuSplit takes the publish profiling from
Nx2Profile::Enabled() when the split starts, instead of reading
EDEN_NX2_GPU_PROFILE at static initialization. Since zortos293#714 the profile
switch is read again when the GPU is made, so a switch set in
nx2_switches.txt now profiles the publishes too.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The test48b GPU cost run fell from 59 to 29 fps on the split GPU thread's back. The upload classification caused it, not the timestamps: Buffer::MarkUsage noted a tick for every 4 KiB page of each new range at each tick, and the largest buffer is 224 MB. "bind stage buffers" took 4.8 us a call instead of 0.43. EDEN_NX2_GPU_COST=1 now times the GPU only. =2 also keeps the page ticks and logs the "Uploads" line, as =1 did before. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…loads An upload that can't go to the upload command buffer is recorded in order: a barrier, the copy, a barrier (BufferCacheRuntime::CopyBuffer). Ocarina of Time makes about 600 of them a frame, nearly all inline writes into GPU-written pages (InlineMemoryImplementation, ~200 bytes each). The GPU cost trace counted 614 copies and 1,212 barriers a frame there. They come outside render passes: only 2.2 a frame end one. So they come in runs, for instance between a wait for idle and the next pass. Scheduler::RecordUpload keeps such uploads from staging memory (the stream buffer) pending in an UploadBatch. They are recorded as one command: the same barrier before, all their copies, and the barrier after one, which here also reaches transfers. The batch is recorded before any other command, before the chunk goes to the worker, and before a submission takes its tick. These record the batch first instead of joining it: - an upload that writes bytes a pending one writes; - one that reads a buffer a pending one writes, or writes one it reads; - one that would take the batch past 64 copies. That bounds the overlap check each upload makes on the recording thread. Why nothing loses its order: - The batch holds only uploads recorded with no other command between them. So the barrier before it covers all earlier work for each copy, as the barrier before each upload did. - One by one, the barrier after an upload ordered its copy ahead of later graphics and compute work. The barrier before the next upload ordered it ahead of every later transfer, in later submissions too. The barrier after the batch does both for all its copies, so it names the transfer stage as well. - Nothing orders two copies of one batch against each other. They don't need it: no two write the same bytes, and none writes a buffer another reads. - Each upload still lands in the tick it was recorded in, so staging regions and usage ticks stay right. - Other copies are unchanged: downloads, buffer-to-buffer copies, uploads from dedicated staging buffers, and copies without barriers. It also closes a gap for these uploads. The last upload of a run was ordered ahead of no later transfer, such as a reordered upload of the same bytes in the next submission's upload command buffer once TickFrame has cleared the usage bits. The barrier after the batch orders it. The recording itself (UploadBatch::Record) is what the scheduler runs. A lavapipe harness runs it under synchronization validation: - 400 small uploads, some adjacent, some rewriting bytes, over two buffers, between earlier and later transfer work. Batched and one by one give the same bytes, which match a CPU model, and no hazard is reported. So do 200 uploads that never overlap, split into batches of 64. - A transfer with no barrier in a later submission, writing over the uploads: no hazard after a batch. It was already ordered after every upload but the last when they were recorded one by one. - Controls reported as hazards: - a batch with two copies writing the same bytes (WRITE_AFTER_WRITE); - that later transfer over the last upload recorded one by one; - that later transfer after a batch closed with the old barrier. - The layer (1.3.275) takes ALL_GRAPHICS for every stage of a graphics queue, transfers included, which the specification's ALL_GRAPHICS does not include. So these cases name the graphics stages of the old barriers one by one. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A deferred clear becomes the next render pass. It is begun with a variant of the framebuffer's render pass that clears instead of loading (Framebuffer::RenderPassVariant). The first draw runs in it. The next draw's RequestRenderpass compared the render pass handles and found the framebuffer's own pass, not the variant. So it ended the pass and began another on the same framebuffer: the attachments were stored only to be loaded back. The GPU cost trace counts 10.9 such ends a frame at GraphicsPipeline::ConfigureDraw in Ocarina of Time, nothing recorded between them and 79% resuming the same framebuffer. RequestRenderpass now keeps the pass when it is such a variant on the same framebuffer and render area. Why it's the same: - The variant is made from the framebuffer's own RenderPassKey with only its clear masks set. Without discard masks it differs only in load ops: same attachments, formats, samples, store ops, layouts and resolve attachments. - Ending it and beginning the framebuffer's pass stored every attachment and loaded it back. Staying in the pass keeps the same contents, and the pass ends with the same store ops. - Within one pass, rasterization order puts each draw's attachment accesses after the earlier draws'. The fragment barrier and feedback loop checks, which end a pass when a draw reads what an earlier one wrote, run before RequestRenderpass as before. - Pipelines stay compatible: render pass compatibility ignores load and store ops, and the first draw already ran in the variant. - Queries and conditional rendering continue in the pass instead of closing and reopening around a break. - Variants that discard (MSAA resolve on tilers) store differently and keep the old behavior. Tested on lavapipe under synchronization validation: - a framebuffer made with its own render pass, a clear variant, and two attachment writes; - the old end-and-reload sequence and the kept pass give the same pixels; - validation reports nothing for either. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Every compute dispatch recorded its read barrier as a command of its own, right before the dispatch's: Ocarina of Time's 147 dispatches a frame made 147 more commands for the back thread to record and the worker to run. The barrier now opens the dispatch's command. The Vulkan commands and their order stay the same: - The only command recorded between the two was TraceComputePipeline's, and only when profiling. It counts on the CPU and touches no command buffer. - The dispatch's command records the barrier before anything else, so it still runs when the pipeline isn't bound or DLSS replaced the dispatch. - When DLSS may replace the dispatch, its evaluation is recorded between the barrier and the dispatch. That case keeps the separate barrier. - NoteComputeWrites is a flag for the next render pass, and no pass begins between it and the dispatch. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A buffer copy recorded in order ends with a barrier from transfers to graphics and compute. Later transfers are not in it. A reordered upload in a later submission goes into that submission's upload command buffer with no barrier of its own before it. So when it writes bytes the earlier copy wrote, nothing orders the two writes, and the older one can land last. In-order copies don't count as uses when CanReorderUpload decides. Once TickFrame clears the usage marks, the same bytes can get a reordered upload in the next submission. Uploads from the stream buffer already end with a barrier that reaches transfers (they go out in batches). This adds transfers to the barrier after the remaining copies: buffer-to-buffer copies, uploads from dedicated staging buffers and downloads. It can only add a little waiting. Found while checking the upload batching against the synchronization validation layer. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
zortos293
added a commit
that referenced
this pull request
Nov 8, 2026
Test gate EDEN_NX2_EGLA66_ACQUIRE=1, read once; off, nx2 behaves as before (the two commits below run only behind it). On: - #639 (23457cc, rdenno): Drain waits for the back's idle hook, and buffer cache downloads stay on the scheduler's owner thread. - fb4d4e4 (jpkto, PR #805): greater-or-equal acquires that are already met, and the polls of suspended ones, skip the drain. The front notes the releases it sends only while the gate is on. - The back is fed and settled, which removes the back CPU perf-4's acquire-met-skip added in Mario Kart World (35-38 to 40-43 ms a frame, with about 85k polls/s of waiting acquires). Why the back spent more. The polls run on the front and are the GPU thread's existing pending-channel loop (yields for 500 us, then one-tick sleeps); the back never sees them. But each poll used to drain, and a drain did two things for the back that nothing did once the polls stopped draining: 1. It reset publish_every to 8 words, so the back got the front's next words at once. Without it the front publishes every 256 words, and a back that ran dry in the middle of a parse spins until then. 2. With #639 it asked a back that had run everything for its idle hook, and the back then slept. Without it the back spins its full 8192 PAUSEs each time it runs dry while the front only polls. The back's thread time (EDEN_DKB_FRAME_TRACE, GetThreadTimes) counts that spinning. Now: - Commit publishes to a back that waits for work once the front holds 8 words: the back stores where it ran dry (back_idle_at) and the front compares it with its last publish. This is the publish part of 62db242 (jpkto, PR #805), "publish to the split's back once it waits". - The GPU thread's pending-channel loop asks a back that ran everything and hasn't run its hook yet for it (SettleIdleBack), as a poll's drain did, but without waiting for it. EDEN_NX2_EGLA66_ACQUIRE_PERF4=1 (A/B only) keeps the gate's skips without the feeding and settling, as perf-4 ran them. EDEN_NX2_EGLA66_ACQUIRE_VERIFY=1 drains anyway after each decision made without a drain and counts the words that disagree. Logged every 10 s while the gate is on: "EGLA66 acquire: S passed without a drain (Q by a queued release), L resumed by a latch, D drained, P polls waited without a drain in 10 s; back: W waits for work, Z slept, X ms spinning; E early publishes, H idle hooks asked by polls", and with _VERIFY a warning when any check disagrees. Tests: a waiting back gets 9 words without a publish (and does not as perf-4), the settle decision, and Drain returning only after the idle hook with the gate on.
anthonyloukinas
pushed a commit
to anthonyloukinas/Lichii
that referenced
this pull request
Nov 9, 2026
…ortos293#805) A deferred clear becomes the next render pass. It is begun with a variant of the framebuffer's render pass that clears instead of loading (Framebuffer::RenderPassVariant). The first draw runs in it. The next draw's RequestRenderpass compared the render pass handles and found the framebuffer's own pass, not the variant. So it ended the pass and began another on the same framebuffer: the attachments were stored only to be loaded back. The GPU cost trace counts 10.9 such ends a frame at GraphicsPipeline::ConfigureDraw in Ocarina of Time, nothing recorded between them and 79% resuming the same framebuffer. RequestRenderpass now keeps the pass when it is such a variant on the same framebuffer and render area. Why it's the same: - The variant is made from the framebuffer's own RenderPassKey with only its clear masks set. Without discard masks it differs only in load ops: same attachments, formats, samples, store ops, layouts and resolve attachments. - Ending it and beginning the framebuffer's pass stored every attachment and loaded it back. Staying in the pass keeps the same contents, and the pass ends with the same store ops. - Within one pass, rasterization order puts each draw's attachment accesses after the earlier draws'. The fragment barrier and feedback loop checks, which end a pass when a draw reads what an earlier one wrote, run before RequestRenderpass as before. - Pipelines stay compatible: render pass compatibility ignores load and store ops, and the first draw already ran in the variant. - Queries and conditional rendering continue in the pass instead of closing and reopening around a break. - Variants that discard (MSAA resolve on tilers) store differently and keep the old behavior. Tested on lavapipe under synchronization validation: - a framebuffer made with its own render pass, a clear variant, and two attachment writes; - the old end-and-reload sequence and the kept pass give the same pixels; - validation reports nothing for either. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Taken from upstream PR zortos293#805 (zortos293/Lichii, open draft PR, split-thread decoupling) in place of upstream zortos293#699 ac166eb, which fixes the same restart but keeps any active pass on the same framebuffer: on this line a clear variant can discard MSAA attachments at the end (storeOp DONT_CARE, vk_render_pass_cache.cpp DiscardsMsaaColor / DiscardsMsaaDepthStencil), and zortos293#699 would keep later draws in such a pass. This commit keeps the pass only for variants without discards. Only this commit of zortos293#805 is taken. (cherry picked from commit 1457bd0) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
anthonyloukinas
pushed a commit
to anthonyloukinas/Lichii
that referenced
this pull request
Nov 9, 2026
…threads stop The test49g verify run hung at shutdown. ForceStop came at 117.684 and the emulation thread was still running 10 s later, when the frontend terminated it. In the other shutdowns, ~TimeWorker logs SetUserSystemClockContext within a few milliseconds of ForceStop. It runs during service teardown, after KernelCore::SuspendEmulation. That line never came, so System::ShutdownMainProcess stopped at or before the kernel's suspension. A guest thread that reads GPU-written memory (GPU::OnCPURead) queues a sync operation, asks the GPU threads to run it, and waits in WaitForSyncOperation. ShutdownMainProcess first stops the GPU threads (NotifyShutdown) and only then the guest threads: - The split's back leaves its loop when asked to stop, without running a tick asked for after its last one. - Once it has stopped, RequestTick does nothing. TickGPU queues a command for the front thread, which has already been joined. - NotifyShutdown woke sync_cv, but not sync_request_cv, and WaitForSyncOperation had no other way out. So a read made around the stop never returned. KernelCore::SuspendEmulation waits until no core runs a thread of the process, and that thread stays current on its core in host code. The wait never ended. In the verify runs, reads waited 38 to 47% of the time (750 to 1,000 reads a second), and they went on until ForceStop. The race is older than the split: the single GPU thread ran these requests before and stopped the same way. Nothing in the upload reordering waits. What changed: once the GPU threads have stopped, NotifyShutdown drops the queued requests and releases their waiters. The requests are dropped because they can refer to their waiters' frames (OnCPURead's timing), and nothing runs them after that point. WaitForSyncOperation returns when its request ran or when the requests were dropped. A read released this way goes on without its flush, which no longer matters. The release comes after the threads stop, so no request runs once its waiter has left. Tested with a harness on the real gpu.cpp and gpu_split.cpp (agents/upload-reorder/hshutdown): a Tegra::GPU, the split's back thread with GPU::TickWork as its tick hook (as GPU::Impl::Start sets it), and a guest thread reading as OnCPURead does. The back served 15,000 to 32,000 reads, then NotifyShutdown ran and the guest read again. Without this change the guest was still waiting 3 s later: 5 of 5 runs on this branch, and 5 of 5 on the oot-amd-test49g sources. With it, the reads return and the request queued before the stop never runs: 30 of 30 runs here, and 30 of 30 with this change applied to the oot-amd-test49g sources (it applies with line offsets only). Also a syntax check and strict warnings on the changed lines. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> (cherry picked from commit 758b5aa) Lichii l25: from upstream PR zortos293#805 (open draft), this commit only. Conflict in gpu.cpp NotifyShutdown resolved without the SetTickHook(nullptr, nullptr) line: this tree has no GpuSplit::SetTickHook (it comes with zortos293#805's decouple commits, not taken). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
anthonyloukinas
pushed a commit
to anthonyloukinas/Lichii
that referenced
this pull request
Nov 9, 2026
…and conditional rendering An upload may run in the upload command buffer, ahead of its submission's work, only when no work recorded before it uses the bytes it writes. The buffer cache learns of such uses from Buffer::MarkUsage, and some GPU work reads or writes guest buffers without calling it: - Indirect draws read their arguments and their count. BindHostDrawIndirectBuffers synchronized both and marked nothing. - ObtainBuffer and ObtainCPUBuffer marked only for MarkAsWritten. These went unmarked: indirect dispatch arguments (DiscardWrite), DMA buffer-to-image copies, query result copies, conditional rendering predicates and their resolve pass (all DoNothing). - Conditional rendering on a guest predicate (CompareValueImpl) reads it again at every render pass that resumes it, in later submissions too. The usage marks were cleared only when TickFrame found a whole buffer idle, so other marks on the same pages usually hid these gaps. Once marks are kept per submission, an upload could run ahead of such a read or write. What changed: - ObtainCPUBuffer marks the range it hands out, for every operation. - BindHostDrawIndirectBuffers marks the arguments and the count. - The query cache keeps the address of a guest predicate. Each time it begins conditional rendering again, it marks the predicate through BufferCache::MarkUsed. MarkUsed finds the buffer without creating one. Marks only keep uploads in order, so nothing that was right before becomes wrong. Tested with syntax checks and strict warnings on the changed lines. The upload reorder check that follows (EDEN_NX2_UPLOAD_REORDER_CHECK) is the in-game proof that no upload runs ahead of a use. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> (cherry picked from commit 90e3342) Lichii l25: from upstream PR zortos293#805 (open draft). Conflict in buffer_cache.h ObtainCPUBuffer: kept our 3-argument MarkWrittenBuffer (this tree has no GpuWaitStats write sites) and dropped our MarkUsage after it, which the commit's MarkUsage before the switch now covers. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
anthonyloukinas
pushed a commit
to anthonyloukinas/Lichii
that referenced
this pull request
Nov 9, 2026
…them The read download check of test49b found 11 bytes of one landed download that differed from its check copy: 0x2b648a20, three bytes of each of four words and two of the last, the top bytes equal. The download and its check copy are recorded back to back and read the same bytes, so guest memory changed between the landing and the check. The only writer that neither takes the buffer cache's lock first nor is left out of the check is a guest core. A guest CPU write into GPU-written memory is collected (HandleRasterizerWrite, GPUDirtyMemoryManager::Collect), then stored, without the buffer cache's lock. After a core's first write to such a page, its later writes there skip OnCPUWrite too: Ocarina of Time collects about 1.3 million writes a second this way, and the check sees about 11 KB a second of them over bytes that downloads landed. A download keeps the bytes of the writes collected when it looks at them (PendingCpuWriteRanges), then writes the rest. A write collected and stored in between was handled wrong: - after the landing's look and before its write, the download wrote the GPU's older bytes over the guest's. The guest's write was lost, in what the game reads. - after the check's look and before its read of guest memory, the check compared the guest's newer bytes with the GPU's copy: the test49b lines. The synchronous downloads (DownloadRangesMemory, DownloadBufferMemory) looked before they waited for the host GPU, so a write collected during that wait was overwritten as well. None of this is new in test49b; its verify run only lands more downloads a second. Each landing is now published (VideoCore::DownloadLanding: its range, and a sequence that is odd while it lasts). A download looks at the collected writes only inside its landing, and writes in it. HandleRasterizerWrite calls DownloadLanding::Await after Collect: a write in the range of a landing under way is stored once it ends. Each side fences between its store and its load, so either the landing's look finds the write and keeps its bytes, or the write finds the landing and comes after it. Downloads land under the buffer cache's lock, one at a time. A read download's landing also covers its check (EDEN_NX2_READ_CHECK), which then compares exactly what the download wrote. Every landing takes part: read downloads (WriteReadDownloads, early ones included), fence downloads (PopAsyncBuffers), and the synchronous ones, which now look once the host GPU has run their copies. A guest write pays a fence and a load. It waits only while a landing over its bytes is under way, a few microseconds. Tested in the read download harness (the real BufferCache, device memory manager and MemoryManager, with the fake runtime) with new tests in buffer_cache_ports.cpp. In them a second thread collects and stores a write as HandleRasterizerWrite does, right after a chosen look: - after a read download's look: the write was lost before this change; - after the check's look: before this change the check printed test49b's shape exactly (0x20a20+0x3 (0xcd for 0x68), 0x20a24+0x3, 0x20a28+0x3, 0x20a2c+0x2, 11 bytes differ); - an early download landed at a draw: the write was lost before; - during a synchronous download's wait for the host GPU, staged and mapped: the write was lost before; - 2000 writes racing read downloads at random delays: before, 2 of 3000 were lost and 326 checks reported differences. All 30 harness tests pass in 21 runs, 10 of them pinned to one CPU. Each of these breaks a test: the landing not published, the look before the landing, the check outside it, the synchronous look before the wait, an empty landing range. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> (cherry picked from commit 4942eb8) Lichii l25: from upstream PR zortos293#805 (open draft), adapted. Taken: VideoCore::DownloadLanding (rasterizer_download_area.h) and the DownloadLanding::Await after Collect in memory.cpp's HandleRasterizerWrite. The landings wrap every download this tree has: DownloadRangesMemory (the snapshot of collected writes now taken after the wait for the host GPU), DownloadBuffersMemory, DownloadBufferMemory (mapped, staged and immediate: snapshot after the wait) and PopAsyncBuffers (fence downloads, under the buffer cache's lock in the fence manager). This tree has no read downloads (WriteReadDownloads) and no EDEN_NX2_READ_CHECK, so their landing, the read-download state in rasterizer_download_area.h (g_read_downloads, ReadDownloadsMayCover, ReadCheckWrites) and the four read-download tests are not taken; the synchronous-download test is, with its GuestCpuWrites helper. Not taken: 1072d4a's applying_writes. Here the gather (GPU::InvalidateGPUCache) and every landing run on the GPU back thread, so the window it closes (a landing on another thread between gather and apply) does not exist until landings move off that thread. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
anthonyloukinas
pushed a commit
to anthonyloukinas/Lichii
that referenced
this pull request
Nov 9, 2026
A thread's translation cache (t_cache, thread_local and shared by every DeviceMemoryManager<Traits> on the thread) names its manager by address and generation. The generation was a counter of each manager, starting at 1 and bumped by Map and Unmap. A manager made where a destroyed one was started at the same generation the old one had after its first Map. The thread then kept using the old manager's host pointers, into memory the old DeviceMemory had freed or another one now holds. A test fixture on the stack does this for every test case, and so would a System made again in the same process. This branch's CI hit it at d4e91e8. Six buffer_cache_ports cases read guest memory wrong, two of them with every compared byte off, while the same cases passed in the read download harness. There, each test's DeviceMemory happened to get the same address back. Keeping every test's DeviceMemory alive in the harness, so that each gets a new address, reproduces two of the failures. Generations now come from one counter for all managers of a type, so no manager's translations ever look valid to another. Tested: read download harness, 14311 checks, 0 failures, both as it is and with each test's DeviceMemory at a new address (2 failures before this change). gpu_device_memory_manager.cpp and device_memory.cpp compile. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> (cherry picked from commit d8988e7) Lichii l25: from upstream PR zortos293#805 (open draft), this commit only (applied clean). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
anthonyloukinas
pushed a commit
to anthonyloukinas/Lichii
that referenced
this pull request
Nov 9, 2026
A Windows clang-cl build of this branch crashed in the command the ASTC decoder pass records (reported by lev92 on zortos293#805, with a minidump): a movaps on its BlockLinearSwizzle2DParams, whose alignas(16) members were at an address 8 mod 16. CommandChunk::Record aligns a command's offset within data to the command's own alignment, so data itself must be aligned at least as much. It was declared alignas(std::max_align_t), which is 8 on MSVC and clang-cl (16 with GCC on x86-64). Before, data followed two pointers, a size_t and a bool, and happened to start 16-aligned. The EDEN_NX2_GPU_COST site list (e11c916) put a 24-byte std::vector before it. That moved data to 8 mod 16, so every command with a 16-byte aligned type was constructed misaligned. data now asks for 16 bytes itself (or std::max_align_t, if that is more), and Record checks at compile time that no command needs more. Tested: every Vulkan file that records commands compiles with GCC and -fno-rtti, including vk_compute_pass.cpp with the ASTC decoder pass, so the new static_assert holds for every command type recorded there. MSVC builds already disable C4324 (structure padded for alignment). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> (cherry picked from commit d4e91e8) Lichii l25: from upstream PR zortos293#805 (open draft), this commit only. Conflict in vk_scheduler.h resolved without the RecordSite list (EDEN_NX2_GPU_COST, not in this tree). Inert on Linux x86-64 (std::max_align_t is 16 there); it protects a clang-cl build. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
anthonyloukinas
added a commit
to anthonyloukinas/Lichii
that referenced
this pull request
Nov 9, 2026
The l23, l24 and l25 upstream pick lines on top of the pinned l22 line already on dev. l23: zortos293#337 dropped (+1.6 ms Kokiri); l24: zortos293#805 zortos293#425 zortos293#816 zortos293#817 zortos293#687 zortos293#752 zortos293#818 zortos293#296 zortos293#784; l25: zortos293#621 (front-thread channel poll), zortos293#805's four follow-ups (sync waiters released at shutdown, buffer uses of indirect draws marked, guest writes wait for a download landing, device memory generations) with command-chunk alignment, W52 (stale compute pipelines rebuilt without a frame stall), zortos293#784 (faster subrect swizzle), zortos293#806 (NVN draw parameters: reeds, tall grass, far tree cards), zortos293#820 (noperspective varyings; translation cache format 9). zortos293#825 not taken: no Ocarina of Time shader uses it. The GPU-thread pinning series (zortos293#418 zortos293#530 zortos293#769 + the GameMode re-pin) is carried as the same patches. Gates: tests 931: 928 pass (2 known nx2_video_render failures, 1 known skip). 151, picks vs l24: Kokiri -0.1/+0.1 ms, Castle Town plaza laps 0.0/0.0 (stand p95 -1.3), 600 s plaza stand p99 28.6 ms with 0 GPU waits, close/reopen clean, pictures Field/River/Castle Town unchanged, 0 shots flagged; pinned vs unpinned on 151: Kokiri -1.2/-1.0 ms. On the 5080 this build measured Kokiri 16.6/17.4 ms and plaza 18.1/17.9 ms. Play test (release configuration, no env): 8.2 min, 22,512 frames, median 17.1 ms, p99 26.5 ms, Kokiri and Zora's River at 60 fps, the market "way smoother" than before, no water flicker; the only stalls were first-visit shader builds at the Zora's River entry. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|
Crash report from a Steam Deck OLED (RADV, Mesa 26.2, integrated RDNA2 "VanGogh", SteamOS 3.9). Ocarina of Time (0400486024DEC000), build E0F3CCFA, handheld, 1x, Max Resolution add-on off. With this PR's head (
The same Log, last critical line: Stack of the aborting thread (systemd-coredump, symbol names from the binary): So an image reaches an upload with a 🤖 Generated with Claude Code |
Frizti
added a commit
to Frizti/Fritchii
that referenced
this pull request
Nov 9, 2026
…threads stop (port of zortos zortos293#805 758b5aa) A guest thread reading GPU-written memory (OnCPURead), and InvalidateRegion, WaitForIdle and the capture paths, queue a sync request and wait in WaitForSyncOperation. NotifyShutdown stopped the GPU thread and the split's back first and only woke sync_cv, so a request queued around the stop never ran and its waiter stayed in host code, where KernelCore::SuspendEmulation waits for it forever ("emulation thread failed to stop"). Now NotifyShutdown, after the GPU threads stopped, drops the queued requests (they can refer to their waiters' frames) and releases the waiters. Adapted: our NotifyShutdown has no SetTickHook(nullptr) (that hook comes from upstream d86bffd, not ported); the split's back is already joined by ThreadManager::NotifyShutdown -> GpuSplit::Stop. Also clears our EXP29 & 2 has_sync_requests flag with the queue. Beyond upstream: a request made after the drop is not queued at all (its waiter returns at once), so a stray TickWork during teardown can't run a lambda whose waiter's frame is gone. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Follow-up with more detail on the Steam Deck abort above. I added a temporary guard at the top of That is the texture cache's null image (
So with this PR the null image ends up marked Skipping the upload doesn't save the run: the next step is an unhandled SIGSEGV ( On 🤖 Generated with Claude Code |
|
Found it, and confirmed the fix on the Deck. The memoized texture path added here ( Two-line change in const ImageViewBase& image_view = slot_image_views[image_view_id];
- if (image_view.IsBuffer()) {
+ // Views of empty descriptors point at the null image: there is nothing to upload or track.
+ if (image_view.IsBuffer() || image_view.image_id == NULL_IMAGE_ID) {
return;
}
PrepareImage(image_view.image_id, is_modification, invalidate);With it, this PR's head ( Numbers so far are one run on the balcony outside Link's house, Kokiri Forest, 15 s of frame times standing, then fps while panning the camera:
I'll run more rounds and post them if they change the picture. 🤖 Generated with Claude Code |
|
More Steam Deck rounds with the
Against plain 🤖 Generated with Claude Code |
This was referenced Nov 9, 2026
|
Data point on an 8 GB card limit, in motion, on AMD, in case it helps with the remaining parts after #1028 and #1056. RX 9070 XT, Ryzen 7 5800X, Windows 11, AMD driver 2.0.416, card limited to 6,500 MiB (what an 8 GB card reports). PGO build (clang-cl, ThinLTO,
These numbers mix the three PRs; I did not measure #874 and #836 apart. |
This was referenced Nov 9, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR is the result of a long measurement-driven pass over the split GPU thread pipeline (the front parses and runs macros, the back runs methods and records Vulkan). It was profiled and verified on Ocarina of Time on an RX 7900 XTX + Ryzen 7 7800X3D. Two testers have since reported on an RTX 2070 and an RX 6900 XT: see Tester reports.
The goal was not a single hot spot. Before this work the frame rate was capped by a chain of waits: guest CPU → front → back → host GPU. Each link waited on the next, so removing one wait only moved it to the next link. The PR removes those waits one by one, makes each removal exact, and adds checks that prove it in game.
Result at the same save spot (Hyrule Field, standing still, Low GPU accuracy, fence behaviour Default):
A heavier spot in Kakariko Village, which ran at a consistent 33 fps before this work, now averages 47–48 fps on test49m2. It is not part of the profiled measurements below.
In the final verify runs of the branch head, every check added here reads 0 differences in what the game sees. Twice the read download check reported bytes that were its own mistake: one byte a newer write had replaced after a download brought it, and after the second merge 55 bytes an upload had replaced. Both are fixed in the check, with regression tests. The Verification section has the numbers.
Along the way the checks found three races that predate this work. They are fixed here in their own commits:
ObtainCPUBuffer, conditional rendering's predicate), so an upload could be reordered ahead of GPU work that read the bytes it overwrites.All new diagnostics are behind
EDEN_NX2_*switches and cost a single flag test when off.Test setup and what the measured builds contained
aae0b063f7without any of those PRs. nx2 was merged in three times since, most recently atbbf3e62974, which brings video_core: cheaper per-draw work on the split back thread (DK Bananza 11.65 -> 14.83 fps) #858's fast draws.f3702f3b97, before the second merge, was built with MSVC on Windows, measured and verified in game: see Measurements.611a09d4a0, with Import from Frizti's Fritchii v4: OoT water flicker, staging buffer reuse, per-method switch reads #778 and the last QMD view fix, was built with MSVC too and verified in two runs of about 11 minutes each: see Verification.894a9e8386and802c5e4592were built with MSVC and verified in runs of about 8.5 and 17 minutes. After the third,0758309bccwas verified in one run of about 11 minutes, with the fast draw check too (the head,34621dcd6b, only adds a scope guard): see Verification.Background: where the time went
The split runs guest GPU commands on two host threads joined by a ring:
At the start of this work the profiler (
EDEN_NX2_GPU_PROFILE) showed:Each of those waits serialises the threads, and the back went idle after each one. So the fixes come in four groups: the guest read path, the front's waits, the back's CPU cost, and the host GPU's work.
Changes
1. Guest reads of GPU-written memory
Asynchronous read downloads. The reading thread waits only for its own download tick, and pages stay GPU-modified until the bytes land (
video_core: let the reading thread wait for its downloads).Early read downloads. The buffer cache learns which pages the guest reads back after GPU writes. Learning rules:
The copy is then recorded right behind the writing dispatch, so the bytes are already on their way when the guest asks.
EDEN_NX2_EARLY_READ_DOWNLOADS=0): 51.9 → 59.3 fps (+14%).Query results. Read downloads land under query results, and query copies outdate them.
Fix (race 1). A guest write that overlaps a landing download in progress now waits for it (
DownloadLanding, a few µs, ~8 ns per recorded write otherwise). Before, the landing could write the GPU's older bytes over the guest's newer ones.2. The front no longer waits for the back
Semaphore view. The front decides most Switch 2 semaphore acquires from what it knows has been released or queued. It doesn't drain the split. Releases it cannot value suspend only their channel, and the back's latch resumes it.
894a9e8386).802c5e4592).Suspended channels. While channels wait on semaphores, the front hands the back what it sent before it sleeps between polls. Without that, the few words it held waited out the sleep while the back ran out of work. Found and measured by hellasleeper108: 44.2 → 48.0 fps at their Lon Lon Ranch spot (
0758309bcc).Back macros. Macros that read guest memory (for example the DMA-reading macro 0xe2c) run on the back with their own macro engine, so the front no longer drains for them.
Command lists on GPU-written pages. These wait only for the words GPU work wrote, not for everything queued.
QMD view. The front runs the QMD scheduling on its own table (
FrontQmdView). It notes what compute launches release (payloads, maxima, reductions), so acquires on those words pass without suspending. To keep it exact and in sync:MaxwellDMA::WritesOfLaunch,Upload::UploadedRange).NativeQmdScheduler.0x398bvs0x398c).0x0noted where the back released0x70ce). The view now reads only once the back is past the write, and a harness case reproduces the old order.Late publishes. When the back waits for work, the front now publishes what it holds once it has 8 words. Resumed channels no longer copy already-run command list entries again: late publishes went from 49 to 35 ms/s, and recopied words from 97M to 75M per second.
GPU-written list parameters stay on the GPU. Ocarina's culling compute shaders write the group counts of the next dispatch, and macros 0xe74/0xe7a pass them unchanged into the next QMD's grid and constant buffer. When a tagged dry run of the real macro code proves that, the front no longer reads them:
vkCmdDispatchIndirectfrom the snapshot.Any other use of the words falls back to the old path.
EDEN_NX2_LIST_INDIRECT=0turns it off.3. Back thread CPU cost
TickWorkskips its lock when no sync request is pending.4. Host GPU work
ObtainCPUBufferusers (indirect dispatch, DMA buffer↔image, query copies, conditional rendering), and the predicate re-read when conditional rendering resumes.5. Shutdown and build fixes
NotifyShutdownreleases threads waiting inWaitForSyncOperation. Before, a guest thread reading GPU-written memory at the moment of shutdown kept its core busy, soSuspendEmulationnever returned ("Emulation thread failed to stop").std::max_align_t, which is 8 on MSVC and clang-cl. This PR's GPU cost site list moved that buffer to 8 mod 16, so commands with 16-byte aligned members were misaligned. The buffer is now 16-byte aligned itself, andRecordchecks at compile time that no command needs more (d4e91e81de).d8988e7032).Measurements
The same save, standing still, profiler on. Each fps is the mean of the steady 10-second windows.
aae0b063f7Castle Town, a heavier spot with ~2,000 draws per frame, ran at ~33 fps on test49m2 and runs at ~45 fps on this branch. The difference comes from upstream #268 in the new base. On test49m2 the texture cache was capped at 5 GiB, and there the collector evicted textures every frame, with ~3 full GPU waits per frame. On this branch those waits are gone and texture uploads per frame are halved.
Host GPU
GPU busy time is measured with
EDEN_NX2_GPU_COST=1(timestamps around every command in sampled frames).Tester reports
From the PR comments, with the testers' own builds (clang-cl 22, ThinLTO, PGO), GPU accuracy Low and
EDEN_NX2_GPU_PROFILE=1:RTX 2070, Ryzen 5 3600X (@Maximel3105), nx2
aae0b063fagainst the PR atf01be636d, the same outdoor area (separate runs, not a strict A/B):About 9 minutes of play, no crash and no new visual problems.
RX 6900 XT, i7-9700K (8 cores, no SMT; @lev92), at
894a9e8386, Lon Lon Ranch standing still:The first run had fence behaviour Balanced in its per-game config, which turns the semaphore view off. The front then drained 690 ms/s at the same frame time. No flashes or hangs with Default.
Verification
Every optimisation that skips a wait, or moves work, has a check that runs the old, safe path next to the new one and compares the results in game. Every verify run used an MSVC build on the user's machine, with all checks at once plus
EDEN_NX2_ACQUIRE_STATS. The table is from611a09d4a0, before the second merge: two runs of about 11 minutes each in Hyrule Field (22.5 minutes in all). The runs after the later merges follow the table.EDEN_NX2_…)DRAIN_CHECKREAD_CHECKeb64bfbf87, with a regression case for each way it can happen); the second run read 0LIST_VERIFYBACK_MACRO_VERIFY=NUPLOAD_REORDER_CHECKENTRY_CHECKLIST_INDIRECT_CHECKDMA_READ_VERIFYAfter the third merge,
0758309bccwas verified the same way, withEDEN_NX2_FAST_DRAW_VERIFYtoo, in one run of about 11 minutes. Every check read 0:After the second merge,
802c5e4592(17 minutes, 3,311,001 acquires passed) and894a9e8386(8.5 minutes through Goron City, Dodongo's Cavern and Kakariko Village, 1,427,736 acquires passed) were verified the same way: 0 not met after the drain, and every other check read 0 too.The first verify run after the second merge had found two things, both fixed before these runs. The semaphore view passed 74,721 of 739,427 acquires that were not met after the drain, because of nx2's #337 (see the semaphore view,
894a9e8386). The read download check reported 55 bytes an upload had replaced after an early download landed them, which was the check's mistake (f01be636de).The branch at
f3702f3b97was verified the same way before, in Hyrule Field and Castle Town (6.5 minutes): every check read 0 except the QMD view note fixed since.Each new mechanism also has a harness that runs the real code, used during development:
GpuSplitwith its back thread;MaxwellDMAandUpload::Stateon a realMemoryManager;BufferCacheread downloads;How to test, and how to turn things off
Switches
Profiler:
EDEN_NX2_GPU_PROFILE=1: per-thread zones, waits by site, and late publishes, every 10 s.=2adds fine zones and counters.EDEN_NX2_GPU_COST=1withEDEN_DKB_FRAME_TRACE=1: host GPU time by command kind and site, and render pass breaks.=2adds an upload classifier, which is slow.Checks: see the table above. They are heavy. Don't measure fps with them on.
Every switch also works from
nx2_switches.txt(#714).Off switches:
EDEN_NX2_EARLY_READ_DOWNLOADS=0: early read downloads.EDEN_NX2_LIST_INDIRECT=0: GPU-kept list parameters.EDEN_NX2_BACK_MACROS=0: back-thread macros.disable_buffer_reordersetting forces every upload back in order.Known limits
Commits
54 commits, oldest first (20 of them
TEST:commits that add checks, profiler zones and stats)fc42430f51TEST: log every 10 s how long guest threads wait for the GPUd86bffda46video_core: serve a guest read's flush on the split's back after its current entry6229487c8cvideo_core: wait for a command list's writers only up to the last host sync4a143da957TEST: count what the split's front would miss without its waits (EDEN_NX2_DRAIN_CHECK)fb4d4e47a7video_core: check Switch 2 semaphore acquires without draining the split when they can34659468f2TEST: verify acquires decided without the drain (EDEN_NX2_DRAIN_CHECK)1d9b6ef31dvideo_core: wait only for the work that writes what a macro DMA reads936c05e149video_core: check macro DMA reads against a drained split (EDEN_NX2_DMA_READ_VERIFY)1072d4a1edvideo_core: let the reading thread wait for its downloadsf6fc343a77TEST: compare read downloads with a synchronous flush (EDEN_NX2_READ_CHECK)4c6864af91video_core: land read downloads under query results, outdate them for query copies7693bb232bTEST: read download check: compare kept bytes as they landed783a9a3409video_core: download read-back pages right behind their GPU writes00a62b9189TEST: read trace and read check for early read downloads739a6ed95cvulkan: cast the LSFG mip level binding count to u32a442c38fe6video_core: run the macros that read memory on the split's back0ba202649aTEST: check back macros against the old way (EDEN_NX2_BACK_MACRO_VERIFY)34656e8225video_core: command lists on GPU-written pages wait only for the words GPU work wrote3a665f9f27TEST: command list read stats and a check against the old reads (EDEN_NX2_LIST_STATS, EDEN_NX2_LIST_VERIFY)188baa1448video_core: decide Switch 2 acquires from the split's semaphore view557a13242cTEST: verify and count the semaphore view's acquire decisions87a44ccecevideo_core: share one ring position wait between back macros and acquirese11c916acaTEST: time the host GPU's work outside draws and dispatches (EDEN_NX2_GPU_COST)bd737eba16video_core: value QMD releases on the split's front, publish sooner after waits2c59c7bb58TEST: check the QMD view's releases, count it and the late publishes4edd776ba4TEST: classify uploads only with EDEN_NX2_GPU_COST=2d612710df3vulkan: one command and one pair of barriers for a run of in-order uploads1457bd01fbvulkan: keep a deferred clear's render pass for the draws after itb62f49104dvulkan: record a dispatch's barrier in the dispatch's own command8b5be87f79vulkan: order later transfers after a copy that asked for barriers62db242b34video_core: publish to the split's back once it waits, copy fewer list words at resumes78f08aed9evideo_core: keep the split front's QMD view in sync, place copies and uploads3319c785cbTEST: check the QMD view's descriptors and table, count why it stopsb964e6b40cvideo_core: profile the 3D methods on the path unprofiled runs take47af075e49engines: write 3D state registers in one call7c382303cebuffer_cache: compare a small uniform buffer where it lies in guest memory20628c90aevideo_core: leave TickWork without its lock when every sync request ran20eba75f80TEST: fine profile zones for the split's back (EDEN_NX2_GPU_PROFILE=2)90e3342981video_core: mark the buffer uses of indirect draws, obtained buffers and conditional rendering3406508a75TEST: mark the read check's own download copy as a use of its bytesddf01f582avulkan: put uploads ahead of a submission's work unless that work uses their bytesa0ecdb0401TEST: check each upload's place against an exact model of buffer uses (EDEN_NX2_UPLOAD_REORDER_CHECK)d4e8d85d4dvideo_core: command entries don't wait for the writers before the last syncf7d4131e1eTEST: check the command entries parsed without waiting for GPU work (EDEN_NX2_DRAIN_CHECK, EDEN_NX2_ENTRY_CHECK)4942eb8a39core, video_core: let guest writes wait for a download landing under themcf3c7f0099TEST: count guest writes held by download landings, explain check differences758b5aae3avideo_core: release threads waiting for sync operations once the GPU threads stop15efb1116cvideo_core: keep a QMD entry from the split's back until the front noted it3cee6b61a0video_core: keep the split's placed writes apart where their granules share a slot3c7cfea4d1video_core: have a QMD signal the front can't read yet wait, not lose the viewb57e486f2cTEST: check the QMD view's late valuing and publish hold, name what its signals wait fora59f7dc6e4video_core: let the host GPU keep GPU-written compute list words9e0cd4f2d7TEST: count and check the GPU-written list words the host GPU keepsf3702f3b97cmake: reuse a cached CI package when its checksum can't be downloadedOn top of them:
443eee0623merges nx2's Import from Frizti's Fritchii v4: OoT water flicker, staging buffer reuse, per-method switch reads #778. Its only conflict was the switch members inmaxwell_3d.h; the merge commit explains it.611a09d4a0video_core: have the QMD view read a descriptor only once the back ran its writeseb64bfbf87TEST: read download check: leave out landed bytes newer writes replacedf9f3bc6394TEST: explain Outdated read download check results59f05abc8fmerges nx2 atbf48fe92cc. It resolves 13 conflicts and takes cmake: reuse the recorded hash of cached CI packages instead of downloading their sha512sum #792 and vk_scheduler: keep drawing in the render pass a deferred clear began (Ocarina of Time) #699 over this branch'sf3702f3b97and1457bd01fb. It also fixes the interaction of video_core: mark native compute storage bindings only as far as the shader's stores reach #816 with the GPU-kept list words; the merge commit lists each resolution.d4e91e81devulkan: align a command chunk's data for every command it holdsd8988e7032core: give device memory translations generations no other manager hasf01be636deTEST: read download check: leave out landed bytes an upload took from the guest894a9e8386gpu_split: count fence-flagged releases in memory once their fence ran802c5e4592gpu_split: decide acquires on a word with a pending fenced release by memorya8db1705f5merges nx2 atbbf3e62974. Its nine conflicts all come from video_core: cheaper per-draw work on the split back thread (DK Bananza 11.65 -> 14.83 fps) #858 (fast draws, always on); the merge commit explains each.0758309bccgpu_thread: publish what the front sent before it sleeps on suspended channels34621dcd6bgpu: take the applied writes out of applying_writes on every exitRelated PRs
git merge-tree).🤖 Generated with Claude Code
Summary by CodeRabbit