Skip to content

Fold release layers into first-party source on current main - #85

Closed
swhitt wants to merge 64 commits into
ThatGuyMcd:mainfrom
swhitt:refactor/fold-followups-20261031
Closed

swhitt wants to merge 64 commits into
ThatGuyMcd:mainfrom
swhitt:refactor/fold-followups-20261031

Conversation

@swhitt

@swhitt swhitt commented Oct 31, 2026

Copy link
Copy Markdown
Collaborator

What

The release assembled first-party code through configure-time source layers, making ordinary edits depend on staging and source replacement. This fold makes the compiled implementation the editable source, moves backend modules to services/, and keeps dependency fixes as registered patches. Generated guest edits use declared hooks and generated symbol tables.

The branch includes main at d457741 as an ancestor and retains its v1.0.12 profile/account UI, portraits, directory, Store history, live FOV and online-performance changes in their source owners. Both owners approved the shipped v1.0.12 identity values; they were verified in the official Windows and both Linux binaries. Historical v1.0.11 equivalence evidence remains preserved.

Pattern checklist

  • First-party code is edited in place; backend modules live under services/. No first-party layer remains under patches/.
  • Third-party fixes are checksum-pinned patches registered in patches/manifest.json.
  • Guest hooks use guest-hooks/hooks.toml; symbol tables use symbols/*.toml.
  • Native UI/services retain their source owners and updated contract documentation.
  • netplay/legacy_identity.hpp deliberately adopts the shipped IDs with both-owner approval; all four engine salts remain unchanged.
  • Tests added or updated; contributor checks and platform validation are detailed below.
  • Platforms considered: Windows, Linux, macOS, iOS and Android. Windows/Android acceptance remains pending CI and the documented owner runs.

Validation

  • 162 host CTest tests pass, including 151 party checks, both owned heap prefixes and 514 native menu-dispatch checks per revision with historical negative controls.
  • The prepared macOS application links; both production owned native translation units compile with their revision headers.
  • Linux x86_64 full-nogen core and focused native fixtures pass at the follow-up checkpoints; all six final menu/heap checks pass after integrating main.
  • Local Worker checks pass: 95 directory, 101 running-spectator and 94 profile/API checks with the native X25519 proof helper. Five bundler tests pass.
  • python3 scripts/check.py --suite all passes, including guard/tooling, metadata, assets, Track Lab and host-tools. Current-document links and explicit pre-commit checks pass; workflow edits pass actionlint and zizmor.

Risk

This is a large source-ownership change. Windows/MSVC and Android full-nogen PR builds, the Windows preparation/build owner run and cross-machine release comparison remain merge/acceptance gates; see docs/fold-in/open-items.md. Live multiplayer and gameplay acceptance are not established by synthetic tests.

Cross-play targets v1.0.12. Its approved contracts differ from the historical v1.0.11 checkpoint. Five C5 release fixtures remain pending, and release finalization continues to reject an incomplete release. No backend was deployed and no release package is being published.

The migration map and validation/provenance notes are in docs/fold-in/MIGRATION.md, docs/notes/2026-10-31-fold-followups.md and docs/notes/2026-10-31-v1012-identity-proposal.md. The original B9 branch and the pre-main integration branch are preserved locally for comparison.

swhitt added 30 commits October 25, 2026 18:58
apply_closure.py writes the shipped v1.0.11 first-party closure back into
runtime-recomp/ and respells staged includes. Bases it rewrites are released
from the capsule pin via scripts/fold/released-bases.txt until B8.
Output of scripts/fold/apply_closure.py against the v1.0.11 reference:
101 changed files at their original paths, 206 new files at their shipped
paths (flat), staged absolute includes respelled relative. Guest and
dependency transforms are left for B3/B4.
run_equivalence.sh configures this tree with the reference's options and
no overlay, then compares compile settings and link lines, header closures
and bytes, preprocessed tokens, and header-variant objects.
Add the TUs the layers added to DKRNetplayCore, DKRPortGame and the mod
libraries, the shared-saves include order, and the task_submission.cpp
wrapper in librecomp. Keep the v1.0.11 network version and launcher defines.
DKRCameraPresentation with per-revision scene kernels from prepared inputs,
DKRCustomVehicles with its original-texture checks (ARM64 -ffp-contract=off
from v1.0.11), and the store, camera and vehicle links in release order.
71 reviewed strings are gone from the shipped launcher; the test asserts
they stay absent until the Phase C copy review in docs/fold-in/.
DKRPortGameCore carries the sources and settings; the exe links it only with
a generated payload, so every launcher module compiles without one.
Each change cites the layer that made it. Removed features (legacy sync
modes, host prediction, per-mod host saves) keep a must-stay-gone check.
The base CMake no longer includes patches/startup-memory/RuntimeMemory.cmake.
open-items.md holds maintainer questions, B6 notes and the patches/ readers.
The guard and the pinned-bytes check read them; PG007 now also flags a
registered staged patch no runtime-recomp CMake applies. Baseline shrinks.
dependency_stages.py derives six patches from the chain output; the gate
pairs the 11 shipped files with the staged copies and checks their bytes.
scripts/guest_hooks.py applies 14 declared hooks at configure time; each must
hit once per payload. The 40 hooked files match v1.0.11; the gate pairs them.
camera_shadow_kernel.c is first-party source; the stubs it implements are
dropped from the prepared kernel's list, so no capsule role is read.
gen_guest_symbols.py writes the four headers into the build tree from the
pinned symbol tables; the committed copies go. Golden hashes guard them.
Unpaired TUs fail anywhere; guest hooks resolve through GuestHooks.cmake's
pointer; only identity digests are masked; literals stay whole; shader
pending TUs are listed; the reference is verified and read-only; level 3
always runs.
legacy_identity.hpp holds them and CMake reads them; no source hash chain
feeds identity. The gate no longer exempts identity defines.
windows-full-nogen and android-folded build RT64 with the stage patches
and the game TUs. prepare-dependencies gains sparse decomp symbols and
an android-full profile. Neither job has run yet.
run_equivalence.sh writes fold-equivalence.json for signing; level 3 fails
if a header-variant TU goes uncompared; the fold-equivalence workflow
checks closure.json.
Five sites use AtomicSnapshot, file-clock counts go through int64_t, the
water residency include is skipped on Apple. portability.json pins the
files; their Linux TUs must compile to identical objects (level 3).
swhitt added 29 commits October 26, 2026 06:52
AGENTS.md and CONTRIBUTING.md reserve guest-code writes for hooks.toml except
save_layout_pipeline.py and character_portrait_pipeline.py, which feed the
owned payload byte for byte. The guard names exactly those two
(OWNED_INPUT_STAGES), and a test keeps the set from growing.
compose_payloads.py checks a ROM and composes and recompiles one revision's
production payload; both prepare scripts use it, then run prepare_payloads.py.
Prepare-DKR-Runtime.ps1 now builds both revisions (one WSL decomp tree each)
and Build-DKR-R-Windows.ps1 defaults to build\prepared. The hand-built v77
config and the diagnose script that reran it are gone. The PS1 parses under
pwsh 7.4 but hasn't run on Windows; the macOS payloads are byte-identical.
patches/ keeps the manifest and registered patches (release-prep aside).
Water natives move to runtime-recomp/src/water; the engine identity digests
to legacy_identity.hpp. services/ holds the directory, Mod Store and library
backends, each with a build.py over its external base, tests and a CI job.
open-items lists every removed or renamed entrypoint.
The keepers (package, sign, seal, updater qualification, finalize, doc and
ELF checks, worker probe) read VERSION instead of exec'ing older copies.
finalize.py gates on required_checks.toml and refuses while its 15 C5-pending
v1.0.11 fixtures are missing. The capsule and its pins are gone, the guard
baseline is empty and the transition-layer label rule is removed.
format_sources.py runs pinned clang-format, gersemi and ruff in a pinned
image; --prove checks a commit range from git: same C/C++ tokens, CMake
words, Python trees, and only re-pins of reformatted files. .clang-format-ignore
holds the never-format list. prepared.py checks bridge digests against the
committed bridge, and REFERENCE.md records the two-link evidence chain.
Six tests assert what first-party code contains by exact substring, so a
format pass broke them without changing code. They read sources through
source_text.Source, which compares with whitespace removed on both sides;
the launcher copy test reads literal contents from the raw text.
Output of scripts/format_sources.py --write at ef8ef5e, plus re-pins of the
reformatted files in the three digest-pinning scripts. Every C/C++ file lexes
to the same tokens; format_sources.py --prove checks the commit.
The proof record from format_sources.py --prove 8beac7c 52e7ae4 is committed,
REFERENCE.md names both evidence links, clang-format and gersemi run in the
hooks and in CI (format_sources.py --check --local), and runtime-recomp/src
and tests/replay_probe are no longer excluded from the formatters.
release-<platform> presets build the full game with input verification;
build-inputs.lock.json pins the prepared inputs and BuildInputs.cmake checks
them at configure. build.py builds and tests from a clean tree with a record,
evidence.py writes and compares RELEASE-EVIDENCE.json, and release.yml runs
the ROM-free gates on Version* tags (tag == VERSION, registry not pending).
@swhitt swhitt closed this Oct 31, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant