Complete HEAPMNG page-map coverage and high-address coalescing

[zhanghm18GIT]

The memo

Summary

This PR completes the existing HFix4GB HEAPMNG bookkeeping across the full 32-bit page-index range.

It makes two same-size immediate-value changes in the existing hooks/HFix4GB.cpp patch:

1. allocate 0x400000 bytes for the page map instead of 0x3FF000;
2. use 0x100000 as the right-neighbour one-past-end page bound instead of 0xC0000.

No function is replaced and no new runtime code path is introduced.

Closes #163.

Related game-level evidence:

• [Performance] Late-game HEAPMNG Total grows far above Committed memory fa#7195
• [Performance] Cache same-tick command-queue length in UpdateRecordedOrders maudlin27/M28AI#394

---

Allocator limitations

Incomplete page-map capacity

At 0x00957E35, the current HFix4GB patch emits:

PUSH 0x3FF000

A complete 32-bit address space contains:

2^32 / 2^12 = 0x100000

4 KiB page indices.

At four bytes per page-map entry, complete arithmetic capacity is:

0x100000 * 4 = 0x400000 bytes

0x3FF000 provides 0xFFC00 entries and leaves the highest 0x400 pages, corresponding to 4 MiB of virtual-address range, without a page-map entry.

3 GiB right-neighbour bound

At 0x00958107, the allocator computes the page index immediately after a freed region and compares it with:

CMP EAX,0xC0000
JAE skip_right_neighbour

0xC0000 4 KiB pages corresponds to 3 GiB.

The replacement uses 0x100000, the one-past-end page index for the complete 4 GiB 32-bit range. The existing JAE therefore continues to reject an index at or beyond the end of the page map while allowing valid indices in the 3–4 GiB range to participate in the right-neighbour lookup.

---

Implementation

This PR modifies the repository's existing hooks/HFix4GB.cpp file.

Page-map allocation

At 0x00957E35:

PUSH 0x3FF000
->
PUSH 0x400000

Exact bytes:

68 00 F0 3F 00
->
68 00 00 40 00

Right-neighbour guard

At 0x00958107:

CMP EAX,0xC0000
->
CMP EAX,0x100000

Exact bytes:

3D 00 00 0C 00
->
3D 00 00 10 00

Both instructions retain their original five-byte length.

The pre-existing Lua-GC control-flow patches in HFix4GB.cpp are unchanged.

Changelog

The existing 4 GB improvement entry is updated to describe complete page-map coverage and high-address free-region coalescing.

---

Safety and runtime impact

• The page map grows by exactly 4 KiB at allocator initialization.
• Both patched instructions retain their original five-byte length.
• The PUSH keeps the same stack behavior and changes only the allocation-size immediate.
• The CMP keeps the same flag semantics and changes only the upper-bound immediate.
• The existing JAE prevents lookup at or beyond page index 0x100000.
• No new allocation loop, branch, thread, import, state object, file I/O, registry I/O, or periodic work is introduced.
• Per-allocation instruction count is unchanged.

The original rationale for the smaller constants is not documented in the repository. The arithmetic and one-past-end boundary behavior are exact; maintainer review can determine whether either legacy constant represented an additional invariant outside the visible HEAPMNG bookkeeping.

---

Evidence

An equivalent local executable candidate was produced from the tested baseline by changing only these two immediates and recalculating the PE checksum.

Tested baseline:

SHA256 db05b62768fd1cea4cbc2fdce4bbf0778fe85353cd1fc3b1048db330914d08b6

Equivalent allocator candidate:

SHA256 22cb537385a402ad2ad1671c5cc63da8a1a29e9af9c91769d6dc0ddc27ac855e

The equivalent binary participated in the lowest-growth documented stress run:

Heap Total / Committed = 1.406 GiB / approximately 1.30-1.33 GiB

Heap Total remained at 1.406 GiB from session time 00:39:07 through at least 00:48:08.

A separate run with the paired M28-side cache active reached:

1.930 GiB / approximately 1.24-1.26 GiB

The executable identity of that counterexample is not recoverable from its log. The runtime evidence therefore establishes successful operation of a configuration containing this patch while the independent contribution of these two immediate changes remains to be isolated by controlled A/B testing.

The intended engine-level effect is complete page-map bookkeeping and the ability to consider valid high-address right neighbours during free-region coalescing.

---

Validation performed

Source and opcode validation

• source based on repository master commit 96af320e872a083fdb302fde7406d6d7f17c418d;
• original HFix4GB.cpp content checked against that commit;
• original executable bytes verified at both target virtual addresses;
• replacement opcodes independently assembled as 32-bit x86;
• both generated instructions are exactly five bytes;
• patch applies and reverses cleanly against the checked source context;
• whitespace / diff checks pass.

Binary reconstruction

A verifier applied the two instruction replacements to the tested baseline, recalculated the PE checksum, and reproduced the known candidate byte-for-byte:

reconstructed SHA256:
22cb537385a402ad2ad1671c5cc63da8a1a29e9af9c91769d6dc0ddc27ac855e

byte-for-byte candidate match:
PASS

The reconstructed output differs from the baseline only at:

• three immediate-value bytes;
• three PE-checksum bytes.

The PE entry point, image base, section table, Large Address Aware flag, and unrelated patch bytes remain unchanged.

Boundary model

• full map entries: 0x100000;
• old map entries: 0xFFC00;
• missing old range: 0x400 pages / 4 MiB;
• final valid page index: 0xFFFFF;
• one-past-end index: 0x100000;
• deterministic 2,000,000-region model: valid regions fit the new map and high-address cases exceeding the old map are identified.

Runtime evidence

The equivalent local binary has been used in real FAF games. The documented low-growth run is linked through fa#7195.

Repository workflow / patcher execution remains pending on the upstream PR.

---

Regression test instructions

1. Build the patched executable from the repository.
2. Verify the output bytes:
   • 0x00957E35: 68 00 00 40 00;
   • 0x00958107: 3D 00 00 10 00.
3. Start a vanilla sandbox and verify normal allocation, unit creation, save, load, and shutdown.
4. Run a long vanilla AI game and compare Heap Total / Committed with the current build.
5. Run the stress workload from fa#7195.
6. Repeat the four cells below at least three times each:

Cell	HEAPMNG patch	M28 same-tick cache
A	Off	Off
B	On	Off
C	Off	On
D	On	On

7. Instrument or inspect:
   • VirtualAlloc growth reservations;
   • page-map write start/end around 0x0095854F;
   • largest reusable free region before growth;
   • right-neighbour lookups and merges at page indices >= 0xC0000;
   • any out-of-range map access or free-list corruption.
8. Compare simulation correctness and performance before and after.

---

Files changed

hooks/HFix4GB.cpp
changelog.md

Checklist

• Read the repository contribution guidance and existing engine patches
• Linked the research issue closed by this PR
• Exact patch addresses, old/new instructions, and bytes documented
• Original instructions verified against the tested executable
• Same-size instruction replacements; no resume-address change
• Stack, EFLAGS, and control-flow effects audited
• No new public structure or reusable engine symbol requiring moho.h, global.h, or Info.txt
• Changelog updated
• Test and regression instructions included
• No executable or proprietary game artifact included
• Equivalent binary reconstruction and byte audit completed
• Equivalent local runtime evidence documented
• Repository workflow / patcher build
• Maintainer review of the legacy allocator invariants

[coderabbitai]

📝 Walkthrough

Walkthrough

The HEAPMNG hook now allocates a full 4 MiB page map for 32-bit address-space coverage and uses 0x100000 as the one-past-end coalescing boundary. The changelog documents free-region coalescing across the 3–4 GiB x64 range.

Changes

HEAPMNG 4 GiB support

Layer / File(s)	Summary
Page-map sizing and coalescing boundary hooks/HFix4GB.cpp, changelog.md	The page-map allocation constant changes from 0x3FF000 to 0x400000, and the coalescing comparison uses 0x100000; the changelog records the expanded coverage and 3–4 GiB coalescing.
Estimated code review effort: 2 (Simple)	~10 minutes

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Linked Issues check	✅ Passed	The patch matches #163 by changing both immediates to the requested full-range page-map and coalescing bounds.
Out of Scope Changes check	✅ Passed	The diff appears limited to the requested allocator immediates plus related changelog and comments, with no unrelated code changes.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Title check	✅ Passed	The title clearly summarizes the main changes: complete HEAPMNG page-map coverage and enable high-address coalescing.
Description check	✅ Passed	The description includes a clear memo, implementation details, evidence, validation, regression instructions, linked issue, and checklist coverage.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}