KZT, refactor: split loader object tracking and patch planning#3
Draft
LaurenIsACoder wants to merge 19 commits into
Draft
KZT, refactor: split loader object tracking and patch planning#3LaurenIsACoder wants to merge 19 commits into
LaurenIsACoder wants to merge 19 commits into
Conversation
Document the KZT loader refactor as an engineering plan instead of a progress report. The document explains why the loader path needs to be split, why the staged rewrite is feasible, and how the work should be organized across the planned stages. Series organization: - Patch 1 documents the overall design and review structure. - Patch 2 belongs to stage 1 and introduces guest object identity tracking. - Patch 3 belongs to stage 2 and adds in-memory Dynamic Table parsing with legacy comparison. - Patch 4 belongs to stage 3 and introduces patch planner decisions for GOT writes. - Patch 5 belongs to stage 4 and compares guest-owner targets with maplib-selected targets. - Patch 6 belongs to stage 4 and selects successful guest-owner targets while preserving maplib fallback. Progress reports and latest validation numbers are intentionally left out of this design document because they change as the series evolves.
Stage 1 of the loader refactor records guest ELF object identity in a dedicated registry instead of relying on scattered loader state. The loader callback registers object name, base, load range, and dynamic table information while preserving the legacy processing path as a compatibility fallback. This creates an explicit address-to-guest-object boundary that later stages use for dynamic parsing and guest-owner patch target selection.
Stage 2 of the loader refactor parses guest Dynamic Table data from guest memory instead of relying only on the legacy file and section header path. The new parser materializes dynamic symbols, relocations, version indices, version needs, version definitions, and string table views from the runtime dynamic information. Keep the legacy parser active and add comparison coverage so parser differences are observable before the old file-based dependency is removed.
Stage 3 of the loader refactor materializes GOT writes as patch decisions before applying them. The planner records the guest object, relocation, symbol version, old target, old owner, selected bridge, target source, and decision reason. Immediate relocations, lazy setup, PLT resolver writes, and unresolved slots all go through the same decision formatting path. This does not change the selected target yet. It makes the existing maplib-based behavior observable so later guest-owner changes can be compared against a recorded decision.
Stage 4 of the loader refactor starts using the current guest GOT value to identify the guest object that owns a resolved target. Add guest-object address lookup, guest-owner wrapper lookup, owner relation classification, and shadow planner logs. The shadow mode records whether the guest-owner target matches the maplib-selected target and classifies failures such as missing guest owners, self PLT targets, missing wrappers, missing libraries, and missing symbols. The selected bridge still comes from maplib in this patch. Keeping the old target source authoritative makes the guest-owner path observable before it is allowed to replace maplib resolution.
Complete stage 4 by allowing a successful guest-owner probe to select the native wrapper bridge directly. The relocation resolver now probes the guest object that owns the current GOT value before repeating the global maplib lookup. When that probe resolves a wrapper target, the planner records the decision as a guest-owner target and skips the maplib reparse for that slot. Failed probes keep the existing maplib fallback and carry the classified failure into the patch decision. Lazy binding remains outside this stage and keeps its existing resolver path.
Stage 5 of the loader refactor lets the guest loader own ordinary guest dependencies. KZT now registers wrappers only for objects that the guest loader has already reported. It no longer expands guest RPATH/RUNPATH, recursively loads guest DT_NEEDED entries, probes wrappers by constructing temporary libraries, or side-loads libGL for SDL/CgGL objects. Remove the old LoadNeededLibs() guest dependency path while keeping AddNeededLib() available for native wrapper registration and wrapper host dependencies.
LaurenIsACoder
force-pushed
the
kzt-refactor
branch
from
4060d91 to
45e8f9d
Compare
Complete stage 6 by moving lazy JUMP_SLOT policy and resolver state out of RelocateElfRELA(). Move lazy slot classification, binding metadata, resolver eligibility, deferred patch decisions, PLT resolver installation, PLT frame handling, resolver actions, patch planning inputs, and lazy logging behind guestlazy helpers. Keep the selected target, patch decision reasons, resolver slot writes, and first-call KZT resolver behavior unchanged. This creates the boundary needed to later let guest ld.so bind first and replace the slot after binding without modifying generic RelocateElfRELA().
Document the stage 7 rule that guest dynamic loader results are authoritative for dlopen, dlsym, dlclose, dlinfo, dladdr, and dlvsym. The local wrapper side should keep only the metadata needed to bridge native wrappers back to guest-owned handles. It must not synthesize independent guest handles, duplicate guest reference counts, or reload guests behind the guest loader.
Split the common guest dl helper calls out of the wrapper entry points. Keep the behavior unchanged while naming the handle lookup path, RunFunctionWithState() forwarding, and guest dlclose forwarding points. These helpers provide the mechanical boundary used by the following patches to make guest loader results authoritative.
LaurenIsACoder
force-pushed
the
kzt-refactor
branch
2 times, most recently
from
99f8a9a to
81ab7db
Compare
Make dlopen reuse and close handling follow the handles returned by the guest loader. Track recycled handles, reopen-after-close, self handles, local maplib metadata, and closed-handle deactivation without reloading guests behind the guest loader.
Route dlsym and dlvsym through guest-owned lookup results before using local wrapper metadata as a compatibility fallback. Handle RTLD_DEFAULT, RTLD_NEXT, wrapped link_map lookups, missing symbol errors, versioned dlvsym requests, dlinfo delegation, and symbol sync failures through named helpers.
Finish stage 7 by pairing local wrapper metadata with guest-owned handle lifetime. Close guest handles before local release, clear guest identity on final close, return guest link maps from dlinfo and dladdr1, forward non-base dlmopen, preserve dlerror semantics, and validate native wrapper dlsym and dlvsym symbols before returning them.
Document the stage 8 rule that loader synchronization should consume loader events instead of depending directly on a private glibc hook. The glibc hook remains as a fallback event source during this stage, but it should only report link_map events. Guest object registration and legacy compatibility processing live behind the event boundary.
Convert the loader callback path into a loader-event submission path. Build immutable event snapshots from link_map data, validate missing or empty link_map records before dispatch, tag event sources in logs, and keep the glibc callback as an adapter that submits events rather than running the whole synchronization flow directly.
Move guest object setup and legacy processing behind the loader-event boundary. Centralize guest object state updates, debug logging, legacy ELF preparation, successful finalization, missing ELF state, object key selection, and observation logging. This lets future event sources share the same registration and compatibility path.
Represent the private glibc hook as a loader-event source object. Name the hook metadata, fixed-candidate and pattern scanning helpers, source tag, hook resolver, cached hook state, callback bridge helpers, and bridge install path. The hook still exists, but the rest of KZT now sees it through the event-source boundary.
Finish Stage 8 by keeping the private glibc hook behind the loader event source boundary. Harden the fallback source that still scans the glibc loader by bounding fixed hook checks, pattern probes, scan cursors, and backtracking. Also validate loader images, reject invalid hook registers, free resolved loader paths, and disable KZT cleanly when no fallback source is found. Keep Dynamic parser comparison logs attached to the active event source instead of hard-coding kzt_tb_callback. This lets future r_debug, mprotect, mmap, or smaller fallback sources reuse the same dispatcher without carrying the old hook name through the compatibility path.
LaurenIsACoder
force-pushed
the
kzt-refactor
branch
from
81ab7db to
a24da1a
Compare
Ask the guest loader before using local wrapper or global-symbol fallback results for dlsym. Stage 7 makes guest dl API results authoritative. RTLD_DEFAULT now uses the LATX_RELOCATION_SAVE_SYMBOLS fallback only after guest dlsym misses. Wrapper handles also ask the guest link_map first, and only replace a successful guest result with the local native bridge address. Keep local metadata failure paths non-fatal as well. Missing local library metadata now reports ordinary dl errors instead of asserting or calling GetElfIndex() with a NULL library. dladdr1 LINKMAP fallback also checks that extra_info and a local link map are available before writing fallback results. Keep dlvsym on the versioned guest lookup path for wrapper handles too. The guest version check now returns the guest result directly unless a local bridge can replace that successful result. Missing local metadata also reports the handle-specific miss instead of falling back to a plain dlsym lookup. Keep guest dlopen and dlclose references paired for wrapper handles. Explicit dlopen now retains the guest loader object even when the link_map was already known from a dependency load. If local symbol registration fails after guest dlopen succeeds, roll back the guest reference and clear the cached guest identity. Initialize all parallel handle-table arrays when growing dlprivate state. This keeps future debug and failure paths from observing stale metadata in unused slots.
LaurenIsACoder
force-pushed
the
kzt-refactor
branch
from
a24da1a to
bb8e12e
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This draft PR carries the staged KZT loader refactor through stages 1 to 8.
The series documents the overall design, introduces guest object identity,
adds in-memory Dynamic Table parsing, records GOT patch decisions, prefers
guest-owner wrapper targets where possible, stops side-loading ordinary guest
dependencies, isolates lazy binding policy, makes guest dl APIs authoritative,
and narrows the private glibc hook into a fallback loader-event source.
The current local head has passed a debug build and the focused KZT loader
regression script. A full
dEQP-EGL.*A/B run from the earlier convergencepoint matched the pre-refactor baseline over 4111 EGL CTS cases; the latest
head should still go through a final full CTS A/B comparison before review is
considered complete.
中文详细说明
中文详细说明
设计意图
这组补丁的目标不是一次性重写整个 loader，而是把原来耦合在一起的 KZT
loader 路径拆成可验证、可替换的边界。
旧方案把 guest loader 通知、ELF 文件解析、maplib 重解析、wrapper 查找、
GOT 修改、lazy binding、
dlopen/dlsym/dlclose状态，以及 glibc 私有hook 同步流程混在一起。核心问题是：guest loader 已经知道对象和绑定结果，
但 KZT 仍经常通过文件、全局符号范围或 maplib 重新推导一次，导致对象身份、
目标来源和引用生命周期都不清晰。
本次重构按阶段解决这些问题：
link_map/l_ld解析运行时 ELF 元数据。dlopen/dlsym/dlclose等 API 返回结果成为权威。当前补丁组织
当前系列按 review 粒度整理为 19 个提交：
是否符合设计预期
总体符合预期。
这组补丁已经把 KZT 从“自己重做 guest loader 决策”推进到“观察 guest loader
结果，并在明确边界后做 wrapper 替换”的方向。对象身份、Dynamic Table 解析、
GOT patch 决策、guest-owner wrapper 选择、dependency loading、lazy binding、
dl API 语义和 loader event source 都已经拆出明确边界。
Stage 8 仍是过渡形态。glibc 私有 hook 还没有完全移除，但它已经不再承担整个
装载同步流程，只作为 fallback event source 上报 link_map 事件。后续可以在
这个边界后替换成
r_debug、mprotect、QEMU loader/mmap event，或更小的版本隔离 fallback hook。
已解决的原有缺陷
已解决主要结构缺陷：
l_ld的内存 parser。symbol、version、old target、old owner、new bridge 和 reason。
GOT 当前值所属 guest object。
DT_NEEDED会被 KZT 旁路加载；现在交回 guest loader。dlopen/dlsym/dlclose有合成句柄、重复引用计数和旁路 reload；现在 guest loader 返回结果成为权威。
最近收敛的修复
最新本地提交补充了 Stage 7/8 收敛后的健壮性修复：
RTLD_DEFAULT先询问 guestdlsym，只有 guest miss 后才走兼容 fallback。dlsym/dlvsym先由 guest link_map 决定符号是否存在，成功后才允许 local native bridge 替换返回值。
dlvsym保持版本化 guest lookup 路径，缺失 metadata 时不再退回普通非版本
dlsym。dladdr1(RTLD_DL_LINKMAP)fallback、关闭后的 handle等路径返回普通 dl error，而不是 assert 或写入无效结果。
dlopen会持有 guest loader 引用，使后续dlclose与 guest 侧引用计数配对。
dlprivatehandle 表扩容时初始化所有并行数组，避免未使用槽位残留状态。异常 env/hook/reg 不再读取坏寄存器。
新风险点
本次修改也引入或暴露了需要继续关注的风险：
RTLD_NEXT、RTLD_DEFAULT、RTLD_NOLOAD、dlmopennamespace、versioned symbol、dladdr1等边角语义仍需要更多真实应用覆盖。
方案更清晰，但仍是高风险区域。
pattern validation 和 missing-source fallback，但不同 glibc 版本仍可能需要
额外适配。
dEQP-EGL.*，但最新 HEAD 还需要最终整体 CTS 对比；GLX、Vulkan、Wine 和真实应用路径仍需要后续覆盖。
已完成验证
最新本地 HEAD 已完成：
此前收敛过程中的完整 EGL CTS A/B 对比：
结果：
后续工作
后续工作主要有三类：
r_debug通知、RELROmprotect、QEMU mmap/loaderevent 等方案，逐步减少 glibc 私有 hook 依赖。
English Detailed Description
English Detailed Description
Design Intent
This series does not try to rewrite the whole loader in one step. It splits
the existing KZT loader path into explicit, testable, and replaceable
boundaries.
The old path mixed guest loader notification, ELF file parsing, maplib
re-resolution, wrapper lookup, GOT patching, lazy binding, dl API bookkeeping,
and the private glibc hook synchronization path. The central problem was that
the guest loader already knew the object and binding result, while KZT often
derived the target again through files, global symbol ranges, or maplib. That
made object identity, target provenance, and handle lifetime difficult to
reason about.
The staged refactor addresses this by:
link_map/l_ld;value;
dlopen/dlsym/dlcloseresults authoritative;Current Patch Organization
The current series is organized into 19 reviewable patches:
Design Conformance
The current implementation matches the refactor direction.
The code now has explicit boundaries for guest object identity, runtime dynamic
metadata parsing, GOT patch decisions, guest-owner wrapper lookup, dependency
loading, lazy binding, guest dl API authority, and loader event sources. This
moves KZT away from redoing guest loader decisions and toward observing guest
loader results before applying wrapper replacement.
Stage 8 is still transitional. The private glibc hook has not been fully
removed, but it has been narrowed to a fallback event source. It reports
link_mapevents and no longer owns the whole synchronization flow. Thatboundary is the point where future
r_debug, mprotect, QEMU loader/mmap event,or smaller version-isolated fallback sources can be plugged in.
Defects Addressed
The series addresses the main structural defects of the old scheme:
Headers.
instead of redoing global maplib resolution first.
DT_NEEDEDdependencies are no longer side-loaded by KZT.duplicated reference counting, and side reload logic.
whole loader sync path.
Recent Hardening
The latest local commit adds final stage 7/8 hardening:
RTLD_DEFAULTasks guestdlsymfirst and uses compatibility fallback onlyafter a guest miss.
dlsym/dlvsymask the guest link_map to decide symbolexistence before replacing a successful result with a local native bridge.
dlvsymstays on the versioned guest lookup path and no longer falls back toa plain non-versioned
dlsymwhen local metadata is missing.dladdr1(RTLD_DL_LINKMAP)fallback, and closed-handlepaths report ordinary dl errors instead of asserting or writing invalid data.
dlopenretains the guest loader reference so laterdlclosecalls are paired on the guest side.dlprivatehandle-table growth initializes all parallel arrays.failure disables KZT, and invalid env/hook/reg state no longer reads a bad
guest register.
New Risks
The refactor also introduces or exposes risks that still need attention:
as
RTLD_NEXT,RTLD_DEFAULT,RTLD_NOLOAD,dlmopennamespaces,versioned symbols, and
dladdr1still need more real-application coverage.This is clearer than the old model but remains a sensitive area.
pattern validation, and missing-source fallback reduce the risk, but new glibc
layouts may still require additional adaptation.
dEQP-EGL.*A/B run has matched during convergence, but the latesthead still needs a final full CTS comparison. GLX, Vulkan, Wine, and real
application paths also need follow-up coverage.
Validation Completed
Latest local head:
Earlier full EGL CTS A/B comparison during convergence:
Result:
Follow-up Work
Follow-up work is focused on three areas:
r_debug, RELROmprotect, QEMUmmap/loader events, or a smaller version-isolated fallback hook.