Refactor generated kernels into an immutable registry

[onlyxItachi]

Summary

• replace the add-specific generated-ELF singleton with a closed manifest-driven registry
• bind each generated kernel by both device-artifact cache identity and immutable receipt identity
• preserve native capability authority, legacy evidence schemas, transactional XRT publication, and the exact BF16[1024] add-only support boundary

Why

The current singleton could not safely coexist with a second physically qualified generated kernel. This change creates the narrow runtime and artifact-admission foundation needed by the matmul work without claiming matmul or changing the active product surface.

Validation

• complete hardware-free suite: 726 passed, 15 expected skips in the final installed-extension environment
• focused evidence, registry, initialization, and Inductor suite: 161 passed
• independent review suite: 80 passed
• Ruff check and format: passed
• strict mypy: passed
• shell syntax and diff checks: passed
• rebuilt CPython 3.12 native extension SHA-256: c810e2ff4bd89938a6e2a4145fffed6e1c149f29b6f8bc57e1f4a99bb73ba5b7

Physical evidence

The exact commit tree was exercised from a fresh compiler cache on RyzenAI-npu4 with XRT 2.20:

• cold, warm, and trapped-wrapper BF16[1024] Strategy B launches completed
• 1024 of 1024 raw BF16 words matched on every launch
• output stayed on xdna:0 until explicit copy-back
• ERT_CMD_STATE_COMPLETED with positive device intervals
• one validated direct three-BO ELF load across all launches
• alias rejection changed no allocation, cache, execution, or runtime state
• post-run XRT platform health passed

Boundary

This PR does not add matmul, a second manifest entry, a new operator, a public API, fallback behavior, a compiler fork, or artifact distribution. The manifest and native validators remain restricted to the one already qualified BF16 add until the next receipt is independently proven.

AI-assisted development: OpenAI Codex assisted implementation, testing, and review. The human maintainer retains responsibility for the change.

[Copilot]

Pull request overview

This PR refactors Strategy B generated-kernel artifact admission from a single-purpose service into a manifest-driven, immutable registry, enabling safe coexistence with additional physically-qualified generated kernels while preserving the existing BF16[1024] add-only boundary and native capability authority.

Changes:

• Replaced the Strategy B artifact singleton/service with a closed XdnaGeneratedKernelRegistry keyed by (device-artifact cache digest, receipt identity digest).
• Updated the Python ↔ native binding contract and runtime state to support a per-entry generated-kernel registry (including a new native state introspection endpoint).
• Migrated and expanded tests/docs/workflow gating to cover the new registry and renamed native binding entrypoint.

Reviewed changes

Copilot reviewed 20 out of 20 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
torch_xdna/inductor.py	Switches Strategy B artifact binding from the old service to the generated-kernel registry during codegen/binding.
torch_xdna/compiler.py	Threads the generated-kernel registry through compile-contract registration.
torch_xdna/init.py	Initializes/adopts the generated-kernel registry during runtime initialization (replacing Strategy B service admission).
torch_xdna/_generated_kernel_registry.py	Adds the manifest-driven registry implementation and key/entry types.
torch_xdna/_strategy_b_artifact_service.py	Removes the old singleton-style Strategy B artifact service.
torch_xdna/_compiler_artifacts.py	Renames/updates the native binder entrypoint to _bind_generated_kernel.
torch_xdna/csrc/XdnaRuntime.h	Replaces single generated-ELF state with a keyed generated-kernel registry and new query API.
torch_xdna/csrc/XdnaRuntime.cpp	Implements composite-keyed publication/binding and per-entry state tracking for generated kernels.
torch_xdna/csrc/bindings.cpp	Renames the pybind binding entrypoint and adds _generated_kernel_registry_state.
tests/test_workflow_contracts.py	Updates workflow contract assertions for the new registry module.
tests/test_strategy_b_initialization.py	Updates initialization tests to validate registry-based admission.
tests/test_strategy_b_artifact_service.py	Removes tests for the deleted Strategy B artifact service.
tests/test_inductor_adapter.py	Updates adapter tests to use the registry path and per-family binding.
tests/test_generated_kernel_registry.py	Adds focused unit tests for manifest/registry behavior and failure modes.
tests/test_generated_elf_native_contract.py	Updates native contract tests for composite-key registry and renamed entrypoint.
tests/test_compiler_artifacts.py	Updates binder naming to match the renamed native entrypoint.
pyproject.toml	Updates hardware-free Python file list to include the new registry module.
docs/COMPILER_CACHE.md	Documents the new registry structure and its invariants/boundary.
docs/alpha/STRATEGY_B_COMPILER_CONTRACT.md	Updates the contract documentation to describe keyed binding semantics.
.github/workflows/hardware-free.yml	Updates workflow file allowlist for the new registry module.

Suppressed comments (2)

torch_xdna/csrc/XdnaRuntime.cpp:737

• bind_generated_kernel() still logs/checks the fork-safety operation as "bind an XDNA generated ELF". Since the API and Python entrypoint were renamed to "generated kernel", this message is now inconsistent and makes debugging harder.

GeneratedElfObservation XdnaRuntime::bind_generated_kernel(
    CompilerElfSpec spec,
    std::vector<char> elf_data) {
  check_not_in_bad_fork("bind an XDNA generated ELF");
  validate_generated_elf_spec(spec);

torch_xdna/csrc/XdnaRuntime.cpp:846

• launch_generated_bfloat16_add() still uses the fork-safety operation string "launch an XDNA generated ELF" even though the binding/publication surface was renamed to "generated kernel". Aligning the message avoids confusion when diagnosing bad-fork failures.

void XdnaRuntime::launch_generated_bfloat16_add(
    XdnaAllocation& lhs,
    XdnaAllocation& rhs,
    XdnaAllocation& output,
    const std::string& expected_device_artifact_cache_key_sha256,
    const std::string& expected_artifact_receipt_identity_sha256) {
  check_not_in_bad_fork("launch an XDNA generated ELF");
  TORCH_CHECK(

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.