feat(neural): registration-seam delta — typed arity/arg_sorts validated against the program, opaque artifact identity, queryable network_metadata - #160
Conversation
…ated against the declarations, sorts and artifact hash carried opaquely
Add CompiledProgram.network_metadata(name), modeled on neural_predicate_info: returns the registration-time arity/arg_sorts/ artifact_hash plus every nn/4 declaration bound to the network name (predicate, predicate_arity, input_arity, labels), so a consumer can check its own idea of a network's shape against the program's actual declarations rather than a naming convention. Declared-but-unregistered and undeclared names get typed PyValueErrors. This metadata was previously write-only: register_network stored it on NetworkConfig, but NetworkHandle::from_config never copied it over, so it was dropped the instant registration completed. Extend NetworkHandle with the same three fields, populated in from_config, so there is something to read back.
…-declared networks An embedding-declared network registers via register_embedding(), not register_network(); calling register_network() on it hits the existing cross-registration guard. But network_metadata's "not registered" error told the caller to "call register_network() first" regardless — wrong for embeddings, misdirecting them into a call that will always fail. Branch on declared_network_forms (the same map the register_network guard at line 218 already consults) before the generic "not registered" error: if the name is embedding-declared, say so explicitly and name register_embedding() as the actual registration path, and state that arity/arg_sorts/artifact_hash metadata is out of scope for embeddings by design. Keep the existing message for the genuine classification-declared-but-unregistered case. Doc comment now lists all three failure modes: undeclared / embedding-declared / declared-but-unregistered.
…ls, getter boundaries, OR byte-invariance probe TestRegistrationMetadata in test_network_registry.py (CUDA-gated, class skipif since the file has no prior CUDA gate) covers register_network's arity/arg_sorts/artifact_hash kwargs and the network_metadata() getter: roundtrip, byte-compat with the legacy no-kwargs call, the three typed refusals (arity vs. declaration, arg_sorts without arity, length mismatch), and the getter's three failure modes (undeclared, declared- but-unregistered, embedding-declared). Writing these tests surfaced that _compiled_program_register_network_with_lineage in crates/pyxlog/python/pyxlog/__init__.py (the nn4-lineage monkeypatch installed over CompiledProgram.register_network) still had the pre-Task-1 fixed signature, with no path for arity/arg_sorts/artifact_hash to reach the underlying native call — every public register_network() call carrying them would TypeError before reaching the Rust validation this surface exists to test. Threaded the three kwargs through positionally, ahead of the wrapper's own keyword-only lineage params. test_neural_credit.py gets one CPU test, test_masked_row_probability_is_byte_invariant_in_the_or, the byte- invariance probe promised to the external reviewer in #157: two p_event vectors differing only at a masked witness row (0.01 vs 0.99) produce bitwise-identical noisy_or_from_index results, with a sanity check that changing an active row does change the OR.
…tract — int sort ids with the bool trap, keyword-only bridge kwargs The consumer's published contract (#155) requires arg_sorts to be catalog sort ids (tuple[int, ...]), not sort names, with bool explicitly excluded even though isinstance(True, int) holds in Python; and requires arity, arg_sorts, and artifact_hash to be keyword-only, matching register_network(..., cache_size, *, arity, arg_sorts, artifact_hash). - pyxlog::neural::register_network now accepts arg_sorts as a Python sequence (Vec<PyObject>), validating each element is an int and not a bool before extracting to i64 -- the bool check runs first since pyo3 extracts bool into i64 silently otherwise. - NetworkConfig::arg_sorts and NetworkHandle::arg_sorts change from Option<Vec<String>> to Option<Vec<i64>>; network_metadata returns ints. - The three kwargs move behind `*` in the pyo3 signature; the __init__.py nn4-lineage wrapper mirrors this and forwards them as keywords instead of positionally (its old positional forward would have broken once the native signature went keyword-only). - Rust unit tests in registry.rs/handle.rs and the Python tests in TestRegistrationMetadata are updated to int sort ids, with new tests for the bool trap, a non-int element, and keyword-only enforcement.
register_network gains the keyword-only bridge kwargs and network_metadata is declared -- the stub was stale since before the seam delta.
Engine/seam deep validation (xlog-side / DTS consumer seat) — PASSReviewed the diff against our published field-level contract (#155) and our landed consumer adapter. The registration-seam mechanics are correct and the convergence with the contract is faithful. Verified specifics:
arity validated against the program (the two-layer check we recommended): Identity / Duplicate-name refusal — border correctly placed: the engine permits re-registration (retrain); duplicate refusal is intentionally left to our single-registry adapter, as our published contract stated. A raw-engine consumer gets retrain semantics; our adapter adds single-registry uniqueness on top. Consistent.
Compatibility with our bridge: the typed args here (the low-level torch-module ↔ Your GPU-verified 23/23 (A40) is consistent with our own first real-compiled-program exercise of the downstream engine-mode path today (which reached and correctly enforced the Engine/seam seat: PASS — no changes requested. Contract-fidelity / adversarial pass is covered separately by @xlog-claude-2. |
|
Contract-fidelity review (consumer side) — PASS, no changes requested. Complementing the engine/seam review above: I verified this delta point-by-point against the field-level contract we published in #155, and independently executed the suite plus adversarial probes on a wheel built locally from this PR's head (torch 2.12.1+cu130, CUDA). Contract fidelity — verbatim on every term:
Independent execution (not just reading): wheel built from this head on my machine;
One informational note on evidence lines: Also confirmed the byte-invariance probe promised in the #157 review is present and non-vacuous (the active-row counter-assertion). Thanks @niveousdragon — this is a precise delivery of the contract. |
|
Both optional items from the contract-fidelity pass are landed in a981ddc: Probe 6 (bool arity) — hardened. Probe 2 (retrain path) — pinned. Both new tests are collect-verified locally (no GPU here); they follow the exact fixtures your probes exercised, so your re-probe or the next pod run confirms execution. The evidence-line note is taken: the 25/25 figure is With both seats PASS and #155 closed, the branch is ready for the mechanical merge on our side. |
The problem
Ask #2 of the #155 seam-mapping — the last piece the consumer's production bridge needs end-to-end: the network registration seat carries no typed signature (
arity,arg_sorts) and no artifact identity (artifact_hash), so the consumer's registration-mapping validates adapter-side against name conventions instead of against the engine, and the identity law ("a retrained network is a NEW rule") is adapter-asserted rather than upstream-anchored.The slice this branch implements
Converged with the consumer's published field-level contract (#155, law-reviewed on their side) before this PR opened:
register_network(..., *, arity=None, arg_sorts=None, artifact_hash=None)— the three bridge kwargs are keyword-only (matching the consumer's registration signature). Declaredarityis validated against the program's own nn declarations: everyNeuralPredicateInfobacked by the network must agree (predicate_arity), mismatch is a typed refusal naming the predicate and both arities — the seat validates against the program, not the adapter (the consumer's structurallen(arg_sorts) == aritycheck composes on top; both layers kept per their recommendation).arg_sortsare catalog sort ids: ints, with the bool trap. Elements are checkedisinstance(elem, bool)before any int extraction — pyo3 silently coercesTrue → 1, so ordering is the trap (the sameisinstance(True, int)law both sides already enforce elsewhere). A bool or non-int element is a typed refusal naming the index. Stored asVec<i64>; length must equalarity;arg_sortswithoutarityis refused.artifact_hash— a single opaque identity anchor, stored and returned untouched, never interpreted (converged: the consumer's seam-sideMappingreduces to one hash at this boundary). Re-registration with a new hash is the legal retrain path of their identity law, deliberately not refused.network_metadata(name)— the queryable registry:{arity, arg_sorts, artifact_hash, declared: [{predicate, predicate_arity, input_arity, labels}]}, so the consumer's registration-mapping verifies against the real declarations. Three honest failure modes: undeclared; embedding-declared (namesregister_embedding— embeddings carry no registration metadata by design); declared-but-unregistered.NetworkConfig→NetworkHandle→ getter — the handle copy was a mid-branch review catch: the fields were silently write-only), the Python lineage wrapper threads the kwargs keyword-only (another mid-branch catch: it would haveTypeErrord before Rust), and the type stubs cover the new surface.Verified
TestRegistrationMetadata23/23 passed on this exact head — roundtrip, legacy byte-compat, arity-vs-declaration refusal, bool-trap, string-element refusal, keyword-only enforcement, all three getter failure modes. Full engine-mode + identifiability suites: 62 passed on the branch (registration-only commits since).cargo test -p xlog-neural25/25 (defaults, clone/from_config retention);cargo check -p pyxlogclean; fmt clean.torch.equal), while an active row's change moves it — physical absence from the index, pinned.Nonewith byte-identical legacy behavior through both the wrapper and the native path.cargo test --workspaceon a fresh-rustup pod hit an LLVM/lld linker crash in the unrelatedxlog-integratione2e binary (toolchain bug in rust 1.97.1's bundled LLVM, "please submit a bug report to llvm-project") — not reproducible against this branch's crates; our crates' tests pass.Notes for review
With this, all three #155 asks are delivered: mask channel (#157), frozen entry (#156), registration seam (this PR) — the consumer's loop "register typed → enumerate with sort-matched slots → frozen_select with the mask channel" closes against a real registered surface.