Skip to content

feat(pyxlog): expose canonical symbol registry - #286

Open
levi770 wants to merge 1 commit into
mainfrom
codex/pyxlog-symbol-bridge
Open

feat(pyxlog): expose canonical symbol registry#286
levi770 wants to merge 1 commit into
mainfrom
codex/pyxlog-symbol-bridge

Conversation

@levi770

@levi770 levi770 commented Aug 30, 2026

Copy link
Copy Markdown
Contributor

Context

LogicProgram declares symbol columns as physical u32 values at the DLPack boundary. XLOG already owns the process-wide symbol registry in xlog_core::symbol, but Pyxlog did not expose a supported way for an external consumer to encode strings before CUDA upload or decode query results afterward.

Without this boundary, consumers either cannot submit real symbol-valued relations or must reproduce private registry behavior outside XLOG. Mistaber's isolated production worker needs XLOG itself to own both directions of that conversion.

Change

  • Add pyxlog.intern_symbols(list[str]) -> list[int].
  • Add pyxlog.resolve_symbols(list[int]) -> list[str].
  • Route both functions directly through xlog_core::symbol.
  • Preserve input order and duplicate identity.
  • Reject unknown IDs with a deterministic ValueError that includes the ID and input ordinal.
  • Export the API in both native and package-root type stubs.
  • Add runtime tests for batch round-trip, duplicate stability, empty symbols, order preservation, and unknown-ID rejection.

Boundary and semantics

The returned IDs belong to the XLOG symbol registry in the current worker process. Consumers must intern input strings and resolve output IDs through the same isolated worker; this PR does not define a portable persisted-ID format and does not alter logical or CUDA execution semantics.

No CUDA kernel, evaluator, parser, or resolver behavior changes in this PR.

TDD evidence

The new runtime test was first run against Pyxlog 0.12.0 without this API and failed because intern_symbols and resolve_symbols were absent. After implementation, the branch wheel passed the focused runtime test.

Verification

Local checkout, commit 247c10f:

  • cargo fmt --all -- --check
  • git diff --check
  • python -m ruff check --config python/pyproject.toml python/tests/test_pyxlog_symbol_registry.py
  • cargo test -p xlog-core symbol --locked
    • 10 passed, 0 failed

RunPod upmqk0ltyhijt6, NVIDIA RTX PRO 4500 Blackwell, CUDA 13.0, Python 3.12, Torch 2.9.1+cu130:

  • Built the release wheel with CUDA kernels:
    • pyxlog-0.12.0-cp312-cp312-linux_x86_64.whl
  • XLOG_REQUIRE_CUDA=1 python -m pytest -q python/tests/test_pyxlog_symbol_registry.py
    • 2 passed
  • Real DLPack/CUDA/XLOG integration:
    1. compile a symbol-typed Datalog program;
    2. intern dairy, meat, and neutral;
    3. upload torch.uint32 on cuda:0;
    4. evaluate through LogicRelationSession;
    5. capture transfer counters while strict deterministic D2H is enabled;
    6. disable the gate only for final result export;
    7. decode output IDs with resolve_symbols.

Observed:

{
  "cuda_device": "NVIDIA RTX PRO 4500 Blackwell",
  "hot_loop_d2h_violations": 0,
  "hot_loop_stats": {
    "dtoh_bytes": 0,
    "dtoh_calls": 0,
    "htod_bytes": 0,
    "htod_calls": 0
  },
  "output_symbols": ["dairy", "meat", "neutral"],
  "query_relation_name": "__xlog_query_0",
  "rows": 3
}

The zero-transfer statement above is scoped only to the measured post-upload evaluation hot loop. Final deterministic result export is outside that scope.

Stack topology

This PR is the root of the Pyxlog/runtime stack. Review and merge order is #286 -> #287 -> #288 -> #289.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant