CrossGL Translator includes a project-level orchestration layer for repositories that contain shader or GPU source files across one or more supported source backends. The project pipeline discovers translation units, invokes the existing single-file translator for each unit, writes translated artifacts under a separate output directory, and emits a machine-readable portability report.
The project pipeline translates shader and kernel source artifacts. It does not rewrite host runtime code, application build systems, resource binding setup, or framework-specific backend integration. Those migration steps are reported as manual follow-up work in the portability report.
Use the project pipeline as an audit-first migration workflow:
- Start with a scan-only report to confirm discovery, source backend detection, configured targets, include directories, source overrides, and diagnostics before writing translated artifacts.
- Add or refine
crosstl.tomlso repository-relative source roots, include/exclude patterns, source overrides, include directories, defines, named variants, optional entry-point selections, output directory, targets, and optional external corpus manifest are explicit. - Run
translate-projectinto a separate output directory and keep the generated portability report with the translated artifacts. - Run
validate-projecton the generated report. Use the JSON output for automation, text output for local triage, or SARIF output for code-scanning systems. - Run
inspect-reportwhen the raw report is too large to review directly. The inspection output keeps bounded samples for diagnostics, failed artifacts, source maps, source remaps, validation records, external corpus entries, and migration actions. - Treat
migrationactions as manual host-integration work. They identify runtime API, resource binding, build-system, and backend framework review that remains outside shader/kernel source translation.
A typical first pass looks like:
python -m crosstl scan /path/to/repo \
--target metal \
--target opengl \
--output scan-report.json
python -m crosstl translate-project /path/to/repo \
--target metal \
--target opengl \
--output-dir crosstl-out \
--report crosstl-out/portability-report.json
python -m crosstl validate-project \
crosstl-out/portability-report.json \
--format text
python -m crosstl inspect-report \
crosstl-out/portability-report.json \
--format textThe same project APIs are available to Python callers that need to integrate with existing automation:
from pathlib import Path
from crosstl.project import inspect_project_report, translate_project
report_path = Path("crosstl-out/portability-report.json")
report = translate_project(
"/path/to/repo",
targets=["metal", "opengl"],
output_dir=report_path.parent,
validate=True,
)
report.write_json(report_path)
inspection = inspect_project_report(report_path)
print(inspection["success"])Use these report fields to decide the next action:
| Report field | Triage use |
|---|---|
diagnosticCounts, diagnosticsByCode,
diagnosticsByTarget, diagnosticsBySourceBackend, and
diagnosticsByVariant/diagnosticsByCheckKind |
Separate configuration errors from source/backend translation failures, then group actionable diagnostics by target backend, source backend, named variant, and validation check kind before reviewing artifacts. |
missingCapabilityCounts |
Group unsupported source features, include resolution gaps, define forwarding gaps, artifact manifest issues, provenance issues, and optional toolchain validation gaps. |
artifactMatrix |
Confirm the expected unit, target, and named-variant artifact plan before translation, then identify missing or extra artifacts after translation. |
project.entryPointSelections and artifacts[].entryPoint |
Confirm that requested materialized source entries were each packaged under deterministic entry paths and identify their reflected target entries and stages. |
units[].entryDiscovery |
Review source-frontend discovery availability, host-visible entry names, stages, declaration provenance, and unresolved-name diagnostics before constructing an entry-scoped artifact plan. |
validation |
Check current source hashes and byte sizes, generated artifact hashes and byte sizes, source maps, source remaps, optional toolchain availability, and opt-in artifact or availability smoke test results after translation. |
externalCorpus |
Compare pinned reduced corpus entries with discovered units and emitted artifacts without treating the manifest as whole-repository semantic parity. |
migration |
Track manual runtime, binding, build-system, and backend integration follow-up work separately from translated shader/kernel artifacts. |
The legacy single-file command remains available:
python -m crosstl examples/graphics/SimpleShader.cgl --backend metalLegacy single-file options may appear before or after the input path.
The explicit single-file subcommand is equivalent:
python -m crosstl translate examples/graphics/SimpleShader.cgl --backend metalBoth single-file forms also accept --source-backend, repeatable
--include-dir, and repeatable --define overrides. Use them when a file
has a nonstandard extension or when the selected source frontend exposes
include-path and preprocessor define options. Use --output - to write the
translated source to stdout instead of creating an output file.
Scan a repository and print a JSON report:
python -m crosstl scan /path/to/repo --target metalEmit the same scan-only portability report with an explicit output path:
python -m crosstl report /path/to/repo \
--target metal \
--output crosstl-out/portability-report.jsonScan and report commands exit nonzero when the generated report contains error
diagnostics, while still writing the JSON report to stdout or the requested
output file.
Use --output - on single-file translation, scan, report, validation, and
inspection commands, or --report - on translate-project, when stdout
should be selected explicitly in scripts.
Translate every discovered unit to one or more targets:
python -m crosstl translate-project /path/to/repo \
--target metal \
--target opengl \
--output-dir crosstl-out \
--report crosstl-out/portability-report.json \
--run-toolchainsProject translation exits nonzero when the report contains failed artifacts or
error diagnostics.
--validate records artifact existence, source and generated hash checks,
source-map and source-remap status, and configured toolchain availability
without invoking external compiler tools.
Embedded toolchain availability records name the configured validation hook
tools for each target; paths and availability remain environment-specific.
--run-toolchains implies artifact validation and records any available
bounded toolchain smoke-check results in the generated portability report.
Smoke-check records and generated toolchain-failure diagnostics include a check
kind so report consumers can distinguish artifact checks from target tool
availability checks.
OpenGL smoke checks invoke glslangValidator with --stdin and an
explicit -S stage inferred from the artifact extension or common GLSL
builtins, defaulting to compute for generic .glsl outputs.
Vulkan smoke checks validate binary .spv artifacts with spirv-val and
assemble textual .spvasm artifacts with spirv-as -o pointed at the
platform null device.
Project translation is sequential by default. Use --workers to run
independent artifact jobs in isolated processes:
python -m crosstl translate-project /path/to/repo \
--target directx \
--target opengl \
--output-dir crosstl-out \
--report crosstl-out/portability-report.json \
--workers 2Python callers use the corresponding max_workers argument to
translate_project. The value must be a positive integer; 1 retains the
sequential path.
At most N jobs are submitted at once. Each worker translates one planned
unit, target, variant, and selected entry combination in a separate process so
mutable frontend and generator state is not shared between concurrent jobs.
Before workers start, the pipeline traverses the canonical plan once to reject
output-path collisions. The execution pass then regenerates that plan lazily
and retains at most N scheduled requests instead of retaining every
unit, target, variant, and entry combination in memory.
The parent process consumes results in the deterministic project plan order,
publishes each staged artifact/source-remap pair, writes the corresponding
checkpoint completion, and assembles diagnostics, artifacts, and the artifact
matrix with the same ordering as a sequential run. Workers do not replace
published outputs directly.
Artifact validation and optional toolchain smoke checks run only after all
translation jobs have returned, avoiding nested toolchain concurrency.
Use --job-timeout-seconds to place a finite wall-clock limit on each
artifact job:
python -m crosstl translate-project /path/to/repo \
--target directx \
--target opengl \
--output-dir crosstl-out \
--report crosstl-reports/portability-report.json \
--checkpoint crosstl-reports/translation-checkpoint.json \
--workers 2 \
--job-timeout-seconds 300Python callers use the corresponding job_timeout_seconds argument. The
value must be a positive finite number. A configured timeout uses process
isolation even when max_workers=1 so the active translation can be stopped.
The budget starts when a job is submitted. A result that has already completed
is accepted even when its budget has elapsed before the coordinator reaches it.
When a job exceeds its budget, the coordinator terminates that worker-pool
generation, removes its private staging, and resubmits unaffected scheduled
jobs in canonical order. The timed-out coordinate is retained as a failed
artifact with a project.translate.timeout diagnostic and, when enabled, a
completed checkpoint record. Previously published output for that coordinate
is not replaced. Translation then continues so the final report describes the
rest of the repository instead of relying on an outer process timeout.
Changing the configured timeout changes checkpoint invocation identity and
therefore requires a new run rather than an incompatible resume.
An interruption stops further submission, cancels work that has not started,
and terminates pool processes that are still translating. Each concurrent run
adds a private token to its staging directories. After the pool settles, the
coordinator removes unconsumed results and any token-owned staging left by a
terminated process without touching another invocation's files. Previously
published outputs remain unchanged. Completed checkpoint records remain
available for a verified resume, and only coordinator-published results are
recorded as complete.
An ordinary worker failure raises ProjectTranslationWorkerError with the
source, target, output path, optional entry point, original exception type, and
original exception attached for programmatic inspection. An enabled checkpoint
records the same active coordinate and typed interruption message.
Spawned workers independently load installed backends and plugins. A backend
registered only in the current Python process is not inherited on every
platform; use max_workers=1 or install that backend through the supported
plugin discovery mechanism.
Each translation job writes its generated artifact and source-remap sidecar to a temporary directory on the destination filesystem. The pipeline computes the reported hashes, byte sizes, source maps, and placeholder diagnostics from the staged files before publication. It then publishes the sidecar followed by the artifact with atomic file replacement, treating the artifact replacement as the pair's commit step.
If generation or publication fails, temporary files are removed and any previously published artifact/remap pair is retained. If replacement has started, the pipeline restores both prior files before reporting the job failure. Consumers should still use the portability report as the authoritative result of the latest run; a retained pair represents the last successful translation, not the failed attempt.
translate-project can persist progress independently of the final
portability report. Use a checkpoint for repository runs that may be interrupted
by a local process limit or CI timeout:
python -m crosstl translate-project /path/to/repo \
--target directx \
--target opengl \
--output-dir crosstl-out \
--report crosstl-reports/portability-report.json \
--checkpoint crosstl-reports/translation-checkpoint.jsonThe checkpoint is replaced atomically as work advances. It records the
running, interrupted, or complete state; project and invocation
identity hashes; the deterministic job plan; completed, active, and pending
coordinates; scan and translation diagnostics; and a partial artifact matrix.
Generated source is not embedded in the checkpoint.
Resume an unfinished run with the same project configuration and translation options:
python -m crosstl translate-project /path/to/repo \
--target directx \
--target opengl \
--output-dir crosstl-out \
--report crosstl-reports/portability-report.json \
--checkpoint crosstl-reports/translation-checkpoint.json \
--resumeBefore skipping a completed job, resume verifies the project identity, complete job plan, current source identity, generated artifact hash and size, and source remap hash and size. Stale, modified, missing, or mismatched outputs stop the resume instead of being trusted. A checkpoint path must be outside the artifact output directory and cannot replace the project configuration or a registered source file.
The default writes each job transition. For projects with large report metadata,
--checkpoint-interval-jobs N persists batches of completed jobs. A larger
interval reduces write overhead, but an ungraceful termination may cause up to
N - 1 already-emitted jobs to be translated again because only persisted
completions are trusted.
Python callers use the corresponding checkpoint_path, resume, and
checkpoint_interval_jobs arguments to translate_project. Worker count
does not affect checkpoint identity, but a configured per-job timeout does.
A progress checkpoint is not a final portability report. Only a complete
checkpoint contains the canonical final report, and neither state establishes
host runtime integration or numerical parity.
Project scans record an entryDiscovery object on every translation unit.
Its status distinguishes a frontend that supports discovery (available),
a frontend that does not yet expose discovery (unavailable), and a
frontend failure (failed). An available result contains ordered,
deduplicated entry records with the exported name, canonical stage, source
location, and declaration provenance. Discovery diagnostics are also promoted
to the project diagnostic stream under the source-entry-discovery check
kind.
Metal discovery expands configured includes, defines, and active
preprocessor branches without materializing template bodies or invoking a
target generator. It reports concrete entry functions and explicit
host-named template materializations. Commented examples, inactive branches,
ordinary helpers whose surrounding comments mention kernels, and unresolved
dynamic host names are not converted into entries. Locations currently use
the preprocessed-source coordinate space, which is recorded explicitly in
the report.
The public ProjectScan.discovered_entry_points() method returns a
repository-relative mapping compatible with ProjectConfig.entry_points:
from dataclasses import replace
from crosstl.project import scan_project, translate_project
scan = scan_project("/path/to/repo")
config = replace(
scan.config,
entry_points=scan.discovered_entry_points(),
targets=("directx", "opengl"),
)
report = translate_project(config)Discovery does not change translation behavior by itself. This explicit step
lets callers review or filter a potentially large entry set before scheduling
artifacts. Source frontends without a discovery provider report
unavailable rather than returning an empty result that could be mistaken
for a source file with no entries.
For repositories where selected source files should expand automatically, add
repository-relative source patterns to translate_discovered_entry_points:
[project]
translate_discovered_entry_points = [
"kernels/arange.metal",
"kernels/normalization/*.metal",
]Only matching units whose discovery status is available and which contain
concrete entries are expanded. Each discovered entry uses the existing
entry-scoped artifact planner, output path, checkpoint coordinate, source map,
provenance, and target behavior. Source and entry ordering remain the ordering
recorded by the scan.
An explicit exact or glob selector in project.entry_points takes
precedence for every source it matches. Sources outside the configured
translate_discovered_entry_points patterns retain aggregate translation.
This source-scoped contract avoids turning a repository-wide scan into an
unbounded artifact matrix; choose broad patterns only after reviewing the
discovered entry count in a scan report.
The same selection can be added for one invocation with a repeatable CLI option:
python -m crosstl translate-project /path/to/repo \
--translate-discovered-entry-points "kernels/arange.metal" \
--target directx --target openglInvalid or unmatched patterns and matching sources with unavailable, failed, or empty discovery produce structured configuration diagnostics. They are not reported as successful per-entry expansion.
Entry discovery identifies shader and kernel declarations only. It does not infer dispatch dimensions, resource bindings, host call sites, backend initialization, or numerical parity.
Repositories can request standalone artifacts for one or more materialized
source entries by adding a repository-relative selector table to
crosstl.toml:
[project]
include = ["kernels/arange.metal"]
include_dirs = ["."]
targets = ["directx", "metal", "opengl"]
output_dir = "crosstl-out"
[project.entry_points]
"kernels/arange.metal" = ["arangeuint32", "arangec64"]Each value may be one entry name or an ordered array of entry names. An array creates one independently checkpointed artifact per entry in the declared order. Empty arrays and duplicate names are rejected.
For DirectX compute output, each selected source entry is emitted as target
entry CSMain. A source such as kernels/arange.metal produces
crosstl-out/directx/kernels/arange/arangeuint32.hlsl and
crosstl-out/directx/kernels/arange/arangec64.hlsl. Each standalone HLSL
artifact retains only that entry's reachable helpers, resources, constants,
and execution contract. Explicit registers and spaces remain unchanged, while
runtime-loader metadata records the selected cs_6_0 entry profile.
For OpenGL compute output, the same selection produces one .glsl artifact
per entry with target entry main. For Metal compute output, it produces one
.metal artifact per entry and retains only the selected kernel, reachable
helpers, referenced declarations, and recursively referenced struct families.
The emitted Metal entry name is recorded exactly rather than normalized to a
fixed main. Runtime manifests reflect the standalone Metal source and keep
buffer, texture, and sampler index spaces independent.
For all three targets, the portability report records every source entry, target entry, and reflected stage; embedded validation records carry the same identities. Runtime artifact manifests then reflect only the selected stage interface from each standalone output.
Selection is exact after source materialization. Missing or ambiguous entries
fail with structured diagnostics and no target file. Targets that do not yet
implement standalone entry generation also fail explicitly instead of pruning
an aggregate artifact. When project.entry_points is absent, project
translation keeps the existing aggregate output path and behavior.
Entry selection scopes shader or kernel translation; it does not infer host dispatch dimensions, runtime bindings, or backend integration. Record those requirements through the corresponding dispatch and runtime contracts.
The current-pinned MLX integration exercises entry-scoped translation for all
877 discovered entries from the include-expanded unary.metal source. The
finite split is 183 each for v_, v2_, gn1_, and gn4large_, plus
145 vn_ entries, spanning 37 operators, 20 concrete input/output type pairs,
and 16 semantic families. v_ uses explicit N=1; v2_ and vn_
retain N=WorkPerThread<T>::n from the source default; gn1_ uses
N=1, IdxT=int; and gn4large_ uses N=4 plus the source-default
IdxT=int64_t.
Every independently translated artifact has one selected operator implementation
and one kernel. Vector shapes materialize one specialization and reflect input,
output, and size resources. Gather shapes additionally materialize the reachable
elem_to_loc<int> or elem_to_loc<int64_t> helper and reflect constant
shape/stride buffers plus a read-only device dimension binding. The resulting
877 artifacts contain 1,243 exact materializations, preserve a host-owned
[1, 1, 1] workgroup contract, and reject residual templates, decltype,
call operators, unsupported placeholders, and non-selected operator bodies.
Required macOS CI compiles all 877 exact artifacts with
xcrun -sdk macosx metal -Werror -c and requires a non-empty AIR output from
each with no warning exemption. The generic path resolves source typedef chains
and bfloat reconstruction, materializes source-compatible constrained free
operators, infers aggregate aliases, recognizes branch-complete returns,
preserves narrow as_type storage, and admits only a proven read-only
immediate scalar-parameter view as a matching thread-local single-field
aggregate. The non-scalar path also retains
constant-resource provenance, const-device references, postfix update position,
and native Metal union aliasing while ambiguous address spaces and unsupported
reinterpretation continue to fail closed. This complete discovered-unary
selected-entry proof does not claim Metal numerical execution, host-runtime
redirection, or the MLX test suite.
The same entry-scoped pipeline now translates all 877 current-pinned unary
entries to standalone OpenGL main artifacts. The schema-v2
unary.opengl-translation.json contract preserves the same five-shape,
37-operator, 20-type-pair classification and all 1,243 exact materializations,
while pinning 4,060,696 generated GLSL bytes and all 3,363 target-reflected
resources. Vector artifacts expose read-only input, read-write output, and an
entry-scoped size uniform block. Gather artifacts expose input, output, shape,
stride, and the read-only ndimBuffer storage resource; scalar uses of the
source device const int& ndim alias element zero without changing expression
typing.
OpenGL's 32-bit index profile cannot represent every source 64-bit index
implicitly. The complete contract therefore records explicit host/runtime
bounds of [0, 2147483647] for offset + i, out_idx++, and idx.
These are declared portability preconditions, not inferred facts or generated
runtime checks; absent proof continues to fail closed. GLSL also has no native
log10 overload, so reachable Metal base-ten logarithms lower through a
single evaluation of log2(value) * 0.3010299956639812 while user-defined
log10 functions remain ordinary calls.
Required CI partitions the family across five disjoint Linux shards. Each shard
retranslates its exact entries, verifies generated identity, materialization,
main workgroup metadata, and reflected ABI, compiles with
glslangValidator --target-env opengl --target-env spirv1.3 -S comp, validates
with spirv-val --target-env spv1.3, and requires every SPIR-V module to be
non-empty. This closes whole-family OpenGL translation, reflection, and native
compiler coverage; it does not claim OpenGL numerical execution, MLX host
runtime redirection, or MLX test-suite parity.
The same entry-scoped pipeline now translates all 877 current-pinned unary
entries to standalone DirectX CSMain artifacts. The schema-v2
unary.directx-translation.json contract retains the exact five-shape,
37-operator, 20-type-pair classification and 1,243 materializations. v_ and
vn_ artifacts reflect input, output, and entry-scoped size resources;
v2_ artifacts additionally reflect the generated CrossGLDispatchInfo
workgroup-count cbuffer at b3. Gather artifacts reflect input, output,
shape, stride, and read-only ndim structured buffers plus their generated
CrossGLDispatchInfo cbuffer at b0, for 3,912 reflected HLSL resources
in total. Source scalar uses of
device const int& ndim alias ndim[0] and source out_idx++ remains a
postfix update.
DirectX bfloat support covers every unary intrinsic required by the family.
The inverse-hyperbolic acosh, asinh, and atanh paths decode bfloat
to float, invoke their portable float helpers, and round back to the exact
bfloat payload representation. Explicit contextual HLSL constructors preserve
float-to-native-16 return and initializer narrowing without -Wconversion
diagnostics when warnings are fatal. All artifacts derive
-enable-16bit-types from retained native 16-bit declarations and target
shader model cs_6_2. As with OpenGL, explicit host/runtime bounds of
[0, 2147483647] for offset + i, out_idx++, and idx remain
portability preconditions rather than inferred or generated checks.
Required CI partitions the family across five disjoint Windows shards. Each
shard retranslates its exact entries, verifies deterministic identity,
materialization, CSMain workgroup metadata, and reflected ABI, then compiles
with pinned DXC using -enable-16bit-types -WX -T cs_6_2 -E CSMain and
requires non-empty DXIL modules. Together, the DirectX and OpenGL contracts
close complete discovered-unary translation, reflection, and native compiler
coverage on both targets; they do not claim numerical execution, MLX host
runtime redirection, or MLX test-suite parity.
The current-pinned MLX copy integration proves all 2,496 discovered entries
from copy.metal through Metal-to-CrossGL-to-Metal translation. The family
covers 30 shapes, 16 concrete templates, 13 input and output types, and all 169
conversion pairs. Its schema-v2 contract pins every artifact and shape ABI,
including source/default parameter provenance. The 2,496 artifacts contain
6,566 exact materializations and 8,684 reflected resources. Every artifact has
one exact cast_to materialization; only the 14 complex-to-Boolean entries
add the nested cast_to<bool, float> body, so materialization accounting is
explicitly data-dependent rather than a false fixed per-shape count.
The generic lowering preserves MLX's float and bfloat16 Boolean bit tests,
recovers native ushort width after frontend uint16_t normalization, and
projects registered complex64_t values only after validating the ordered
real/imag float representation. Wrong registered shapes fail closed and
unregistered lookalikes remain untouched. Reflected interfaces contain three to
eight exact resources and retain host-owned [1, 1, 1] workgroup metadata.
Required macOS CI uses 24 disjoint 104-entry shards and compiles all 2,496 exact
artifacts with xcrun -sdk macosx metal -Werror -c, requiring a non-empty AIR
object for each. This proves translation, reflection, and native compiler
acceptance; it does not claim Metal numerical execution, MLX host-runtime
redirection, or MLX test-suite parity.
The same selected-entry pipeline translates all 2,496 copy entries to
standalone OpenGL main artifacts. The schema-v2
copy.opengl-translation.json contract preserves all 30 shapes, 16 concrete
kernel templates, 13 input/output types, all 169 conversion pairs, 6,566 exact
materializations, and 8,684 reflected target resources. Scalar and vector forms
expose source and destination storage buffers plus an entry-scoped size block;
fixed generalized forms add scalar stride blocks or stride storage buffers;
rank-generic forms expose shape and stride buffers plus an entry-scoped rank
block; dynamic forms add exact source and destination offset blocks.
The generic registered-structure contract recognizes both canonical
complex64_t and its emitted complex_t_float representation. The 150
complex-to-scalar entries project the real field only after validating the exact
ordered real/imag float shape, evaluate the source once, and retain
narrow target conversion semantics. A malformed registered representation
fails closed rather than emitting an invalid GLSL constructor.
OpenGL's 32-bit index profile cannot implicitly preserve every source 64-bit
buffer index. The complete contract therefore declares explicit host/runtime
bounds of [0, 2147483647] for offset + i, src_idx, dst_idx,
dst_idx + i, src_idx + src_offset, dst_idx + dst_offset, idx.x,
and idx.y. These are portability preconditions rather than inferred facts
or generated checks; absent proof continues to fail closed.
Required Linux CI partitions the family into 24 disjoint 104-entry shards. Each
shard retranslates its entries, verifies deterministic artifact identity,
materialization provenance, standalone main workgroup metadata, and exact
target reflection, compiles with
glslangValidator --target-env opengl --target-env spirv1.3 -S comp, validates
with spirv-val --target-env spv1.3, and requires a non-empty SPIR-V module.
Together with the Metal contract this closes complete discovered-copy
translation, reflection, and native compiler coverage for those two targets; it
does not claim numerical execution, MLX host-runtime redirection, or MLX
test-suite parity.
The DirectX path translates the same 2,496 entries to standalone CSMain
artifacts. Its compact schema-v2 copy.directx-translation.json contract
preserves all 30 shapes, 16 concrete templates, 13 input/output types, all 169
conversion pairs, and 6,566 exact materializations. Exact target reflection
contains 10,036 resources. The g2, g2large, g3, g3large,
gn2, gn4large, s2, and v2 shapes additionally expose generated
CrossGLDispatchInfo metadata at their exact shape-specific b0, b6,
or b3 binding.
The shared registered-structure conversion contract admits emitted
complex_t_float values only after validating their ordered real and
imag float fields. Exactly 150 complex-to-scalar entries project the real
field, malformed registered representations fail closed with structured
project diagnostics, and unregistered lookalikes remain untouched. Unlike the
OpenGL lowering, this native HLSL path introduces no additional source-scoped
32-bit index-range portability promise.
Required Windows CI partitions the DirectX family into 24 disjoint 104-entry
shards. Each shard retranslates its exact entries, verifies deterministic HLSL
identity, materialization provenance, CSMain workgroup metadata, and exact
reflected ABI, then compiles with checksum-pinned DXC using
-enable-16bit-types -WX -T cs_6_2 -E CSMain and requires a non-empty DXIL
module. Together with the Metal and OpenGL contracts this closes complete
discovered-copy translation, reflection, and native compiler coverage on all
three targets; it does not claim numerical execution, MLX host-runtime
redirection, or MLX test-suite parity.
The current-pinned MLX binary integration independently proves all 4,122
discovered entries from binary.metal. Fifteen 238-entry base shapes and
three 184-entry work-per-thread shapes span 18 shapes, 11 concrete kernel
templates, 24 operators, and 25 input/output type pairs. Each selected artifact
contains one operator implementation and one kernel, rejects residual templates,
decltype, call operators, unsupported placeholders, and non-selected
operator bodies, and preserves explicit or source-default template provenance.
Scalar, size-bounded vector, fixed-dimensional generalized, and rank-generic
forms reflect exact three-, four-, five-, or seven-resource interfaces. The
generalized forms additionally materialize only their reachable
elem_to_loc_1, elem_to_loc_2, elem_to_loc_3, or
elem_to_loc_2_nd helper for the exact 32- or 64-bit index type. In total,
4,122 artifacts contain 6,026 exact materializations and 19,106 reflected
resources while preserving a host-owned [1, 1, 1] workgroup contract. The
238-entry scalar contract remains an exact subset of the complete contract.
The generic path maps concrete signed and unsigned 64-bit vectors to native Metal vector types, rewrites dependent free operators only to already-emitted exact helpers, and applies known non-explicit contextual aggregate constructors without admitting explicit-only or ambiguous conversions. Bfloat minimum and maximum calls promote their arguments to float before typed reconstruction; discarded type constructors remain evaluated through unambiguous void casts; and scalar Boolean relational expressions receive explicit C++ integral promotion. Focused tests retain conservative rejection boundaries for each contextual operation.
A schema-v2 hash-pinned contract records every identity, shape, template,
classification, byte count, materialization, and resource ABI. Required macOS
CI compiles all 4,122 exact artifacts in 24 disjoint shards with
xcrun -sdk macosx metal -Werror -c and requires non-empty AIR without a
warning exemption. This closes discovered selected-entry translation,
reflection, and native compiler coverage. It does not claim Metal numerical
execution, host-runtime redirection, or MLX test-suite parity.
The same selected-entry pipeline translates all 4,122 binary entries to
standalone OpenGL main artifacts. The schema-v2
binary.opengl-translation.json contract preserves all 18 shapes, 11 kernel
templates, 24 operators, 25 input/output type pairs, and 6,026 exact
materializations. It pins 16,276,504 generated GLSL bytes
and 19,106 reflected resources across exact three-, four-, five-, and
seven-resource target interfaces. Scalar forms expose three storage buffers;
size-bounded forms add an entry-scoped size uniform block; fixed generalized
forms expose scalar stride blocks or stride storage buffers; and rank-generic
forms expose shape and stride buffers plus an entry-scoped rank block.
OpenGL's 32-bit index profile cannot implicitly preserve the source's runtime
64-bit buffer indices. The complete contract therefore declares host/runtime
bounds of [0, 2147483647] for offset + i, a_idx, b_idx,
out_idx, out_idx++, idx.x, and idx.y. These are explicit
portability preconditions, not inferred facts or generated runtime checks;
unproven wide-index translation remains fail-closed.
Required Linux CI partitions the family into 24 disjoint shards. Each shard
retranslates its exact entries, checks deterministic identity, materialization
provenance, standalone main workgroup metadata, and target reflection,
compiles every artifact with
glslangValidator --target-env opengl --target-env spirv1.3 -S comp, validates
it with spirv-val --target-env spv1.3, and requires a non-empty SPIR-V
module. This closes complete discovered-binary OpenGL translation, reflection,
and native compiler coverage; it does not claim numerical execution, MLX
host-runtime redirection, or MLX test-suite parity.
The same selected-entry pipeline translates all 4,122 binary entries to
standalone DirectX CSMain artifacts. The schema-v2
binary.directx-translation.json contract preserves all 18 shapes, 11
kernel templates, 24 operators, 25 input/output type pairs, 6,026 exact
materializations, and the seven explicit index-range portability preconditions.
DirectX resource namespaces preserve source buffer coordinates. The nine shapes
that consume threads_per_grid add explicit CrossGLDispatchInfo
reflection, producing 21,248 resources across exact three- through
eight-resource target interfaces.
Computed-result bfloat ArcTan2, LogAddExp, and Power paths expand
both operands to float, compute in float, and reconstruct the result with exact
round-to-nearest-ties-to-even. Maximum and Minimum also expand for the
comparison, but preserve and return the selected original low-16-bit bfloat
payload without requantization. Unproven bfloat builtins remain fail-closed,
and native 16-bit storage remains a Shader Model 6.2 requirement.
Required Windows CI partitions the family into 24 disjoint shards. Each shard
retranslates its exact entries, checks deterministic identity, materialization
provenance, standalone CSMain workgroup metadata, and target reflection,
then compiles every artifact with checksum-pinned DXC using
-enable-16bit-types -WX -T cs_6_2 -E CSMain and requires a non-empty DXIL
module. This closes complete discovered-binary DirectX translation, reflection,
and native compiler coverage; it does not claim numerical execution, MLX
host-runtime redirection, or MLX test-suite parity.
Metal compute sources that require fixed 32-lane SIMD semantics can opt into a bounded OpenGL shared-memory lowering when the deployment device does not provide the KHR subgroup extensions:
[project.entry_points]
"kernels/quantized.metal" = "affine_quantize_float_gs_32_b_2"
[project.entry_workgroup_size_rules."kernels/quantized.metal"]
affine_quantize_float_gs_32_b_2 = [32, 1, 1]
[project.source_options.metal.target_options.opengl]
software_subgroup_width = 32This option is target-scoped and explicit; it does not change Metal parsing or
other target artifacts. The only accepted width is 32. The selected output
must contain exactly one compute entry with concrete positive local dimensions,
a local X dimension divisible by 32, and no more than 1,024 total invocations.
CrossTL partitions the linear invocation range into an exact compile-time count
of independent 32-lane software subgroups. Subgroup count, subgroup index,
subgroup width, and lane index lower respectively to the workgroup invocation
count divided by 32, gl_LocalInvocationIndex / 32,
CROSSTL_SOFTWARE_SUBGROUP_WIDTH, and
gl_LocalInvocationIndex % 32. The one-subgroup case retains the simpler
1u, 0u, and gl_LocalInvocationIndex forms.
The bounded mode supports scalar float, int, or uint sum, minimum,
maximum, and shuffle-down operations. Shared scratch spans the complete
workgroup, while every helper derives a subgroup-local lane and base so reads
and reductions cannot cross a 32-lane boundary. Subgroup operations normally
must execute in workgroup-uniform control flow. They may appear directly in the
entry or in a uniquely identified helper only when every helper call is direct,
unconditional, top-level, and owned by that sole entry. Compile-time branches,
canonical constant loops, and canonical runtime loops whose integer bounds are
proven workgroup-uniform remain valid.
One narrow lane-dependent reduction form is also supported: a direct
WaveActiveSum, WaveActiveMin, or WaveActiveMax assignment in a
top-level entry-owned if with no else. The condition must be
side-effect-free, the branch must contain exactly that one subgroup operation,
declarations and escaping control flow cannot precede it, and the payload and
target must be matching 32-bit numeric scalars. CrossTL evaluates the branch
prefix only for active lanes, contributes a typed identity from inactive lanes,
invokes the barriered subgroup helper uniformly across the workgroup, and
exposes the result only to active lanes. Sum uses zero. Minimum uses positive
infinity, INT_MAX, or UINT_MAX; maximum uses negative infinity,
INT_MIN, or unsigned zero. Conditional shuffle, nested or multi-operation
branches, and other unproven shapes continue to fail closed.
Direct source references to raw gl_Subgroup* or subgroup* builtins,
empty operation sets, unsupported payloads or operations, and unresolved helper
ownership also fail closed. Successful lowering emits barriered shared
memory helpers and the marker CROSSTL_SOFTWARE_SUBGROUP_WIDTH. It
deliberately emits no GL_KHR_shader_subgroup* extension,
gl_Subgroup* use, CROSSTL_REQUIRED_SUBGROUP_WIDTH marker, or hardware
subgroupWidth execution metadata. This keeps the software execution
contract distinct from the default hardware-subgroup path and its host
preflight. When the option is absent, KHR subgroup generation and exact-width
enforcement are unchanged.
The lowering establishes shader semantics only for this constrained contract.
It does not infer dispatch counts, prove arbitrary divergent control flow,
rewrite an application's loader, or claim parity for other entries. Invalid
requests fail with
project.translate.opengl-software-subgroup-invalid and include the width,
workgroup size, operation, and reason available at the rejection site.
Some source index types cannot be represented directly by a target's legal
scalar index types. When the application already constrains an index at the
host or runtime boundary, record that precondition in crosstl.toml:
[[project.index_range_assertions]]
source = "kernels/*.metal"
function = "gather_values"
expression = "element_index"
minimum = 0
maximum = 1023Each assertion table has these fields:
| Field | Meaning |
|---|---|
source |
Repository-relative source glob. The assertion is considered only for
matching translation units; omitting it defaults to *. |
function |
Optional exact source function name. When omitted, the assertion can apply in any function containing the matching expression. |
expression |
Source index expression covered by the assertion. Matching ignores whitespace but otherwise preserves the expression identity. |
minimum |
Inclusive integer lower bound for the expression. |
maximum |
Inclusive integer upper bound for the expression. It must not be less
than minimum. |
Index-range assertions are explicit host/runtime portability preconditions. CrossGL does not infer, emit, or enforce them at runtime. It uses an assertion only to justify a semantics-preserving target index conversion when the full asserted range is legal for the target representation and indexed extent. An assertion does not clamp, wrap, or otherwise redefine out-of-range source values; the application remains responsible for satisfying the precondition on every execution.
OpenGL cannot represent a source workgroup pointer directly. It specializes
pointer-free helpers against a concrete entry-owned shared array and must
prove that every composed access remains within that backing array. When the
source runtime already enforces an entry-specific absolute element range,
record that precondition explicitly:
[[project.workgroup_access_assertions]]
source = "kernels/fft.metal"
entry_point = "fft_mem_256_*"
function = "ReadWriter_*"
parameter = "crosstl_ptr_buf"
minimum = 0
maximum = 255Each assertion table has these fields:
| Field | Meaning |
|---|---|
source |
Repository-relative source glob. Omitting it defaults to *. |
entry_point |
Required source entry-point pattern. The assertion cannot cross entry ownership boundaries. |
function |
Helper-function pattern. Omitting it defaults to *. |
parameter |
Workgroup-pointer parameter pattern. Omitting it defaults to *. |
minimum |
Inclusive absolute element offset into the concrete backing array. |
maximum |
Inclusive absolute element offset. It must not be less than minimum. |
The assertion does not provide backing identity, extent, element type, or a pointer offset. OpenGL must still derive those properties from the source call graph and emits the original composed runtime offset expression. Matching assertions are intersected with any statically derived access range; a contradiction or an asserted range outside the concrete backing array fails before artifact emission. Entries without a matching assertion continue to require a complete static proof.
Workgroup-access assertions are host/runtime portability preconditions. CrossGL records them in the project report but does not emit runtime checks or change source indexing behavior. The application is responsible for satisfying every assertion on each dispatch.
The portability report records the configured tables under
project.workgroupAccessAssertions and their count under
project.workgroupAccessAssertionCount. Report consumers can therefore audit
the host/runtime assumptions used during translation alongside the generated
artifacts.
Exact DirectX bfloat16 lowering preserves bfloat16 storage and conversion
semantics instead of substituting IEEE half precision. When generated HLSL uses
native 16-bit storage, its artifact and runtime metadata advertise DirectX 12,
a minimum Shader Model of 6.2, and entry profiles such as cs_6_2. DXC
validation and runtime loader commands for that HLSL require
-enable-16bit-types; application-specific compiler wrappers and build
commands must preserve the same option.
Successful artifacts that use this path record a bfloat16Lowering object
with status set to exact, approximationUsed set to false, and
the register, storage, and rounding representations used by the generated
HLSL. This keeps exact lowering distinct from a compile-only half-precision
substitution in machine-readable reports.
If an operation cannot be lowered with exact bfloat16 semantics, translation
fails closed with a structured
project.translate.directx-bfloat16-unsupported diagnostic. Its
details.bfloat16Lowering record identifies available context such as the
target profile, operation, source type, and reason instead of silently changing
precision or behavior.
These compiler requirements and diagnostics define an artifact contract only. They do not imply automatic host runtime or backend integration: CrossGL does not modify application loader code, configure a DirectX backend, bind resources, or wire generated artifacts into a framework.
Targets that cannot represent a source pointer directly must prove its backing storage and composed offset before emitting an artifact. For workgroup storage, that proof includes the concrete backing declaration, entry-point ownership, element extent and type, offset composition through helper calls, and the affected materialization or specialization. Dynamic backing selection, unresolved offsets, incompatible declarations, escaped identity, and cross-entry ownership fail closed instead of producing target code with altered aliasing or synchronization behavior.
When the target exception provides provenance, an OpenGL workgroup-pointer
diagnostic records the available function, parameter, backingName,
offsetExpression, materializationName, and reason values under
details.workgroupPointer. Unavailable values are omitted so consumers can
distinguish retained evidence from assumptions. The surrounding diagnostic
also identifies the source path, intended artifact, target, and missing target
capability.
This report contract localizes translation work and preserves actionable evidence. It does not establish whole-repository semantic parity, rewrite host runtime integration, or prove execution correctness for a framework or corpus.
Project scan, report, and translation commands also accept repeatable
--source-root, --include-dir, --define, and --source-override
overrides. CLI source roots replace the configured source roots for that
command. CLI defines use NAME or NAME=VALUE syntax and override matching
names loaded from crosstl.toml. CLI source overrides use PATTERN=BACKEND
syntax and override matching source patterns loaded from crosstl.toml.
These overrides are recorded in the emitted project report.
Scan, report, and translation commands accept repeatable --variant NAME
selectors. crosstl.toml can set selected_variants = ["debug"] as the
default scoped variant list for project runs; explicit --variant arguments
override that configured default for the command. Scoped scan and report output
evaluates only the selected declared variants for variant-aware include and
define metadata, records the selected variant list, and does not claim omitted
variants as scanned.
Unsupported target backend names are reported as configuration diagnostics in scan, report, and translation output. Translation still records per-artifact failures for any artifact attempt that cannot be generated.
Validate artifacts referenced by a report:
python -m crosstl validate-project crosstl-out/portability-report.json \
--format textValidation exits nonzero when the report metadata is malformed, artifact
records, source-map records, or preserved diagnostics are malformed, source-map
mapping lists are empty, file-granularity source maps do not contain one
file-level mapping, finer-grained mappings are not positive-length or fall
outside artifact-level file anchors, source-map, diagnostic location, or
diagnostic originalLocation spans are internally inconsistent, diagnostic
location file paths or artifact source paths are not repository-relative, project target
lists are not normalized and deduplicated, diagnostic or artifact targets are
not declared by the report, artifact sources are not declared translation units,
embedded validation records reference artifacts not declared by the report,
embedded toolchain runs reference failed report artifacts,
validation records contain duplicate identities or inconsistent status fields,
full reports with embedded validation artifacts omit validation summaries,
summarized embedded validation omits declared artifacts, embedded toolchain-run
coverage omits OK validation artifacts for available toolchains, failed
embedded toolchain runs omit matching diagnostics, external corpus entry presence,
discovery, or source-backend fields do not match the project root and declared
units, full reports omit units or skipped files that the current
project scan discovers, translated outputs are missing, artifact paths resolve
outside the repository, generated artifact hashes no longer match the files on
disk, source files with recorded hashes are missing or changed, or opt-in
toolchain smoke checks fail. The report file being validated is ignored during
that freshness scan, and files under the configured output directory remain
excluded from discovery.
Toolchain smoke checks only run for translated artifacts that still exist inside
the repository. Each smoke check is bounded by a short subprocess timeout, and
timeouts are reported as failed toolchain runs. Targets that need backend-
specific entry points, profiles, package metadata, or SDK context record bounded
tool availability runs instead of claiming artifact compilation. Validation
reports include
severity, diagnostic-code, and missing-capability rollups for generated and
preserved diagnostics, plus artifact target, artifact source-backend,
artifact variant, hash-status, source-size status, generated-size status,
source-map status, source-remap status, toolchain status, toolchain-run status,
toolchain-run target, toolchain-run source backend, diagnostic check kind,
toolchain-run check kind, toolchain-run tool, and toolchain-run variant rollups
for validation results.
The JSON validation report uses schema version 1 with a fixed top-level field
set so automation can detect contract drift. It includes compact project
context with the project root, output directory, configured targets, source
roots, include/exclude patterns, include directories, selected variants,
define/variant names without exposing raw define values, and the source report
hash used for validation provenance.
The default output is JSON; --format text prints a concise validation
summary with validation report identity metadata, source report hash, project
context, and the same rollups, and --format sarif emits validation
diagnostics as SARIF with project context and source report hash metadata in
invocation properties.
Inspect an existing report as a concise JSON, text, or SARIF summary:
python -m crosstl inspect-report crosstl-out/portability-report.json \
--format text \
--max-diagnostics 20 \
--max-failed-artifacts 20 \
--max-source-map-artifacts 20 \
--max-artifact-matrix-artifacts 20 \
--max-artifact-provenance-artifacts 20 \
--max-define-processing-artifacts 20 \
--max-include-path-processing-artifacts 20 \
--max-include-dependencies 20 \
--max-skipped-sources 20 \
--max-validation-artifacts 20 \
--max-toolchain-runs 20 \
--max-migration-actions 20 \
--max-runtime-references 20 \
--max-external-corpus-entries 20Report inspection includes inspection identity, SARIF invocation metadata, and
source report schema/kind metadata, source report hash metadata,
source report generation metadata,
validation status,
invalid/unavailable report status, project counts, project configuration path,
project root, output directory, configuration counts, normalized source-root,
include-pattern, exclude-pattern, and include-directory lists, runtime-reference
rollups and bounded runtime-reference samples, failed artifacts
with variant labels when present, diagnostic code and missing-capability rollups,
validation diagnostic-code, missing-capability, artifact target,
artifact source-backend, artifact variant, hash-status, source-map status,
source-remap status, toolchain status, and toolchain-run target,
source-backend, check-kind, tool, and variant rollups, report
source-backend, source override mappings, file-extension, and artifact
target rollups, source-map count, granularity, target, and source-backend
rollups, source-remap count, mapping-count, granularity, target, and
source-backend rollups,
artifact matrix completion counts, matrix source provenance, target and
variant completion rollups, sampled missing and extra artifact identities,
bounded validation artifact and validation toolchain-run samples with
truncation counts, failed validation metadata on artifact provenance samples,
include-directory status counts, inactive source-root and include-directory
record details, diagnostics, configurable diagnostic and failed-artifact
truncation counts, external corpus rollups, sampled missing and
present-but-undiscovered external corpus entries with retained provenance
metadata and configurable sample limits, and migration actions.
Inspection sample-limit options accept non-negative integer counts and default
to 20 for each sampled report section.
The JSON inspection report uses schema version 1 with a fixed top-level field
set so automation can detect contract drift while optional report sections
remain present with available: false until their source report data exists.
Migration action inspection is bounded and records truncation counts for large
reports.
--format sarif emits the inspection diagnostics as SARIF for
code-scanning workflows. SARIF invocation properties include the source report
path, source report hash, report identity metadata, project root, output
directory, configured targets, source roots, include/exclude patterns, include
directories, and selected variants. SARIF locations include line and column
metadata and positive-length character spans when diagnostics carry source
offsets.
Build a metadata-only runtime integration plan from a portability report:
python -m crosstl plan-runtime crosstl-out/portability-report.json \
--format text \
--max-runtime-references 20Runtime planning emits a crosstl-runtime-integration-plan JSON document
with source report hash metadata, validation diagnostics from the source
report, project target summaries, runtime-reference rollups and bounded
samples, per-target compiler runtime-plan request commands, and manual actions
for runtime references found in host or build files. The compiler request
entries point at the metadata-only runtime-loader-plan-v1 contract request
in the compiler repository. This is planning evidence only: it does not import
compiler internals, execute device code, or rewrite host application code.
Build a runtime artifact manifest for downstream host or package tooling:
python -m crosstl runtime-manifest crosstl-out/portability-report.json \
--format textRuntime artifact manifests emit a crosstl-runtime-artifact-manifest JSON
document from a validated portability report. The manifest lists translated
artifacts by target with source/backend/variant identity, generated artifact
hash and byte-size metadata, source-map anchors, optional compiler
source-remap sidecars, and the runtime planning contract summary required
by downstream packaging or host integration tooling. Invalid source reports
produce diagnostic-only failed manifests. The manifest is a handoff contract;
it does not generate runtime framework code, execute device code, or rewrite
host application code.
Build a backend-neutral runtime binding manifest for host integrations:
python -m crosstl runtime-binding-manifest crosstl-out/portability-report.json \
--output crosstl-out/runtime-bindings.jsontranslate-project can write the same binding manifest beside the
portability report in one run:
python -m crosstl translate-project /path/to/repo \
--target cgl \
--output-dir crosstl-out \
--report crosstl-out/portability-report.json \
--runtime-binding-manifest crosstl-out/runtime-bindings.jsonRuntime binding manifests emit a crosstl-runtime-binding-manifest JSON
document derived from the validated portability report and runtime artifact
metadata. Each entry is backend-neutral and includes sourceFile,
sourceBackend, targetBackend, artifactPath, entryPoint,
resourceBindings, bufferMutability, scalarConstants,
specializationConstants, dispatchDimensions, sourceProvenance, and
validation. Resource bindings include set/binding coordinates, access, and
derived mutability.
Dispatch dimensions record reflected workgroup size data when available while
leaving workgroup, global, and grid counts unset for host code to provide.
Reflection, runtime artifact manifests, and runtime binding manifests keep
function and specialization constants in dedicated specializationConstants
records with their own counts. They are not reported as resources or
resourceBindings, nor as ordinary constants or scalarConstants.
Build a deterministic runtime handoff package from a runtime artifact manifest:
python -m crosstl package-runtime crosstl-out/runtime-manifest.json \
--package-dir crosstl-runtime-package \
--format textRuntime packages emit a crosstl-runtime-package JSON report and write a
package manifest, translated artifacts, source-remap sidecars, and a short
integration guide into the package directory. Packaging revalidates artifact
hash and byte-size metadata before copying files so stale generated outputs are
reported as structured diagnostics instead of hidden. The package is a handoff
artifact for host or build-system tooling; it does not rewrite host application
code, execute device code, generate runtime framework code, or install target
SDKs.
Inspect a runtime handoff package before host binding:
python -m crosstl inspect-runtime-package \
crosstl-runtime-package/runtime-package.json \
--format textRuntime package inspections emit a crosstl-runtime-package-inspection JSON
document with ready and failed host-binding records. The inspection is read-only
and verifies copied packaged artifacts and source-remap sidecars against the
package manifest's recorded paths, hashes, and byte sizes. Missing, stale, or
malformed package contents are reported as structured diagnostics before host
loader or build-system tooling consumes the handoff package. Inspection
preserves the runtime-loader-plan-v1 summary linkage and does not rewrite
host application code, execute device code, generate runtime framework code, or
install target SDKs.
Build a host binding plan from a runtime package manifest:
python -m crosstl plan-host-bindings \
crosstl-runtime-package/runtime-package.json \
--format textHost binding plans emit a crosstl-runtime-host-binding-plan JSON document
with per-target packaged artifact paths, package-inspection readiness metadata,
bind-runtime-artifact actions for host loader or build-system tooling, and
review-runtime-references actions when the source repository contained
runtime API references. The planner reuses runtime package inspection and only
emits bind actions for ready package records; missing or stale package artifacts
remain diagnostics instead of host-integration work items. The plan preserves the
runtime-loader-plan-v1 summary linkage from earlier reports. It is an action
plan only; it does not rewrite host application code, execute device code,
generate runtime framework code, or install target SDKs.
Build a target-scoped runtime adapter plan from a runtime package manifest:
python -m crosstl plan-runtime-adapters \
crosstl-runtime-package/runtime-package.json \
--format textRuntime adapter plans emit a crosstl-runtime-adapter-plan JSON document
from the same package handoff metadata used by package inspection. The plan
lists ready package bindings by target with adapterKind, artifactFormat,
requiredTools, hostResponsibilities, source-remap handoff paths,
parser-derived hostInterface entry point and resource summaries where the
packaged artifact frontend is available, and wire-runtime-adapter actions
for host loader or build-system tooling. When host interface metadata is
unavailable or not ready, the plan emits resolve-host-interface-metadata
actions so host and build tooling can provide reflection or backend-specific
binding metadata before wiring the adapter. Source targets with registered
frontends can contribute parser-derived interface summaries; formats that need
compiled reflection, such as SPIR-V handoff artifacts without reflected entry
point/resource data, remain explicit follow-up actions. The plan also carries
through package inspection diagnostics and
review-runtime-references actions when the source repository contained
runtime API references. The plan is a target-scoped integration contract; it
does not rewrite host application code, execute device code, generate runtime
framework code, or install target SDKs.
Materialize runtime adapter descriptor files from a runtime package manifest:
python -m crosstl materialize-runtime-adapters \
crosstl-runtime-package/runtime-package.json \
--adapter-dir crosstl-runtime-adapters \
--format textRuntime adapter descriptor packages emit a
crosstl-runtime-adapter-package JSON document and write a deterministic
runtime-adapters.json manifest, an ADAPTERS.md summary, and one
adapters/<target>/*.adapter.json descriptor per ready or blocked runtime
adapter plan record. Each descriptor preserves the packaged artifact path,
target adapter identity, source-remap handoff path, host-interface metadata,
required tools, host responsibilities, and validation readiness for downstream
host loader or build-system tooling. The descriptor package is metadata only:
it does not rewrite host application code, execute device code, generate
runtime framework code, or install target SDKs.
Runtime fixture execution uses a backend-agnostic adapter contract carried on
each RuntimeExecutionRequest as adapter_contract. The contract can be
loaded from a fixture's runtimeAdapter object and merged with manifest
metadata already recorded on a translated artifact. This keeps execution
fixtures stable across downstream runtimes while letting package inspection
provide reflected entry points, resource bindings, and dispatch workgroup
sizes when they are available.
The contract fields are intentionally limited to kernel execution metadata:
| Field | Purpose |
|---|---|
entryPoints |
Names, stages, execution config, optional parameter records, and workgroup-size metadata for callable translated kernels or shaders. |
resourceBindings |
Backend-neutral resource names, kinds, types, set/binding numbers, access modes, and optional fixture value names that an adapter maps to runtime buffers, textures, samplers, or parameter blocks. |
specializationConstants / functionConstants |
Specialization or function constant identifiers, dtypes, values, defaults, and required flags needed before launching the entry point. |
dispatch |
Entry point, workgroup size, workgroup count, global size, or grid size for compute-style launches. Fixture dispatch counts can augment artifact-manifest workgroup sizes. |
validationHooks |
Expected pre-run, runtime, post-run, or comparison checks that a downstream executor should perform or report as skipped/unavailable. |
Runtime planning scopes artifact execution metadata to the selected compiled
entry before merging the fixture's requested adapter contract. When both the
selected entry and requested dispatch provide a concrete workgroup size, the
values must match. A mismatch produces
project.runtime-verification.workgroup-size-mismatch during runtime setup,
records both sizes and the selected-entry provenance, and leaves the test case
unplanned. When only one side provides the size, the planner carries that value
into the merged dispatch, so either missing side of the runtime contract can be
completed from the other. This completion does not hide a disagreement when
both sides are present.
Downstream runtimes implement the RuntimeAdapter protocol or subclass
RuntimeExecutor and receive the merged contract in run(request). For
example, an MLX validation adapter can consume translated Metal artifacts and
map neutral fixture buffers and function constants to MLX runtime objects, but
the fixture contract remains expressed in terms of entry points, bindings,
constants, dispatch geometry, and validation hooks rather than MLX APIs.
Saved project test-runner plans can execute deterministic runtime fixtures when callers supply adapter implementations explicitly:
python -m crosstl execute-test-runner \
crosstl-out/project-test-runner-plan.json \
--runtime-executor native-vulkan=tools.runtime.vulkan:VulkanRuntimeAdapter \
--output crosstl-out/project-test-runner-report.json--runtime-executor is repeatable and uses
EXECUTOR=MODULE:OBJECT. MODULE may be a dotted Python module name or a
.py file path. OBJECT may be an adapter instance, adapter class, or
factory returning an object with run(request) or the parity-adapter methods
prepare_buffers(state), dispatch(state, buffers), and
collect_outputs(state, result).
For the built-in DirectX, OpenGL, and Vulkan native parity adapters, callers can
also pass --native-runtime-adapter TARGET or
--native-runtime-adapter TARGET=MODULE:OBJECT. The optional object is the
backend runtime driver consumed by the native adapter after artifact validation.
Use --no-native-runtime-validation only when the caller has already handled
toolchain validation or is running a controlled test fixture.
CrossTL includes optional DirectXComputeRuntime, OpenGLComputeRuntime,
and VulkanComputeRuntime reference drivers for bounded compute fixtures.
They import target dependencies lazily and report structured unavailability or
setup failures when the required API, loader, device, or resource contract is
not available. Their supported resource shapes are intentionally narrower than
the translated shader languages; a successful translation does not imply that
one of these reference drivers can execute the complete host workload.
A runtime fixture value can include an optional allocation object to keep
native allocation identity separate from the reflected resource name and
binding coordinate. The allocation object has the following fields:
| Field | Purpose |
|---|---|
id |
Required, stable allocation identity. Values attached to separate bindings refer to one allocation only when this value is identical. |
byteOffset |
Byte offset of the typed resource view. The default is 0. |
byteLength |
Bounded view length in bytes. When omitted, runtime planning derives it from the fixture shape and exact scalar layout when possible. |
allocationByteLength |
Total allocation size in bytes. When omitted, runtime planning derives
it from the greatest known end offset among views with the same id. |
The view remains typed by the surrounding fixture value's kind, dtype,
shape, and physical layout metadata. Its access mode and binding coordinates
remain part of the corresponding resourceBindings entry. The allocation
object does not replace those contracts or permit a fixture to reinterpret an
incompatible physical layout.
For example, separate reflected input and output bindings can intentionally refer to the same full allocation while retaining distinct binding coordinates:
{
"inputs": [
{
"name": "source",
"kind": "buffer",
"dtype": "float32",
"shape": [2],
"values": [1.0, 2.0],
"allocation": {
"id": "working-set",
"byteOffset": 0,
"byteLength": 8,
"allocationByteLength": 8
}
}
],
"expectedOutputs": [
{
"name": "destination",
"kind": "buffer",
"dtype": "float32",
"shape": [2],
"values": [2.0, 4.0],
"allocation": {
"id": "working-set",
"byteOffset": 0,
"byteLength": 8,
"allocationByteLength": 8
}
}
],
"runtimeAdapter": {
"resourceBindings": [
{
"name": "source",
"kind": "buffer",
"binding": 0,
"access": "read"
},
{
"name": "destination",
"kind": "buffer",
"binding": 1,
"access": "write"
}
]
}
}Runtime planning preserves the explicit allocation ID on each bound resource and validates the complete group before adapter execution. It rejects malformed ranges, conflicting total sizes, out-of-bounds or misaligned views, explicit input/output views that disagree for one binding, incompatible overlapping physical layouts, and overlapping writable views without a synchronization plan. Validation diagnostics identify the allocation and affected binding coordinates or byte ranges. Driver-specific failures also identify the applicable target constraint. A plan containing these errors is not dispatched.
The DirectX and OpenGL reference drivers group compatible bindings by allocation ID, combine non-conflicting fixture uploads, create one physical device allocation for the group, and bind that allocation at each reflected coordinate. Conflicting upload bytes fail setup instead of causing an implicit conversion or per-binding allocation. DirectX currently requires every structured-buffer view to cover the complete allocation, requires one dtype and stride across the group, and rejects shared constant-buffer allocations. OpenGL supports bounded uniform-block and storage-buffer ranges, subject to the offset-alignment limits reported by the active context. It rejects mixed uniform/storage groups, incompatible overlapping scalar layouts, and overlapping writable ranges.
The built-in Vulkan driver does not currently realize shared allocation IDs or bounded allocation views, and no native shared-allocation support is claimed for Metal, WebGL, WGSL, CUDA, HIP, Mojo, Rust, or Slang targets. Runtime-plan serialization on those targets is not evidence that one physical allocation was reused. Target-specific synchronization, resource-state transitions, allocation lifetime, and framework memory planning remain host-runtime responsibilities.
The allocation field is optional for backward compatibility. When it is
absent, runtime planning assigns a deterministic per-binding allocation ID, so
existing independent bindings remain independent. A single initialized
read_write binding continues to use one allocation for upload and readback.
Aliasing between separate bindings is never inferred from equal values or
similar names; it requires an explicit shared id.
The translator stops at this contract boundary. Full framework rewrites, non-kernel host API ports, application command scheduling, target SDK installation, build-system migration, memory lifetime policy, and production runtime framework generation remain downstream integration work.
The runtime execution graph API represents a bounded multi-operation workload
without embedding target runtime calls in the project report. Use
parse_runtime_execution_graph to load a versioned graph,
validate_runtime_execution_graph to obtain structured diagnostics,
inspect_runtime_graph_package to verify packaged artifact and interface
references, and execute_runtime_graph to run a supported native graph. The
types and functions are available from crosstl.project.
A graph declares resources separately from operations. Resource records carry
their role, kind, exact physical layout, optional allocation view, and bounded
lifetime. Nodes use stable IDs and explicit dependsOn edges and have one of
the following operation records:
| Node kind | Contract |
|---|---|
dispatch |
Selects one translated artifact and entry point, maps named bindings to graph resources with explicit access, and records dispatch geometry and constants. |
copy |
Describes bounded source and destination byte ranges between compatible resources. |
fill |
Describes a bounded byte range and scalar fill value for one resource. |
barrier |
Makes an explicit write-to-read or write-to-write visibility transition for named resources between dependency-ordered operations. |
Dispatch, copy, and fill nodes may also carry bounded repeat or condition records. Validation rejects unbounded controls, dependency cycles, missing resource or artifact references, unsafe access ordering, missing visibility barriers, incompatible layouts or ranges, and temporary-resource lifetimes that do not cover their producers and consumers. Failures are returned as structured diagnostics with graph, node, resource, path, and missing-capability context; unsupported constructs are not silently removed.
Package inspection validates the graph before reading package metadata. For each dispatch it resolves the artifact selector deterministically, requires one ready packaged artifact, checks the requested entry point, and verifies named binding presence, uniqueness, and access compatibility against the reflected host interface. Inspection is read-only and records explicitly that device execution was not performed.
The DirectX and OpenGL reference runtimes currently execute dependency-ordered
dispatch and barrier paths. All requests are validated before device
work begins, resources are allocated once for the graph sequence, and a
payload-free temporary resource can retain one physical allocation from its
producer through its consumer. Intermediate results remain on the device and
output readback is deferred until the sequence completes. The native proofs
execute a two-stage reduction: an input of shape [4] containing
[0, 1, 7, 42] produces two partial sums in a temporary of shape [2],
then an output of shape [1] containing [50]. Direct3D 12 and OpenGL 4.3
both verify the exact result.
copy and fill nodes and bounded control records are part of the
serializable and validated graph model, but the native executor currently
returns explicit unsupported-capability diagnostics for them. Vulkan native
graph execution is not supported, and no native graph execution is claimed for
Metal, WebGL, WGSL, CUDA, HIP, Mojo, Rust, or Slang. The backend-neutral graph
contract can still be parsed and validated independently of those targets.
The pinned MLX revision
4367c73b60541ddd5a266ce4644fd93d20223b6e is a corpus and CI reference for
the runtime-porting work, not a special case in the graph schema or executor.
This capability does not rewrite host applications or runtime frameworks and
does not claim that the complete MLX runtime or upstream test suite has been
ported.
Curated repository fixture metadata can be converted into a standard
crosstl-project-runtime-test-manifest document without adding a native
runtime adapter. This lets project ports describe parity cases as deterministic
inputs, expected outputs, tolerances, artifact selectors, runtime adapter
contracts, resource bindings, function or specialization constants, and dispatch
geometry, then reuse the existing runtime test manifest parser, planner, and
report writer.
Build a project runtime test manifest from a translated artifact report or runtime artifact manifest plus curated fixture metadata:
python -m crosstl runtime-test-manifest \
crosstl-out/runtime-manifest.json \
fixtures/runtime-fixtures.json \
--format textThe fixture metadata convention is a small repository-agnostic input document:
{
"kind": "crosstl-project-runtime-fixture-metadata",
"fixtures": [
{
"id": "reduced-binary-add-f32",
"selector": {
"source": "mlx/backend/metal/kernels/binary.metal",
"target": "metal",
"path": "out/metal/reduced_binary_add.metal"
},
"inputs": [{"name": "lhs", "values": [1.0, 2.0]}],
"expectedOutputs": [{"name": "out", "values": [3.0, 4.0]}],
"runtimeAdapter": {
"entryPoints": [{"name": "binary_add", "stage": "compute"}],
"resourceBindings": [
{"name": "lhs", "kind": "buffer", "binding": 0, "value": "lhs"},
{"name": "out", "kind": "buffer", "binding": 1, "value": "out"}
],
"functionConstants": [
{"name": "element_count", "id": 0, "value": 2}
],
"dispatch": {"entryPoint": "binary_add", "globalSize": [2, 1, 1]}
}
}
]
}The generator validates each fixture selector against the translated artifacts.
Incomplete fixture data, duplicate ids, missing expected outputs, unresolved
artifacts, and ambiguous selectors are emitted as structured diagnostics on the
generated manifest. Valid fixture records remain in the manifest so
plan_runtime_test_manifest and verify_runtime_test_manifest can apply
the same adapter dependency checks and runtime planning used by hand-authored
manifests.
Generated test records also include metadata.runtimeMetadata. When the
selected artifact carries runtimeDataStatus from a runtime artifact
manifest, that status is preserved; otherwise the generator derives readiness
from the merged runtime adapter contract. The manifest summary includes
runtimeMetadataStatusCounts so downstream tooling can separate incomplete
fixture data from incomplete artifact metadata before attempting native runtime
execution.
Build a runtime loader manifest from a runtime package manifest:
python -m crosstl runtime-loader-manifest \
crosstl-runtime-package/runtime-package.json \
--format textRuntime loader manifests emit a crosstl-runtime-loader-manifest JSON
document derived from runtime adapter planning. The manifest groups per-target
load units with package-relative artifact paths, adapter kind, artifact format,
source-remap handoff paths, parser-derived hostInterface metadata when
available, required target tools, host responsibilities, ordered loader steps,
and blockers that must be resolved before a host loader or build-system adapter
can consume the artifact safely. Unavailable interface reflection remains an
explicit resolve-host-interface-metadata blocker instead of being hidden or
treated as generated host code. The manifest carries package inspection
diagnostics and runtime-reference review actions forward, and it remains a
metadata contract only: it does not rewrite host application code, execute
device code, generate runtime framework code, or install target SDKs.
Source reflection records a scalarLayout only when the target-language
resource has an exact physical representation covered by the project runtime
contract. HLSL reflection supports StructuredBuffer and
RWStructuredBuffer resources whose element type is a scalar or up-to-four
component vector of float, int, or uint, plus scalar int64_t
and uint64_t elements. Single-member cbuffer declarations support the
same types. GLSL reflection supports explicit std430 buffer blocks
containing one scalar runtime-array member and explicit std140 uniform
blocks containing one scalar member of type float, int, uint,
int64_t, or uint64_t.
The reflected layout records physicalType, elementType,
elementSizeBytes, elementStrideBytes, alignmentBytes,
memberOffsetBytes, storageLayout, and runtimeSized. It also records
memberName when the source block provides one and blockSizeBytes for a
fixed scalar block. These fields are preserved through runtime packages and
native loader ABI descriptors. Descriptor-to-request validation rejects
missing, incomplete, or mismatched layouts instead of inferring a host ABI.
Native runtime allocation consumes the same physical contract. DirectX
constant-buffer views require a fixed HLSL scalar block and allocate at least
the reflected block size, rounded to the 256-byte API alignment. OpenGL
uniform buffers require a fixed std140 scalar block and zero-pad the upload
to blockSizeBytes; std430 scalar runtime arrays retain their logical
payload size for storage-buffer readback.
Standard GLSL vec, ivec, uvec, and bvec block members now
receive exact component widths, vector widths, and std140/std430
alignment metadata; i64vec and u64vec use the supported 64-bit physical
tables. Runtime vec2 and vec4 storage arrays remain tightly packed,
while vec3 records its logical element size and padded array stride
separately. The current native loader rejects padded storage vectors rather
than uploading a falsely tight layout. GLSL dvec values, HLSL 64-bit
vectors, matrices, fixed arrays, aggregates, unsupported narrow or
floating-point scalar widths, implicit GLSL block layouts, arbitrary member
offsets, and multi-member blocks do not receive usable loader metadata. Those
shapes remain unresolved or fail closed when a native loader request requires
a physical layout. Native requests range-check signed and unsigned 64-bit
values and preserve them with little-endian 8-byte packing; 64-bit
specialization constants remain intentionally unsupported.
At pinned MLX commit 4367c73b60541ddd5a266ce4644fd93d20223b6e, the
arangeuint32 entry from arange.metal is translated to DirectX and
OpenGL, reflected, packaged, converted through the public native loader bridge,
and executed in Windows Direct3D and Linux EGL CI. With start = 3,
step = 2, and four invocations, both readbacks are exactly
[3, 5, 7, 9]. This is an end-to-end proof for one scalar kernel contract;
it is not a claim of vector or aggregate layout support, full MLX runtime
integration, or MLX test-suite parity.
At current pinned MLX commit
846d176227a0ac13d2667e58d2bb68b322109ab0, the selected
fft_mem_256_float2_float2 entry also passes an entry-scoped OpenGL proof.
Five unsigned index assertions and one 256-element workgroup-access assertion
bound its host contract. Translation materializes 37 specializations from 42
reachable records, prunes 2,120 candidates, and preserves 21 reachable
function constants for deferred specialization. The deterministic GLSL is
82,045 bytes with SHA-256
a1ab0c346d9143e6749e391fb971aeaed71bd84e15fedaf7a7e92808a56449bb;
glslangValidator and spirv-val accept it, and its SPIR-V has 19 control
barriers with no group-nonuniform instruction.
The reflected ABI has two std430 float32 vec2 arrays at 8-byte size,
stride, and alignment plus two 16-byte std140 integer blocks. The runtime
variant registry has no blocked keys and produces a verified deferred SPIR-V
request for workgroup size [1, 1, 64]. Linux Mesa llvmpipe executes one
workgroup for an index-1 complex unit impulse and compares all 256 complex
outputs with the analytical forward DFT at 2e-4 absolute and relative
tolerance. The measured maximum absolute error is
9.264554161336758e-08. This is one selected current-pinned workload; it does
not redirect the MLX host runtime, cover other FFT plans or dtypes, prove a
Metal round trip, or establish full backend parity.
At the same current pin, a bounded GEMV proof selects
gemv_t_float32_bm1_bn2_sm8_sn4_tm4_tn4_nc0_axpby0 for one contiguous
float32 vector-matrix product with M=1, N=32, and K=32. The
host-derived contract fixes workgroup [32, 2, 1], subgroup width 32, and
one dispatched workgroup. Entry-scoped translation materializes the selected
GEMV and elem_to_loc_uint only. Its 8,188-byte HLSL has SHA-256
f300bbea75b2ed9e47c29313a56f882ed848cbb93858f1347fbc97a60e167223
and passes official DXC 1.9.2602.24 under cs_6_6,
-enable-16bit-types, and warnings as errors. DirectX explicitly enables a
32-lane target-scoped software subgroup because physical waves need not contain
contiguous flattened SV_GroupIndex values. Logical subgroup and lane IDs
are SV_GroupIndex / 32 and SV_GroupIndex % 32. A 64-float
groupshared array carries each shuffle between two
GroupMemoryBarrierWithGroupSync calls; the source lane is validated before
addition so an extreme unsigned delta cannot wrap the scratch index, and an
out-of-range source returns the calling invocation's value. The HLSL contains
no WaveReadLaneAt, WaveGetLaneIndex, or physical-wave atomic allocator.
[WaveSize(32)] remains the source/reflection contract without making the
reduction depend on physical lane topology.
The superseded 8,410-byte physical-wave artifact remains recorded as rejected
evidence under SHA-256
f8f1107d0de251fd300c7a16ce6638796bd08dd2eadd8f7959e37c78d0aa170d.
Windows workflow run 33268998061, job 99143984804 mismatched every output: the
reduction replaced logical lanes 5 through 8 with physical lanes 21 through
24, reaching maximum absolute error 1.90625. This exact substitution rules out
a tolerance adjustment or merely guarding invalid high-lane reads.
OpenGL uses two logical 32-lane software subgroups in the 64-thread workgroup.
Both target-specific fail-closed analyses admit the source's
sm >= 1; sm >>= 1 loop as the integral-equivalent positive-to-zero form of
sm > 0. DirectX also requires one bounded compute entry, concrete
width-compatible dimensions, explicit calling-invocation fallback, a supported
scalar shuffle, unambiguous helper identity, logical invocation identity, and
statically uniform control flow before artifact emission. OpenGL retains
rejection for wider bounds, mutation, nontermination, escaping control flow,
and indirect or nested helper calls. The 7,705-byte GLSL has SHA-256
f5ef8900ee65d63a6df2818ef111f56b4f269c6366c82d82a9d97c967042f562;
glslangValidator and spirv-val accept it, and its SPIR-V contains three
control barriers with no group-nonuniform instruction.
The reflected DirectX and OpenGL ABIs each contain 15 resources, including
signed 64-bit batch strides and seven scalar argument blocks. A deterministic
binary-fraction workload compares all 32 output columns at 1e-5 absolute
and relative tolerance. Linux Mesa llvmpipe executes the software-subgroup
artifact in required mode, and Windows CI requires the same workload through
Direct3D 12 WARP. This proof covers one host-valid current-corpus entry; gather,
wide, batched, axpby, and remaining GEMV entries, MLX host redirection, selected
Metal compilation, and the full MLX suite remain outside the claim. It does
not change the separate historical 224-entry aggregate compiler gates.
The same current pin has a bounded MXFP4 quantize/dequantize contract for
mxfp4_quantize_dequantize_float_gs_32_b_4_hgs_false. Host provenance from
quantized.cpp and fp_quantized.h fixes float32 input, group size 32,
four payload bits, no global scale, workgroup [32, 1, 1], and one dispatched
workgroup. Entry-scoped translation materializes only that specialization. The
9,123-byte HLSL has SHA-256
3fe38e171ba8c8ea1adfc8efad20b242ca02dd05e1a5a53a9b9d1e18459d8c7d and
passes DXC under cs_6_6, -enable-16bit-types, and warnings as errors.
Its 4,716-byte DXIL uses the explicit DirectX-only
project.source_options.metal.target_options.directx.widen_native_float16
mode. Source as_type<float16_t>(uint16_t) reconstructs its payload directly
as float32 with integer IEEE-754 masks, while logical float16_t locals,
parameters, and returns stay widened through arithmetic, sign application, and
the consuming conversion. DXIL contains uitofp i32 and fmul float but
no LegacyF16ToF32, half, fptrunc, or fpext. The default native
binary16 contract remains exact asfloat16/asint16/asuint16 and is
unchanged when this target-scoped option is absent.
Four Windows failures bound this contract. The 7,809-byte HLSL under SHA-256
3591e38d20a612b4061fe3154ef0ea3deb035283294fbd27376ef90627569361
produced numeric uitofp i16 ... to half; workflow run 33271117475, job
99149649480 collapsed all 28 nonzero values to signed zero on WARP. Exact
bitcast i16 ... to half produced same-size HLSL under SHA-256
4e8044758d65b6b2c189092ce56fff3c5ba7948221883de490c1a4b9c5563352,
but run 33272842347, job 99154326814 consumed the constructed subnormal in
fmul half and produced the same result. Moving arithmetic to float32
produced 7,909-byte HLSL under SHA-256
938ca6fac1c47ea633453836b5d76833c294853bb92d6a410a2c4772dd7fa627;
dx.op.legacyF16ToF32 still failed in run 33274360343, job 99158370210.
Integer decoding produced 9,240-byte HLSL under SHA-256
936088a24a6b575e50dc97e16a4c0dca63a76200ddd94d5211e4bf312fec1625,
but the remaining fptrunc float to half, half sign operation, and
fpext half to float returned the identical 28 signed zeros in run
33275550062, job 99161501105. Removing every half instruction produced a
9,118-byte artifact under SHA-256
7afdc612f9091ae47abca8c4fd9d2171e8ea42c6539e02a40bbad2de7d1a7c6a,
but run 33277494856, job 99166677942 still returned the same signed zeros.
DXIL then exposed the actual remaining defect: absent Metal/C++ scalar integer
promotion reduced uint16_t(bits) << 23 to a 16-bit shift by 7 plus an
0xffff mask. The corrected HLSL emits int(uint16_t(bits)) << 23 and
DXIL shifts the 32-bit value by 23 before asfloat, with no native-half
operation anywhere in the selected path.
The 9,571-byte explicit-software-subgroup GLSL has SHA-256
cbbe989c40317c04ffe915f1f314f55db8896edfd38f04ad4b8882be53b2a4da;
glslangValidator and spirv-val accept its three-barrier SPIR-V, which
contains no group-nonuniform instruction. GLSL widens binary16 values to
float32, so the same source bitcast preserves the exact low 16-bit payload
through unpackHalf2x16 rather than a float32 bitcast; inverse forms use
packHalf2x16 with exact low-bit extraction.
Exact scale semantics require the source fp8_e8m0(float) constructor
factory before conversion back to float. Qualified metal::round maps to the
portable math intrinsic, while OpenGL metal::isfinite and signbit use
single-evaluation IEEE-754 bit tests. Read-only private scalar-to-struct views
are accepted only for exact one-member layouts. Unsupported predicate types,
receiver mutation, writes, and unresolved constructor branches remain
fail-closed.
The reflected data ABI is float32 input at binding 0, an inert declared
global_scale input at binding 1, and float32 read-write output at binding 2.
DirectX also uses b0 for generated dispatch input, legally distinct from
t0 in the HLSL register namespaces. The real host omits global_scale
for this specialization; the generic loader allocates it because reflection
retains the declaration, and the selected code contains no read. A 32-element
workload uses only exactly representable FP4 E2M1 values with maximum magnitude
6. The scale divisor is 6 and the MX scale is exactly 1, so Windows WARP and
Linux Mesa required-mode tests demand bit-exact float32 readback at zero
tolerance. Other quantized entries and parameter families, MLX host redirection,
selected-entry Metal compilation, and full-suite parity remain outside this
bounded proof.
At current pinned MLX commit
846d176227a0ac13d2667e58d2bb68b322109ab0, a bounded arg-reduce proof
selects argmin_float32 and argmax_float32 for two axis-32 rows. The
checked host dispatch formula produces workgroups [32, 1, 1] and dispatch
[1, 2, 1] with subgroup width 32. Signature-aware source instantiation
materializes the scalar elem_to_loc<int64_t> helper rather than its
uint3 overload. DirectX generation explicitly sets
project.source_options.metal.target_options.directx.relative_wave_shuffle_out_of_range
to "self" for these artifacts. Relative source lanes outside the wave then
retain the calling lane's value; generated helpers select a proven in-range
source before an unconditional WaveReadLaneAt. This deterministically
refines source undefined behavior without changing in-range shuffle semantics.
The default policy remains "undefined" so unrelated project artifacts are
unchanged. The generated HLSL artifacts pass official DXC 1.9.2602.24 under
cs_6_6 with -enable-16bit-types and warnings as errors. OpenGL uses the
explicit software subgroup and admits direct shuffle-helper calls only inside
a proven canonical workgroup-uniform halving loop. Both GLSL modules pass
glslangValidator and spirv-val, contain five control barriers, and
contain no group-nonuniform SPIR-V instruction.
The reflected runtime contract includes exact signed and unsigned 64-bit
scalar resources: float32 input, uint32 output, int32 shape, int64 stride, and
uint64 size buffers or scalar blocks. Linux arm64 Mesa EGL executes rows with
tied extrema and reads back argmin indices [5, 7] and argmax indices
[3, 2], proving lowest-index tie behavior; Windows CI requires the same
workloads through Direct3D 12 WARP. This bounded proof does not unblock the
24-entry aggregate DirectX/OpenGL artifact, which remains fail-closed with
project.translate.workgroup-size-entry-ambiguous. Other axis sizes,
dtypes, and entries, MLX host redirection, and the full MLX test suite remain
outside the claim. Entry-scoped Metal also remains unavailable because the
per-entry workgroup specialization rule fails explicitly with
project.translate.workgroup-size-rule-unsupported-target; this proof does
not claim a Metal round trip.
The same current pin now has a bounded one-pass scaled-attention proof for
sdpa_vector_float_64_64. The checked host contract selects batch/head/query
counts of one, key length 4, dimensions D=64 and V=64, scale 0.125,
and no mask, causal mode, or sinks. It fixes [1024, 1, 1] with 32 logical
subgroups and one dispatched workgroup. Function constants 20 through 25 are
all false; two-pass-only ID 26 is not part of this artifact. The exact HLSL is
8,721 bytes with SHA-256
003c8b9e85bad7363bae2e3d80380d979cbe0b8988d0d98751131c3acfbff6b6
and passes DXC 1.9.2602.24 under cs_6_6, -enable-16bit-types, and
warnings as errors. Its 32 physical waves receive unique, wave-uniform IDs
through the synchronized allocator; no lane-varying flattened-index quotient
remains.
OpenGL uses an explicit 32-lane software artifact across the full 1,024-thread
workgroup. The subgroup-ID-strided runtime loop is synchronized round by round;
inactive subgroups supply typed reduction identities so all barriers remain
uniform. Its 12,089-byte GLSL has SHA-256
9b7cb7dc9a76b9fb93c30fd93d13ad639f5493f60fd97b965514db0fe6b4840b.
The validated SPIR-V has nine control barriers, six false specialization
constants, local size 1024 1 1, and no group-nonuniform instruction.
The native-loader ABIs contain 19 DirectX and 18 OpenGL resources. Deferred
optional resources receive placeholders, including uint32 physical storage for
bmask. DirectX concretizes the six constants and executes through WARP;
OpenGL builds a verified deferred compilation request, specializes the six
constants, and executes through Mesa surfaceless EGL. Both compare all 64
outputs with a stable CPU reference at 2e-4 absolute and relative
tolerance. The local Mesa proof measured maximum absolute error
4.082320426146424e-08 and maximum relative error
4.2163276126605175e-06. Masked, causal, sinks, two-pass and full-attention
paths, other dimensions and dtypes, MLX host redirection, selected Metal
round-trip validation, and the full MLX suite remain outside this bounded
claim. The historical 42-entry aggregate DirectX/OpenGL run remains fail-closed
because it does not consume this entry-scoped contract; it is not evidence that
a bounded attention runtime proof is absent.
At current pinned MLX commit
846d176227a0ac13d2667e58d2bb68b322109ab0, a bounded LayerNorm VJP proof
selects vjp_layer_normfloat32 for one axis-32 row with function constant
has_w=true, workgroup and subgroup width 32, and eight reflected resources.
DirectX materializes the constant and executes the generated HLSL through
Direct3D 12 WARP. OpenGL retains constant ID 20 in GLSL, lowers the reductions
to an explicit software subgroup, derives a verified deferred-compilation
request, compiles to SPIR-V, specializes has_w=true, and executes through
Mesa EGL. Both paths compare all 32 input-gradient and 32 per-row
weight-gradient values. The one-row boundary means the per-row weight-gradient
temporary is also the final host-reduced result. This proof excludes the
separate bias-gradient dispatch, multi-row weight reduction, other dtypes and
axis sizes, MLX host redirection, and the full MLX test suite.
The same current pin also has a bounded RMSNorm VJP proof for
vjp_rmsfloat32 with one axis-32 row, has_w=true, one 32-thread
workgroup, and ten reflected resources. DirectX concretizes the function
constant and executes HLSL through WARP. OpenGL retains constant ID 20, lowers
four reductions through the explicit software subgroup, and admits the
kernel's runtime row loop only after proving its initializer and bound
workgroup-uniform. The deferred path compiles GLSL to SPIR-V, specializes the
constant, verifies the interface, and executes through Mesa EGL. Both targets
compare all 32 input-gradient and 32 one-group weight-gradient values. The
one-row/one-group boundary makes the group-local weight gradient equal to the
final host reduction; multi-row reduction, other dtypes and axes, MLX host
redirection, and the full MLX test suite remain outside this proof.
The current pin additionally has a bounded Softmax proof for
block_softmax_float32. Its checked dispatch contract applies the pinned
host formula 32 * ceilDiv(ceilDiv(axisSize, 4), 32) to two rows of axis 32
and one row of axis 2049, producing workgroups [32, 1, 1] and
[544, 1, 1]. Both DirectX artifacts retain WaveSize(32). OpenGL keeps
the default guarded hardware-subgroup artifacts and separately packages
explicit 32-lane software-subgroup artifacts; the wide artifact partitions 544
invocations into 17 logical subgroups and uses typed inactive-lane identities
for masked maximum and sum reductions. Official DXC accepts both HLSL artifacts
under cs_6_6 with -enable-16bit-types and warnings as errors. The
32-thread entry is provably one wave and retains the zero-ID quotient fast path;
the 544-thread entry allocates one uniform ID for each physical wave through a
workgroup-synchronized counter. Both software modules pass
glslangValidator and spirv-val, contain 11 control barriers, and contain
no group-nonuniform SPIR-V instruction.
Windows CI requires both workloads to execute through Direct3D 12 WARP, and
Linux CI requires both to execute through surfaceless Mesa EGL. Every output is
compared with a stable float32 CPU Softmax reference at 5e-5 absolute and
relative tolerance. This evidence covers only the two finite float32 block
workloads: axis sizes above 4096, half and bfloat16 entries, MLX host
redirection, and the full MLX test suite remain outside the claim. Selected
entry-scoped Metal generation also remains unavailable and fails explicitly
with project.translate.entry-point-target-unsupported; this bounded proof
does not claim a Metal round trip.
Build a versioned native loader ABI descriptor and optional C declarations for one ready load unit:
python -m crosstl native-loader-abi \
crosstl-runtime-package/runtime-loader-manifest.json \
--load-unit copy:directx \
--output copy.directx.abi.json \
--declarations-output copy.directx.abi.h \
--execution-output copy.directx.native-loader-execution.h \
--target-adapter-output copy.directx.native-loader-adapter.hppnative-loader-abi selects exactly one ready load unit. --load-unit is
optional only when the input manifest contains one unit. The command emits a
schema-v1 crosstl-native-loader-abi-descriptor JSON document containing
the target entry point, artifact identity and hash, source identity and remap,
binding namespaces and coordinates, access modes, scalar layout,
specialization constants, and provenance. A blocked unit, incomplete host
interface, malformed artifact identity, ambiguous selection, or invalid
binding coordinate produces a structured diagnostic instead of declarations.
--declarations-output writes deterministic C declarations for the same
descriptor. The header contains a guarded, versioned ABI type contract and
immutable unit and binding descriptor data. The declarations compile as C11
and C++17 and can be included for more than one load unit without redefining
the shared ABI types. They describe what downstream host integration must
load and bind; they do not execute the unit.
--execution-output writes a deterministic, allocation-free C11/C++17
execution wrapper for the selected unit. The header includes the declaration
contract and adds request, adapter-callback, result, and structured-error
types plus a unit-specific *_execute function. Before consulting the
adapter, that function validates the ABI version, target, exact binding
identity and coordinates, access modes, specialization identities and types,
payload presence, and dispatch geometry.
After validation, the wrapper loads the artifact, applies specialization values to that artifact, creates the pipeline, binds resources in descriptor order, dispatches, synchronizes, and reads back writable resources. It then releases resources in reverse binding order, destroys the pipeline, and unloads the artifact on both success and failure paths. The wrapper uses fixed-size stack storage derived from the descriptor and performs no heap allocation.
The caller owns the execution request, binding and specialization payloads,
readback destinations, and adapter context for the duration of the call. The
adapter owns each native artifact, pipeline, and resource handle that its
callbacks return; the wrapper passes those handles to the corresponding
unload, destroy, and release callbacks. A nonzero callback return value is
preserved as adapter_status with the failing phase and binding or
specialization index. error records the primary execution failure, while
cleanup_error records the first cleanup failure without replacing an
earlier primary failure. A cleanup failure becomes the primary error only
when all preceding execution phases succeeded.
--target-adapter-output writes the deterministic C++17 reference adapter
for the selected target. Include the unit execution header before this adapter
header. The adapter fills the shared callback table rather than replacing the
unit-specific validation wrapper, so request validation, execution order,
structured errors, and cleanup behavior remain defined by the common ABI.
Direct3D 12 and OpenGL compute have reference adapters. A target without a
reference adapter produces a structured
project.native-loader-target-adapter.target-unsupported diagnostic.
Build descriptors and declarations for every ready unit in one operation:
python -m crosstl native-loader-abi-package \
crosstl-runtime-package/runtime-loader-manifest.json \
crosstl-native-loader-abinative-loader-abi-package validates every unit and generates every
available target adapter before writing output. It emits target-scoped
descriptor, declaration, and execution header files, one adapter header per
supported target, plus native-loader-abi-package.json and prints that
package manifest as JSON.
Each packaged unit records executionABIPath and executionABIHash;
the generated-file inventory identifies execution headers as
native-loader-execution-abi and reference adapters as
native-loader-target-adapter. The schema-v3 targetAdapters array records
the target, availability, generated path, and SHA-256 hash. Targets without a
reference implementation remain in that array with
reason: target-adapter-unavailable rather than being silently omitted.
Schema-v3 packages also publish runtime/runtime-variant-registry.json and
record its file hash, registry identity, and exact variant count in
runtimeVariantRegistry. When that registry is ready, the package verifies
and copies every referenced translated artifact, recording each as
runtime-target-artifact; descriptor byte sizes and hashes remain available
for independent verification. A metadata-only loader manifest that lacks the
provenance required for exact lookup remains valid, but marks the registry and
native header unavailable instead of claiming dispatch readiness.
When every registry target has a reference adapter, the package also emits
native-runtime-variant-registry.hpp. This deterministic C++17 header maps
canonical runtime variant keys to unit execution wrappers and retains the exact
target, entry point, workgroup size, subgroup width, and specialization
payloads selected during packaging. Packages containing targets without a
reference adapter keep the JSON registry but mark the native header unavailable.
A ready OpenGL GLSL-source variant that still carries specialization constants
also keeps its JSON registry ready, copied artifact, and exact lookup key, but
marks the native header unavailable with reason
specialization-requires-deferred-compilation. The C++ header cannot express
that compile-then-specialize step; callers instead derive and execute the
verified deferred SPIR-V compilation request. Other registry-generation errors
remain fatal and are not converted into this fallback.
The same operations are available through the public project API:
from crosstl.project import (
NATIVE_LOADER_ABI_PACKAGE_KIND,
NATIVE_LOADER_ABI_PACKAGE_MANIFEST,
NATIVE_LOADER_ABI_PACKAGE_VERSION,
NATIVE_RUNTIME_VARIANT_REGISTRY_HEADER_PATH,
NATIVE_RUNTIME_VARIANT_REGISTRY_PATH,
NATIVE_LOADER_TARGET_ADAPTER_KIND,
NATIVE_LOADER_TARGET_ADAPTER_VERSION,
NativeRuntimeVariantRegistryError,
build_native_loader_abi_descriptor,
build_native_loader_abi_package,
build_runtime_variant_dispatch_request,
generate_native_loader_declarations,
generate_native_loader_execution_abi,
generate_native_loader_target_adapter,
generate_native_runtime_variant_registry,
native_loader_target_adapter_targets,
)
descriptor = build_native_loader_abi_descriptor(
loader_manifest,
load_unit_id="copy:directx",
)
declarations = generate_native_loader_declarations(descriptor)
execution_abi = generate_native_loader_execution_abi(descriptor)
target_adapter = generate_native_loader_target_adapter(
descriptor["target"]
)
package = build_native_loader_abi_package(
"crosstl-runtime-package/runtime-loader-manifest.json",
"crosstl-native-loader-abi",
)build_native_loader_abi_descriptor validates and normalizes one loader
unit before returning deterministic JSON-compatible metadata.
generate_native_loader_declarations validates that descriptor before
rendering the C representation. generate_native_loader_execution_abi
validates the same descriptor before rendering the executable callback
wrapper. generate_native_loader_target_adapter renders the target callback
implementation for a supported canonical target, while
native_loader_target_adapter_targets reports the available target names.
build_native_loader_abi_package validates all load units and adapter output
before writing, then emits one descriptor, declaration header, and execution
header per unit, one reference adapter per supported target, and a
deterministic package manifest. The manifest records content hashes, source
loader-manifest identity, generated paths, target, adapter, and unit counts,
and uses the exported
NATIVE_LOADER_ABI_PACKAGE_KIND, NATIVE_LOADER_ABI_PACKAGE_VERSION, and
NATIVE_LOADER_ABI_PACKAGE_MANIFEST constants for its kind, schema version,
and file name. A blocked, incomplete, or duplicate unit prevents package
publication rather than producing a partially described package.
The generated Direct3D adapter owns its device, queue, fence, pipelines, and resource allocations within an explicit caller-created context. The generated OpenGL adapter consumes a caller-owned current desktop context and an explicit function table; it does not create a window-system context or choose EGL, GLX, WGL, or another loader. Both adapters reject unsupported artifact, resource, layout, and capability contracts through nonzero adapter statuses. They are reference implementations for one unit execution lifecycle, not repository schedulers or host-runtime rewrites. Host application rewriting, primitive selection, graph policy, full MLX runtime integration, and MLX test-suite parity are not provided or claimed.
The Direct3D 12 adapter accepts packaged HLSL compute source and DXIL
containers. HLSL source compilation uses the DXC API with shader model 6.2 and
native 16-bit types enabled. Hosts that compile HLSL source need the official
dxcapi.h header; dxcompiler.dll and dxil.dll must be discoverable
at execution time. DXIL-only hosts do not need the DXC API, but cannot apply
source specializations. Generated HLSL source specialization requires both the
reflected constant name and numeric ID so the adapter can replace the exact
CrossTL fallback declaration before passing a numeric definition to DXC.
Structured-buffer SRV and UAV bindings and constant-buffer CBV bindings are
supported when the descriptor includes a compatible scalar layout. Other
resource kinds fail closed.
The OpenGL adapter accepts GLSL compute source with entry point main and
OpenGL SPIR-V compute binaries. GLSL source can run on a current OpenGL 4.3
desktop context but does not support specialization. SPIR-V specialization
requires OpenGL 4.6 or caller-confirmed GL_ARB_gl_spirv support and the
matching glShaderBinary and glSpecializeShader entry points. The
adapter supports set-zero shader-storage and uniform-buffer bindings; nonzero
sets and texture, image, sampler, scalar-uniform, and shared-allocation
contracts fail closed.
Native CI translates reduced source fixtures through CrossTL, verifies packaged artifact hashes, binds distinct input and output buffers, applies one specialization, dispatches on Direct3D 12 and surfaceless OpenGL, and compares deterministic readback with the expected values. These checks prove the generated adapter lifecycle for the reduced contracts. They do not establish semantic parity for every translated kernel or execute an upstream repository's complete test suite.
For a complete DirectX or OpenGL compute descriptor, the public project API can construct and preflight the backend-neutral runtime request consumed by the native parity adapters:
import json
from pathlib import Path
from crosstl.project import build_native_loader_dispatch_request
package_root = Path("crosstl-native-loader-abi")
descriptor = json.loads(
(package_root / "descriptors/directx/copy.abi.json").read_text()
)
request = build_native_loader_dispatch_request(
descriptor,
package_root,
input_values={
"input_values": {
"dtype": "float32",
"shape": [4],
"values": [1.0, 2.0, 3.0, 4.0],
},
# The descriptor reflects this binding as read_write.
"output_values": {
"dtype": "float32",
"shape": [4],
"values": [1.0, 2.0, 3.0, 4.0],
},
},
output_values={
"output_values": {
"dtype": "float32",
"shape": [4],
"values": [2.0, 4.0, 6.0, 8.0],
}
},
dispatch_geometry={"workgroupCount": [1, 1, 1]},
specialization_values={3: 4},
expected_target="directx",
)build_native_loader_dispatch_request supports compute-stage DirectX HLSL
and OpenGL GLSL artifacts. It validates the descriptor, requires exact
reflected binding names, verifies the artifact size and SHA-256 digest inside
the package root, validates specialization values and dispatch geometry, and
returns a preflighted RuntimeExecutionRequest. Buffer bindings require a
complete, tightly packed 32-bit scalar layout; missing or ambiguous physical
layout metadata is a structured error rather than an inferred ABI.
The returned request also carries a frozen RuntimeArtifactIdentity copied
from the validated descriptor's byte size, SHA-256 digest, and unit ID. Native
execution uses this pinned record instead of rereading identity fields from the
request's public artifact mapping. Later mutation of that mapping therefore
cannot replace the expected identity used at execution. The record pins
identity metadata only; it does not freeze other artifact metadata or the file
at the artifact path.
An exact binding name may appear in both input_values and output_values
only when the descriptor reflects that resource as read_write. In the
example above, the input payload initializes one native allocation and the
expected output contract marks that same allocation for readback and
comparison. Request construction requires the two roles to agree on dtype and
shape and validates both against the complete reflected physical scalar layout.
An overlap for another access mode, or an incompatible contract, produces a
structured diagnostic.
This API prepares one request and its native resource contract. Repository
scheduling, host application rewriting, and full MLX test-suite parity remain
outside its scope. Generated descriptors for the exact HLSL and GLSL scalar
forms described above carry the complete physical layout into request
construction; unsupported resource shapes remain rejected rather than
inferred. The pinned MLX arangeuint32 proof establishes native DirectX and
OpenGL execution for that one contract, not full MLX runtime integration or
numerical parity across the MLX test suite.
Build a deterministic runtime variant registry from either a ready runtime package or loader manifest:
crosstl runtime-variant-registry \
crosstl-runtime-package/runtime-loader-manifest.json \
--output runtime-variant-registry.jsonRuntime variant registries emit a schema-v1
crosstl-runtime-variant-registry JSON document whose runtime variant key
schema is version 2. Each variants entry is indexed by a canonical
crosstl-rvk2: key: URL-safe base64 without padding over canonical JSON
containing the source unit and source entry, target and target profile, the
selected binding-interface entry point's execution identity, type and value
template arguments, specialization constant IDs and values, and defines. The
execution identity contains workgroupSize and subgroupWidth; each
field remains null when the selected entry does not provide an exact value.
Unselected entry points and project-level aggregate metadata do not affect the
key. Key fields are sorted before encoding, registry records and target
summaries are ordered by key, and registryHash covers the key schema and
records. Equivalent input records therefore produce the same registry
regardless of package or loader record order.
Each registry record preserves source and target names separately and maps the
exact key to the target artifact path, format, hash and byte size, target entry
point, binding resources and ordinary constants, pipeline specialization
constants, and translation and source provenance. Inputs use closed package
and loader field sets. Malformed schemas fail before records are emitted;
duplicate keys and keys with conflicting artifacts or metadata are diagnosed
and rejected. Package inspection hash or size failures remain explicit
stale records, and loader blockers remain blocked records. Both are
listed as available exact keys but have lookup.eligible set to false.
The public build_runtime_variant_registry API builds the document,
encode_runtime_variant_key and decode_runtime_variant_key expose the
key contract, and lookup_runtime_variant performs exact lookup with the
available keys included in not-found diagnostics. Lookup validates the closed
registry schema, registryHash, canonical key-to-record identity, and record
eligibility before returning a ready artifact. Modified or malformed registry
records fail as invalid rather than participating in selection. When the
non-execution identity matches but the requested execution identity does not,
the diagnostic reports requestedExecution and the exact
availableExecutionAlternatives with their keys, status, workgroup size,
and subgroup width. There is no fallback to one of those alternatives. Legacy
crosstl-rvk1: keys are rejected with guidance to regenerate both the key
and registry. This remains deterministic selection and packaging metadata;
target compilation, deferred compilation, host runtime dispatch, device
execution, and numerical parity are not established by the registry.
A schema-v3 native loader package binds the exact JSON registry to the descriptors, execution wrappers, translated artifacts, and optional native registry header by SHA-256 identity. Build a preflighted request from one canonical key:
import json
from pathlib import Path
from crosstl.project import build_runtime_variant_dispatch_request
package_root = Path("crosstl-native-loader-abi")
package = json.loads(
(package_root / "native-loader-abi-package.json").read_text()
)
registry = json.loads(
(package_root / package["runtimeVariantRegistry"]["path"]).read_text()
)
key = registry["lookup"]["readyKeys"][0]
request = build_runtime_variant_dispatch_request(
registry,
key,
package_root,
input_values,
output_values,
{"workgroupCount": [1, 1, 1]},
)build_runtime_variant_dispatch_request performs exact lookup and rejects a
registry that does not belong to the package. It verifies the packaged registry
hash and, when a native header is published, the native header hash. It also
checks descriptor size and hash, source and target provenance, artifact
identity, entry point, workgroup size, specialization identity and value, and
translated artifact bytes before delegating to
build_native_loader_dispatch_request. Runtime callers provide resource
values and dispatch counts but cannot replace the selected workgroup size or
specialization values.
Exact variant selection extends the request's frozen artifact identity with the selected variant name and canonical variant key. Native execution retains the pinned descriptor size, hash, and artifact ID plus those variant fields even if the request's public artifact mapping is later mutated.
At native runtime setup, the DirectX and OpenGL adapters read the selected
translated artifact once and compute its byte size and SHA-256 digest. When the
request records both values, a mismatch produces a structured setup diagnostic
with the expected and observed identities before target validation, compilation,
or runtime loading. Partial identity metadata and malformed fields fail closed.
An artifact without either recorded field is still captured, but the result is
reported as not-recorded rather than verified. Artifact read and snapshot
materialization failures also produce structured setup diagnostics. The
captured bytes are materialized in an adapter-owned temporary directory, and
subsequent source validation and target compilation use that snapshot instead
of reopening the original artifact path. Source-format loading reads the
snapshot, while compiled modules are produced from it. Replacing the original
file after capture therefore cannot change the bytes consumed by that
execution.
This snapshot isolates one execution from later changes to the original path; it is not a filesystem security boundary. It does not prevent another process with access to the temporary directory from changing the snapshot, attest compiler output, or verify artifacts consumed by a caller-supplied runtime outside these DirectX and OpenGL adapters.
The generated C++17 header exposes
crosstl_native_runtime_variant_lookup,
crosstl_native_runtime_variant_make_request, and
crosstl_native_runtime_variant_execute. Execution accepts only a pointer
returned by the generated registry, checks ABI and target identity, and compares
the exact specialization payloads before invoking the selected unit wrapper.
DirectX HLSL and OpenGL SPIR-V specializations are supported according to the
target adapter contracts; precompiled DXIL specialization and GLSL source
specialization remain fail-closed.
This bridge selects and prepares one native compute dispatch from an existing ready variant. It does not synthesize a missing variant, perform deferred target compilation, rewrite a repository's host runtime, schedule multiple kernels, translate framework control flow, or establish full MLX test-suite parity.
Repositories that select from a finite set of source variants can compile one
fully resolved variant after packaging without accepting arbitrary source
generation or compiler arguments. This path is defined by the schema-v1
crosstl-native-deferred-compilation-request contract and the public project
API:
from crosstl.project import (
build_native_deferred_compilation_dispatch_request,
build_native_deferred_compilation_request,
compile_native_deferred_compilation_request,
execute_native_deferred_compilation_request,
)
request = build_native_deferred_compilation_request(
source,
includes,
target,
variant,
expected_loader_descriptor,
)
compilation = compile_native_deferred_compilation_request(
request,
package_root,
cache_root,
)
dispatch = build_native_deferred_compilation_dispatch_request(
compilation,
input_values,
output_values,
{"workgroupCount": [1, 1, 1]},
)An exact ready runtime variant can form the same closed request directly from a schema-v3 native loader package:
from crosstl.project import (
build_runtime_variant_deferred_compilation_request,
execute_native_deferred_compilation_request,
)
request = build_runtime_variant_deferred_compilation_request(
registry,
key,
package_root,
)
result = execute_native_deferred_compilation_request(
request,
package_root,
cache_root,
input_values,
output_values,
{"globalSize": [count, 1, 1]},
)build_runtime_variant_deferred_compilation_request performs the same exact
lookup and package-to-registry identity checks as native variant dispatch. It
then verifies the selected source artifact bytes, loader descriptor size and
digest, source and target identity, entry point, execution configuration, and
specialization interface before deriving the request. The canonical variant
key, type and value arguments, compile definitions, specialization values, and
execution identity are copied from the selected registry record rather than
accepted from the runtime caller.
This bridge accepts DirectX HLSL source and OpenGL GLSL source records. Binary
DXIL or SPIR-V records remain ahead-of-time artifacts and cannot be treated as
compiler source. Native CI selects the exact record, compiles it with DXC or
glslangValidator, and dispatches the resulting binary on Direct3D 12 or a
software OpenGL device. The current package bridge emits an empty include list,
so the selected target source must be self-contained. A source artifact with
literal includes fails during verified materialization until the native loader
package carries a complete include closure.
source and every includes record carry a portable package path, source
format, byte size, and SHA-256 digest. target fixes the backend, profile,
compute entry point, and binary output format. variant carries the canonical
runtime variant key, finite type and value arguments, compile definitions,
specialization values, workgroup size, and optional subgroup width. The request
also pins the expected native loader ABI descriptor by path, size, and digest.
A canonical requestHash covers the complete closed-schema document.
Unknown fields, arbitrary compiler flags, unresolved or non-finite values,
source/target format mismatches, and inconsistent variant keys fail contract
validation.
Before a compiler is consulted,
materialize_native_deferred_compilation_inputs verifies the source, every
include, and the loader descriptor against their recorded size and digest. It
rejects symbolic links, path escapes, portable case collisions, non-regular
files, undeclared or unreachable includes, ambiguous angle includes, and
dynamic include operands. Only a complete literal include closure is copied
into an isolated source tree; the compiler receives include directories derived
from that verified closure.
Compilation then validates the loader descriptor against the request, including target, stage, entry point, source identity, execution configuration, and exact specialization identity and value rules. It separately reflects the source and compares entry-point execution, bindings, scalar layout, constants, and the OpenGL specialization interface with the descriptor. Any drift fails before target compilation or dispatch. Source reflection currently evaluates the primary translation unit. Include files remain byte-verified compiler inputs, but declarations visible only after preprocessing an include are not folded into host-interface reflection in this path and therefore cannot satisfy the interface check.
DirectX requests compile packaged HLSL compute source to DXIL with dxc.
OpenGL requests compile packaged GLSL compute source to SPIR-V with
glslangValidator. Compiler commands are derived from the validated request;
callers cannot append arbitrary flags. The schema-v1 compilation result records
the tool name, reported version, executable hash, probe and compile commands,
source and include provenance, compiler diagnostics, and the compiled output's
format, size, and SHA-256 digest.
Successful outputs use a deterministic cache key derived from the complete request hash and the toolchain name, version, and executable hash. Cache entries retain the expected interface identity and output identity, are published atomically, and are revalidated on lookup. The toolchain executable is also rechecked before a cached or newly compiled result is returned. Compiler failures, missing or malformed output, interface drift, and partial cache entries are never published as successful cache entries.
build_native_deferred_compilation_dispatch_request converts a successful
compiled result into the same preflighted RuntimeExecutionRequest used by
the native loader path. The result retains the normalized source request;
dispatch revalidates its hash and rejects target, specialization, or descriptor
provenance drift. It preserves exact binding, specialization, execution, and
compiled-artifact identity. execute_native_deferred_compilation_request
performs compilation or exact cache reuse and then dispatches through the
DirectX or OpenGL native runtime adapter. Callers still supply resource values
and dispatch counts. An injected OpenGL context remains caller-owned when
OpenGLComputeRuntime is constructed with release_context=False.
This is a bounded compute compilation and dispatch contract. It does not generate arbitrary source, rewrite host applications or runtime frameworks, choose repository scheduling policy, or provide framework-specific runtime integration. It does not claim complete MLX kernel coverage, MLX runtime integration, or parity with the MLX test suite. The implemented scope is tracked in GitHub issue #1854.
Build deterministic host loader scaffold metadata from a runtime loader manifest:
python -m crosstl scaffold-host-loaders \
crosstl-runtime-package/runtime-loader-manifest.json \
--scaffold-dir crosstl-host-loaders \
--format textHost loader scaffolds emit a crosstl-runtime-host-loader-scaffolds JSON
document and write a small metadata bundle with host-loader-scaffolds.json,
HOST_LOADERS.md, and one target-scoped *.loader.json file for each
ready load unit. The scaffold files preserve the loader manifest's artifact
paths, source-remap handoff paths, host interface metadata, required tools,
host responsibilities, and ordered load steps so host loader or build-system
tooling can consume the package contract deterministically. Load units with
unresolved blockers, such as missing host interface reflection metadata, remain
listed in the scaffold report and guide but do not get target loader files.
The bundle is metadata for integration tooling only: it does not rewrite host
application code, execute device code, generate runtime framework code, or
install target SDKs.
Inspect host loader scaffold files before runtime tooling consumes them:
python -m crosstl inspect-host-loader-scaffolds \
crosstl-host-loaders/host-loader-scaffolds.json \
--format textHost loader scaffold inspections emit a
crosstl-runtime-host-loader-scaffolds-inspection JSON document that verifies
the scaffold manifest, integration guide, and target-scoped loader metadata
files are present, readable, and consistent with the scaffold manifest. Ready
loader metadata files are parsed and checked for matching scaffold identity,
target, adapter kind, package path, and status. Blocked load units remain
explicitly blocked without requiring target loader files. Missing, malformed,
or mismatched scaffold files are reported as structured diagnostics before host
loader or build-system tooling consumes the metadata. Inspection remains
read-only and does not rewrite host application code, execute device code,
generate runtime framework code, or install target SDKs.
Build a read-only host loader consumption plan from scaffold metadata:
python -m crosstl plan-host-loader-consumption \
crosstl-host-loaders/host-loader-scaffolds.json \
--format textHost loader consumption plans emit a
crosstl-runtime-host-loader-consumption-plan JSON document. Planning runs
scaffold inspection first, reads only ready target-scoped host loader unit JSON
files, carries required tools and host responsibilities forward, and promotes
ordered loadSteps into actionable records for host build or runtime
integration tooling. Blocked scaffold records remain actionable
resolve-loader-scaffold-blockers entries, and failed scaffold inspection
diagnostics are reported without reading unsafe unit files. The plan remains
metadata only: it does not rewrite host application code, execute device code,
generate runtime framework code, or install target SDKs.
Write a deterministic host integration handoff bundle from a consumption plan:
python -m crosstl host-integration-handoff \
crosstl-host-loaders/host-loader-consumption-plan.json \
--handoff-dir crosstl-host-integration \
--format textHost integration handoff bundles emit a
crosstl-runtime-host-integration-handoff JSON report and write
host-integration.json, HOST_INTEGRATION.md, and one
targets/*.integration.json file per target. The bundle is designed as a
stable handoff for build-system and runtime integration tools: it preserves the
validated loader units, promoted actions, required tools, host responsibilities,
package paths, scaffold files, and blocked-unit records from the consumption
plan. It remains a metadata bundle only and does not rewrite host application
code, execute device code, generate runtime framework code, or install target
SDKs.
Inspect host integration handoff files before downstream tooling consumes them:
python -m crosstl inspect-host-integration-handoff \
crosstl-host-integration/host-integration.json \
--format textHost integration handoff inspections emit a
crosstl-runtime-host-integration-handoff-inspection JSON document that
verifies the handoff manifest, guide, and per-target
targets/*.integration.json files are present, readable, and consistent with
the handoff manifest. Target files are parsed for matching kind, target, status,
loader-unit counts, and action counts. Missing, malformed, wrong-kind, or
mismatched handoff files are reported as structured diagnostics. Inspection is
read-only and remains bundle-local: it does not rewrite host application code,
execute device code, generate runtime framework code, install target SDKs, or
re-run host integration.
Build a read-only host integration execution plan from an inspected handoff:
python -m crosstl plan-host-integration-execution \
crosstl-host-integration/host-integration.json \
--host-root . \
--format textHost integration execution plans emit a
crosstl-runtime-host-integration-execution-plan JSON document. Planning
runs handoff inspection first, records optional host-root readiness, and
normalizes per-target handoff actions into stable phase-ordered execution
steps. The step phases cover tool preparation, loader consumption, artifact
loading, host responsibility handling, blocker resolution, and other host
actions. Plans carry required tools, host responsibilities, package paths,
scaffold files, target status, and structured diagnostics so downstream host
or build-system tooling can decide what to run next. Plans also include a
deviceExecution readiness block that declares the adapter-backed dispatch
inputs a target will need, including the runtime package root, runtime adapter
descriptor root, and an external target runtime runner. The plan remains
metadata only: it does not rewrite host application code, execute device code,
generate runtime framework code, or install target SDKs.
Execute deterministic host integration checks from a saved execution plan:
python -m crosstl execute-host-integration \
crosstl-host-integration-execution-plan.json \
--host-root . \
--scaffold-root crosstl-host-loader-scaffolds \
--package-root crosstl-runtime-package \
--adapter-root crosstl-runtime-package/runtime-adapters \
--format textHost integration execution results emit a
crosstl-runtime-host-integration-execution-result JSON document. Execution
validates the plan, revalidates the planned host root or an explicit
--host-root override, records scaffold-root and package-root readiness,
verifies generated loader scaffold files when --scaffold-root is provided,
checks package artifact and source-remap paths when --package-root is
provided, verifies runtime adapter descriptor manifests and descriptor files
when --adapter-root is provided, checks required host tools on PATH, and
reports project-specific host responsibilities as skipped actionable steps.
Blocked plan steps remain blocked in the result, and missing files, stale
descriptor hashes, or invalid paths are emitted as structured diagnostics.
The result deviceExecution block reports whether each target has a ready
runtime package and verified runtime adapter descriptor for an external runner.
Pass --runner-manifest with a
crosstl-runtime-device-runner-manifest JSON file to record target runner
readiness alongside package and adapter readiness. Runner manifests list
target-specific runner ids, statuses, optional capabilities, and optional
commands; they are validated as readiness metadata only. This command still
does not rewrite host application code, dispatch device work, generate runtime
framework code, or install target SDKs.
Diagnostics with originalLocation keep the generated or validation
location as the primary SARIF location and attach the original source span as a
related location. Remapped diagnostics also expose sanitized
diagnosticLocation and originalLocation SARIF properties for tools that
filter or group results without walking related locations.
Inspection exits nonzero when validation finds report errors.
When present, crosstl.toml is loaded from the repository root. The initial
configuration contract is intentionally small:
[project]
source_roots = ["shaders", "kernels"]
include = ["**/*"]
exclude = ["third_party/**", "build/**"]
targets = ["metal", "opengl"]
output_dir = "crosstl-out"
include_dirs = ["shaders/include"]
external_corpus_manifest = "external-corpus.json"
workgroup_size = [32, 8, 4]
[project.sources]
"legacy/**/*.shader" = "cgl"
[project.defines]
USE_FAST_PATH = "1"
[project.workgroup_size_rules]
"kernels/gemv.metal" = ["32", "BN", "BM"]
[project.entry_workgroup_size_rules."kernels/gemv.metal"]
"gemv_wide*" = ["32", "k_lanes / 8", "1"]
[project.subgroup_width_rules]
"kernels/wave.metal" = "WIDTH"
[project.source_options.metal]
max_template_specializations = 2048
max_template_materialization_work = 131072
[project.source_options.metal.source_patterns."kernels/scan*.metal"]
max_template_specializations = 4096
[project.source_options.metal.source_patterns."kernels/proven-layout.metal"]
cooperative_matrix_fragment_mapping = "tile_4x4_row_pair"
cooperative_matrix_fragment_mapping_provenance = "project_source_contract"
[project.source_options.metal.target_options.opengl]
max_template_materialization_work = 65536
[project.source_options.metal.target_options.opengl.source_patterns."kernels/gemv.metal"]
max_template_materialization_work = 131072
[project.variants.debug]
USE_FAST_PATH = "0"
workgroup_size = [32, 4, 8]
[project.specialization_constants]
useFastPath = true
"2" = 16
[project.source_specialization_constants."kernels/*.metal"]
"2" = 32
[project.source_specialization_constants."kernels/gemv.metal"]
"2" = 16
[project.variants.debug.specialization_constants]
useFastPath = false
"2" = 8An explicit --config path may be absolute or repository-relative. Relative
config paths are resolved against the repository root passed to the command, not
against the shell's current working directory. When --config is provided,
the referenced file must exist; otherwise the command exits with an error
instead of silently falling back to default scan settings.
Function and specialization constant values are configured under
[project.specialization_constants]. Each key selects a source declaration
by its exact name or by a quoted, non-negative numeric ID such as "2".
Configured values are checked against the declaration's scalar source type. If
both selectors address the same declaration, their values must agree; a
name/id conflict fails the artifact instead of choosing one silently.
Repository-wide numeric IDs and source names do not need to be unique. Use
[project.source_specialization_constants."<repo-relative-pattern>"] to
override specialization selectors only for matching translation units. For
example, unrelated sources can configure the same numeric ID as a Boolean in
one table and an integer in another without offering either value to the other
source.
Source patterns use normalized repository-relative paths and merge per
selector. project.specialization_constants provides the base values. Every
matching source table can add selectors; for a selector present in more than
one table, an exact normalized path wins over a glob. Otherwise, fewer wildcard
operators take precedence, followed by the longer normalized pattern. Patterns
with equal specificity may provide the same scalar value; lexical pattern order
then selects one stable provenance path without changing the value. Equal-
specificity patterns that provide different values fail closed with a
ValueError identifying the source, selector, matching patterns, and values.
An exact table only overrides selectors it contains, so less-specific matching
tables can still supply other selectors.
Source function_constant and constant_id identifiers use C-family
integral literal rules. Decimal, leading-zero octal, hexadecimal, and binary
forms are accepted, together with apostrophe digit separators and the standard
u/l/ll integer suffix combinations supported by the source
frontend. IDs are normalized before duplicate checks, configuration lookup,
reflection, or target emission. Metal function_constant indices use the
native inclusive range 0 through 65535; GLSL constant_id values use
0 through 2147483647. Consequently, 1, 01, and 0x1 all
select the canonical configuration key "1" and collide if used by separate
declarations. A non-canonical source form is retained as idSpelling beside
the numeric id in artifact specialization records. Invalid digits, negative
values, constant expressions, and values outside the applicable source range
produce project.translate.specialization-constant-id-invalid with the source
span, original spelling, and a structured reason.
[project.variants.<name>.specialization_constants] applies after the
project-level and matching source tables and overrides the same selector for
that named variant.
Artifact specializationConstants records retain the effective value and
valueProvenance with the project or variant configuration path, selector,
selector kind, and variant name. Values selected from a source table use
project-source-pattern provenance with the normalized sourcePattern and
the complete configuration path. The report's
project.sourceSpecializationConstants,
project.sourceSpecializationPatternCount, and
project.sourceSpecializationConstantCounts fields preserve the normalized
configuration for deterministic report validation. Using a name in one table
and the corresponding ID in another still creates two matches, so different
values fail as a conflicting contract rather than relying on table precedence.
A declaration without a source initializer is required; one with an initializer has a source default. Explicit project or variant values override source defaults. Targets with native specialization retain required declarations for the host runtime to provide, while targets without it must receive a concrete configured value or a materializable source default.
OpenGL defers specialization natively as
layout(constant_id = N) const ... and does not lower these declarations to
uniforms or resources. A required declaration without a source default receives
only an encoding initializer needed for valid GLSL; its report record remains
required and the host must provide the value before execution. DirectX
instead materializes a concrete CrossGL variant before HLSL generation. It uses
the selected variant override, project value, or source default and fails closed
without emitting HLSL when a required value is missing, conflicting, or
incompatible with the source type.
project.workgroup_size defines one concrete local workgroup size as exactly
three positive integers in X, Y, Z order. A named variant can override it with
project.variants.<name>.workgroup_size. Workgroup-size entries are execution
metadata, not preprocessor defines. They therefore do not enter the artifact
define map, while each named size still produces its own variant path and
deterministic execution identity.
For DirectX and OpenGL translation from Metal, a concrete
project.workgroup_size or
project.variants.<name>.workgroup_size can specialize every compute entry
produced by deterministic, host-named template materialization. This requires
a complete one-to-one join between emitted compute stages and materialization
records using their stable hostName and materializedName identities.
Every joined entry receives the configured size for that project variant.
DirectX preserves the entries in one HLSL library artifact, while OpenGL emits
one standalone main artifact per entry. Their execution records retain the
source, materialized, and target entry identities together with
project-config or project-variant provenance.
[project.workgroup_size_rules] defines repository-relative, source-specific
workgroup sizes for materialized compute entries. Each key is a source path
pattern and each value contains three integral expressions in X, Y, Z order.
An exact path takes precedence over a glob; otherwise the most specific matching
pattern is selected. Expressions may use the concrete parameters recorded for
each host-named template materialization and the integer operators supported by
the Metal constant-expression evaluator. They are evaluated with signed 64-bit
intermediate bounds. Calls, casts, member access, unknown identifiers,
non-integral parameters or literals, unsigned width-dependent arithmetic,
overflow, division by zero, and non-positive results fail closed with a
structured workgroup-rule diagnostic.
Rule evaluation joins emitted compute entries to materialization records by the
stable hostName and materializedName identities. Record order is not
significant. Every host-named record must be matched exactly once; missing,
duplicate, conflicting, or unmatched records fail the artifact. Helper
materializations without hostName remain provenance records and are not
reported as runnable entries. The source is materialized and parsed once per
target, after which a distinct size is applied to each matched compute stage.
[project.entry_workgroup_size_rules.<source-pattern>] handles source files
that contain template families with different dispatch formulas. Its keys are
host entry-point patterns and its values use the same three-expression format.
The most specific matching entry pattern overrides
[project.workgroup_size_rules] for that entry; the source-wide rule remains
the fallback for entries without an override. Without a source-wide fallback,
every host-named materialization must match an entry rule. Every configured
entry pattern must match at least one host-named materialization. Missing entry
coverage and stale patterns fail closed with
project.translate.workgroup-size-entry-rule-unmatched.
Metal also consumes source-wide and entry-specific workgroup-size rules, but as
host-dispatch contracts rather than source specialization. MSL does not encode
a fixed numthreads or local_size attribute: the generated kernel keeps
its exact emitted entry name, execution.entryPoints records each evaluated
size, and report validation verifies that the reflected compute-entry identities
match the contract. DirectX and OpenGL continue to encode the same canonical
sizes in HLSL and GLSL respectively. This distinction prevents a host dispatch
requirement from being mistaken for Metal source metadata.
For DirectX and OpenGL project translation, a consumed Metal
[[threads_per_threadgroup]] parameter requires this concrete configuration
or equivalent concrete source execution metadata. Translation emits
[numthreads(x, y, z)] and the matching OpenGL local-size declaration from
the same canonical value. A scalar source parameter observes .x, a
two-component parameter observes .xy, and a three-component parameter
retains all components. Missing, malformed, non-positive, target-limit-
exceeding, or conflicting values fail the artifact with an
execution-specialization diagnostic instead of using a default local size.
DirectX can package several materialized compute entries in one HLSL artifact.
Each exported entry receives its own numthreads declaration and retains its
source, materialized, and target entry identities. Standard OpenGL GLSL exposes
one runnable main entry, so project translation emits one independently
runnable artifact per source entry. Each OpenGL artifact records only its own
entry in execution and maps that entry to main. Its source-wide template
materialization metadata retains the complete host identity set so report
validation and artifact-matrix inspection reject a missing split artifact.
Helper wrappers are not presented as runnable OpenGL entries.
The fixed project.workgroup_size contract remains available for genuinely
single-entry artifacts, source metadata proving a shared size, and the complete
host-named materialization join described above. Merely configuring one size
does not prove an ordinary multi-entry aggregate safe: sources without that
deterministic materialization identity remain ambiguous and fail closed instead
of applying the value to every entry. Missing, duplicate, conflicting, or
unmatched host records also fail closed. A multi-entry OpenGL source is still
packaged as separate runnable artifacts even when every entry uses the same
size, and translation fails if the artifact model cannot represent that split.
Targets outside DirectX, Metal, and OpenGL reject a matching workgroup-size
rule before source materialization or target generation. Metal accepts the rule
as host-dispatch metadata; DirectX and OpenGL additionally specialize target
source dimensions. A failed artifact and structured
execution-specialization diagnostic retain the selected rule, target, and
supported target set so the configuration cannot be silently ignored.
Successful artifact records include an execution object with the canonical
workgroupSize, affected sourceEntryPoints, configuration or source
provenance, and a SHA-256 identity. Report validation recomputes that
identity, and runtime artifact manifests preserve the execution object alongside
reflected target dispatch metadata. Workgroup size is independent of subgroup
width; the project pipeline does not infer or record a subgroup requirement from
any workgroup dimension.
Rule-based artifacts instead record an execution.entryPoints array. Each
entry includes the source, materialized, and target entry names, evaluated
dimensions, exact expression rule, concrete parameter values and provenance,
the joined materialization identity, and a deterministic SHA-256 identity. The
aggregate execution identity covers the complete entry array and rule
provenance. Entry-specific rules additionally retain the selected entry pattern
and its nested configuration path. Report validation selects each source and
entry pattern again, re-evaluates every expression, verifies the materialization
join and hashes, and checks the generated target entry metadata. These records
describe shader or kernel translation and dispatch requirements; they do not
rewrite framework runtime code or establish numerical runtime parity.
[project.subgroup_width_rules] defines repository-relative,
source-specific exact subgroup widths for materialized compute entries. Each
key is a source path pattern and each value is one bounded integral expression.
Pattern selection, materialization joins, expression syntax, parameter
provenance, and signed 64-bit evaluation follow the per-entry workgroup rule
contract. The expression must resolve independently for every host-named
materialized entry; unknown or non-integral parameters, invalid arithmetic,
non-positive results, missing materializations, and ambiguous joins fail the
artifact with a structured execution-specialization diagnostic.
DirectX currently enforces this contract for exact widths 4, 8,
16, 32, 64, and 128. Every generated target entry receives one
single-value [WaveSize(width)] attribute, and its execution metadata records
a cs_6_6 profile requirement. Report validation re-evaluates the expression
against the recorded template materialization, verifies deterministic entry and
execution identities, and checks the generated WaveSize and shader-profile
contract. A subgroup-width rule can accompany a per-entry workgroup-size rule;
both must resolve to the same materialized entry identities.
OpenGL accepts exact widths 1, 2, 4, 8, 16, 32, 64,
and 128 through a device-compatibility contract. Generated GLSL requires
GL_KHR_shader_subgroup_basic, declares
CROSSTL_REQUIRED_SUBGROUP_WIDTH, and guards the compute entry before any
translated work. Execution metadata requires the host extension
GL_KHR_shader_subgroup and the GL_SUBGROUP_SIZE_KHR query. The built-in
Python and generated C++ OpenGL adapters compare that query with the artifact
contract before shader compilation, resource allocation, or dispatch, and
report a structured mismatch instead of running on an incompatible device.
Report validation verifies the extension, marker, guard, execution metadata,
and deterministic identities. The shader guard is a defensive fallback; hosts
must honor the recorded pre-dispatch check.
Every other target currently fails closed before generation with
project.translate.subgroup-width-enforcement-unsupported and reason
target-not-supported. These failures record the missing
execution.subgroup-width-specialization capability, rule provenance, and
the supported target set without emitting a misleading target artifact.
Subgroup-width specialization establishes a compiler-facing shader contract only. It does not dispatch device work, verify hardware support, integrate a host runtime, or establish numerical parity. Workgroup dimensions also remain independent and do not imply a subgroup width.
For example, the host code at pinned MLX commit
4367c73b60541ddd5a266ce4644fd93d20223b6e selects GEMV tile parameters per
entry and dispatches (32, BN, BM). That is evidence for distinct per-entry
workgroup variants. The leading 32 remains the X workgroup dimension and is
not evidence of a required subgroup width. This repository example does not
change the backend-neutral configuration contract.
The pinned MLX project-porting gate applies this contract to
mlx/backend/metal/kernels/rms_norm.metal at commit
4367c73b60541ddd5a266ce4644fd93d20223b6e. Its DirectX project declares two
named variants, selecting has_w=false by declaration name and "20"=true
by numeric ID. The gate checks report provenance and concrete materialization,
then compiles a reflected compute entry from each generated HLSL artifact with
DXC on Windows. Its current OpenGL proof leaves the subgroup rule unconfigured,
checks deferred layout(constant_id = 20) emission, and validates generated
OpenGL SPIR-V on Linux. The separate bounded LogSumExp proof exercises the
OpenGL exact-width contract. These checks prove translation and native compilation only; they do not claim
RMSNorm numerical runtime parity or full MLX test-suite support. Numerical
execution also requires host dispatch values to match each compiled artifact's
workgroup-size and subgroup-width contracts.
A separate current-corpus proof pins rms_norm.metal at commit
846d176227a0ac13d2667e58d2bb68b322109ab0 and selects the forward
rmsfloat32 entry for an axis-size-32, two-row workload. Entry-scoped runtime
reflection excludes the unreachable VJP-only function constant has_w while
preserving constants used by selected entries. The proof packages the same six
resources for HLSL and GLSL, requires WaveSize(32) on DirectX, and uses the
explicit target-scoped 32-lane software subgroup on OpenGL. Windows CI executes
the package with Direct3D 12 WARP; Linux CI validates the GLSL through
glslangValidator and spirv-val and executes it with surfaceless Mesa
EGL. Both compare 64 float32 outputs against the independent RMSNorm formula at
3e-5 absolute and relative tolerance. This evidence covers only the
selected forward float32 workload; VJP, looped, half-precision, other axis-size,
host-runtime redirection, and full MLX test-suite coverage remain outside its
claim.
source_roots limits discovery to selected directories. include and
exclude use shell-style patterns against repository-relative paths. Project
reports include order-preserving source-root status records and status counts
so active, missing, non-directory, outside-project, and scan-visible roots can
be triaged without re-running discovery. Missing source roots, source roots
that resolve to files or other non-directory paths, and roots that resolve
outside the repository are reported as scan or configuration diagnostics.
Include, exclude, and source override patterns must also be
repository-relative; absolute patterns or patterns containing parent-directory
segments are reported as configuration diagnostics and skipped. Source
overrides allow extensionless or non-standard files to be assigned to a
registered source backend. Override patterns are also considered during default
discovery, so override-only files do not require broad include globs. CLI source
roots replace the configured source roots before scan, report, or translation.
CLI source overrides are merged with this configuration before scan, report, or
translation.
Known override backend aliases are canonicalized in reports; invalid override
backend names are reported as configuration diagnostics.
Explicit broad include patterns may also match compiled shader artifacts or
known source formats that CrossTL cannot parse yet. Project scans keep those
files in the skipped-file rollups and emit structured diagnostics with the
same specific guidance as single-file translation, while continuing to discover
supported translation units in the repository.
Include directories, defines, and named
variants are recorded in project reports. Source frontend options can also be
set under [project.source_options.<source-backend>] and are forwarded only
to source frontend and reverse-codegen callables that expose matching keyword
options. Metal source imports
support max_template_specializations as the project-specific unique concrete
helper specialization cap and max_template_materialization_work as the
project template materialization work budget. Materialization work is charged
to the reachable concrete graph and the type information actually resolved. It
counts each unique reachable source entry, concrete helper specialization, and
concrete struct specialization once, together with uncached function and type-
environment resolution and concrete struct-field type resolution performed for
that graph. Shared transitive helpers and repeated occurrences of the same
concrete signature are therefore deduplicated. The budget does not precharge a
whole-source source instantiations x template declarations Cartesian
estimate, and repeated scans of progressively expanded source text are not work
items merely because the text was scanned again.
Metal cooperative-matrix fragment mappings are opt-in. Configure
cooperative_matrix_fragment_mapping together with
cooperative_matrix_fragment_mapping_provenance only for source files whose
lane-coordinate contract has been established independently. The built-in
tile_4x4_row_pair profile is exact for an 8x8 matrix distributed over 32
lanes with two adjacent row elements per lane. The Metal
thread_elements() identity and matching cardinality do not select that
profile automatically. Unknown, incomplete, or shape-incompatible profiles
fail before target emission, and selected profile metadata is retained in
project diagnostics.
Direct translate() calls from Metal to template-hostile targets use the
same reachable-specialization preparation as one-unit project translation.
Explicit instantiations, host_name attributes, template defaults, include
paths, defines, and materialization budgets are therefore applied before
DirectX or OpenGL code generation. If a reachable declaration still requires
template arguments, direct translation raises a ValueError carrying the
project.translate.template-materialization-unsupported diagnostic code,
missing-capability list, materialization metadata, and source location instead
of returning an artifact with unresolved target resource types. Metal,
CrossGL, and already-preprocessed source paths retain their existing behavior.
This contract does not infer variants for which the source supplies no concrete
evidence, and it is not a full-corpus or runtime-parity claim.
During project translation, Metal template-member inference preserves a
generic pointer template parameter as a pointer rather than reducing it to its
pointee type. For a parameter such as Pointer src, a bare tracked pointer,
legal pointer-plus-integral or pointer-minus-integral expression, or the address
of a directly subscripted pointer or array element binds Pointer to the
complete pointer type. Pointer identity, the proven Metal address space, and
const/volatile qualification are retained. Legal offset forms include
ptr + offset, offset + ptr, and ptr - offset when offset has a
known integral type.
This does not change deduction for a parameter declared as device U* or
threadgroup U*: after pointer compatibility is established, U is still
deduced as the pointee type. Inference remains conservative. An unknown base or
address space, a non-integral offset, pointer-pointer arithmetic, an offset
minus a pointer, or address-taking outside a proven &base[index] shape
fails closed with project.translate.metal-struct-method. Addressed-element
indices must also be known integral expressions without unsupported calls,
assignments, side effects, nested subscripts, or ambiguous expression forms.
Concrete pointee comparisons resolve visible non-template using and
typedef chains at the method declaration and call site before deciding
whether two pointer types are compatible. Declaration order, nested shadowing,
and sibling scopes are preserved. Forward references, alias cycles, and chains
whose equivalence cannot be proved remain failed bindings; they are not treated
as matching merely because their unresolved spelling is the same.
For DirectX, a supported storage-pointer helper parameter is emitted as a
StructuredBuffer or RWStructuredBuffer resource together with a signed
element offset. Passing &buffer[index], a previously rebased alias, or an
alias forwarded through another supported helper composes that offset without
emitting HLSL pointer syntax or mutating the resource handle. The generated
helper applies the offset to every indexed load or store. Element-type
changes, insufficient read or write access, pointer-to-pointer parameters, and
arguments without a concrete structured-buffer root fail with
project.translate.directx-resource-pointer-parameter-unsupported rather
than producing an invalid call.
A local read-only auto* alias whose initializer resolves to a concrete
StructuredBuffer or RWStructuredBuffer root deduces its element type
from that backing resource before DirectX alias lowering. Direct and nested
aliases retain the accumulated signed element offset and read-only contract.
Explicit pointee types remain authoritative: incompatible declared element
types are rejected rather than being replaced with the backing type.
Metal reverse translation preserves conditional and assignment expressions as
lower-precedence operands when serializing binary and postfix expressions. For
resource-backed pointer aliases, a dynamic offset such as
buffer + (enabled ? first : second) therefore retains the conditional as
the offset expression before DirectX or OpenGL resource-plus-offset lowering.
This guarantees expression-tree preservation for the translated artifact; it
does not provide host dispatch, resource binding, or numerical runtime parity.
The contract applies generally to Metal sources handled by project translation.
The pinned MLX BaseMMAFrag::load(&(src[index])) call shape is a focused
acceptance example for retaining the qualified pointer through nested template-
member materialization; it is not evidence of runtime parity or completion of
the pinned or full MLX corpus.
Metal struct-method lowering preserves direct mutable and read-only reference
accessors when the returned lvalue is receiver-owned scalar storage or an
exactly indexed fixed-array element. Simple receiver declarations can use a
lexically visible using or typedef chain that resolves to the concrete
struct; alias scope and declaration order are honored before the accessor is
rewritten to original storage. A proven thread const receiver selects one
matching const accessor, including implicit calls from a const struct method. A
local thread const auto& binding may also read through an accessor on a
nested value member when the member path contains no pointer, reference, or
array traversal. The binding is replaced with the original fixed-array storage
only when every use is an indexed read and the accessor arguments cannot change
through a reference, member mutation, or subsequent call during the binding's
lifetime.
Constant-address-space receivers, unresolved aliases, pointer-member
receivers, ambiguous overloads, mutable or escaping local reference bindings,
side-effectful or unstable indices, non-indexed alias uses, and indirect storage
remain fail-closed with
project.translate.metal-struct-method rather than being converted to
value-returning helpers.
Non-entry Metal const reference parameters retain their input-only contract
in the shared representation. DirectX and OpenGL receive value inputs, mutable
references remain inout, and Metal round-trip generation reconstructs a
const address-space reference. Stage-entry buffer references continue
through the resource binding path instead of being rewritten as helper values.
The default materialization work budget is derived from the active template
specialization limit, so larger finite source-instantiated kernels can complete
without raising the unique helper specialization cap. Use
[project.source_options.metal.source_patterns."<repo-relative-glob>"] to
raise or lower Metal budgets for matching sources. Use
[project.source_options.metal.target_options.<target>] and its nested
source_patterns table to override budgets only for one target, such as
OpenGL. Both limits remain fail-closed. If the next unique entry, helper, struct
specialization, or required type-environment resolution would cross a configured
limit, translation fails without emitting the target artifact. The structured
diagnostic identifies the concrete item or resolution that crossed the limit
and reports the requested count, active limit, configuration field that set it,
source location, and suggested remediation. Project reports include
order-preserving include-directory status records and status counts so missing,
non-directory, outside-project, and frontend-visible active include directories
can be triaged without re-running discovery. Missing include directories,
include entries that resolve to files or other non-directory paths, and include
directories that resolve outside the repository are reported as non-blocking
configuration diagnostics so reports retain portability and provenance context.
Existing include directories that remain inside the repository, plus configured
defines, are passed to source frontends that expose preprocessor options. CLI
include and define overrides are merged with this configuration before scan,
report, or translation. Translation artifacts record defineProcessing
metadata so reports distinguish define maps that were forwarded to the source
lexer from define maps that were not requested or could not be consumed by that
frontend. Report inspection samples include effective define names and
deterministic define fingerprints without exposing define values.
When configured defines cannot be forwarded, translation reports emit a
non-blocking warning diagnostic with a missing-capability rollup so the
limitation appears in validation and inspection summaries.
Scan reports also emit diagnostics for active #error and #warning
directives after evaluating project and selected variant conditionals. Active
#error directives are reported as errors, while active #warning
directives are reported as warnings.
Scan reports also emit non-blocking warning diagnostics when active
#define or #undef directives in translation units or resolved include
files shadow configured project or selected variant define names. The
diagnostics identify the source location and define name without reporting
configured define values.
Summary, inspection payloads, and text reports also include define-processing
rollups by target, source backend, and named variant when variants are
configured, so target-specific, frontend-specific, and variant-specific
preprocessing gaps are visible without reading every artifact record. Report
inspection also includes sampled artifact
define-processing metadata with status, frontend support, and define counts,
but not define values, so artifact-level preprocessing state can be triaged
without exposing configuration values.
Define-processing inspection summaries also include redacted project define
names, deterministic define fingerprints, selected variant names, and
per-variant define records without exposing configured define values. Text
issue lines for unsupported define forwarding include define names and the same
fingerprint so the affected define set can be identified without revealing
values.
They also record includePathProcessing metadata so active include-directory
paths can be distinguished from include paths that were not requested or could
not be consumed by the selected source frontend. Include-path processing
warnings are also emitted when active include paths cannot be forwarded, so
the report diagnostics identify affected source frontends without failing the
batch translation.
Summary, inspection payloads, and text reports also roll up by target, source
backend, and named variant when variants are configured. Report inspection
includes sampled artifacts whose active include paths could not be forwarded, so
the affected source, target, and frontend are visible without reading every
artifact record. It also includes sampled artifact include-path processing
metadata with status, frontend support, and include path counts, so
artifact-level include forwarding state can be triaged from the report summary.
Inspection summaries also include configured include-directory status records
plus frontend-visible and inactive directory counts, so report consumers can
distinguish directories that reached the frontend from missing, non-directory,
or outside-project entries. Text issue lines for unsupported include-path
forwarding name the frontend-visible configured include directories so the
affected configuration is visible next to the artifact identity.
During scan, project reports also record #include directives discovered in
translation units. Each dependency record keeps the include target, local,
system, or dynamic kind, line and column, and a status of resolved,
missing, system, dynamic, or outside-project. Resolved
dependencies record the repository-relative resolved path and whether the match
came from the source directory or a configured include directory. A directive
that uses one project define, such as #include PROJECT_HEADER, is resolved
when that define's value is a quoted or angle-bracket include target; the
dependency keeps resolvedFromDefine so the report remains actionable.
When named variants are configured, include discovery evaluates those
define-backed include targets with the same base-plus-variant define maps used
for translation, and variant-scoped dependency records keep variant.
Scan-time include discovery also honors simple #if, #ifdef,
#ifndef, #elif, #else, and #endif branches using the same
project and variant define maps. Supported #if expressions include
defined checks, boolean operators, parentheses, integer and boolean define
values, and simple integer comparisons. Unsupported conditional expressions
remain open so discovery does not hide possible dependencies.
Resolved include files are scanned recursively for additional dependencies.
Nested dependency records keep source when the directive came from a
resolved include file rather than the root translation unit, so diagnostics and
inspection output can point to the include file that introduced the dependency.
If a resolved nested include cannot be read, the already discovered dependency
is kept and project scan emits project.scan.include-read-failed with
include.resolution capability metadata.
Unresolved system includes are recorded without warning because they often
refer to SDK or toolchain headers. Missing local includes, dynamic include
expressions, and include paths that resolve outside the repository emit
structured include.resolution diagnostics. When the failed include came
from a project or selected variant define, diagnostics identify that define
and the variant context when applicable.
Report inspection samples resolved include dependencies, unresolved system
include dependencies, and include issues, including the source location,
source backend, include kind, unit source hash and byte size, resolved path,
resolved include hash and byte size, and resolution source where available.
Define-backed include samples also retain the project define name that supplied
the include target and the variant name when the dependency came from a named
variant define map.
output_dir must resolve inside the repository root; paths that escape the
repository are reported as configuration diagnostics and artifacts are not
written. When named variants are configured, project translation emits one
artifact attempt per variant and passes base defines merged with the variant's
define overrides to the source frontend. Variant artifacts are written under a
variant path segment inside each target output directory, and the original
variant name plus applied define map are recorded on the artifact and variant
name is recorded on validation records. --variant NAME can be repeated to
scope scan, report, or translation runs to declared variants; when no explicit
--variant arguments are provided, selected_variants in crosstl.toml
sets the default scoped variant list. Scoped reports declare only the selected
variants, de-duplicate repeated selections before planning, and do not claim
omitted variants as scanned or attempted.
CrossGL source translation applies object-like define expansion and conditional
branch selection for #if/#ifdef/#ifndef/#elif/#else/#endif
when defines are provided. Project translation also passes selected variant
define maps into native source frontends that expose define options; current
project coverage includes OpenGL/GLSL and Vulkan angle include expansion,
DirectX/HLSL, Metal/MSL, Slang, and CUDA/HIP local header expansion, CUDA/HIP
runtime system include preservation, conditional branches, and project include
directories through those paths. Other native preprocessor behavior remains
backend-dependent.
Configuration scalar values and define/source-override maps are type checked
when crosstl.toml is loaded. Define names, source override patterns, source
override backend names, named variants, and variant define names must be
non-empty strings. Malformed values are rejected before scan or translation so
reports do not silently stringify invalid project metadata.
external_corpus_manifest points at an optional repository-relative JSON
manifest of pinned upstream shader or GPU-source reductions. When configured,
the manifest path must be a non-empty string. Project reports use the manifest
for coverage accounting only: they record declared paths, present and missing
entries, discovered translation units, source-backend and target rollups, valid
and invalid manifest-entry counts, and artifact outcomes for entries included
in the project run. CrossTL does not clone upstream repositories, run native
build systems, or claim whole corpus semantic parity from this manifest.
The bundled support manifest is a reduced, fixture-backed coverage manifest
with one pinned entry per registered source backend; those entries support
provenance and accounting checks rather than corpus-wide semantic parity
claims.
Malformed manifest entries are reported as configuration diagnostics and
skipped from retained corpus entries. Duplicate manifest paths or explicit
entry ids are also reported and skipped so generated reports do not inflate
corpus coverage. The summary still records how many manifest entries were
skipped. Inspection samples for missing and present-but-undiscovered entries
retain repository, commit, and source URL metadata when the manifest provides
those provenance fields.
Project reports include configured define, variant, and specialization constant selectors and values, and artifact records include the applied define map used for that translation attempt. Review reports before sharing them outside the repository if those values include private build metadata. Compact inspection summaries list configured define names, deterministic define fingerprints, variant names, per-variant define counts, variant define names, and deterministic per-variant define fingerprints without printing define values.
Projects can import versioned JSON host dispatch contracts through
project.dispatch_contracts in crosstl.toml:
[project]
dispatch_contracts = [
"contracts/layer-norm.dispatch.json",
"contracts/copy.dispatch.json",
]The scan, report, and translate-project commands also accept a
repeatable --dispatch-contract PATH option. Command-line imports augment
the configured contract list for that invocation:
python -m crosstl scan /path/to/repo \
--dispatch-contract contracts/layer-norm.dispatch.json
python -m crosstl report /path/to/repo \
--dispatch-contract contracts/layer-norm.dispatch.json \
--dispatch-contract contracts/copy.dispatch.json \
--output crosstl-out/portability-report.json
python -m crosstl translate-project /path/to/repo \
--dispatch-contract contracts/layer-norm.dispatch.json \
--output-dir crosstl-outContract expressions are evaluated over finite declared domains in a deterministic order. The resulting project metadata includes:
| Report field | Contents |
|---|---|
dispatchContractFiles |
Ordered configured and command-line contract paths. |
dispatchContractCount |
Number of imported contract manifests. |
dispatchVariantCount |
Total number of deterministically evaluated dispatch variants. |
dispatchContracts |
Embedded normalized manifests, content identities, provenance, and evaluated variant records. |
The embedded manifests and evaluations are machine-readable and self-contained for project report validation, including deterministic replay without the external contract files. During scanning, evaluated records are converted into a deterministic source-scoped artifact plan. Compile-equivalent records share one artifact job while their distinct dispatch geometries remain available as dispatch variants. A contract that names an undiscovered source, an unknown entry point, or a conflicting artifact identity fails before target emission.
translate-project applies each planned job only to its referenced source
unit. The job selects the source entry point and carries its workgroup size,
required subgroup width, specialization constants, stable artifact identity,
and contract provenance into target generation. Unreferenced source units retain
their ordinary project configuration. Generated reports expose this contract
through:
| Report field | Contents |
|---|---|
project.dispatchArtifactCount |
Number of source-scoped compile jobs in the deterministic plan. |
project.dispatchArtifactPlan |
Planned artifacts, dispatch variants, source units, stable identities, entry points, execution specialization, and provenance. |
artifacts[*].dispatchArtifact |
The exact planned job applied to an emitted or failed artifact record. |
artifacts[*].execution.provenance |
A closed reference back to the matching artifact-plan record. |
artifactMatrix.variantMode |
source-scoped when dispatch-derived and ordinary artifacts coexist. |
Report validation deterministically rebuilds the plan from the embedded contracts and discovered units, rejects unknown or tampered dispatch variants, and checks emitted execution metadata against the planned entry point and specialization. DirectX and OpenGL can emit source-scoped artifacts when their target contracts are representable. Requirements such as an exact subgroup width still fail closed on a target that cannot enforce them.
Named project variants cannot yet be composed with dispatch-derived variants; that work remains tracked in GitHub issue #1798. Imported contracts do not execute the host dispatch path, allocate or bind runtime resources, or establish numerical parity. Those responsibilities remain with repository integration and runtime adapters.
Project reports are JSON documents with:
- top-level metadata: report schema version, report kind, generation timestamp, and generator name/pipeline/package-version fields.
projectmetadata: root, config path, optional config hash, source roots, source-root status records and status counts, include/exclude patterns, targets, output directory, source override map, include directories, include-directory status records and status counts, define and variant maps, project and per-variant specialization constant maps, project and per-variant workgroup sizes, per-variant define and specialization constant counts, and counts for source roots, include patterns, exclude patterns, source overrides, include directories, defines, variants, and project specialization constants.summary: total unit/artifact/diagnostic/source-map counts plus rollups by unit source backend, unit file extension, skipped reason, skipped file extension, unit source override, skipped source override, artifact source backend, variant, target backend, source-map granularity, source-map target, source-map source backend, source-map variant, source-remap mapping count, source-remap granularity, source-remap target, source-remap source backend, source-remap variant, include dependency kind, include dependency status, include dependency source backend, include dependency source-backend status, include dependency resolution source, include dependency variant, artifact provenance pipeline, intermediate, source backend plus intermediate, target plus intermediate, variant plus intermediate, diagnostic severity (diagnosticCounts), diagnostic code (diagnosticsByCode), diagnostic target backend (diagnosticsByTarget), diagnostic source backend (diagnosticsBySourceBackend), diagnostic variant (diagnosticsByVariant), diagnostic check kind (diagnosticsByCheckKind), and missing capability (missingCapabilityCounts).units: discovered translation units with stable repository-relative POSIX paths, source backend names, path-derived extensions, source hashes, source byte sizes, and source overrides. Units that contain#includedirectives also includeincludeDependenciesrecords for project-level include triage. Include scans ignore directives inside C-style block comments while still recognizing active directives after same-line block comments. Resolved include dependencies record repository-relative include paths, resolution source, SHA-256 hashes, and byte sizes so report validation can detect include file content or size drift after scan. Full report validation also re-scans current source files and rejects missing or extra include dependency records. Recursive include scans stop at include cycles and emitproject.scan.include-cyclediagnostics withinclude.resolutionmissing-capability rollups while preserving the dependency that closes the cycle for triage.skipped: stable repository-relative POSIX paths for files intentionally left untranslated with reason codes and source override metadata when an override selected an unsupported source backend. Known unsupported source or binary artifact extensions are recorded withunsupported-extensionand a matching scan diagnostic so broad repository scans remain auditable. Full reports require skipped source override metadata to match the configured source override map.artifacts: attempted outputs with stable repository-relative POSIX source and output paths, source backend, target, applied define map, optional variant name, target/variant-scoped output path with the target backend suffix, status, source hash, source byte size, generated artifact hash, generated artifact byte size, pipeline provenance, and file-span source-map anchors for successful translations. Full reports require every artifact to carry a source hash and source byte size, artifact source hashes to match their declared translation-unit source hashes, artifact source byte sizes to match their declared translation-unit source byte sizes, artifact output paths to match the target/variant directory plus the source-relative path with the target backend suffix, artifact source paths to match declared translation units, unit source backend names to be registered canonical source backend names, unit source override metadata to match the configured source override map, and artifact source backend names to match those units. Full reports with translated or failed artifacts must include the expected artifact matrix for each declared translation unit, target, and configured variant. Full reports also require artifact define maps to match the project-level defines merged with the artifact variant's define overrides, and requiredefineProcessingmetadata to match the artifact define map, registered source frontend support, and summary rollups including named-variant rollups. Full reports also requireincludePathProcessingmetadata to match active include-directory records, registered source frontend support, and summary rollups including named-variant rollups. Artifacts with function or specialization constant declarations also carry dedicatedspecializationConstantsrecords for identity, required/default state, effective values, and value provenance, plusspecializationMaterializationmetadata that distinguishes native deferred specialization from a concrete CrossGL variant. Artifacts with a concrete workgroup-size contract carry anexecutionrecord with canonical dimensions, source entry points, provenance, and a deterministic identity. Full report validation rejects malformed dimensions, unknown variant provenance, or an identity that does not match the artifact source, target, variant, entries, and dimensions. Successful artifact records in full reports must include file-level source-map anchors. Generated CrossGL artifacts also include a compiler-compatiblesource-remapsidecar with a file-level generated/original mapping for compiler--source-remapconsumers. The source-map and source-remapoffset,length, andendOffsetfields are UTF-8 byte offsets. Source maps use a closed schema withfile,line,column,offset,length,endLine,endColumn, andendOffsetspan fields.mappingGranularitymay befile,line,statement, ortoken. File-granularity source maps must contain one mapping that exactly matches the artifact-level source and generated anchors. Finer-grained source maps keep those artifact-level anchors as file spans and may include one or more positive-length mappings whose source and generated files match the anchors. Line-preserving source and generated artifacts include line-granularity mappings with UTF-8 byte offsets. The report records the sidecar path, per-artifact mapping count, aggregate source-remap mapping count, hash, generated-file identity, summary rollups by target and source backend, and bounded inspection samples for source-map and source-remap artifacts with declared target, source and generated hash, and byte-size metadata. Validation checks that artifact-level source-map spans still cover the current source and generated files, recomputes line-preserving mappings, requires source-remap metadata mapping counts to match source-map mappings, and checks that compiler source-remap sidecars use the closed schema-1 field set. The project pipeline emits line-granularity source maps only when the generated artifact preserves the same logical lines after newline normalization, allowing a final-newline-only difference; translated artifacts keep file-granularity source maps until backend pipelines expose generated line, statement, or token provenance. Artifact provenance records thesingle-file-translatepipeline and usescrossglas the intermediate marker only when both source and target backends route through the CrossGL bridge. Report summaries and inspections include provenance rollups by pipeline, intermediate marker, source backend with intermediate marker, target with intermediate marker, and variant with intermediate marker, plus bounded artifact provenance samples for direct and bridge artifacts. Inspection samples include failed validation status, existence, hash, source-map, and source-remap status metadata when the validated artifact no longer matches the report. Metal artifacts that are materialized before translation can includetemplateMaterializationmetadata. That metadata records whether materialization succeeded, configured template parameters, unsupported templates with missing parameter names, and concrete specializations with the original template name, materialized function name, parameter map, specialization source, and optionalhostNamefor source-instantiated kernels. Source-instantiated Metal artifacts can also include anaccountingobject.reachableSpecializationCountcounts unique concrete function and struct specializations selected for the artifact,dependencyDiscoveryWorkCountcounts uncached type-environment and type resolution charged separately from those specializations, andprunedCandidateCountrecords source-instantiation/template-declaration pairs from the former eager candidate space that were not selected. The same accounting object is included in a materialization-work budget diagnostic when all three counts are available. Report validation rejects missing, negative, boolean, or unknown accounting fields. Full reports require failed artifacts to carry an actionable error string and reject failed artifacts that claim generated hashes or source-map records. Full reports also reject translated artifacts that carry error metadata. Invalid project output directories are recorded as failed artifacts without writing files.artifactMatrix: scan and translation metadata with expected, emitted, translated, failed, missing, extra, and completion counts plus target, source-backend, and variant completion rollups for the unit, target, and variant matrix. Scan-only reports include the expected artifact plan with zero emitted, translated, and failed artifacts so automation can review planned outputs before artifact generation. Report inspection also includes sampled missing and extra artifact identities from report-provided or translation artifact-derived matrix metadata, and text inspection identifies which matrix source was used, so incomplete batch outputs are visible without opening every artifact record.externalCorpus: optional manifest-backed corpus accounting with declared entries, present/missing and discovered-unit status, source-backend and target rollups, valid/invalid manifest-entry counts, and translated/failed artifact outcome counts for manifest entries. Validation checks entry presence against the project root, checks discovered/source-backend fields against declared translation units, and rejects missing or inconsistent retained-entry and manifest-entry summary counts.diagnostics: structured diagnostics using severity, code, message, location, optionaloriginalLocation, target, source-backend, variant, and missing-capability fields aligned with the compiler diagnostic contract.locationidentifies the report or generated artifact span that produced the diagnostic, whileoriginalLocationpreserves the original repository source span when diagnostics are remapped through generated artifacts. Project-level include and define forwarding limitations are warnings, not translation failures. Scan-time#defineand#undefdirectives in translation units or resolved include files that shadow active project or selected variant define names are also reported as warnings; directives inside C-style block comments are ignored.validation: report contract checks, generated timestamp and generator metadata checks, report inspection summaries, failed source artifact checks, project metadata, target normalization, and config count checks including compact variant-name and variant-define-count inspection summaries, unit and skipped record shape checks, artifact record shape checks, source and generated hash checks, duplicate artifact identity checks, required source/generated hash, source-size, generated-size, and source-map/source-remap status fields for summarized validation artifacts, aggregate validation artifact and validation status summary counts, direct validation report project context, source report hash metadata, artifact target, source backend, variant, hash-status, source-size status, generated-size status, source-map status, source-remap status, toolchain status, toolchain-run status rollups, toolchain-run check-kind metadata, toolchain-run tool rollups, and a closed standalone validation-report field set, failed-artifact text with source-backend context plus non-OK hash, source-size, generated-size, source-map, and source-remap statuses, bounded validation artifact samples with source-backend context, bounded validation toolchain-run inspection samples, source-root and include-directory status record and count consistency checks, config hash shape and current-file checks, unit source hash and byte-size shape and current-file checks, full-report artifact matrix coverage and artifact define map checks, artifact define-processing metadata and status/target/source-backend/variant rollup checks, artifact include-path processing metadata and status/target/source-backend/variant rollup checks, artifact matrix emitted/translated/failed/missing/extra/completion count and target/variant rollup checks, full-report source-map granularity, target, source-backend, and variant rollup checks, source-remap granularity, target, source-backend, and variant rollup checks, source hash and source byte-size checks, source-size validation status checks, generated artifact byte-size checks, failed artifact error metadata checks, translated artifact error metadata rejection, required artifact provenance and provenance value checks, artifact provenance source-backend, target, and variant rollup checks, failed artifact generated metadata rejection, required translated artifact source maps, required CrossGL artifact source remaps, source-map record shape, non-empty mapping list, file-level mapping cardinality, positive-length finer-grained mappings, finer-grained mapping containment within artifact-level anchors, span consistency, anchor consistency, current file-level source-map span coverage, source-remap metadata shape, mapping-count consistency, sidecar hash and byte size, closed compiler sidecar field sets, and sidecar content checks, external corpus record, per-entry artifact count, required manifest-entry accounting, and summary checks, summary consistency checks, migration action shape, rollup, and target declaration checks, preserved diagnostic shape, repository-relative file path, location andoriginalLocationspan consistency, target declaration checks, diagnostic severity rollup checks, scan-scope count consistency, diagnostic check-kind rollup consistency, validation toolchain status consistency checks, validation artifact and toolchain run record shape and duplicate identity checks, validation artifact coverage, required validation summary records for embedded validation artifacts, embedded toolchain-run coverage for available toolchains, failed embedded toolchain-run diagnostics, toolchain-run target, source-backend, check-kind, tool, and variant rollups, toolchain target coverage and status consistency checks, include dependency record shape and include dependency summary consistency, current include dependency status, resolved-path, resolved-hash, resolved-size, source-backend status rollup, resolution-source checks, and project-define include provenance checks, current include-scan diagnostic presence checks, resolved and unresolved include inspection samples, artifact source, source-backend, target, variant, and source-relative output layout declaration checks, current project scan coverage for omitted unit and skipped-source records, translated artifact existence checks, escaped output directory and artifact-path checks, source artifact existence and hash mismatch checks, generated artifact hash and byte-size mismatch checks, optional external toolchain availability, and opt-in toolchain smoke results including bounded timeout failures.migration: actionable manual follow-up work outside shader/kernel translation. The report records documented non-goals for runtime API migration, build-system rewrites, and backend framework integration. Each action has a documented kind, severity, message, and target list, plus action count and kind, severity, target, and runtime-reference rollups. Scan-only reports include supported requested targets when translation units are present. Translation reports scopemanual-runtime-integrationto targets that produced translated artifacts, covering host runtime API, resource binding, build script, and backend integration review. Runtime actions can includeruntimeReferencesentries for detected host or build files, with repository-relative path, line, column, backend, kind, and symbol metadata. Reports also include runtime-reference count, backend, kind, and path rollups so inspection tools can summarize host integration evidence without parsing each action. These references are evidence for follow-up integration work; they are not host-code rewrites. Reports with unresolved system include dependencies also emitmanual-include-resolutionactions so target SDK or toolchain header assumptions remain visible without claiming automatic header rewriting.