Skip to content

Design reproducible runtime artifacts - #77

Draft
Tenshock wants to merge 3 commits into
Hkshoonya:mainfrom
Tenshock:design/issue-60-runtime-artifact-design
Draft

Design reproducible runtime artifacts#77
Tenshock wants to merge 3 commits into
Hkshoonya:mainfrom
Tenshock:design/issue-60-runtime-artifact-design

Conversation

@Tenshock

Copy link
Copy Markdown
Contributor

Description

Documents the proposed reproducible runtime artifact architecture requested in Issue #60.

The design defines:

  • the initial CPU/CUDA target matrix and versioned matrix/manifest examples;
  • uv as the proposed lock tool, with target-specific PEP 751 locks and hash-verified offline wheelhouses;
  • complete runtime layouts that include the matching Python interpreter and do not modify system Python;
  • immutable payload manifests, SPDX SBOMs, SLSA provenance, and separate digest-bound verification attestations;
  • CI trust boundaries and package-consumer flows;
  • self-contained and split DEB/RPM lifecycle alternatives, including Provides/Conflicts, upgrade, variant switching, rollback, and uninstall behavior; and
  • a staged implementation plan with explicit validation gates.

This is a design-only change. It does not add downloader, activation, UI, worker, installer, workflow, or runtime-changing code. It keeps requires-python = ">=3.11", preserves the OpenAI Whisper compatibility path, starts with CPU/CUDA, and leaves TensorRT and the DEB/RPM package-shape choice outside the first milestone.

References and standards sources are included in the design document.

Refs #60.

Type of Change

  • Bug fix
  • New feature
  • Performance improvement
  • Documentation update
  • Other (describe): Design proposal

Checklist

  • I have tested my changes locally
  • My code follows the existing code style
  • I have NOT removed or modified any author attribution (doczeus copyright headers, UI credits, LICENSE)
  • I have added comments for any non-obvious logic (not applicable; no executable logic is added)
  • I have updated documentation if needed

Screenshots / Video (if UI change)

Not applicable; this PR changes documentation only.


All contributions to NVIDIA Broadcast for Linux are reviewed by maintainers before merging.
By submitting this PR, you agree to the GPL-3.0 License terms.

@Hkshoonya

Copy link
Copy Markdown
Owner

I reviewed the design against current main and the current uv 0.11.7 CLI/documentation. The ownership boundaries, staged CPU/CUDA scope, private-interpreter gate, package-manager separation, and no-network assembly model are the right foundation.

Two specification gaps need to be resolved before this moves out of draft:

  1. The signing flow and evidence subject currently contradict each other. The trust diagram runs lifecycle, hardware, and rebuild checks against the unsigned package, then signs/notarizes in the protected release step. RPM signing and macOS signing/notarization/stapling change the enclosing artifact bytes and therefore its digest, while the document also requires every verification statement to name the exact final artifact digest. Please choose and document one consistent model: either test the final signed/stapled artifacts, or attest the reproducible unsigned payload/content-set digest and add protected evidence that the final signed wrapper contains exactly that tested payload. Update the diagram, attestation schema, and acceptance tests together.

  2. faster-whisper is not actually an input to the example locks. On current main, it is installed out of band with --no-deps; neither meeting nor meeting-support declares it, so the shown extras cannot place it or its complete closure in the PEP 751 file. Make it a direct requirement in a maintained packaging extra, and define the exact scoped policy that removes only faster-whisper -> onnxruntime. Current uv supports scoped tool.uv.exclude-dependencies, which is more precise than replacing the package's complete metadata and lets the remaining upstream dependency metadata continue to resolve normally. The lock validation should prove that every other declared dependency remains present.

One implementation note should also be made explicit: if native-runtime includes bindings such as PyGObject that do not have an acceptable upstream wheel for a target, the controlled wheel-build step has to run before the --only-binary :all: lock/assembly path and publish that wheel as a pinned input. The current text allows this generally, but tying it to the private-interpreter prototype will prevent the sample commands from being read as directly executable.

With those points tightened, this becomes an actionable Stage 0 design rather than only a direction of travel.

@Tenshock
Tenshock force-pushed the design/issue-60-runtime-artifact-design branch from a4e1012 to 6210346 Compare August 25, 2026 19:47
@Tenshock

Copy link
Copy Markdown
Contributor Author

The design now

  • tests finalized signed/notarized/stapled artifacts
  • defines the scoped faster-whisper -> onnxruntime exclusion with full closure validation
  • requires private-ABI wheels to be built and pinned before binary-only locking.

I updated trust diagram, attestation schema, and acceptance tests were updated accordingly.

@Hkshoonya Hkshoonya left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I re-read the revised design against the three points in the earlier review. They are now resolved coherently:

  • finalized signed/notarized/stapled artifacts are the exact lifecycle and hardware-test subjects, while unsigned payload reproducibility remains separate evidence;
  • faster-whisper==1.2.1 is a direct managed-runtime input, only its onnxruntime edge is excluded, and acceptance tests require the rest of its declared closure;
  • missing private-ABI wheels must be built from pinned inputs with provenance before binary-only lock generation.

That removes the specification blockers I raised. Keeping this as a draft is still correct because the private-interpreter supplier and the DEB/RPM package shape remain explicit Stage 0 choices that need evidence-backed prototypes. The next useful step is those non-release prototypes and oldest-platform results, not merging the design into v1.5.2.

Define target-specific locks, complete runtime payloads, interpreter ownership, package lifecycle alternatives, trust boundaries, and staged release gates for issue Hkshoonya#60.
@Tenshock
Tenshock force-pushed the design/issue-60-runtime-artifact-design branch from 6210346 to 202ac61 Compare September 1, 2026 17:04
@Tenshock

Tenshock commented Sep 1, 2026

Copy link
Copy Markdown
Contributor Author

Added the v1.5.2 evidence from Issue #60:

  • Independent Snap build digests and the distinction between provenance and reproducibility are documented under deterministic assembly.
  • Connected Snapcraft builder behavior is documented under CI trust boundaries.
  • Stage 2 and acceptance tests now require complete builder inventory and reject artifacts published outside the protected promotion flow.

I can start a separate linux-x86_64-cp313-cpu private-interpreter feasibility prototype that will cover: archive verification, relocation, isolated import paths, private-ABI PyGObject/pycairo bindings, dependency closure, and runtime size.

Which exact minimum releases should be used for acceptance testing across the advertised linux distributions (Debian/Ubuntu, Fedora/RHEL-family,etc)? Do I test on latest LTS or stable releases where available?

@Hkshoonya

Copy link
Copy Markdown
Owner

Use the oldest release we are prepared to support, plus a current release. Testing only the newest LTS would miss the compatibility floor that this prototype is meant to prove.

For the first linux-x86_64-cp313-cpu feasibility pass, use this matrix:

  • Ubuntu 22.04 LTS as the Ubuntu floor, with Ubuntu 26.04 LTS as the current endpoint. We should also add 24.04 to the release matrix before shipping, but it does not need to block the first relocation/import prototype.
  • Debian 12 as the Debian floor while it remains under LTS, with Debian 13 as current stable.
  • Fedora 43 as the current Fedora floor and Fedora 44 as the current release. Make this entry lifecycle-driven: Fedora 43 reaches EOL in December 2026, so the floor should advance rather than remain hard-coded indefinitely.
  • Enterprise Linux 9 as the minimum major family. Rocky Linux 9 is suitable for the initial freely reproducible CI prototype, but passing there must not be treated as proof for RHEL, AlmaLinux, or CentOS Stream. Each name we advertise still needs its own dependency-transaction validation before release.
  • openSUSE Leap 16.0 only after the Zypper dependency mapping is explicit. Keep that as a separate adapter result rather than inferring it from Fedora RPM success.

Do not include Ubuntu derivatives, rolling distributions, NixOS, aarch64, CUDA, or TensorRT in this first prototype. The Stage 0 result should prove archive verification, relocation, isolated import paths, private-ABI PyGObject/pycairo, complete dependency closure, and size on clean systems. GUI launch, camera/audio devices, v4l2loopback, package upgrade/removal, and CUDA execution remain later release gates on real systems.

Please record the image identifiers and update dates with the results so the evidence is reproducible. If one floor fails because of a native library rather than the private interpreter itself, report that boundary instead of relaxing or hiding it; then we can either add a separate cell or narrow the advertised package support honestly.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants