feat: one-command engineer entrypoint — scan → signed bundle → the auditor's command - #49
Open
bobrapp wants to merge 1 commit into
Open
feat: one-command engineer entrypoint — scan → signed bundle → the auditor's command#49bobrapp wants to merge 1 commit into
bobrapp wants to merge 1 commit into
Conversation
…tor's command
The honest answer to "how do I get a signed AI inventory?" was: install Node,
compile a native SQLite addon, run a server, run a Vite dev server, walk a
five-step wizard, then click Export. That is a good afternoon for a governance
lead in a workshop. It is far too much for an engineer deciding whether this is
worth their time — and the first step could fail outright, because the server's
SQLite driver does not build on current Node.
python3 -m src.beacon_scan # or `beacon-scan` once installed
now does the whole loop: scan -> signed receipts -> audit bundle -> the exact
command the auditor runs, printed at the end.
Python on purpose. `cryptography` is a wheel on every platform Beacon supports
with nothing to compile, so this path cannot hit the native-module wall the Node
quickstart hits. It builds on the beacons/ collectors that are already the
supported way to observe AI activity; it does not touch the server, the
database, or the Studio.
The bundle is byte-compatible with server/src/services/export.js — same
manifest fields, same layout, same shipped verifier — so starting here does not
put anyone on a side track, and an auditor cannot tell which producer wrote it.
`inventory.json` is DERIVED from the receipts rather than hand-maintained, so no
inventory row can exist that signed evidence does not support.
Also fixed: beacons/{model,artifact,prompt}_beacon.py did `from _common import
...`, which only worked when run as a script from inside beacons/. As package
members they raised ModuleNotFoundError, so nothing could build on the scanners
programmatically even though pyproject declares `beacons` a package. They now
work both ways, and beacons/README.md's documented `python beacons/x.py` still
works.
Two honesty properties, both tested:
* "never looked" is never rendered as "looked and found nothing". model_beacon
imports fine without psutil and its scan() then returns an empty list, so the
entrypoint checks the CAPABILITY, not the import, and reports
`egress scan: SKIPPED — psutil is not installed`.
* an empty scan is a valid answer, not an error path, and says out loud that a
clean scan is not evidence of coverage.
tests/unit/test_beacon_scan.py (11 tests) crosses the producer/verifier boundary
deliberately — a producer that verifies its own output proves nothing:
* the bundle passes src/beacon_verify.py, the AUDITOR's tool, run as a
subprocess the documented way
* it still passes with `cryptography` stubbed out to raise on import, so
VERIFY.md's "a bare Python 3 is enough" promise is exercised rather than
assumed (asserts the report says `backend: pure-python`)
* a tampered receipt is REFUSED — otherwise the check is worthless
* manifest.sha256 is the digest of manifest.json's BYTES, so `sha256sum -c`
passes (an earlier exporter hashed the canonical form instead, which failed
that command on every otherwise-intact bundle)
* receipts carry no file contents — planted file text must not appear anywhere
in the NDJSON
Verified on this branch:
python3 -m pytest 78 passed, 3 skipped (the 3 are lab-service parity
tests needing npm install in lab-service/; the Node
E2E layer RAN rather than self-skipping)
python3 verify.py All checks passed
end-to-end by hand scan -> bundle -> `python3 verify_bundle.py .` -> OK,
exit 0; `shasum -a 256 -c manifest.sha256` -> OK
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
Adds a new Python-first engineer entrypoint (beacon-scan) that performs an end-to-end loop (scan → signed receipts → audit bundle) and finishes by printing the exact auditor verification command, reducing reliance on the Node/server path for first-time usage.
Changes:
- Introduces
src/beacon_scan.pyCLI to scan provenance markers + optional egress, then assemble a server-shaped audit bundle. - Adds end-to-end unit tests that verify produced bundles using the auditor tool (
src/beacon_verify.py) as a subprocess. - Updates docs and packaging to advertise and expose the new
beacon-scanscript, and fixesbeacons/*_beacon.pyimports to work as package members and as direct scripts.
Reviewed changes
Copilot reviewed 8 out of 8 changed files in this pull request and generated 8 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/unit/test_beacon_scan.py | New end-to-end tests validating beacon-scan output via the auditor verifier and key properties (tamper detection, privacy posture, empty scan semantics). |
| src/beacon_scan.py | New CLI implementation: scanning, inventory derivation from receipts, bundle assembly, and user-facing output with auditor command. |
| README.md | Documents the new “one command” signed inventory path and expected output. |
| QUICKSTART.md | Adds a “short path first” section pointing engineers to python3 -m src.beacon_scan. |
| pyproject.toml | Registers the new beacon-scan console script entrypoint. |
| beacons/prompt_beacon.py | Fixes imports so the beacon works both as beacons.* modules and as direct scripts. |
| beacons/model_beacon.py | Same import fix for the model beacon. |
| beacons/artifact_beacon.py | Same import fix for the artifact beacon. |
Suppressed comments (1)
tests/unit/test_beacon_scan.py:212
- This test docstring starts with four quote characters, which leaves an extra leading '"' in the docstring text. It’s harmless but looks accidental and makes grepping/doc rendering noisy.
""""Never looked" must never render as "looked and found nothing"."""
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+214
to
+230
| from cryptography.exceptions import InvalidSignature | ||
| from cryptography.hazmat.primitives.asymmetric.ed25519 import Ed25519PublicKey | ||
|
|
||
| pub = Ed25519PublicKey.from_public_bytes(raw_public_key) | ||
| ok = 0 | ||
| failures: list[str] = [] | ||
| for r in receipts: | ||
| body = {k: v for k, v in r.items() if k != "signature"} | ||
| canonical = _common.canonicalize(body).encode("utf-8") | ||
| sig = base64.b64decode(r.get("signature", {}).get("sig_b64", "")) | ||
| try: | ||
| pub.verify(sig, canonical) | ||
| ok += 1 | ||
| except (InvalidSignature, ValueError): | ||
| if len(failures) < 10: | ||
| failures.append(r.get("id", "<no id>")) | ||
| return { |
| row["first_seen_utc"] = ts | ||
| if ts and ts > row["last_seen_utc"]: | ||
| row["last_seen_utc"] = ts | ||
| return sorted(rows.values(), key=lambda r: (r["vendor"], r["model"], r["version"])) |
|
|
||
|
|
||
| def test_empty_scan_still_produces_a_valid_bundle(tmp_path, monkeypatch): | ||
| """"Nothing found" is a real answer and must not be an error path.""" |
| "bundle_path": str(bundle), | ||
| "manifest_sha256": manifest_sha, | ||
| "verifier_included": verifier_included, | ||
| "public_key_fingerprints": [hex_fpr], |
Comment on lines
+265
to
+277
| # Public key. Named with the 16-hex-character fingerprint, not the SSH-style | ||
| # one from docs/RECEIPT_SCHEMA.md — the SSH form is base64 and can contain | ||
| # `/`, which is not a filename. beacon_verify indexes a key under every | ||
| # spelling plus its filename stem, so receipts carrying the SSH-style | ||
| # `key_fpr` still resolve to this file. | ||
| _priv, pub = _common.load_or_create_keypair() | ||
| raw_pub = pub.public_bytes( | ||
| encoding=serialization.Encoding.Raw, | ||
| format=serialization.PublicFormat.Raw, | ||
| ) | ||
| hex_fpr = hashlib.sha256(raw_pub).hexdigest()[:16] | ||
| (bundle / "public_keys" / f"{hex_fpr}.pem").write_bytes(_public_key_pem(raw_pub)) | ||
|
|
Comment on lines
+300
to
+302
| "active_key_fingerprint": _common.key_fingerprint(pub), | ||
| "public_key_fingerprints": [hex_fpr], | ||
| "signing_algorithm": "Ed25519", |
Comment on lines
+42
to
+45
| 0 a bundle was written and every receipt in it verifies | ||
| 1 the bundle was written but self-verification found a problem | ||
| 2 usage error, or nothing could be scanned | ||
| 3 key material or signing backend unavailable |
| from beacons import _common | ||
|
|
||
|
|
||
| BEACON_VERSION = "0.1.0" |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
HOLD — open for Bob's review, not to be merged by the agent.
The friction this removes
The honest answer to "how do I get a signed AI inventory?" was: install Node, compile a native SQLite addon, run a server, run a Vite dev server, walk a five-step wizard, click Export. That is a good afternoon for a governance lead in a workshop. It is far too much for an engineer deciding whether this is worth their time — and the first step could fail outright, because the server's SQLite driver does not build on current Node (see #48).
pip install cryptography python3 -m src.beacon_scan # or `beacon-scan` after `pip install -e .`does the whole loop: scan → signed receipts → audit bundle → the exact command the auditor runs, printed at the end.
Why Python
cryptographyis a wheel on every platform Beacon supports, with nothing to compile — so this path cannot hit the native-module wall the Node quickstart hits. It builds on thebeacons/collectors that are already the supported way to observe AI activity, and touches neither the server, the database, nor the Studio.The bundle is byte-compatible with
server/src/services/export.js— same manifest fields, same layout, same shipped verifier — so starting here does not put anyone on a side track, and an auditor cannot tell which producer wrote it.inventory.jsonis derived from the receipts rather than hand-maintained, so no inventory row can exist that signed evidence does not support.Drive-by fix
beacons/{model,artifact,prompt}_beacon.pydidfrom _common import ..., which only worked when run as a script from insidebeacons/. As package members they raisedModuleNotFoundError— so nothing could build on the scanners programmatically, even thoughpyproject.tomldeclaresbeaconsa package. They now work both ways, andbeacons/README.md's documentedpython beacons/x.pystill works.Two honesty properties, both tested
model_beaconimports fine withoutpsutiland itsscan()then returns an empty list. The entrypoint checks the capability, not the import, and reportsegress scan: SKIPPED — psutil is not installed.The tests cross the producer/verifier boundary on purpose
A producer that verifies its own output proves nothing.
tests/unit/test_beacon_scan.py(11 tests):src/beacon_verify.py— the auditor's tool — run as a subprocess the documented waycryptographystubbed out to raise on import, so VERIFY.md's "a bare Python 3 is enough" promise is exercised rather than assumed (asserts the report saysbackend: pure-python)manifest.sha256is the digest ofmanifest.json's bytes, sosha256sum -cpasses (an earlier exporter hashed the canonical form instead, which failed that command on every otherwise-intact bundle)Gate
python3 -m pytestlab-serviceparity tests needingnpm installinlab-service/. The Node E2E layer ran rather than self-skipping.python3 verify.pypython3 verify_bundle.py .→ OK, exit 0;shasum -a 256 -c manifest.sha256→ OKNote for review
This is additive — no existing entrypoint changed behaviour. It stacks cleanly with #48 but does not depend on it.
🤖 Generated with Claude Code