Repository navigation
fix(ingest): name the event types and sources no detector reads, not unknown - #103
Merged
Merged
Conversation
…unknown The bus ingest report called every event type outside idryx's knownTypes "unknown". knownTypes is the set a detector reads (frozen as events.consumed), so a registered type that no detector reads yet, such as identity_mismatch or dependency_failed, was reported as unknown. It is ingested as it is. The stderr line now says what is true: N event type(s) no idryx detector reads, ingested as they are, plus N source(s) no idryx detector reads, a source being one whose ingested events carry no type in knownTypes across the whole load. Names are sorted, escaped, cut at 64 bytes, and at most eight are listed, then "+K more". The formatter is tokenfuse.StderrSummary, so its branches are covered in that package; cmd/idryx makes one call. knownTypes and events.consumed are unchanged. The README sentence and the test badge (374 to 381, measured by scripts/readme-numbers.sh) follow. Co-Authored-By: Claude Haiku 5.5 <noreply@anthropic.com>
The case removed features/source-must-match-its-stream.feature by name and expected features-are-bound.sh to say it measured nothing. A second feature file, added for the unread-type report, kept scenarios alive, so the gate passed on the fault it exists to catch and the teeth run reported TOOTHLESS. The case means "no scenario left in features/", so it now removes every features/*.feature file. Measured on this tree: the case reports ok (fail) with the fix and TOOTHLESS without it. Co-Authored-By: Claude Haiku 5.5 <noreply@anthropic.com>
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What and why
The bus ingest called every event type outside
knownTypes"unknown".knownTypesis the set a detector reads, and it is frozen asevents.consumedincompat/1.0.json. A type the bus registry names but no detector reads yet (identity_mismatch,taint_raised,unit_cap_exceededanddependency_failedfrom tokenfuse,tool_callfrom mcp,policy_allowfrom wardryx) was therefore reported as unknown, although the registry names it and idryx ingests it as it is. The word was false, and the operator was told the wrong thing.The stderr line now says what is true, and names what it counts:
A source counts as read when any of its ingested events carries a type in
knownTypes, judged across the whole load. Names are sorted, escaped with%q, cut at 64 bytes, and at most eight are listed, then+K more. The formatter istokenfuse.StderrSummary, so its branches are covered ininternal/ingest/tokenfuse.cmd/idryxmakes one call.Not changed:
knownTypes,events.consumed, the frozen surface. No detector is added. Anidentity_mismatchdetector is a follow-up.Tier: T2 (user-visible stderr output).
Red first
Before the fix,
TestARegisteredTypeNoDetectorReadsIsNotCalledUnknown(cmd/idryx) fails on the assertion that the registered types are not called unknown:The formatter tests in
internal/ingest/tokenfusewere red on the unfixed code at compile time (undefined: StderrSummary,rep.Sources undefined), not by assertion.Mutants, planted one at a time in the formatter and the connector, each turned a test red, and the file was restored byte for byte:
TestStderrSummaryBoundsAndEscapesEveryNameTestAnUnreadSourceIsJudgedAcrossEveryFileOfALoadTestAnUnreadSourceIsJudgedAcrossEveryFileOfALoad%qescaping removed:TestStderrSummaryBoundsAndEscapesEveryNameTestStderrSummaryBoundsAndEscapesEveryNameSeven new test functions (the gate counts 374 before, 381 now). Scenarios in
features/unread-is-named-not-unknown.feature, four of them, each bound to a test.Teeth case changed
features-are-bound: no scenario leftremovedfeatures/source-must-match-its-stream.featureby name. The new feature file kept scenarios alive, so that case reported TOOTHLESS. The case now removes everyfeatures/*.featurefile, which is what "no scenario left in features/" means. This is a separate commit (test(gates): ...). Measured: ok (fail) with the change, TOOTHLESS without it.Coverage (
go test -cover, statements)internal/ingest/tokenfuse: 96.0% before, 97.0% after. The uncovered paths were already there and are not in the changed code:verifyChainerror path (85.7%),LoadWith(93.1%),parse(97.1%).cmd/idryx: 40.1% before, 40.1% after. Below the 90% bar and not raised here: the change in this package is one call, which the end-to-end test covers. The formatter lives intokenfusefor that reason.scripts/readme-numbers.shmeasures.Gates (local, go1.27.2; staticcheck under GOTOOLCHAIN=go1.27.1)
gofmt -l .,go vet ./...,go build ./...: clean.go test -race -count=1 ./...: every package ok.staticcheck2026.2.1: one finding on darwin only,procPidInitInounused (U1000) ininternal/ebpfcapture/self.go, used bycapture_linux.go.GOOS=linuxrun is clean; CI runs on Ubuntu. Not from this change.go test -tags integration: compiles. NeedsDATABASE_URL, not run here.readme-numbers(381),detectors-complete(27),ebpf-optional,diagrams-match-detectors,compat-surface(29 frozen names),features-are-bound(17 scenarios, 17 bindings),reproducible-build: all pass.govulncheckv1.7.0: no vulnerabilities.gosecv2.29.0: clean.gates-have-teeth.sh: 20 cases ok, 0 failed. 4object-matches-its-sourcecases NOT RUN here (they need clang 18; theebpfjob runs them).preflight.sh pr: clean.Ran it
idryx detect --load "tokenfuse:<dir>/*.ndjson"over a tokenfuse file with five types (one read, four not) and a typryx file with one valid and one malformed line:Exit 0. The chain verdict line follows, as before.
NOT proven
ebpfandintegrationCI jobs have not run on this branch.knownTypesmembership. Two cases fall outside it. (a) A bus event whose type is spelledlogin,egressormfa_challengereaches a detector through the model constant, but is not inknownTypes, so it is named as unread. No registry producer in SPEC 6.2 emits those. (b)claimed_agent_unattestedchecks only whether an event has any Source; that is not counted as reading a source.unrouted_egressreads scopyx by source equality. The rule here gives the same answer for scopyx's current types (web_fetch,web_blocked, both inknownTypes).identity_mismatch; that is a follow-up, not part of this change.🤖 Generated with Claude Code