gc events lies to a machine reader three ways: --since discards a timed-out page walk, a comma --type exits 0 with no records and no stderr, and the --json deprecation notice corrupts stdout JSONL (gc-378x4) - #174
Conversation
…ed-out page walk, a comma --type exits 0 with nothing, and the --json notice corrupts stdout JSONL (gc-378x4) Measured in loomington 2026-09-03, all three live: gc events --since 24h 0 records, exit 1, 30.9s wall gc events --since=24h --type=a,b,c 0 records, exit 0, EMPTY stderr gc events --json --type=session.woke stdout line 1 is the deprecation notice --since: doEvents built one 30s context and fetchCityEvents drained the whole window under it at 500 events per page. A page cost 0.18s at the head and ~3.7s deep in the walk, so a 24h window never fit; on the deadline the walk returned nil and every page it had already fetched was discarded. It now returns what it read, paired with an error naming the oldest seq and timestamp it reached, and doEvents prints those events, writes the boundary to stderr and exits non-zero. The city drain also gets a budget sized for a walk rather than for one request; the supervisor list issues a single request whatever the filters say, so it keeps the 30s guard. That same 24h ask now yields 41000 events over 10.5h with the boundary named, where it yielded nothing. --type: both filters that read it compare one exact string -- filterCityEvents tests item.Type != typeFilter, and the API's EventListInput.Type is a scalar query param -- so a comma list matches no event and the server mints no next_cursor. Rejected in RunE beside the existing --after rejection, which covers list, watch and follow. --json: MarkDeprecated makes pflag buffer a notice that cobra drains through Command.Print -> OutOrStderr, and cmd/gc/main.go points that writer at stdout, so it landed as line 1 of a stream documented as JSON Lines. MarkHidden keeps the flag the silent no-op it claims to be. TestEventsJSONFlagIsSilentNoOp could not catch this because it never wired cmd.SetOut; it does now. Checked and not a defect, so the next reader need not: --limit is already rejected as an unknown flag. The junk line attributed to it upstream is the --json notice above. docs/reference/events.md gains the --type cardinality rule and a Window Coverage section: a --since count is a floor, and a caller separating "none occurred" from "not fully read" has to check exit status. engdocs listed "No event retention or rotation. The JSONL file grows without bound" under Known Limitations; rotation is enabled by default and size-triggered, and nothing bounds the active log by time, which is why the reachable window varies with how fast the city emits. Reported as tk-0bltj in the gc-toolkit ledger. Claude-Session: https://claude.ai/code/session_01GDAdvz2wt3D7h4hUoqzqPH
The Known Limitations note claimed gc events visibility tracks the active file's emission rate (a busy city holds minutes, a quiet one days). The list paths read the active file plus every retained sibling archive: List -> ReadFiltered, and ListInFlight -> ReadFilteredWithInFlight for a segment still mid-rotation (fetchEventPageAscending -> listWithInFlight). A query therefore reaches the whole retained history, not just the active file. Rotation is size-triggered and only sets how much history the active file alone holds; how far back a query reaches is governed by archive retention (archive_retain_age, which keeps all archives when empty via reapExpiredArchives). Separate the two in the note accordingly. Addresses the pre-open signoff finding on gc-8au8k (review of polecat/gc-378x4). Pre-commit hook skipped: gc-m7rp0 makes it fail on this host for any diff; docsync gate run by hand under a coherent toolchain (ok). Claude-Session: https://claude.ai/code/session_0173HfdLWDyFjw1ukmH5BdaN
|
Pre-open signoff (comment-only — not an approval): VERDICT: approve Scope checked: Read the full three-dot diff from origin/main to 7385900 across cmd/gc/cmd_events.go, cmd/gc/cmd_events_test.go, docs/reference/events.md, and engdocs/architecture/event-bus.md. Checked anchor bead gc-378x4, review bead gc-qcu53, prior request-changes reviews gc-mp0qv and gc-8au8k, and the rework notes on gc-5zqvk and gc-ytmkf. Checked newEventsCmd, validateEventsType, doEvents, cityEventsListBudget, fetchCityEvents, fetchCityEventsAfterSeq, fetchSupervisorEvents, filterCityEvents, streamCityEvents, streamSupervisorEvents, the Huma city and supervisor event-list input/handler paths, generated event-list query params, FileRecorder.List/ListInFlight, ReadFiltered, ReadFilteredWithInFlight, rotation retention defaults, docs/reference events guidance, engdocs event-bus retention wording, the API control-plane/Huma docs, TESTING.md, the gascity-docs review rules, and the pack work-quality and learning-exemplar fragments. This is pre-open, so there is no PR page to review. I did not run the full project matrix or dashboard checks; the diff does not touch internal/api, OpenAPI, dashboard code, or generated dashboard types. Findings: none. Filed: none. Verification: At the reviewed commit in detached worktree /tmp/gc-review-gc-qcu53.4qaZPE, an initial isolated-cache Anchor: gc-378x4 — check.codex @ 7385900 |
…ed-out page walk, a comma --type exits 0 with no records and no stderr, and the --json deprecation notice corrupts stdout JSONL (gc-378x4) (#174) * gc events lies to a machine reader three ways: --since discards a timed-out page walk, a comma --type exits 0 with nothing, and the --json notice corrupts stdout JSONL (gc-378x4) Measured in loomington 2026-09-03, all three live: gc events --since 24h 0 records, exit 1, 30.9s wall gc events --since=24h --type=a,b,c 0 records, exit 0, EMPTY stderr gc events --json --type=session.woke stdout line 1 is the deprecation notice --since: doEvents built one 30s context and fetchCityEvents drained the whole window under it at 500 events per page. A page cost 0.18s at the head and ~3.7s deep in the walk, so a 24h window never fit; on the deadline the walk returned nil and every page it had already fetched was discarded. It now returns what it read, paired with an error naming the oldest seq and timestamp it reached, and doEvents prints those events, writes the boundary to stderr and exits non-zero. The city drain also gets a budget sized for a walk rather than for one request; the supervisor list issues a single request whatever the filters say, so it keeps the 30s guard. That same 24h ask now yields 41000 events over 10.5h with the boundary named, where it yielded nothing. --type: both filters that read it compare one exact string -- filterCityEvents tests item.Type != typeFilter, and the API's EventListInput.Type is a scalar query param -- so a comma list matches no event and the server mints no next_cursor. Rejected in RunE beside the existing --after rejection, which covers list, watch and follow. --json: MarkDeprecated makes pflag buffer a notice that cobra drains through Command.Print -> OutOrStderr, and cmd/gc/main.go points that writer at stdout, so it landed as line 1 of a stream documented as JSON Lines. MarkHidden keeps the flag the silent no-op it claims to be. TestEventsJSONFlagIsSilentNoOp could not catch this because it never wired cmd.SetOut; it does now. Checked and not a defect, so the next reader need not: --limit is already rejected as an unknown flag. The junk line attributed to it upstream is the --json notice above. docs/reference/events.md gains the --type cardinality rule and a Window Coverage section: a --since count is a floor, and a caller separating "none occurred" from "not fully read" has to check exit status. engdocs listed "No event retention or rotation. The JSONL file grows without bound" under Known Limitations; rotation is enabled by default and size-triggered, and nothing bounds the active log by time, which is why the reachable window varies with how fast the city emits. Reported as tk-0bltj in the gc-toolkit ledger. Claude-Session: https://claude.ai/code/session_01GDAdvz2wt3D7h4hUoqzqPH * docs(event-bus): correct gc events retention/visibility note (gc-ytmkf) The Known Limitations note claimed gc events visibility tracks the active file's emission rate (a busy city holds minutes, a quiet one days). The list paths read the active file plus every retained sibling archive: List -> ReadFiltered, and ListInFlight -> ReadFilteredWithInFlight for a segment still mid-rotation (fetchEventPageAscending -> listWithInFlight). A query therefore reaches the whole retained history, not just the active file. Rotation is size-triggered and only sets how much history the active file alone holds; how far back a query reaches is governed by archive retention (archive_retain_age, which keeps all archives when empty via reapExpiredArchives). Separate the two in the note accordingly. Addresses the pre-open signoff finding on gc-8au8k (review of polecat/gc-378x4). Pre-commit hook skipped: gc-m7rp0 makes it fail on this host for any diff; docsync gate run by hand under a coherent toolchain (ok). Claude-Session: https://claude.ai/code/session_0173HfdLWDyFjw1ukmH5BdaN --------- Co-authored-by: refinery costing <refinery@local>
…ed-out page walk, a comma --type exits 0 with no records and no stderr, and the --json deprecation notice corrupts stdout JSONL (gc-378x4) (#174) * gc events lies to a machine reader three ways: --since discards a timed-out page walk, a comma --type exits 0 with nothing, and the --json notice corrupts stdout JSONL (gc-378x4) Measured in loomington 2026-09-03, all three live: gc events --since 24h 0 records, exit 1, 30.9s wall gc events --since=24h --type=a,b,c 0 records, exit 0, EMPTY stderr gc events --json --type=session.woke stdout line 1 is the deprecation notice --since: doEvents built one 30s context and fetchCityEvents drained the whole window under it at 500 events per page. A page cost 0.18s at the head and ~3.7s deep in the walk, so a 24h window never fit; on the deadline the walk returned nil and every page it had already fetched was discarded. It now returns what it read, paired with an error naming the oldest seq and timestamp it reached, and doEvents prints those events, writes the boundary to stderr and exits non-zero. The city drain also gets a budget sized for a walk rather than for one request; the supervisor list issues a single request whatever the filters say, so it keeps the 30s guard. That same 24h ask now yields 41000 events over 10.5h with the boundary named, where it yielded nothing. --type: both filters that read it compare one exact string -- filterCityEvents tests item.Type != typeFilter, and the API's EventListInput.Type is a scalar query param -- so a comma list matches no event and the server mints no next_cursor. Rejected in RunE beside the existing --after rejection, which covers list, watch and follow. --json: MarkDeprecated makes pflag buffer a notice that cobra drains through Command.Print -> OutOrStderr, and cmd/gc/main.go points that writer at stdout, so it landed as line 1 of a stream documented as JSON Lines. MarkHidden keeps the flag the silent no-op it claims to be. TestEventsJSONFlagIsSilentNoOp could not catch this because it never wired cmd.SetOut; it does now. Checked and not a defect, so the next reader need not: --limit is already rejected as an unknown flag. The junk line attributed to it upstream is the --json notice above. docs/reference/events.md gains the --type cardinality rule and a Window Coverage section: a --since count is a floor, and a caller separating "none occurred" from "not fully read" has to check exit status. engdocs listed "No event retention or rotation. The JSONL file grows without bound" under Known Limitations; rotation is enabled by default and size-triggered, and nothing bounds the active log by time, which is why the reachable window varies with how fast the city emits. Reported as tk-0bltj in the gc-toolkit ledger. Claude-Session: https://claude.ai/code/session_01GDAdvz2wt3D7h4hUoqzqPH * docs(event-bus): correct gc events retention/visibility note (gc-ytmkf) The Known Limitations note claimed gc events visibility tracks the active file's emission rate (a busy city holds minutes, a quiet one days). The list paths read the active file plus every retained sibling archive: List -> ReadFiltered, and ListInFlight -> ReadFilteredWithInFlight for a segment still mid-rotation (fetchEventPageAscending -> listWithInFlight). A query therefore reaches the whole retained history, not just the active file. Rotation is size-triggered and only sets how much history the active file alone holds; how far back a query reaches is governed by archive retention (archive_retain_age, which keeps all archives when empty via reapExpiredArchives). Separate the two in the note accordingly. Addresses the pre-open signoff finding on gc-8au8k (review of polecat/gc-378x4). Pre-commit hook skipped: gc-m7rp0 makes it fail on this host for any diff; docsync gate run by hand under a coherent toolchain (ok). Claude-Session: https://claude.ai/code/session_0173HfdLWDyFjw1ukmH5BdaN --------- Co-authored-by: refinery costing <refinery@local>
Summary
gc eventshad three ways to hand a machine reader an empty answer that lookedlike a real one. All three were measured live in loomington on 2026-09-03, and
each is fixed with a regression guard that fails against
origin/main.--sincethrew away a walk it ran out of time for.doEventsbuilt one30s context and
fetchCityEventsdrained the entire window under it, 500events per page. A page cost 0.18s at the head of the log and ~3.7s deep in
the walk, so a 24h window never fit — and on the deadline the walk returned
nil, discarding every page it had already fetched. Thirty seconds ofsuccessful fetching printed nothing.
It now returns the pages it read, paired with an error naming the oldest seq
and timestamp it reached.
doEventsprints those events to stdout, writes thecoverage boundary to stderr, and exits non-zero, so a caller can tell a window
it searched from one it never reached. The city drain also gets a budget sized
for a page walk instead of for a single request; the supervisor list issues one
request whatever the filters say, so it keeps the 30s hang guard (pinned by its
own test). The same
--since 24hthat returned nothing now returns 41,000events over 10.5h with the boundary stated.
A comma-separated
--typematched nothing, silently. Both filters thatread the value compare one exact string —
filterCityEventstestsitem.Type != typeFilter, and the API'sEventListInput.Typeis a scalarquery param — so
--type=a,b,cmatched no event, the server minted nonext_cursor, and the command exited 0 with no records and an empty stderr.That is indistinguishable from "none of those events occurred", which is
precisely the question a coverage query is asking. It is now rejected in
RunE, beside the existing--afterrejection, so list, watch and follow allget it.
The
--jsondeprecation notice was written to stdout.MarkDeprecatedmakes pflag buffer the notice into cobra's
flagErrorBuf;ParseFlagsdrainsit through
Command.Print→OutOrStderr, andcmd/gc/main.gopoints thatwriter at stdout. The notice landed as line 1 of a stream this command
documents as JSON Lines, where
wc -lcounts it as an event andjqfails onit.
MarkHiddenmakes the no-op flag the silent no-op it claims to be.TestEventsJSONFlagIsSilentNoOpcould not catch this — it never calledcmd.SetOut, so the notice escaped to the process stderr rather than thebuffer it asserts on. It wires both writers now and checks that every stdout
line parses as JSON.
Checked and not a defect, recorded so the next reader need not re-check:
--limitis already rejected as an unknown flag with exit 1. The "single junkline" attributed to it in the originating report is the
--jsonnotice above,seen on a command line that also carried
--json.docs/reference/events.mdgains the--typecardinality rule and a WindowCoverage section stating that a
--sincecount is a floor and that callersdistinguishing "none occurred" from "not fully read" must check exit status.
The pre-commit hook could not run on this host for any diff: it exports
GOTOOLCHAIN,GOROOT_VALthen resolves to the toolchain root, andTEST_ENV'senv -iforwardsGOROOTwithoutGOTOOLCHAIN, socheck-docsdrives a 1.26.6
GOROOTwith the 1.27.0goon PATH. It reproduces on a cleantree at
origin/main; commented ongc-m7rp0, which owns that root cause. Thiscommit therefore used
--no-verifyafter each gate was run by hand:gofmtclean,
go buildandgo vetclean, everycmd_eventstest passing, thehook's own scoped lint (
LINT_CHANGED_SCOPE=staged --new-from-rev=HEAD --whole-files) reporting 0 issues, andcheck-docspassing under a coherenttoolchain.
Reported as
tk-0bltjin the gc-toolkit ledger; filed here asgc-378x4because the defect is in this repo and a
tk-bead cannot route to this rig'srefinery.
Dispatch — what this work was asked to do
Symptom (re-measured 2026-09-03 in loomington, all three confirmed live)
Filed from gc-toolkit tk-0bltj, which reported the first symptom. The store
boundary is why this bead exists: the defect is in this repo, tk-0bltj is in
the gc-toolkit ledger, and a tk- bead cannot be routed to this rig's refinery.
Three distinct defects
1.
--sincediscards a walk it ran out of time for.doEvents(cmd/gc/cmd_events.go) builds ONE 30s context and hands it tofetchCityEvents, which — only when--sinceis set — drains the whole windowat
cityEventsPageLimit(500) per page. One page measured 0.18s / 1.1MB / 8.6minutes of history, so a 24h window is ~167 pages and does not fit in 30s. When
the deadline fires mid-walk,
fetchCityEventsreturnsnil, err: every pagealready fetched is thrown away. 30 seconds of successful fetching prints
nothing.
2. A comma-separated
--typesilently matches nothing.--typeis one exact string on both sides — clientfilterCityEventscomparesitem.Type != typeFilter, and the API'sEventListInput.Typeis a scalarquery:"type".--type=a,b,ctherefore matches no event, and the server mintsno next_cursor, so the CLI exits 0 with zero records and an empty stderr. This
is the false-clean shape tk-0bltj was filed about: a caller cannot tell "no such
events occurred" from "that query could never match". It reproduces with or
without
--since.3. The
--jsondeprecation notice is written to STDOUT.MarkDeprecatedmakes pflag buffer the notice into cobra'sflagErrorBuf;ParseFlagsdrains it throughc.Print->OutOrStderr(), andcmd/gc/main.go:405doesroot.SetOut(stdout). The notice lands as line 1 ofa stream documented as JSON Lines, so
wc -lcounts it as an event andjqfails on it.
TestEventsJSONFlagIsSilentNoOpdoes not catch this: it nevercalls
cmd.SetOut, so in-test the notice escapes to the realos.Stderrinstead of the buffer it asserts on.
Not defects (checked, so the next reader does not re-check)
--limitis correctly rejected:gc: unknown flag: --limit, exit 1. The"single junk line" tk-0bltj attributes to
--limitis defect 3 above,observed on a command line that also carried
--json.sincefilter works. The client walk is what fails.Fix
fetchCityEventsreturns the pages it did fetch when the walk is cut short,with an error naming the coverage boundary;
doEventsprints those events,warns on stderr, and exits non-zero. A coverage query must be able to state
how far it actually looked.
--typevalue containing a comma, next to the existing--after/--after-cursorrejection, so list, watch and follow are allcovered. Same reasoning already recorded there: reject rather than silently
ignore.
MarkDeprecated("json")withMarkHidden("json")so the no-op flagis actually silent and stdout stays pure JSONL; strengthen
TestEventsJSONFlagIsSilentNoOpto wirecmd.SetOutso it can see theregression.
Refinery handoff
gc-378x4polecat/gc-378x4maincodexsigned off pre-open at73859001; PR opened green.