Skip to content

Cover the argument parses that produce a different run rather than an error - #681

Merged
hahwul merged 1 commit into
mainfrom
hahwul/test-increase-coverage
Aug 13, 2026
Merged

Cover the argument parses that produce a different run rather than an error#681
hahwul merged 1 commit into
mainfrom
hahwul/test-increase-coverage

Conversation

@hahwul

@hahwul hahwul commented Aug 13, 2026

Copy link
Copy Markdown
Owner

91 examples over six seams that had none, picked by what they cost when wrong. Every one of them can be wrong and still exit 0: a flag that mis-parses does not raise, it sends different bytes, stores the rule somewhere else, or no-ops a mutation while reporting success.

file examples what it pins
spec/cli/run/rewriter_spec.cr 23 gori run rewriter — 710 lines that had no spec at all
spec/cli/run/fuzz_args_spec.cr 22 the payload flag parsers fuzz and discover share
spec/settings_scan_rules_spec.cr 20 the global scan-rule library
spec/cli/run/probe_helpers_spec.cr 14 probe's flag parses + rule-id decoding
spec/cli/run/list_leftovers_spec.cr 8 the discarded-verb refusal twelve list dispatchers share
spec/rules_spec.cr +4 Rules.normalize_shape

How these were picked

Crystal has no working coverage tooling (kcov is Linux-only, crystal-coverage is dead), so this used an identifier-index heuristic instead: tokenise all of spec/, then count how many of each source file's def names appear in that set.

The useful result was the negative one. gori's public surface is already essentially covered — of 581 source files, exactly one (settings/scan_rules.cr) had its entire public def self. surface unmentioned anywhere under spec/. So the remaining gaps are in two places, and all six seams here come from them:

  • the private def self. helpers inside cli/run/*.cr, whose names never appear in a spec because the parsing and formatting are all private
  • functions whose own comment says they were split out to be spec-able — Run.list_leftover_error says exactly that and had zero coverage

What each one costs when wrong

rewriter — the scope letter leads a rule row because the project and global stores number independently, so #3 alone does not say which rule the next command addresses. The JSON projection is what a script reads, and default_enabled beside enabled is the only way that script can tell an override from a library default. Also the two refusals add makes before a rule reaches a store: an unparseable stub (it would answer every matching request with gori's own 502) and --part=ws on an op that is not replace (normalising it would move the rule to a different PROTOCOL, silently).

fuzz/discover flags — each has already been wrong in a way that produced a quietly different payload set: --numbers -10--5 split on the first hyphen and lost its lower bound, --regex-replace /foo//bar/ dropped everything past the second delimiter, --brute split its charset on the wrong colon. Checked through what each source actually generates, since none of them expose their fields.

list_leftover_error — what stops gori run rewriter --project=t1 rm 1 from listing the rules and exiting 0 with the delete discarded, and what keeps a lone list from being called an unknown subcommand by a message that lists list among the verbs.

probe rule idscustom_p_12 is a row in this project's DB, custom_g_12 lives in settings.json. Reading the global one as a row id would send a delete at whatever project row happened to share the number.

scan_rules — its parse is the boundary a hand-edited settings.json crosses into the match engine, which reads side/region/kind/severity as though they were enums, so the clamp is the contract. A rule missing id/title/pattern is dropped rather than defaulted, because an invented pattern would scan live traffic for it. The delete of the LAST rule is checked on disk, not through a reload: an emptied section stops being serialized, which is the exact shape that has made save's merge copy a stale block forward before.

Limits

The abort branches remain out of reach — abort calls exit, which would take the suite down — so these cover success paths only, the limit spec/cli/run/links_spec.cr already works under. Reaching the refusal paths would need a harness that runs the built binary as a subprocess; there isn't one today, and adding it was left out of scope.

Verification

  • crystal spec: 8592 examples, 8 failures — the same 8 as before this branch (project_registry ×5, capture_status ×3), all from a gori already listening on :8070. No new failures.
  • crystal tool format --check and ameba: clean on every file touched.
  • Source is untouched; this is spec-only, plus a refresh of the now-stale split-status header in spec/cli_run_spec.cr.

… error

91 examples over six seams that had none, picked by what they cost when wrong.
Every one of them can be wrong and still exit 0: a flag that mis-parses does not
raise, it sends different bytes, stores the rule somewhere else, or no-ops a
mutation while reporting success.

`gori run rewriter` had no spec at all — 710 lines managing Match & Replace and
the `extract` sub-CRUD that mints session bindings. The scope letter leads a rule
row because the project and global stores number independently, so `#3` alone
does not say which rule the next command addresses; the JSON projection is what a
script reads, and `default_enabled` beside `enabled` is the only way that script
can tell an override from a library default. Both are now pinned, along with the
two refusals `add` makes before a rule reaches a store: an unparseable stub (it
would answer every matching request with gori's own 502) and `--part=ws` on an op
that is not `replace` (normalising it would move the rule to a different
PROTOCOL, silently).

The payload flag parsers `fuzz` and `discover` share decide what leaves the
machine, and each has already been wrong in a way that produced a quietly
different set: `--numbers -10--5` split on the first hyphen and lost its lower
bound, `--regex-replace /foo//bar/` dropped everything past the second delimiter,
`--brute` split its charset on the wrong colon. Checked through what each source
actually generates, since none of them expose their fields.

`Run.list_leftover_error` — the seam behind the twelve list dispatchers, split out
of `refuse_list_leftovers` precisely so the decision and the wording could be
spec'd, and then never spec'd. It is what stops `gori run rewriter --project=t1
rm 1` from listing the rules and exiting 0 with the delete discarded, and what
keeps a lone `list` from being called an unknown subcommand by a message that
lists `list` among the verbs.

`probe`'s flag parses, and the rule-id decode that says which STORE an id
addresses: `custom_p_12` is a row in this project's DB, `custom_g_12` lives in
settings.json. Reading the global one as a row id would send a delete at whatever
project row happened to share the number.

`Settings.scan_rules` was the only file in the tree whose entire public API went
unmentioned anywhere under spec/. Its parse is the boundary a hand-edited
settings.json crosses into the match engine, which reads side/region/kind/severity
as though they were enums — so the clamp is the contract, and a rule missing
id/title/pattern is dropped rather than defaulted, because an invented pattern
would scan live traffic for it. The delete of the LAST rule is checked on disk,
not through a reload: an emptied section stops being serialized, which is the
exact shape that has made `save`'s merge copy a stale block forward before.

`Rules.normalize_shape`, the one place a rule's {target, part} pair is settled and
every creating surface calls.

The `abort` branches remain out of reach — `abort` calls `exit`, which would take
the suite down — so these cover success paths only, the limit
spec/cli/run/links_spec.cr already works under. Suite: 8592 examples, no new
failures.
@hahwul
hahwul merged commit c516db9 into main Aug 13, 2026
6 checks passed
@hahwul
hahwul deleted the hahwul/test-increase-coverage branch August 13, 2026 15:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant