Context — strength to develop
jrag’s only decisive win was blast-radius (D=0.53 vs A=0.37; D caps 4× vs A’s 12× and B’s 22×) — the one category where grep structurally can’t finish. But blast-radius is only 5 of 50 questions; ~35 of 50 are greppable lookups (“find the X that implements/calls Y”) where grep is competitive or better. The current mix under-samples jrag’s home turf.
Work
- Add graph-native questions where grep structurally struggles: deep transitive impact (>2 hops), cross-cutting concerns, “what is affected if X changes across services,” dead-code / orphan detection, dependency-layering questions.
- Keep the greppable baseline questions (they honestly show where grep is enough), but rebalance so graph-native is not ~10% of the set.
- Regenerate oracle expected-answers for the new questions (jqassistant-grounded where possible).
Acceptance
- New question count + categories documented in
PREREGISTRATION.md (inventory) with an amendment.
- Either jrag’s edge broadens (good) or the benchmark honestly shows the edge stays narrow (also good — both are credible).
Context — strength to develop
jrag’s only decisive win was blast-radius (D=0.53 vs A=0.37; D caps 4× vs A’s 12× and B’s 22×) — the one category where grep structurally can’t finish. But blast-radius is only 5 of 50 questions; ~35 of 50 are greppable lookups (“find the X that implements/calls Y”) where grep is competitive or better. The current mix under-samples jrag’s home turf.
Work
Acceptance
PREREGISTRATION.md(inventory) with an amendment.