Skip to content

Add planning-effort counters and prepare-stage timer to profile notes - #16

Merged
bhensley5 merged 1 commit into
bhensley5:mainfrom
heunghingwan:feat/planner-work-counters
Jul 3, 2026
Merged

Add planning-effort counters and prepare-stage timer to profile notes#16
bhensley5 merged 1 commit into
bhensley5:mainfrom
heunghingwan:feat/planner-work-counters

Conversation

@heunghingwan

Copy link
Copy Markdown
Contributor

Builds on #15. When profile: true (with include_plan: true), the
existing stage-timing note in GqlExecutionExplain.notes also carries:

  • a prepare stage timer, covering the GQL work between lowering and the
    published snapshot (order-by resolution, row-count evaluation, target
    configuration) — previously the only uninstrumented gap around the
    graph-row call;
  • planning-effort counters: node_legal_universe_sources (legal-universe
    candidate sources enumerated across node plans), edge_source_consults /
    edge_source_misses (edge-source cost memo use), and
    secondary_index_followups (secondary-index read followups enqueued).

Why

The stage timers from #15 isolate the graph-row call as the dominant bucket
but say nothing about planning effort. These counters are a cheap
regression canary for planner work — they surface how many candidate
sources the planner evaluated, so a change that balloons enumeration (new
index types, planner refactors) is visible before it ships. Filling the
prepare gap also means the graph-row bucket is the only unexplained
stage, so its isolation is exact.

Design (same contract as #15)

  • No public struct / connector changes — values ride the existing
    notes: Vec<String> line.
  • Does not touch the shared read substrate (read.rs, query_exec.rs,
    ReadView/SourceList); the native hot path is unaffected.
  • Counters are produced via a crate-local thread-local the planner
    increments and the GQL layer snapshots only when profiling.
  • graph_row_plan_and_execute remains a single combined bucket; splitting
    it (plan vs probe vs execution) is intentionally out of scope here.

Overhead

profile: false is unchanged: the timer mark() returns None and
counter values are neither snapshotted nor surfaced. The planner always
increments the thread-local; measured fire count is 3–8 per query
(~22–57 ns on the default path, ~0.001–0.005% of read latency), confirmed
by reading the values back through the note.

Tests

New gql_query_profile_planning_counters_in_explain_note asserts the note
carries every counter/timer field, that a single-node query enumerates

=1 universe source, and that nothing is attached when profile is off.
cargo test --lib gql (358 tests) and cargo clippy --lib are clean.

Extends the profile-gated instrumentation (bhensley5#15) with per-query
planning-effort counters and a `prepare` stage timer.

- `prepare` timer fills the gap between lowering and the published
  snapshot (order-by resolution, row-count evaluation, target config),
  the only previously uninstrumented stage around the graph-row call.
- Counters: node_legal_universe_sources, edge_source_consults,
  edge_source_misses, secondary_index_followups.

Counters use a crate-local thread-local the planner increments (always)
and the GQL layer snapshots when profiling; values ride the existing
GqlExecutionExplain.notes line, so no public struct or connector changes
and the shared read substrate is untouched.

profile=false is unchanged: timer mark() returns None and counter values
are neither snapshotted nor surfaced. Measured fire count is 3-8 per query
(~22-57 ns, ~0.001-0.005% of read latency).
@bhensley5

Copy link
Copy Markdown
Owner

Thanks for the follow-up. I reviewed this on top of #15 and the scope still looks right. It keeps the profiling data on the existing explain notes surface, avoids public/connector API changes, and adds useful planner-effort visibility.

Merging via squash.

@bhensley5
bhensley5 merged commit 0adfd2b into bhensley5:main Jul 3, 2026
3 checks passed
@heunghingwan
heunghingwan deleted the feat/planner-work-counters branch July 3, 2026 20:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants