Skip to content

feat: report scoped source removals - #306

Merged
mikemiles-dev merged 2 commits into
mikemiles-dev:mainfrom
ktsaou:fix/scoped-source-removal-reporting
Jul 21, 2026
Merged

feat: report scoped source removals#306
mikemiles-dev merged 2 commits into
mikemiles-dev:mainfrom
ktsaou:fix/scoped-source-removal-reporting

Conversation

@ktsaou

@ktsaou ktsaou commented Jul 20, 2026

Copy link
Copy Markdown
Contributor

Summary

Add opt-in, synchronous reports for every implicit source removal performed by AutoScopedParser and RouterScopedParser.

Reports identify:

  • the exact removed source;
  • whether it was removed by capacity pressure, idle pruning, or a capacity reduction.

Problem

Scoped parsers bound memory by automatically removing source-specific child parsers. Applications can keep state associated with those sources, but the existing APIs discard the removed key. This leaves applications unable to remove their corresponding state accurately.

Design

  • Add *_with_reporter companions for parsing, iterator parsing, idle pruning, and capacity changes.
  • Keep every existing method as a source-compatible wrapper that discards reports.
  • Isolate reporter errors and unwind panics so a reporting failure cannot interrupt the completed parser-state transition.
  • Retain aggregate removal and reporter-failure counters after child parsers are gone.
  • Keep explicit remove_* methods unchanged as caller-owned parser transfers; they do not emit implicit-removal reports.
  • Preserve every existing TemplateStore operation exactly. This change neither adds nor removes persistence cleanup.

Compatibility

Existing callers require no changes. Reporting is opt-in through the companion methods.

Validation

  • 20 focused integration tests cover exact source/cause reporting, both parser APIs, all source-key variants, all three removal causes, error/panic isolation, malformed and existing-source no-op paths, global LRU selection, explicit removal, and every existing TemplateStore behavior.
  • Full scripts/check-all.sh suite passes.
  • Default and no-default-feature tests pass.
  • Rust 1.88 compatibility tests pass.

Performance

Paired, CPU-pinned, 11-trial base/candidate measurements show:

  • normal parsing: worst candidate/base throughput ratio 98.8%;
  • forced removal on every operation: worst compatibility-wrapper/base ratio 97.3%;
  • explicit reporting on forced removals: effectively equal to baseline;
  • allocation counts and allocated bytes unchanged in every measured scenario.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds an opt-in reporting mechanism for implicit scoped-source removals in AutoScopedParser and RouterScopedParser, so callers can accurately clean up their own per-source state when the parser evicts sources due to pressure, idle pruning, or capacity reduction.

Changes:

  • Introduces SourceRemovalCause, SourceRemoval, SourceRemovalMetrics, and SourceRemovalReporterError, plus internal helpers to deliver reports while isolating reporter errors/panics.
  • Adds *_with_reporter companion APIs for parse, iterator parse, idle pruning, and max-source resizing; keeps existing APIs as wrappers that discard reports.
  • Adds a comprehensive integration test suite for exact source/cause reporting, metrics, and TemplateStore behavior preservation; updates README and public re-exports.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
tests/scoped_source_removal_reporting.rs New integration tests validating reporting behavior, isolation, metrics, and TemplateStore invariants.
src/scoped_parser/source_removal.rs New public reporting/metrics types and internal reporting helpers (with panic/error isolation).
src/scoped_parser.rs Adds reporter-enabled companion methods, records metrics for implicit removals, and introduces AutoSourceKey.
src/lib.rs Re-exports new scoped-parser reporting types and AutoSourceKey.
README.md Documents how to observe implicit source removals via *_with_reporter APIs and metrics.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/scoped_parser.rs Outdated
@ktsaou
ktsaou force-pushed the fix/scoped-source-removal-reporting branch from ede49af to 97359b8 Compare July 21, 2026 08:03
@mikemiles-dev
mikemiles-dev requested a review from Copilot July 21, 2026 15:19

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated no new comments.

@mikemiles-dev
mikemiles-dev merged commit fff2ab0 into mikemiles-dev:main Jul 21, 2026
6 checks passed
@ktsaou ktsaou mentioned this pull request Jul 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants