Skip to content

feat: churn-based hotspot orientation pass for the analyzer - #22

Draft
moyom96 wants to merge 1 commit into
mainfrom
feature/analyzer-orientation
Draft

feat: churn-based hotspot orientation pass for the analyzer#22
moyom96 wants to merge 1 commit into
mainfrom
feature/analyzer-orientation

Conversation

@moyom96

@moyom96 moyom96 commented Jun 26, 2026

Copy link
Copy Markdown
Collaborator

Context

wall-e's findings were capped by what four collectors detect (debride, flog, flay, layer heuristics). Files that are simply large and changing constantly, where defect and review cost concentrate, stayed invisible because no collector was built to flag them. This adds an orientation pass so the LLM triage also reviews those files.

The approach is borrowed from the churn-vs-complexity hotspot literature (Michael Feathers, CodeScene): the intersection of high git churn and large size is where debt accumulates.

What Changed

  • HotspotCollector (fifth collector): counts commits per in-scope Ruby file over a window via git log, multiplies by file size, applies per-file thresholds, and emits the top files as hotspot candidates.
  • hotspot is an input seed only. The LLM triage diagnoses each hotspot file into the existing debt taxonomy (or rejects it) and never emits hotspot as an output debt_type. This means zero changes to issue creation or fingerprinting. The --skip-llm path keeps the raw type and is handled generically.
  • Wired into Analyzer#collect_candidates, with hotspot baseline metrics for verification metadata.
  • Config#hotspot settings with defaults and an enabled toggle: window_months (6), min_commits (5), min_loc (100), max_files (10).
  • score_details label so --skip-llm hotspot issues read honestly.
  • Templates updated: settings block documents the hotspot config; the analysis prompt explains that hotspot candidates are orientation signals to diagnose, not findings to echo.
  • Scan workflow checks out full history (fetch-depth: 0). Required: actions/checkout defaults to depth 1, so without this git log --since sees one commit and the pass finds nothing in CI.
  • Spec covering thresholds, scope filtering, churn-times-size scoring, the max_files cap, and the disabled/no-git paths.

How to Test

  1. mise exec ruby@3.3.6 -- bundle exec rspec spec/tech_debt/collectors/hotspot_collector_spec.rb (system Ruby 2.6 is incompatible). Expect 10 examples, 0 failures.
  2. See it work on real git data:
    mise exec ruby@3.3.6 -- bundle exec wall-e --dry-run --skip-llm in a repo with history. Hotspot candidates appear for the largest, most-churned files.
  3. Edge cases:
    • Shallow checkout (depth 1): the pass yields no hotspots. Confirms why the workflow needs fetch-depth: 0.
    • analysis.hotspot.enabled: false: the collector is skipped entirely (no git log call).
    • Tune min_commits / min_loc down on a small repo to see candidates surface.

Deployment Tasks

  • Consumers re-running rails g wall_e:install get the fetch-depth: 0 scan workflow and the hotspot config block. Existing installs that want the pass to work in CI must add fetch-depth: 0 to their wall_e_scan.yml checkout step. No migrations or env vars.

Notes

  • Renames split churn across old and new paths (no --follow), so a recently-renamed file under-counts. Acceptable for a heuristic.
  • A hotspot file already flagged by another collector produces a redundant file-level candidate; the LLM merges them at triage. Cross-collector dedup is out of scope.
  • Branched off main (not the per-PR debt-delta branch). Both touch the analyzer/collectors, so expect a small, mechanical merge conflict when the second of the two lands.

Add HotspotCollector, a fifth collector that surfaces maintenance
hotspots (files with high git churn and large size) so the LLM triage
also reviews files the AST and dead-code collectors never flag. Hotspots
are input seeds only: the model diagnoses each into the existing debt
taxonomy and never emits hotspot as an output type. The --skip-llm path
keeps the raw type and is handled generically.

- HotspotCollector: git churn over a window times file size, with
  per-file thresholds and a cap, all configurable.
- Wire into Analyzer#collect_candidates and add hotspot baseline metrics.
- Config.hotspot settings with defaults (window_months, min_commits,
  min_loc, max_files) and an enabled toggle.
- score_details label so skip-llm hotspot issues read honestly.
- Settings and analysis-prompt templates document the hotspot signal.
- Scan workflow checks out full history (fetch-depth: 0) so git churn is
  available in CI; without it the pass finds nothing.
- Spec covering thresholds, scope, scoring, cap, and disabled paths.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant