Skip to content

fix: accurate issue creation date backdating via SCM changesets#129

Merged
joshua-quek-sonarsource merged 1 commit into
mainfrom
fix/migration-issue-history-accuracy
Apr 25, 2026
Merged

fix: accurate issue creation date backdating via SCM changesets#129
joshua-quek-sonarsource merged 1 commit into
mainfrom
fix/migration-issue-history-accuracy

Conversation

@joshua-quek-sonarsource

@joshua-quek-sonarsource joshua-quek-sonarsource commented Apr 25, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Rewrote backdateChangesets() to preserve each issue's original SonarQube creation date in SonarCloud, replacing the previous arbitrary 30-day-spaced bucket approach
  • Each issue's creationDate is mapped to per-line SCM blame dates in the changeset protobuf. The CE takes MAX(date) across an issue's textRange, so "oldest wins" for overlapping lines preserves accurate dates
  • Safety split handles calendar days with >5K issues (sub-groups with 1-day spacing, no file splitting)

Algorithm (3 phases)

  1. Phase 0 — Safety split: Pre-assign synthetic dates for any calendar day exceeding 5K issues
  2. Phase 1 — Per-line date map: Map each issue's textRange lines to its effective creation date (oldest wins)
  3. Phase 2 — Rebuild changesets: One changeset entry per unique date per file; non-issue lines default to oldest date

Files changed

  • src/shared/utils/batch-distributor/helpers/backdate-changesets.js — complete rewrite
  • docs/ — CHANGELOG, architecture, bespoke-algorithms, technical-details, troubleshooting, key-capabilities updated

Test plan

  • npm run package builds successfully
  • Migration test with 3 projects (including Angular Framework with 31K+ issues) completes with zero errors
  • Backdating logs show correct unique date counts per project
  • Spot-check issue creation dates in SonarCloud match original SonarQube dates
  • No safety split warnings (no single day exceeded 5K issues in test data)

🤖 Generated with Claude Code


Note

Medium Risk
Changes core SCM changeset rewriting logic that influences SonarCloud issue creation dates; mistakes could skew dates or trigger ES bucket caps on large migrations.

Overview
Replaces the prior file/batch-level SCM date bucketing with per-issue creation-date preservation. backdateChangesets() is rewritten to map each issue’s SonarQube creationDate onto its textRange lines, rebuild per-file changesets[] with one entry per unique date, and set changesetIndexByLine[] accordingly (defaulting non-issue lines to the oldest date).

Adds a safety-split for days exceeding ISSUE_BATCH_SIZE (5K) by assigning 1-day-spaced synthetic dates without splitting files, and updates logging to report unique/top dates by line count. Documentation across docs/ is refreshed to describe the new backdating approach and deprecate the old 30-day bucket strategy.

Reviewed by Cursor Bugbot for commit 43c89a8. Bugbot is set up for automated code reviews on this repo. Configure here.

@sonar-review-alpha

sonar-review-alpha Bot commented Apr 25, 2026

Copy link
Copy Markdown

Summary

Complete rewrite of issue backdating logic, shifting from arbitrary date-bucket assignment to preserving original SonarQube creation dates.

The old approach grouped files into ≤5K-issue batches and assigned sequential dates 30 days apart—spreading issues across multiple SonarCloud date buckets but losing accuracy. The new approach maps each issue's creation date directly to its line range in the SCM changeset, leveraging SonarCloud's CE behavior of taking MAX(date) across an issue's textRange.

Three-phase process: safety split for calendar days >5K issues (pre-assigns synthetic 1-day-spaced dates to sub-batches), per-line date mapping with "oldest wins" collision handling, and changeset rebuild with one entry per unique date per file.

Files changed: core rewrite of backdate-changesets.js (~95 lines → ~212 lines) and updates to 6 documentation files explaining the algorithm.

What reviewers should know

Start here:

  • backdate-changesets.js lines 1–40: read the top comment explaining all three phases—it's the map
  • buildSafetySplitOverrides() lines 96–123: understand when and how synthetic dates are created
  • buildFileLineDates() lines 164–193: see how per-line date assignment works, especially the "oldest wins" logic at lines 187–190

Key things to verify:

  1. CE MAX assumption — The entire approach assumes SonarCloud's ComputeEngine takes MAX(date) across an issue's textRange lines. Confirm this behavior is stable and documented (critical dependency).
  2. Safety split heuristic — When a calendar day exceeds 5K issues, code sub-groups by file without file splitting. This works if files are relatively small. Check test data to confirm no file alone approaches 5K issues.
  3. Edge cases in date parsingparseCreationDate() line 195 has a fallback to extraction time. Confirm this is the right behavior for missing/malformed dates.
  4. Performance with large line rangesbuildFileLineDates() loops every line in every issue's textRange. If migration has issues spanning 1000+ lines, this could add up. Migration test should log total lines processed.

Non-obvious decisions:

  • Synthetic dates go backward in time (line 121): group.dateMs - (totalBatches - 1 - batchIdx) * ONE_DAY_MS. This intentionally creates historical dates when safety split is needed—confirm this won't confuse auditing or downstream tools.
  • Last safety-split batch keeps original date (line 123): Earlier batches get synthetic dates, the last keeps the real one. This ensures at least one issue per day is accurate even under extreme load.
  • Per-file changesets — Each file gets its own changeset with multiple entries per file (one per unique date). This differs from the old single-entry-per-file approach.

Migration test checklist from logs:

  • Look for "subdtyle issues on YYYY-MM-DD exceed 5000 cap" warnings—confirms safety split triggered (or didn't, as expected)
  • "Total unique creation dates" should match expected date diversity
  • Spot-check one issue in SonarCloud: issue creation date should match original SonarQube date, not arbitrary 30-day-spaced values

  • Generate Walkthrough
  • Generate Diagram

🗣️ Give feedback

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 43c89a8. Configure here.

for (let batchIdx = 0; batchIdx < totalBatches - 1; batchIdx++) {
const syntheticDate = group.dateMs - (totalBatches - 1 - batchIdx) * ONE_DAY_MS;
for (const issue of subBatches[batchIdx]) {
effectiveDates.set(issue.key, syntheticDate);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Safety split synthetic dates may collide with existing days

Low Severity

The safety split pushes synthetic dates backward by (totalBatches - 1 - batchIdx) * ONE_DAY_MS from the original day. These synthetic dates can land on calendar days that already have issues — including days that themselves triggered a safety split. When multiple adjacent days each exceed 5K issues, their backward-shifted synthetic dates can accumulate on the same target day, potentially exceeding the 10K ES visualization cap that the safety split aims to prevent.

Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit 43c89a8. Configure here.

@joshua-quek-sonarsource
joshua-quek-sonarsource merged commit deb2e24 into main Apr 25, 2026
1 of 2 checks passed
@joshua-quek-sonarsource
joshua-quek-sonarsource deleted the fix/migration-issue-history-accuracy branch April 25, 2026 11:53
@joshua-quek-sonarsource

Copy link
Copy Markdown
Contributor Author

The migration is I/O-bound (API calls to SonarCloud at ~5 issues/sec).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

2 participants