Skip to content

docs: regression testing protocol for live migration verification#138

Merged
joshua-quek-sonarsource merged 1 commit into
mainfrom
docs/regression-testing-protocol
Apr 29, 2026
Merged

docs: regression testing protocol for live migration verification#138
joshua-quek-sonarsource merged 1 commit into
mainfrom
docs/regression-testing-protocol

Conversation

@joshua-quek-sonarsource

@joshua-quek-sonarsource joshua-quek-sonarsource commented Apr 29, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Adds docs/regression-testing.md — a reusable 7-phase protocol for verifying features/fixes against a live SQ→SC migration
  • Designed as a prompt for AI coding agents to follow after implementation
  • Includes Angular Framework test environment reference and a worked example (parallel issue sync)

Test plan

  • Verified no secrets leaked in docs (grep for tokens, staging URLs are not secrets)
  • Doc is self-contained and actionable

🤖 Generated with Claude Code


Note

Cursor Bugbot is generating a summary for commit 8a19e4c. Configure here.

Codifies the 7-phase test protocol used to verify features against the
Angular Framework project (38K+ issues): adversarial code review,
environment assessment, build & execute, analyze results, investigate
anomalies, fix & re-test loop, and clean pass criteria. Includes test
environment reference table and a worked example from the parallel
issue sync feature.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@sonar-review-alpha

sonar-review-alpha Bot commented Apr 29, 2026

Copy link
Copy Markdown

Summary

This PR adds a comprehensive 7-phase regression testing protocol (docs/regression-testing.md) designed as an AI agent prompt for verifying features and fixes against a live SonarQube→SonarCloud migration. The protocol covers adversarial code review, environment setup, test execution, result analysis, investigation, and remediation. It includes a reference section for the Angular Framework test environment (38K+ issues, 400+ hotspots) and practical scripts for generating test data. Grounded in real experience: a worked example demonstrates the protocol successfully caught silent API failures and guided a fix that improved sync success rates from 14% to 99%+.

What reviewers should know

What to verify:

  • Phase clarity: Each of the 7 phases should be specific enough that an AI agent (or human) can follow it mechanically without judgment calls. Spot-check phases 4 and 5 — they're the most complex and hardest to execute consistently.
  • Worked example credibility: The final example (parallel issue sync) demonstrates that the protocol caught real bugs (silent API failures in catch blocks) and guided a concrete fix. The before/after stats (0% → 99%+ success) validate that the protocol is thorough.
  • Actionability: The reference section (SQ/SC URLs, project keys) and the bulk-tagging Python script should make it easy to set up test conditions. Check that script syntax is correct and the SQ API calls match the API version in use.
  • Scope: This is pure documentation — no code changes, no breaking changes, no risk of regression. The protocol is advisory (a prompt to be given to agents), not enforced.

Why this matters: Live migration testing is error-prone and easy to skip. This protocol makes it systematic and repeatable. The worked example proves it catches bugs that would otherwise ship.


  • Generate Walkthrough
  • Generate Diagram

🗣️ Give feedback

@sonar-review-alpha sonar-review-alpha Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! ✅

Documentation-only PR, nothing to block merge. The protocol is internally consistent and the worked example grounds it in real test results.

🗣️ Give feedback

@joshua-quek-sonarsource

Copy link
Copy Markdown
Contributor Author

Hi @joe-tingsanchali-sonarsource - I will merge this in soon.
@okorach-sonar and I are doing some testing.

@joshua-quek-sonarsource
joshua-quek-sonarsource merged commit 5efa519 into main Apr 29, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants