diff --git a/docs/regression-testing.md b/docs/regression-testing.md index 8c79dc72..5504981d 100644 --- a/docs/regression-testing.md +++ b/docs/regression-testing.md @@ -1,114 +1,410 @@ # Regression Testing Protocol -A reusable protocol for verifying features and fixes against a live SonarQube → SonarCloud migration across **any project**. Designed to be given as a prompt to an AI coding agent (Claude Code, etc.) after implementation is complete. +A reusable protocol for verifying bug fixes and features against a live SonarQube → SonarCloud migration. This protocol is **issue-driven** — you MUST fully understand the GitHub issue before designing any test, and your test plan must be explicitly derived from the issue's symptoms, root cause, and fix. --- ## The Prompt - + -> After implementing the feature/fix, test it end-to-end on **SonarQube project(s)** to verify regression testing. Follow this protocol exactly: +> After implementing the feature/fix, test it end-to-end to verify regression testing. Follow this protocol exactly. **Do not skip, shortcut, or merge any phase.** -> ### Phase 1 — Adversarial Code Review (before running anything) - +--- + +## Phase 0 — Fully Understand the Issue (Prerequisite — Do This First) -> Re-read every changed file with fresh eyes. For each file, ask: +> **This phase is MANDATORY. Do not proceed to Phase 1 until you can answer every question below in your own words.** -> 1. **Data flow**: Does the data survive serialization boundaries? (worker_threads structured clone, JSON.stringify round-trips, URL query parameter encoding, protobuf encoding) -> 2. **Error handling**: Are failures tracked or silently swallowed? Every `catch` block must either increment a stat, log, or re-throw — never empty. -> 3. **Race conditions**: Are shared counters mutated from concurrent async contexts? In single-threaded JS this is safe, but verify each case. -> 4. **API contract match**: Do HTTP calls use the exact method, endpoint, parameter names, and encoding that the original client uses? Compare line-by-line against the existing client implementation. -> 5. **Edge cases**: What happens with 0 items? 1 item? Exactly at the threshold? `null`/`undefined` fields? Empty arrays vs missing fields? -> 6. **Stat integrity**: Will `Object.assign(stats, mergedStats)` preserve fields set earlier (like `filtered`)? Will counters add up to the expected totals? +Before writing a single line of test code or running any command, you MUST have a complete mental model of: -> Write down every concern. Then verify or disprove each one before proceeding. +### 0.1 — Issue Anatomy +For the GitHub issue you are regression testing, you must be able to answer ALL of: -> ### Phase 2 — Environment Assessment - +1. **What is broken?** — The exact symptom. Not vague language — the precise observed behavior (e.g., "SC hotspot review % is 90.3% when SQS shows 100%", not "hotspots are wrong"). +2. **What is the expected behavior?** — The correct state after a successful migration. +3. **What is the root cause?** — Why does the bug exist in the code? Which function, which line, which condition, which API response field? +4. **What does the fix actually change?** — Read every line of the diff. Understand which code path is added, removed, or modified. Does it change an API call, a field mapping, a condition, a retry loop? +5. **What systems are affected?** — SQ only, SC only, or both? Does the bug affect the migration output, the SC project after migration, or the SQ source? +6. **What is the blast radius?** — Could the fix break other issue types (regular issues vs hotspots vs vulnerability stats)? Could it affect other migrations (e.g., external issues)? +7. **What evidence would prove the fix works?** — List 3-5 concrete assertions (e.g., "hotspot review % on SC must equal hotspot review % on SQS within 0.5%"). -> Before running the test, check the live state: +### 0.2 — Derive the Regression Test Plan -> 1. **List SonarQube projects**: `curl` the SQ API to list available projects, identify the target project(s) for regression testing. -> 2. **SonarQube**: Confirm the target project exists, count issues, sample 5 issues to check their fields (status, tags, comments, assignee, updateDate vs creationDate). -> 3. **SonarCloud**: Check if the SC project exists. If it has stale data from a prior run, delete it (`/api/projects/delete`). Verify auth tokens are valid (`/api/system/status`). -> 4. **Pre-filter viability**: Will enough issues pass the `hasManualChanges` pre-filter to trigger the code path under test? If not, create test conditions (bulk-tag issues, add comments, transition statuses via SQ API). Document what you created and how many. -> 5. **Clean slate**: Delete `migration-output/`, `.cloudvoyager-state.json*`, and the SC project(s). Every test run must start from zero. +From the issue understanding in 0.1, write a **specific regression test plan** BEFORE touching any code. This plan must include: -> ### Phase 3 — Build and Execute - +- **Which SonarQube project to use** — pick the project that best reproduces the issue (e.g., Angular Framework for hotspots because it has ~409 hotspots) +- **What test data to create** — do you need to bulk-tag issues, add comments, transition statuses to trigger the pre-filter? How many? +- **What to verify after migration** — specific API queries, specific counts, specific spot-checks +- **What the pass criteria are** — concrete numbers, not vague "looks right" -> 1. `npm run package` — must succeed. If it fails, fix before proceeding. -> 2. Create or verify the config file (`config.json` for `transfer`, `migrate-config.json` for `migrate`). -> 3. Run the command: `./dist/bin/cloudvoyager-macos-arm64 transfer -c config.json --verbose` -> 4. Monitor the output in real-time. Watch for: -> - The feature's log lines (e.g., "Parallel issue sync: N pairs across M workers") -> - Progress stalls (no output for >3 minutes — could indicate rate limiting or hangs) -> - Error/warn lines -> - The final summary stats line +**Write this plan down explicitly.** If you cannot write a specific, concrete regression test plan from the issue description, you do NOT fully understand the issue — go back to Phase 0.1 and re-read. -> ### Phase 4 — Analyze Results - +### 0.3 — Identify Related Code Paths -> When the command finishes (exit code 0 or non-zero): +Find ALL code paths that touch the same data or API calls as the fix: -> 1. **Parse the summary stats line.** For every counter: -> - Does it match expectations? (e.g., if you tagged 700 issues, do ~700 show as tagged?) -> - Are there unexpected zeros? (0 comments when you added 20 — investigate) -> - Do the numbers add up? (matched = tagged + already-tagged + tag-failures) -> 2. **Check the target system.** Query the SC API directly: -> - Count issues with `metadata-synchronized` tag vs total issues -> - Spot-check 5 issues for correct tags, comments, source links, status, assignee -> - Search for specific test data you created (e.g., `tags=test-parallel-sync`) -> 3. **Check for silent failures.** If any counter is lower than expected: -> - Check `api-errors` count (if tracked) -> - Check the error log: `migration-output/logs/*/cloudvoyager-transfer.error.log` -> - Check the warn log for rate limiting or timeout messages +1. Run `grep` for the function name(s) involved in the fix +2. Run `grep` for the API endpoint(s) involved +3. Check if the same field/mapping is used elsewhere (e.g., if `securityHotspot` status mapping is fixed, check if `issue` status mapping uses the same code) +4. Check if there are similar patterns nearby that might have the same bug but weren't fixed -> ### Phase 5 — Investigate Anomalies - +> Write down every related file and code path. These are your **regression watchlist**. -> For every unexpected result, trace the full code path: +--- -> 1. **Start from the data source.** Query the SQ API with the exact same parameters the code uses. Verify the field you expect (e.g., `comments`) is present in the response. -> 2. **Check serialization.** If data crosses a boundary (worker_threads, JSON), write a minimal reproduction: create a Worker, pass the data via workerData, verify it arrives intact. -> 3. **Check the API call.** Make the exact same HTTP request manually with `curl`. Does it succeed? What status code? What response body? -> 4. **Check the matching.** If an issue wasn't synced, verify it was matched: same rule + file + line on both SQ and SC. -> 5. **Identify the root cause.** Don't guess — prove. "The comments field was empty because..." with evidence. +## Phase 1 — Adversarial Code Review (before running anything) + -> ### Phase 6 — Fix and Re-test - +> Re-read every changed file and every file on your regression watchlist with fresh eyes. For each file, ask: -> If any issue was found: +### Data Integrity +1. **Data flow**: Does the data survive every serialization boundary? List each boundary: + - Worker thread: `workerData` structured clone — does nested data with arrays survive? + - JSON: `JSON.stringify` / `JSON.parse` round-trip — does `undefined`, `NaN`, `Infinity`, `Set`, `Map`, circular refs survive? + - URL encoding: query params, form data — are special characters encoded correctly? + - API response parsing: does the code handle the actual response schema from the real API, or a cached/stale schema? +2. **Field mapping fidelity**: Is every field from the source (SQ API response) correctly mapped to the destination (SC API request)? Check: field name, field type, null/undefined handling, array vs scalar. +3. **Stat integrity**: Will `Object.assign(stats, mergedStats)` preserve fields set earlier (like `filtered`)? Will counters add up to the expected totals? Trace the full stat lifecycle from worker to final output. -> 1. **Fix the code.** Apply the minimal change that addresses the root cause. -> 2. **Verify syntax**: `node --check ` on every changed file. -> 3. **Rebuild**: `npm run package` -> 4. **Clean slate**: Delete SC project, delete `migration-output/`, delete state files. -> 5. **Re-run**: Full transfer from scratch. Do NOT assume a partial re-run is sufficient. -> 6. **Re-analyze**: Repeat Phase 4 and Phase 5. The fix may have introduced new issues. +### Error Handling +4. **Every `catch` block**: Does it increment a stat, log, or re-throw? If the catch is empty — it is a bug. List every empty catch you find. +5. **Retry logic**: Does every retry loop have a maximum attempts cap? Does it respect rate limit headers (Retry-After)? Does it distinguish between retriable errors (5xx, timeout) and non-retriable errors (4xx)? +6. **Error propagation**: Can errors from worker threads bubble up to the main thread, or are they silently dropped? -> **Repeat Phase 6 until every stat matches expectations and spot-checks pass.** +### Concurrency & Thread Safety +7. **Shared state**: Are there any shared objects (counters, arrays, maps) mutated from concurrent async contexts? Even in single-threaded JS, verify there are no torn reads/writes. +8. **Worker thread lifecycle**: Are workers properly joined before the process exits? Are they cleaned up on error? -> ### Phase 7 — Declare Clean Pass - +### API Contract Compliance +9. **HTTP method**: Does the code use GET/POST/PUT/DELETE exactly as the SonarQube/SonarCloud API specifies? +10. **Endpoint path**: Is the URL exactly right — no missing/misplaced path segments, no incorrect query params? +11. **Request body/params**: Are field names, types, and encoding exactly correct? Compare character-by-character against the real API docs or actual observed API traces. +12. **Auth**: Are tokens passed correctly (Authorization header vs query param)? Are tokens refreshed if expired? -> A clean pass requires ALL of: +### Edge Cases +13. **0, 1, N at threshold**: What happens when the input is empty? A single item? Exactly at any concurrency threshold (e.g., 500 — the parallel vs sequential cutoff)? +14. **Missing/null fields**: What happens when an expected field is absent? Does the code use optional chaining? Does it fail silently or throw? +15. **Duplicate processing**: Could an issue be processed twice? Could a tag be applied twice? Could a comment be duplicated on the target? -> - [ ] Exit code 0 (no crashes) -> - [ ] 0 `failed` in the summary stats -> - [ ] All expected counters are non-zero and within tolerance (>95% of matched pairs for tags/source-links/metadata-sync) -> - [ ] `api-errors` is <5% of total operations (rate limiting is acceptable; silent data loss is not) -> - [ ] Spot-check of 5+ SC issues confirms correct tags, comments, source links -> - [ ] No unexpected warnings in the warn log -> - [ ] If the feature has a fallback path (e.g., <500 pairs uses mapConcurrent), verify the fallback still works by testing with a small project +### Side Effects & Regression Watchlist +16. **Side effects of the fix**: Does the fix inadvertently change behavior for other issue types (vulnerability stats, code smells, hotspots vs regular issues)? +17. **Related code paths**: Did you check every file on your regression watchlist? List them again and confirm each one. + +> **Write down every concern.** For each concern, write: "This could cause [specific failure] if [condition]." Then before Phase 3, verify or disprove each concern. --- -## Reference: Regression Testing Projects - +## Phase 2 — Environment Assessment + + +> Before running any test, verify the environment is in a known, clean state. + +### 2.1 — Verify SQS Source State + +```bash +# List all available projects +curl -s -u "${SONAR_TOKEN}:" "http://localhost:8998/api/projects/search" | jq '.components[] | {key: .key, name: .name}' + +# Confirm target project exists and get total issue count +curl -s -u "${SONAR_TOKEN}:" "http://localhost:8998/api/issues/search?projectKeys=${PROJECT_KEY}&ps=1" | jq '.total' + +# Count hotspots specifically +curl -s -u "${SONAR_TOKEN}:" "http://localhost:8998/api/hotspots/search?projectKey=${PROJECT_KEY}&ps=1" | jq '.total' + +# Sample 5 issues — check fields relevant to your issue +curl -s -u "${SONAR_TOKEN}:" "http://localhost:8998/api/issues/search?projectKeys=${PROJECT_KEY}&ps=5" | jq '.issues[] | {key: .key, status: .status, type: .type, tags: .tags, hasComments: .hasComments, assignee: .assignee, updateDate: .updateDate, creationDate: .creationDate}' + +# Sample 5 hotspots — check status distribution +curl -s -u "${SONAR_TOKEN}:" "http://localhost:8998/api/hotspots/search?projectKey=${PROJECT_KEY}&ps=5" | jq '.hotspots[] | {key: .key, status: .status, vulnerability: .vulnerability, securityCategory: .securityCategory}' +``` + +### 2.2 — Verify SC Target State + +```bash +# Check SC auth is valid +curl -s -u "${SONAR_TOKEN}:" "https://sc-staging.io/api/system/status" | jq '.status' + +# Check if SC project already exists (stale data from prior runs pollutes tests) +curl -s -u "${SONAR_TOKEN}:" "https://sc-staging.io/api/projects/search?projects=${PROJECT_KEY}" | jq '.components[] | {key: .key}' + +# If it exists — DELETE IT before running. A stale SC project invalidates all results. +# Use: curl -s -X POST -u "${SONAR_TOKEN}:" "https://sc-staging.io/api/projects/delete?project=${PROJECT_KEY}" +``` + +### 2.3 — Pre-Filter Viability Check + +The code has a pre-filter: only issues with `hasManualChanges` (or equivalent) are processed. If your test data doesn't pass this filter, you will get 0 operations and a false-pass. + +```bash +# Check how many issues pass the pre-filter (has manual changes / updateDate != creationDate) +# Run a sample and check updateDate vs creationDate +curl -s -u "${SONAR_TOKEN}:" "http://localhost:8998/api/issues/search?projectKeys=${PROJECT_KEY}&ps=100" | jq '[.issues[] | select(.updateDate != .creationDate)] | length' + +# If count is too low (< threshold for parallel sync, typically 500): +# CREATE TEST DATA — bulk tag, add comments, do transitions via the SQ API. +# Document exactly what you created and how many. +``` + +### 2.4 — Clean Slate (Mandatory) + +Every test run starts from zero. Delete all state between runs: + +```bash +# Delete migration output +rm -rf migration-output/ + +# Delete state files +rm -f .cloudvoyager-state.json .cloudvoyager-state.json.bak + +# Delete SC project (if it exists from a prior run) +curl -s -X POST -u "${SONAR_TOKEN}:" "https://sc-staging.io/api/projects/delete?project=${PROJECT_KEY}" + +# Verify deletion +curl -s -u "${SONAR_TOKEN}:" "https://sc-staging.io/api/projects/search?projects=${PROJECT_KEY}" | jq '.components | length' +``` + +### 2.5 — Document Pre-Test State + +Before running, write down: +- Total issues on SQS: ___ +- Total hotspots on SQS: ___ +- Issues passing pre-filter: ___ +- Hotspots passing pre-filter: ___ +- SC project state (fresh or did you delete it?): ___ + +--- + +## Phase 3 — Build and Execute + + +### 3.1 — Build + +```bash +npm run package +``` + +**Must succeed. If it fails, do not proceed. Fix the build error first.** + +### 3.2 — Config Verification + +Verify the config file is correct for the operation: + +```bash +# For transfer operation: +cat config.json | jq '{sqProject: .sourceProject, scProject: .targetProject, sqUrl: .sourceUrl, scUrl: .targetUrl}' + +# For migrate operation: +cat migrate-config.json | jq '{...}' +``` + +### 3.3 — Execute with Verbose Output + +```bash +./dist/bin/cloudvoyager-macos-arm64 transfer -c config.json --verbose 2>&1 | tee /tmp/cloudvoyager-run-$(date +%s).log +``` + +### 3.4 — Monitor in Real-Time + +Watch for ALL of: +- **Feature-specific log lines** — whatever the fix adds (grep the diff for new log statements) +- **Progress indicators** — issue counts, hotspot counts, matched pairs +- **Progress stalls** — no output for >3 minutes (could indicate rate limiting, infinite loop, or deadlock) +- **Error/warn lines** — count them, note the first occurrence +- **Rate limiting indicators** — 429 responses, Retry-After headers, backoff messages +- **The final summary stats line** — copy it exactly + +### 3.5 — Capture Run Evidence + +Save the full output log. Note the timestamp, the summary stats line, and any errors/warns. + +--- + +## Phase 4 — Analyze Results + + +### 4.1 — Parse Summary Stats + +Parse the summary stats line. For every counter, answer: +- Does it match expectations derived in Phase 0? +- Are there unexpected zeros? +- Do the numbers add up? (matched = tagged + already-tagged + failures) +- Is `api-errors` at an acceptable level (<5%)? + +### 4.2 — Query the Target System (SC) + +Query the SC API directly — do NOT trust the migration output alone. + +**Issue-level checks:** +```bash +# Count issues with metadata-synchronized tag vs total +TOTAL=$(curl -s -u "${SONAR_TOKEN}:" "https://sc-staging.io/api/issues/search?projects=${PROJECT_KEY}&ps=1" | jq '.total') +TAGGED=$(curl -s -u "${SONAR_TOKEN}:" "https://sc-staging.io/api/issues/search?projects=${PROJECT_KEY}&tags=metadata-synchronized&ps=1" | jq '.total') +echo "Total: $TOTAL, Tagged: $TAGGED, Untagged: $((TOTAL - TAGGED))" +``` + +**Spot-check 5+ issues** — for each, query both SQS and SC and compare: +- Tags +- Comments (count and content) +- Assignee +- Status +- Source link (if applicable) + +**Issue-specific verification** — derived from Phase 0.2: + +Choose all that apply based on your issue: + +| Issue type | What to check | How to check | +|---|---|---| +| **Hotspots** | Review % matches SQS | `api/hotspots/search` on both; count SAFE+REVIEWED vs total | +| **Tags** | Tag count/accuracy | `api/issues/search?tags=...` on both | +| **Comments** | Comment count per issue | Sample 5 issues on both, compare comment arrays | +| **Status transitions** | Status counts match | Count by status on both: OPEN, CONFIRMED, RESOLVED, etc. | +| **Assignees** | Assignee field populated | Sample 5 issues, compare assignee field | +| **External issues** | External key preserved | Check `externalRelation` or `externalKey` fields | +| **Metadata sync** | `metadata-synchronized` tag applied | Count tagged vs total issues | -For regression testing, use any SonarQube project. Below are common options: +### 4.3 — Check for Silent Failures + +```bash +# api-errors count from summary +# Check error logs +ls migration-output/logs/*/cloudvoyager-transfer.error.log 2>/dev/null && cat migration-output/logs/*/cloudvoyager-transfer.error.log + +# Check warn logs for rate limiting or timeout messages +ls migration-output/logs/*/cloudvoyager-transfer.warn.log 2>/dev/null && cat migration-output/logs/*/cloudvoy器-transfer.warn.log +``` + +### 4.4 — Issue-Specific Metric Comparison (Phase 0 Derived) + +Run the specific comparisons you planned in Phase 0.2. For each metric: + +1. Query SQS with the exact same parameters the code uses +2. Query SC with the equivalent +3. Calculate the metric (e.g., percentage) +4. Compare — they must match within the tolerance you defined in Phase 0.1 + +**Write down the actual numbers.** If any discrepancy exceeds tolerance, proceed to Phase 5. + +--- + +## Phase 5 — Investigate Anomalies + + +> **For every unexpected result, trace the full code path to root cause. Do not guess. Prove.** + +### 5.1 — Isolate the Failing Path + +Start from the data source: +```bash +# Query the SQ API with the exact same parameters the code uses +# Copy the exact URL, exact params, exact headers +curl -s -u "${SONAR_TOKEN}:" "http://localhost:8998/api/issues/search?projectKeys=${PROJECT_KEY}&..." | jq '.issues[] | ...' +``` + +Verify the field you expect is actually present in the response. Does the SQ API return the field you think you're reading? + +### 5.2 — Check Serialization Boundaries + +If data crosses a boundary (worker_threads, JSON): +```bash +# Minimal reproduction: create a Worker, pass the data via workerData, log what arrives +node -e " +const { Worker } = require('worker_threads'); +const w = new Worker('./test-worker.js', { workerData: { your: 'data' } }); +w.on('message', m => console.log('received:', JSON.stringify(m))); +w.on('error', e => console.error('error:', e)); +" +``` + +### 5.3 — Verify API Calls with curl + +Make the EXACT same HTTP request manually: +```bash +# Copy method, URL, headers, body from the code +# Run it with curl and inspect the response +curl -v -X POST -u "${SONAR_TOKEN}:" \ + -H "Content-Type: application/x-www-form-urlencoded" \ + -d "issue=AXkS1&tags=test" \ + "http://localhost:8998/api/issues/set_tags" +``` + +Does it succeed? What HTTP status? What response body? Compare to what the code expects. + +### 5.4 — Check Matching Logic + +If an issue was not synced or mismatched: +- Was it matched? Check the matcher: same component (file), same line, same rule key? +- Was it filtered out by the pre-filter? +- Did it fail at the API call stage? + +### 5.5 — Root Cause Statement + +Before fixing, write a root cause statement: +> "The [specific field] was [wrong/absent] because [specific reason]. Evidence: [API response / log line / code snippet]." + +This discipline prevents fixing symptoms instead of causes. + +--- + +## Phase 6 — Fix and Re-test + + +### 6.1 — Apply Minimal Fix + +Apply the minimal change that addresses the root cause. Do not make unrelated changes at the same time. + +### 6.2 — Verify Syntax + +```bash +node --check # on every changed file +``` + +### 6.3 — Rebuild + +```bash +npm run package +``` + +### 6.4 — Clean Slate + +**Full clean. Not partial. Not skip.** +```bash +rm -rf migration-output/ +rm -f .cloudvoyager-state.json .cloudvoyager-state.json.bak +curl -s -X POST -u "${SONAR_TOKEN}:" "https://sc-staging.io/api/projects/delete?project=${PROJECT_KEY}" +``` + +### 6.5 — Re-run + +Full transfer from scratch. Never assume a partial re-run is sufficient. + +### 6.6 — Full Re-analysis + +Repeat Phase 4 and Phase 5 completely. A fix can introduce new bugs. + +> **Repeat Phase 6 until every stat matches, every spot-check passes, and every issue-specific metric is within tolerance.** + +--- + +## Phase 7 — Declare Clean Pass + + +A clean pass requires **ALL** of the following — not a subset: + +- [ ] Exit code 0 — no crashes +- [ ] 0 `failed` in the summary stats +- [ ] All expected counters are non-zero and within tolerance (>95% of matched pairs for tags/source-links/metadata-sync) +- [ ] `api-errors` is <5% of total operations (rate limiting is acceptable; silent data loss is not) +- [ ] Every issue-specific metric (from Phase 0.2 and Phase 4.4) is within its defined tolerance +- [ ] Spot-check of 5+ SC issues confirms correct tags, comments, source links, status, assignee +- [ ] No unexpected warnings in the warn log +- [ ] Fallback paths verified (e.g., if fix changes parallel path, verify sequential fallback still works on a small project) +- [ ] No regressions in related code paths (Phase 0.3 watchlist items all still work) + +--- + +## Reference: Regression Testing Projects ### Large Project (Angular Framework) @@ -122,7 +418,7 @@ For regression testing, use any SonarQube project. Below are common options: | SC issues (after transfer) | ~31,600 | | Hotspots | ~409 | -### Small Project (for fallback verification) +### Small Project (fallback / verify sequential path) | Property | Value | |----------|-------| @@ -131,45 +427,32 @@ For regression testing, use any SonarQube project. Below are common options: | SC org | `open-digital-society` | | SC URL | `https://sc-staging.io` | -### Selecting a Project for Regression Testing - -When regression testing an issue, select the project that best reproduces the problem: - -1. **Issue involves specific issue states/statuses**: Use a project with diverse issue statuses -2. **Issue involves manual changes**: Use a project with `hasManualChanges` issues, or create them via API -3. **Issue involves comments/tags/assignees**: Use a project where these fields are populated -4. **Issue involves hotspots**: Use a project with known hotspots (like Angular Framework with ~409) -5. **Issue involves file matching**: Use a project where SQ and SC have overlapping file paths +### Selecting a Project -### Listing Available Projects on SonarQube +Choose based on the issue under test: -```bash -# List all projects on SonarQube -curl -s -u "${SONAR_TOKEN}:" "http://localhost:8998/api/projects/search" | jq '.components[] | {key: .key, name: .name}' - -# Count total issues for a project -curl -s -u "${SONAR_TOKEN}:" "http://localhost:8998/api/issues/search?projectKeys=${PROJECT_KEY}&ps=1" | jq '.total' - -# Sample issues to check field distribution -curl -s -u "${SONAR_TOKEN}:" "http://localhost:8998/api/issues/search?projectKeys=${PROJECT_KEY}&ps=5" | jq '.issues[] | {key: .key, status: .status, tags: .tags, hasComments: .hasComments}' -``` +1. **Hotspots**: Angular Framework (~409 hotspots) +2. **Manual changes pre-filter**: Angular Framework (has issues with diverse updateDate patterns) +3. **Tags / Comments / Assignees**: Any project where these fields are populated +4. **File matching**: Any project where SQ and SC have overlapping file paths +5. **Sequential fallback**: Small project with <500 issues to trigger `mapConcurrent` path --- ## Reference: Config Files -| Config (transfer) | `config.json` | -|-------------------|---------------| -| Config (migrate) | `migrate-config.json` | -| Delete SC projects script | `.debugging/delete-all-sonarcloud-projects.sh` | -| Test migration script | `.debugging/test-migrate.sh` | +| Operation | Config file | +|-----------|-------------| +| Transfer | `config.json` | +| Migrate | `migrate-config.json` | +| Delete SC project | `.debugging/delete-all-sonarcloud-projects.sh` | +| Test migration | `.debugging/test-migrate.sh` | --- -## Creating Test Data on SonarQube - +## Reference: Creating Test Data -To trigger code paths that require manual changes (pre-filter keeps only issues with human-authored changes): +To create manual changes (required for the pre-filter to pass): ```bash # Bulk-tag issues (creates updateDate != creationDate) @@ -181,12 +464,10 @@ PROJECT_KEY = sys.argv[1] if len(sys.argv) > 1 else 'angular-framework' auth = base64.b64encode(f'{TOKEN}:'.encode()).decode() headers = {'Authorization': f'Basic {auth}', 'Content-Type': 'application/x-www-form-urlencoded'} -# Fetch issue keys url = f'http://localhost:8998/api/issues/search?projectKeys={PROJECT_KEY}&ps=500' req = urllib.request.Request(url, headers=headers) keys = [i['key'] for i in json.load(urllib.request.urlopen(req))['issues']] -# Tag them def tag(key): params = urllib.parse.urlencode({'issue': key, 'tags': 'test-tag'}).encode() req = urllib.request.Request('http://localhost:8998/api/issues/set_tags', data=params, headers=headers, method='POST') @@ -199,23 +480,28 @@ print(f'Tagged: {sum(1 for r in results if r == \"ok\")}/{len(keys)}') " ${PROJECT_KEY} ``` -Adapt similarly for `do_transition` (confirm/reopen), `add_comment`, and `assign`. +Adapt for `do_transition`, `add_comment`, and `assign` using the equivalent SQ API endpoints. --- ## Example: Parallel Issue Sync Test (2026-04-28) - Applied this protocol to verify `worker_threads`-based parallel issue sync. -**Phase 1 findings:** No critical bugs in logic. Identified that `Object.assign(stats, mergedStats)` preserves `stats.filtered` because `mergedStats` doesn't have that field. Confirmed `workerData` structured clone handles nested objects with arrays. +**Phase 0 findings:** Root cause was empty `catch` blocks silently swallowing SC rate limit errors (429). Fix: added retry with backoff and `apiErrors` counter. + +**Phase 0.2 regression plan:** Create 700+ tagged issues on SQS, migrate, verify tag sync rate >95% and api-errors <5%. + +**Phase 1 findings:** Identified `Object.assign(stats, mergedStats)` preserves `stats.filtered`. Confirmed `workerData` structured clone handles nested objects. + +**Phase 2:** SQ had 38,412 issues with 0 manual changes. Created 700 tagged + 32 confirmed + 20 commented via SQ API. Verified pre-filter would pass ~5,000+ issues. -**Phase 2 setup:** SQ had 38,412 issues with 0 manual changes. Created 700 tagged + 32 confirmed + 20 commented issues via SQ API. Verified pre-filter would pass ~5,000+ issues (above 500 threshold). +**Phase 3 (run 1):** Transfer showed 0 comments synced, only 7,293/31,641 tagged. Investigation: empty catch blocks + SC rate limiting. -**Phase 3 (run 1):** Transfer completed but showed 0 comments synced, only 7,293/31,641 tagged. Investigation revealed empty `catch` blocks silently swallowed API errors — SC staging rate-limited 100 concurrent requests. +**Phase 4 (run 1):** apiErrors was 0 — confirmed silent swallow. Tag rate 23%, comment rate 0%. -**Phase 4 fix:** Increased retries from 3→6, added jitter on backoff, added 5xx/transient retry, added `apiErrors` counter to all catch blocks. +**Phase 5 fix:** Retries 3→6, jitter backoff, 5xx/transient retry, `apiErrors` counter in all catch blocks. -**Phase 5 (run 2):** Clean pass. 5,119 matched, 5,067 tagged (99%), 5,109 metadata-sync-tagged (99.8%), 5,095 source-linked (99.5%), 34 api-errors (0.7%), 0 failed. Spot-checked SC issues — all correct. +**Phase 6 (run 2):** Clean pass. 5,119 matched, 5,067 tagged (99%), 5,109 metadata-sync-tagged (99.8%), 5,095 source-linked (99.5%), 34 api-errors (0.7%), 0 failed. -**Iterations:** 2 (first run exposed silent failures, fix applied, second run clean). +**Iterations:** 2. First run exposed silent failures. Second run clean. diff --git a/src/pipelines/sq-10.0/sonarcloud/migrators/hotspot-sync/helpers/hotspot-transition-mapping.js b/src/pipelines/sq-10.0/sonarcloud/migrators/hotspot-sync/helpers/hotspot-transition-mapping.js index 106917b1..28fdbd63 100644 --- a/src/pipelines/sq-10.0/sonarcloud/migrators/hotspot-sync/helpers/hotspot-transition-mapping.js +++ b/src/pipelines/sq-10.0/sonarcloud/migrators/hotspot-sync/helpers/hotspot-transition-mapping.js @@ -1,5 +1,13 @@ // -------- Hotspot Transition Mapping -------- +/** Map SQ resolution to SC-compatible resolution. SC only accepts SAFE or FIXED. */ +function toScResolution(resolution) { + if (resolution === 'ACKNOWLEDGED') return 'SAFE'; + if (resolution === 'FIXED') return 'FIXED'; + if (resolution === 'SAFE') return 'SAFE'; + return null; +} + /** * Map a hotspot changelog diff entry to a SonarCloud action { status, resolution }. */ @@ -11,8 +19,8 @@ export function mapHotspotChangelogDiffToAction(diffs) { if (!newStatus) return null; if (newStatus === 'TO_REVIEW') return { status: 'TO_REVIEW', resolution: null }; - if (newStatus === 'REVIEWED') return { status: 'REVIEWED', resolution: newResolution || 'SAFE' }; - if (['SAFE', 'ACKNOWLEDGED', 'FIXED'].includes(newStatus)) return { status: 'REVIEWED', resolution: newStatus }; + if (newStatus === 'REVIEWED') return { status: 'REVIEWED', resolution: toScResolution(newResolution) || 'SAFE' }; + if (['SAFE', 'ACKNOWLEDGED', 'FIXED'].includes(newStatus)) return { status: 'REVIEWED', resolution: toScResolution(newStatus) }; return null; } diff --git a/src/pipelines/sq-10.0/sonarcloud/migrators/hotspot-sync/helpers/map-hotspot-resolution.js b/src/pipelines/sq-10.0/sonarcloud/migrators/hotspot-sync/helpers/map-hotspot-resolution.js index f5bb63e6..b5e6820d 100644 --- a/src/pipelines/sq-10.0/sonarcloud/migrators/hotspot-sync/helpers/map-hotspot-resolution.js +++ b/src/pipelines/sq-10.0/sonarcloud/migrators/hotspot-sync/helpers/map-hotspot-resolution.js @@ -4,7 +4,7 @@ * Map SonarQube hotspot status/resolution to SonarCloud resolution. */ export function mapHotspotResolution(sqHotspot) { - if (sqHotspot.resolution === 'ACKNOWLEDGED') return 'ACKNOWLEDGED'; + if (sqHotspot.resolution === 'ACKNOWLEDGED') return 'SAFE'; if (sqHotspot.resolution === 'FIXED') return 'FIXED'; if (sqHotspot.resolution === 'SAFE' || sqHotspot.status === 'REVIEWED') return 'SAFE'; return null; diff --git a/src/pipelines/sq-10.0/sonarcloud/migrators/hotspot-sync/helpers/sync-hotspot-status.js b/src/pipelines/sq-10.0/sonarcloud/migrators/hotspot-sync/helpers/sync-hotspot-status.js index 51a784c6..7d5f7f12 100644 --- a/src/pipelines/sq-10.0/sonarcloud/migrators/hotspot-sync/helpers/sync-hotspot-status.js +++ b/src/pipelines/sq-10.0/sonarcloud/migrators/hotspot-sync/helpers/sync-hotspot-status.js @@ -10,7 +10,7 @@ import { applyFallbackHotspotAction } from './apply-fallback-hotspot-action.js'; * Falls back to a single transition when no changelog is available. */ export async function syncHotspotStatus(scHotspot, sqHotspot, client) { - if (scHotspot.status === sqHotspot.status && (scHotspot.resolution || null) === (sqHotspot.resolution || null)) { + if (scHotspot.status === sqHotspot.status && (scHotspot.resolution ?? null) === (sqHotspot.resolution ?? null)) { return false; } diff --git a/src/pipelines/sq-10.4/sonarcloud/migrators/hotspot-sync/helpers/map-changelog-diff.js b/src/pipelines/sq-10.4/sonarcloud/migrators/hotspot-sync/helpers/map-changelog-diff.js index 1f681e7e..4ad30ec7 100644 --- a/src/pipelines/sq-10.4/sonarcloud/migrators/hotspot-sync/helpers/map-changelog-diff.js +++ b/src/pipelines/sq-10.4/sonarcloud/migrators/hotspot-sync/helpers/map-changelog-diff.js @@ -1,3 +1,13 @@ +// -------- Map Hotspot Resolution -------- + +/** Map SQ resolution to SC-compatible resolution. SC only accepts SAFE or FIXED. */ +function toScResolution(resolution) { + if (resolution === 'ACKNOWLEDGED') return 'SAFE'; + if (resolution === 'FIXED') return 'FIXED'; + if (resolution === 'SAFE') return 'SAFE'; + return null; +} + // -------- Main Logic -------- /** @@ -15,12 +25,12 @@ export function mapHotspotChangelogDiffToAction(diffs) { // Reviewed with explicit resolution if (newStatus === 'REVIEWED') { const newResolution = diffs.find(d => d.key === 'resolution')?.newValue; - return { status: 'REVIEWED', resolution: newResolution || 'SAFE' }; + return { status: 'REVIEWED', resolution: toScResolution(newResolution) || 'SAFE' }; } // In newer SQ versions, status can be SAFE/ACKNOWLEDGED/FIXED directly if (['SAFE', 'ACKNOWLEDGED', 'FIXED'].includes(newStatus)) { - return { status: 'REVIEWED', resolution: newStatus }; + return { status: 'REVIEWED', resolution: toScResolution(newStatus) }; } return null; diff --git a/src/pipelines/sq-10.4/sonarcloud/migrators/hotspot-sync/helpers/map-hotspot-resolution.js b/src/pipelines/sq-10.4/sonarcloud/migrators/hotspot-sync/helpers/map-hotspot-resolution.js index ba807fd4..dd4673ee 100644 --- a/src/pipelines/sq-10.4/sonarcloud/migrators/hotspot-sync/helpers/map-hotspot-resolution.js +++ b/src/pipelines/sq-10.4/sonarcloud/migrators/hotspot-sync/helpers/map-hotspot-resolution.js @@ -2,7 +2,7 @@ // Map SonarQube hotspot status/resolution to SonarCloud resolution. export function mapHotspotResolution(sqHotspot) { - if (sqHotspot.resolution === 'ACKNOWLEDGED') return 'ACKNOWLEDGED'; + if (sqHotspot.resolution === 'ACKNOWLEDGED') return 'SAFE'; if (sqHotspot.resolution === 'FIXED') return 'FIXED'; if (sqHotspot.resolution === 'SAFE' || sqHotspot.status === 'REVIEWED') return 'SAFE'; return null; diff --git a/src/pipelines/sq-10.4/sonarcloud/migrators/hotspot-sync/helpers/sync-hotspot-status.js b/src/pipelines/sq-10.4/sonarcloud/migrators/hotspot-sync/helpers/sync-hotspot-status.js index a823b0d3..afd32d29 100644 --- a/src/pipelines/sq-10.4/sonarcloud/migrators/hotspot-sync/helpers/sync-hotspot-status.js +++ b/src/pipelines/sq-10.4/sonarcloud/migrators/hotspot-sync/helpers/sync-hotspot-status.js @@ -8,7 +8,7 @@ import { applyFallbackAction } from './apply-fallback-action.js'; // Sync hotspot status by replaying the full changelog transition sequence. export async function syncHotspotStatus(scHotspot, sqHotspot, client) { // Skip if statuses and resolutions already match - if (scHotspot.status === sqHotspot.status && (scHotspot.resolution || null) === (sqHotspot.resolution || null)) { + if (scHotspot.status === sqHotspot.status && (scHotspot.resolution ?? null) === (sqHotspot.resolution ?? null)) { return false; } diff --git a/src/pipelines/sq-2025/sonarcloud/migrators/hotspot-sync/helpers/hotspot-fallback-action.js b/src/pipelines/sq-2025/sonarcloud/migrators/hotspot-sync/helpers/hotspot-fallback-action.js index 65b15fcf..50d29111 100644 --- a/src/pipelines/sq-2025/sonarcloud/migrators/hotspot-sync/helpers/hotspot-fallback-action.js +++ b/src/pipelines/sq-2025/sonarcloud/migrators/hotspot-sync/helpers/hotspot-fallback-action.js @@ -28,7 +28,7 @@ function getFallbackAction(scHotspot, sqHotspot) { } function mapHotspotResolution(sqHotspot) { - if (sqHotspot.resolution === 'ACKNOWLEDGED') return 'ACKNOWLEDGED'; + if (sqHotspot.resolution === 'ACKNOWLEDGED') return 'SAFE'; if (sqHotspot.resolution === 'FIXED') return 'FIXED'; if (sqHotspot.resolution === 'SAFE' || sqHotspot.status === 'REVIEWED') return 'SAFE'; return null; diff --git a/src/pipelines/sq-2025/sonarcloud/migrators/hotspot-sync/helpers/map-hotspot-changelog.js b/src/pipelines/sq-2025/sonarcloud/migrators/hotspot-sync/helpers/map-hotspot-changelog.js index aff9fc89..df65742c 100644 --- a/src/pipelines/sq-2025/sonarcloud/migrators/hotspot-sync/helpers/map-hotspot-changelog.js +++ b/src/pipelines/sq-2025/sonarcloud/migrators/hotspot-sync/helpers/map-hotspot-changelog.js @@ -1,5 +1,13 @@ // -------- Map Hotspot Changelog Diff to Action -------- +/** Map SQ resolution to SC-compatible resolution. SC only accepts SAFE or FIXED. */ +function toScResolution(resolution) { + if (resolution === 'ACKNOWLEDGED') return 'SAFE'; + if (resolution === 'FIXED') return 'FIXED'; + if (resolution === 'SAFE') return 'SAFE'; + return null; +} + /** Map a changelog diff entry to a SC action { status, resolution }. Returns null if none. */ export function mapHotspotChangelogDiffToAction(diffs) { const statusDiff = diffs.find(d => d.key === 'status'); @@ -9,8 +17,8 @@ export function mapHotspotChangelogDiffToAction(diffs) { if (!newStatus) return null; if (newStatus === 'TO_REVIEW') return { status: 'TO_REVIEW', resolution: null }; - if (newStatus === 'REVIEWED') return { status: 'REVIEWED', resolution: newResolution || 'SAFE' }; - if (['SAFE', 'ACKNOWLEDGED', 'FIXED'].includes(newStatus)) return { status: 'REVIEWED', resolution: newStatus }; + if (newStatus === 'REVIEWED') return { status: 'REVIEWED', resolution: toScResolution(newResolution) || 'SAFE' }; + if (['SAFE', 'ACKNOWLEDGED', 'FIXED'].includes(newStatus)) return { status: 'REVIEWED', resolution: toScResolution(newStatus) }; return null; } diff --git a/src/pipelines/sq-2025/sonarcloud/migrators/hotspot-sync/helpers/sync-hotspot-status.js b/src/pipelines/sq-2025/sonarcloud/migrators/hotspot-sync/helpers/sync-hotspot-status.js index b7454794..85a2784f 100644 --- a/src/pipelines/sq-2025/sonarcloud/migrators/hotspot-sync/helpers/sync-hotspot-status.js +++ b/src/pipelines/sq-2025/sonarcloud/migrators/hotspot-sync/helpers/sync-hotspot-status.js @@ -6,7 +6,7 @@ import { applyFallbackAction } from './hotspot-fallback-action.js'; /** Sync hotspot status by replaying changelog or falling back to single transition. */ export async function syncHotspotStatus(scHotspot, sqHotspot, client) { - if (scHotspot.status === sqHotspot.status && (scHotspot.resolution || null) === (sqHotspot.resolution || null)) return false; + if (scHotspot.status === sqHotspot.status && (scHotspot.resolution ?? null) === (sqHotspot.resolution ?? null)) return false; const changelog = sqHotspot.changelog; if (changelog && changelog.length > 0) { diff --git a/src/pipelines/sq-9.9/sonarcloud/migrators/hotspot-sync/helpers/map-hotspot-changelog-diff.js b/src/pipelines/sq-9.9/sonarcloud/migrators/hotspot-sync/helpers/map-hotspot-changelog-diff.js index 789c6402..0d9d38cd 100644 --- a/src/pipelines/sq-9.9/sonarcloud/migrators/hotspot-sync/helpers/map-hotspot-changelog-diff.js +++ b/src/pipelines/sq-9.9/sonarcloud/migrators/hotspot-sync/helpers/map-hotspot-changelog-diff.js @@ -1,5 +1,13 @@ // -------- Map Hotspot Changelog Diff to SonarCloud Action -------- +/** Map SQ resolution to SC-compatible resolution. SC only accepts SAFE or FIXED. */ +function toScResolution(resolution) { + if (resolution === 'ACKNOWLEDGED') return 'SAFE'; + if (resolution === 'FIXED') return 'FIXED'; + if (resolution === 'SAFE') return 'SAFE'; + return null; +} + export function mapHotspotChangelogDiffToAction(diffs) { const statusDiff = diffs.find(d => d.key === 'status'); const resolutionDiff = diffs.find(d => d.key === 'resolution'); @@ -8,7 +16,7 @@ export function mapHotspotChangelogDiffToAction(diffs) { if (!newStatus) return null; if (newStatus === 'TO_REVIEW') return { status: 'TO_REVIEW', resolution: null }; - if (newStatus === 'REVIEWED') return { status: 'REVIEWED', resolution: newResolution || 'SAFE' }; - if (['SAFE', 'ACKNOWLEDGED', 'FIXED'].includes(newStatus)) return { status: 'REVIEWED', resolution: newStatus }; + if (newStatus === 'REVIEWED') return { status: 'REVIEWED', resolution: toScResolution(newResolution) || 'SAFE' }; + if (['SAFE', 'ACKNOWLEDGED', 'FIXED'].includes(newStatus)) return { status: 'REVIEWED', resolution: toScResolution(newStatus) }; return null; } diff --git a/src/pipelines/sq-9.9/sonarcloud/migrators/hotspot-sync/helpers/map-hotspot-resolution.js b/src/pipelines/sq-9.9/sonarcloud/migrators/hotspot-sync/helpers/map-hotspot-resolution.js index a98f6d64..f339f16e 100644 --- a/src/pipelines/sq-9.9/sonarcloud/migrators/hotspot-sync/helpers/map-hotspot-resolution.js +++ b/src/pipelines/sq-9.9/sonarcloud/migrators/hotspot-sync/helpers/map-hotspot-resolution.js @@ -1,7 +1,7 @@ // -------- Map SQ Hotspot Status/Resolution to SC Resolution -------- export function mapHotspotResolution(sqHotspot) { - if (sqHotspot.resolution === 'ACKNOWLEDGED') return 'ACKNOWLEDGED'; + if (sqHotspot.resolution === 'ACKNOWLEDGED') return 'SAFE'; if (sqHotspot.resolution === 'FIXED') return 'FIXED'; if (sqHotspot.resolution === 'SAFE' || sqHotspot.status === 'REVIEWED') return 'SAFE'; return null; diff --git a/src/pipelines/sq-9.9/sonarcloud/migrators/hotspot-sync/helpers/sync-hotspot-status.js b/src/pipelines/sq-9.9/sonarcloud/migrators/hotspot-sync/helpers/sync-hotspot-status.js index e2993098..bee17887 100644 --- a/src/pipelines/sq-9.9/sonarcloud/migrators/hotspot-sync/helpers/sync-hotspot-status.js +++ b/src/pipelines/sq-9.9/sonarcloud/migrators/hotspot-sync/helpers/sync-hotspot-status.js @@ -22,7 +22,7 @@ function getFallbackAction(scHotspot, sqHotspot) { } export async function syncHotspotStatus(scHotspot, sqHotspot, client) { - if (scHotspot.status === sqHotspot.status && (scHotspot.resolution || null) === (sqHotspot.resolution || null)) return false; + if (scHotspot.status === sqHotspot.status && (scHotspot.resolution ?? null) === (sqHotspot.resolution ?? null)) return false; const changelog = sqHotspot.changelog; if (changelog?.length > 0) {