feat: parallel issue sync with worker threads

[joshua-quek-sonarsource]

Summary

• Adds worker_threads-based parallel issue sync for large projects (≥500 matched pairs)
• 20 workers × 5 concurrency each = 100 concurrent API calls (vs 20 with single-process mapConcurrent)
• SEA-compatible: workers use eval: true with inline code using only Node.js built-in https/http modules
• Includes exponential backoff with jitter (6 retries), 5xx/transient retry, and apiErrors stat tracking
• Falls back to existing mapConcurrent path for <500 matched pairs

Test plan

• Built SEA binary (npm run package) — passes
• Ran transfer on Angular Framework (31K+ SQ issues, 5,119 pre-filtered with manual changes)
• Parallel sync activated: 20 workers, 100 concurrent requests
• Results: 5,067 tagged (99%), 5,109 metadata-sync-tagged (99.8%), 5,095 source-linked (99.5%), 0 failed
• 34 api-errors tracked and logged (SC staging rate limiting — 0.7%)
• Spot-checked SC issues: tags, source link comments, metadata-synchronized tag all present
• Run verify command for full cross-system validation
• Test with <500 matched pairs to confirm fallback path unchanged

🤖 Generated with Claude Code

---

Note

Medium Risk
Changes the behavior and concurrency model of SonarCloud issue write operations (new worker-thread path, inline eval, retries/backoff), which could affect rate limiting and sync completeness if edge cases differ from the existing mapConcurrent flow.

Overview
Enables parallel issue metadata sync for large projects by switching syncIssues (sq-2025 pipeline) to dispatch matched issue pairs to a new parallelSyncIssues() implementation once a 500-pair threshold is reached.

Introduces src/shared/utils/concurrency/helpers/parallel-issue-sync.js, which spawns worker_threads (inline eval code) to execute issue transitions/assignments/comments/tags/source-linking with per-worker concurrency and built-in retry/backoff for 429/5xx/transient errors, then aggregates per-worker stats (including new apiErrors) and feeds progress back to the parent.

Exports parallelSyncIssues from the shared concurrency index and updates docs/changelog to describe the new worker-thread sync path and its tuning knobs.

^{Reviewed by Cursor Bugbot for commit dd815f8. Bugbot is set up for automated code reviews on this repo. Configure here.}

[sonar-review-alpha]

Summary

This PR adds worker_threads-based parallelization for issue metadata sync on large projects (≥500 matched pairs). When activated, it spawns 20 workers with 5 concurrent operations each (100 total concurrent API calls vs. 20 previously). The implementation uses SEA-compatible eval: true workers with inline code and only Node.js built-ins, includes exponential backoff retry for rate limiting, and falls back to the existing mapConcurrent path for smaller datasets.

Key changes:

• New parallelSyncIssues() orchestrator in src/shared/utils/concurrency/helpers/parallel-issue-sync.js (325 lines: ~210 worker code + ~100 parent logic)
• Conditional dispatch in issue-sync migrator (500-pair threshold)
• Stats logging extended to include apiErrors count
• Comprehensive documentation updates explaining design and tuning parameters

What reviewers should know

Where to start:

1. Read parallel-issue-sync.js top-level structure: parallelSyncIssues() function + worker code string + helper functions
2. Review the conditional dispatch in issue-sync/index.js (lines 52–67) — note how configs are serialized for workers
3. Scan the CHANGELOG entry and key-capabilities.md for context on problem/solution

Non-obvious decisions:

• Worker code isolation: The inline WORKER_CODE string is evaluated with workerData in scope. It cannot require() bundled modules — only Node.js built-ins (https, http, worker_threads). This is intentional for SEA compatibility.
• Round-robin partitioning: Issues are distributed via index modulo, not by size/weight. Heavy issues (many comments/transitions) may cluster in a single worker — design accepts this for simplicity.
• Progress tracking: Uses worker._lastReported to track delta since last message; the parent increments only the difference to avoid double-counting.
• Changelog filtering: Each worker receives only the changelog entries for its chunk (lines 293–298), not the full map.
• Stats merging: apiErrors count represents API calls that failed after all retries (tags/comments/source-links are partial if rate-limited). The filtered field is not in worker stats because pre-filtering happens before partitioning.

Gotchas to watch:

• The worker code must be valid when evaluated as a function body — no module-level syntax that won't work under eval: true.
• If concurrency per worker changes, adjust expectations for total concurrent requests (20 workers × N concurrency).
• Retry logic only applies to HTTP errors (429, 5xx, transient network issues); application errors (invalid assignee, etc.) are not retried.
• No explicit worker cleanup on parent abort — workers will continue until completion even if parent errors.

---

• Generate Walkthrough
• Generate Diagram

🗣️ Give feedback

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit dd815f8. Configure here.}

[cursor]

Duplicated sync logic risks divergence from originals

Low Severity

The WORKER_CODE string reimplements ~130 lines of business logic (mapChangelogDiffToTransition, extractTransitionsFromChangelog, getFallbackTransition, syncIssueStatus, syncIssueAssignment, syncIssueComments, syncIssueTags, addSourceLink, syncOneIssue) that already exist as separate helper files in issue-sync/helpers/. Future bug fixes to the originals (e.g., adding a new transition case) won't automatically propagate to this inline copy, creating silent behavioral drift between the parallel and non-parallel sync paths.

 

^{Reviewed by Cursor Bugbot for commit dd815f8. Configure here.}

[sonar-review-alpha]

Solid implementation overall, but there are two issues that need to be addressed before merge: a meaningful behavioral regression in failure handling, and a logic duplication that creates a silent maintenance trap for the dominant code path.

🗣️ Give feedback

[sonar-review-alpha]

Logic duplication: WORKER_CODE reimplements the full issue-sync pipeline — mapChangelogDiffToTransition, extractTransitionsFromChangelog, getFallbackTransition, syncIssueStatus, syncIssueAssignment, syncIssueComments, syncIssueTags, addSourceLink, and syncOneIssue — duplicating the same-named helpers that the sequential path calls from the rest of the codebase.

This is the dominant code path (≥500 pairs = effectively all large migrations), so any bug fix or behaviour change made to the main-process helpers will silently diverge from the parallel path. The Angular Framework run (5,119 pairs) only ever executes the worker copy, not the tested one.

The constraint is real (SEA eval: true workers can't require() bundled modules), but the duplication should be made explicit:

• Add a prominent comment at the top of WORKER_CODE stating it mirrors the functions in issue-sync/helpers/ and listing which files are mirrored.

• Consider a build-time step (e.g., an npm run check:worker-sync script) that asserts the worker string and the source helpers produce identical results for a fixed set of inputs, so drift is caught in CI rather than in production.

• Mark as noise

[sonar-review-alpha]

Bug: Promise.all fails fast — if any single worker crashes (OOM, unexpected exception escaping run().catch(), non-zero exit), the entire parallelSyncIssues() call rejects and all progress from the other 19 workers is discarded. The caller (syncIssues) then surfaces an unhandled exception rather than a partial stats object.

This is the opposite of the sequential path's { settled: true } semantics, which absorbs per-item errors and always returns an aggregated stats object. For a 31K-issue run spread across 20 workers, losing 19 healthy workers' results because one worker OOMs is a significant production risk.

Fix: use Promise.allSettled and merge only the fulfilled results, logging a warning for any rejected workers:

const results = await Promise.allSettled(workerPromises);
const failed = results.filter(r => r.status === 'rejected');
if (failed.length > 0) {
  logger.warn(`${failed.length} worker(s) failed: ${failed.map(r => r.reason?.message).join('; ')}`);
}
const allStats = results.filter(r => r.status === 'fulfilled').map(r => r.value);
const merged = mergeStats(allStats);

Suggested change

	const allStats = await Promise.all(workerPromises);
	const results = await Promise.allSettled(workerPromises);
	const failedWorkers = results.filter(r => r.status === 'rejected');
	if (failedWorkers.length > 0) {
	logger.warn(`Parallel issue sync: ${failedWorkers.length} worker(s) failed — their issues may be incomplete. Errors: ${failedWorkers.map(r => r.reason?.message).join('; ')}`);
	}
	const allStats = results.filter(r => r.status === 'fulfilled').map(r => r.value);
	const merged = mergeStats(allStats);

• Mark as noise