diff --git a/README.md b/README.md index 52fe39c..ee90585 100644 --- a/README.md +++ b/README.md @@ -1,9 +1,11 @@ # ☁️ πŸ‹ CloudVoyager -Migrate your data from self-hosted SonarQube to SonarCloud β€” no re-scanning needed. This was done by reverse-engineering SonarScanner (scan report protobuf files) & then fully rebuilding everything from the ground up on Node.js. +Migrate your data from self-hosted SonarQube Server to SonarQube Cloud β€” no re-scanning needed. -CloudVoyager copies everything β€” projects, code issues, security hotspots, quality gates, quality profiles, permissions, and more β€” directly from SonarQube into SonarCloud. It is available as both a CLI tool and a Desktop app with a guided wizard interface. +CloudVoyager migrates everything β€” projects, code issues, security hotspots, quality gates, quality profiles, permissions, and more β€” directly from SonarQube Server to SonarQube Cloud. It is available as both a CLI tool and a Desktop app with a guided wizard interface. + +CloudVoyager is compatible with all SonarQube Server versions starting from 9.9.x, i.e. 9.9.x, 2025.1.x, 2025.4.x, 2026.1.x and the latest version (2026.2 as of April 2026). Intermediate SonarQube Server version may work but this is not guaranteed. ## βœ… Quick Start (Recommended) @@ -13,9 +15,9 @@ CloudVoyager copies everything β€” projects, code issues, security hotspots, qua | πŸ€” Scenario | Click Below ‡️ | |----------|-------| -| Migrate **one project** from SonarQube to SonarCloud | [Single Project Migration](docs/scenario-single-project.md) | -| Migrate **everything** from SonarQube to **one** SonarCloud org | [Full Migration β€” Single Org](docs/scenario-single-org.md) | -| Migrate **everything** from SonarQube to **multiple** SonarCloud orgs | [Full Migration β€” Multiple Orgs](docs/scenario-multi-org.md) | +| Migrate **one project** from SonarQube Server to SonarQube Cloud | [Single Project Migration](docs/scenario-single-project.md) | +| Migrate **everything** from SonarQube Server to **one** SonarQube Cloud org | [Full Migration β€” Single Org](docs/scenario-single-org.md) | +| Migrate **everything** from SonarQube Server to **multiple** SonarQube Cloud orgs | [Full Migration β€” Multiple Orgs](docs/scenario-multi-org.md) | ## πŸ–₯️ Desktop App (New!) @@ -63,7 +65,7 @@ See the [Desktop App Guide](docs/desktop-app.md) for setup instructions and a wa 1. Download the latest release of CloudVoyager from the [releases page](https://github.com/sonar-solutions/cloudvoyager/releases). -2. Ensure that you have full admin access API tokens for your SonarQube server and your SonarCloud organization. +2. Ensure that you have full admin access API tokens for your SonarQube Server and your SonarQube Cloud organization. 3. Create the `migrate-config.json` file with the required information (see the [full migration docs](docs/scenario-single-org.md) for details). 4. Run the following command in your terminal: ```bash @@ -93,7 +95,7 @@ A **lock file** prevents concurrent runs against the same project, so you cannot # Discard checkpoint and start the migration from scratch ./cloudvoyager transfer -c config.json --force-restart -# Re-extract data from SonarQube but keep other cached phases +# Re-extract data from SonarQube Server but keep other cached phases ./cloudvoyager transfer -c config.json --force-fresh-extract # Clear a stale lock file (e.g. after a hard crash) @@ -103,19 +105,19 @@ A **lock file** prevents concurrent runs against the same project, so you cannot See the [Configuration Reference](docs/configuration.md#checkpoint-settings) for checkpoint options (`transfer.checkpoint`). -## πŸ”„ SonarQube Version Compatibility +## πŸ”„ SonarQube Server Version Compatibility -CloudVoyager ships **four fully independent pipelines** β€” one per SonarQube version range. At runtime, `version-router.js` calls `/api/system/status`, detects the server version, and dynamically loads the matching pipeline. No special configuration is needed. +CloudVoyager ships **four fully independent pipelines** β€” one per SonarQube Server version range. At runtime, `version-router.js` calls `/api/system/status`, detects the server version, and dynamically loads the matching pipeline. No special configuration is needed. -| SonarQube Version | Pipeline | Issue Search Param | MetricKeys | Clean Code Source | Groups API | +| SonarQube Server Version | Pipeline | Issue Search Param | MetricKeys | Clean Code Source | Groups API | |-------------------|----------|-------------------|------------|-------------------|------------| | **9.9 LTS** | `sq-9.9` | `statuses` | Batched (15) | SC enrichment map | Standard | | **10.0 – 10.3** | `sq-10.0` | `statuses` | Batched (15) | Native from SQ | Standard | | **10.4 – 10.8** | `sq-10.4` | `issueStatuses` | Batched (15) | Native from SQ | Standard | | **2025.1+** | `sq-2025` | `issueStatuses` | No batching | Native from SQ | Web API V2 fallback | -Each pipeline folder (`src/pipelines/sq-*`) contains its own SonarQube client, SonarCloud client, protobuf builder, and transfer/migrate logic β€” no runtime version checks or `if/else` branches. +Each pipeline folder (`src/pipelines/sq-*`) contains its own SonarQube Server client, SonarQube Cloud client, protobuf builder, and transfer/migrate logic β€” no runtime version checks or `if/else` branches. See [Backward Compatibility](docs/backward-compatibility.md) for technical details on how version differences are handled. @@ -152,7 +154,7 @@ Want to build and test CloudVoyager locally? See the [Local Development Guide](d | [Technical Details](docs/technical-details.md) | Protobuf encoding, measure types, concurrency model | | [Troubleshooting](docs/troubleshooting.md) | Common errors and how to fix them | | [Dry-Run CSV Reference](docs/dry-run-csv-reference.md) | CSV schema documentation for the dry-run workflow (including user mapping) | -| [Backward Compatibility](docs/backward-compatibility.md) | SonarQube version support (9.9 LTS through 2025.1+) | +| [Backward Compatibility](docs/backward-compatibility.md) | SonarQube Server version support (9.9 LTS through 2025.1+) | | [Desktop App Guide](docs/desktop-app.md) | Installation, wizard walkthrough, and building from source | | [Verification](docs/verification.md) | Migration verification checks, pass/fail criteria, and report formats | | [Pseudocode Explanation](docs/pseudocode-explanation.md) | Every feature documented in pseudocode for technical review | diff --git a/docs/Audit & Error Events Logging Strategy.md b/docs/Audit & Error Events Logging Strategy.md index ab74800..68e3ffd 100644 --- a/docs/Audit & Error Events Logging Strategy.md +++ b/docs/Audit & Error Events Logging Strategy.md @@ -187,7 +187,7 @@ With stack traces appended for errors: **Error with stack trace:** ``` 2026-05-07 01:15:00 [error]: Project JIRA-456 FAILED -Error: SonarQube API error +Error: SonarQube Server API error at SonarQubeClient.request (/src/sonarqube/api-client.js:45:12) at async migrateOneProjectCore (/src/pipelines/sq-2025/pipeline/project-core-migrator/helpers/migrate-one-project-core.js:67:20) ``` @@ -225,8 +225,8 @@ CloudVoyagerError (base) β”œβ”€β”€ ConfigurationError <- Invalid configuration values β”œβ”€β”€ ValidationError <- Input validation failures β”œβ”€β”€ StateError <- Invalid state transitions -β”œβ”€β”€ SonarQubeAPIError <- SonarQube API failures -β”œβ”€β”€ SonarCloudAPIError <- SonarCloud API failures +β”œβ”€β”€ SonarQubeAPIError <- SonarQube Server API failures +β”œβ”€β”€ SonarCloudAPIError <- SonarQube Cloud API failures β”œβ”€β”€ AuthenticationError <- Auth/credential failures β”œβ”€β”€ ProtobufEncodingError <- Protobuf serialization errors β”œβ”€β”€ GracefulShutdownError <- Controlled shutdown signal diff --git a/docs/Business Value.md b/docs/Business Value.md index 45fb1de..278a226 100644 --- a/docs/Business Value.md +++ b/docs/Business Value.md @@ -2,7 +2,7 @@ -CloudVoyager delivers a complete SonarQube-to-SonarCloud migration without requiring a single line of code to be re-scanned. By reverse-engineering SonarScanner's internal protobuf report protocol, CloudVoyager extracts all data directly from SonarQube's API and repackages it into the exact binary format that SonarCloud's Compute Engine expects. +CloudVoyager delivers a complete SonarQube Server-to-SonarQube Cloud migration without requiring a single line of code to be re-scanned. By reverse-engineering SonarScanner's internal protobuf report protocol, CloudVoyager extracts all data directly from SonarQube Server's API and repackages it into the exact binary format that SonarQube Cloud's Compute Engine expects. --- @@ -19,14 +19,14 @@ Traditional migration approaches require re-running CI/CD scanners against every - **Days to weeks of pipeline execution** β€” Each scan must complete in sequence or with limited parallelism, constrained by CI/CD runner availability - **Developer interruption** β€” Teams must update branch configurations, trigger manual pipelines, and monitor for failures - **Locked CI/CD resources** β€” Build agents are occupied running scans instead of their normal development work -- **Extended validation cycles** β€” Each re-scan potentially surfaces new issues that did not exist in the original SonarQube, requiring investigation +- **Extended validation cycles** β€” Each re-scan potentially surfaces new issues that did not exist in the original SonarQube Server, requiring investigation ### CloudVoyager Eliminates This Entirely -CloudVoyager connects directly to your existing SonarQube server via API, extracts all project data, and uploads it to SonarCloud as a legitimate scanner submission β€” without touching your source code or CI/CD pipelines. +CloudVoyager connects directly to your existing SonarQube Server via API, extracts all project data, and uploads it to SonarQube Cloud as a legitimate scanner submission β€” without touching your source code or CI/CD pipelines. **Real-world example from production use:** @@ -48,7 +48,7 @@ This same migration via re-scanning would have taken days,占用 significant CI/ - **No CI/CD disruption** β€” Migration runs in the background while your pipelines continue normal operations - **No coordination required** β€” Individual development teams do not need to take any action - **Predictable timeline** β€” A 50-project portfolio migrates in hours, not weeks -- **Zero re-scan risk** β€” Code that passed SonarQube analysis arrives in SonarCloud with the same issue status, not potentially different results from a re-run +- **Zero re-scan risk** β€” Code that passed SonarQube Server analysis arrives in SonarQube Cloud with the same issue status, not potentially different results from a re-run --- @@ -60,7 +60,7 @@ This same migration via re-scanning would have taken days,占用 significant CI/ -CloudVoyager migrates every category of data that SonarQube tracks: +CloudVoyager migrates every category of data that SonarQube Server tracks: | Category | Data Preserved | |----------|---------------| @@ -76,12 +76,12 @@ CloudVoyager migrates every category of data that SonarQube tracks: -Each issue carries its complete lifecycle history from SonarQube: +Each issue carries its complete lifecycle history from SonarQube Server: -- **Original creation date** β€” Preserved via SCM date backdating, so SonarCloud shows the same temporal distribution as SonarQube +- **Original creation date** β€” Preserved via SCM date backdating, so SonarQube Cloud shows the same temporal distribution as SonarQube Server - **Status transitions** β€” Confirmed, False Positive, Accepted, Won't Fix, Resolved, Reopened states all synced -- **Assignments** β€” Issues assigned to specific users transfer to the corresponding SonarCloud user -- **Comments** β€” Full comment history preserved with `[Migrated from SonarQube]` attribution +- **Assignments** β€” Issues assigned to specific users transfer to the corresponding SonarQube Cloud user +- **Comments** β€” Full comment history preserved with `[Migrated from SonarQube Server]` attribution - **Tags** β€” Custom categorization labels maintained @@ -89,7 +89,7 @@ Each issue carries its complete lifecycle history from SonarQube: -CloudVoyager preserves each issue's original SonarQube creation date by backdating SCM changeset blame dates in the protobuf report. Issues appear in SonarCloud with the same creation timestamp they had in SonarQube, maintaining the historical distribution in the creation date facet. +CloudVoyager preserves each issue's original SonarQube Server creation date by backdating SCM changeset blame dates in the protobuf report. Issues appear in SonarQube Cloud with the same creation timestamp they had in SonarQube Server, maintaining the historical distribution in the creation date facet. For calendar days with more than 5,000 issues, CloudVoyager automatically splits into sub-groups with 1-day-spaced synthetic dates to prevent clustering, ensuring a realistic distribution across the project's history. @@ -108,7 +108,7 @@ For calendar days with more than 5,000 issues, CloudVoyager automatically splits -Before any data touches SonarCloud, run the migration in dry-run mode: +Before any data touches SonarQube Cloud, run the migration in dry-run mode: ```bash cloudvoyager migrate -c migrate-config.json --dry-run @@ -116,9 +116,9 @@ cloudvoyager migrate -c migrate-config.json --dry-run This extracts all data and generates 9 CSV mapping files for review. You can: -- Verify which projects map to which SonarCloud organizations +- Verify which projects map to which SonarQube Cloud organizations - Exclude specific projects or branches from migration -- Map SonarQube users to SonarCloud users before migration +- Map SonarQube Server users to SonarQube Cloud users before migration - Review quality gate and profile assignments Only after reviewing the generated CSVs do you proceed to the actual migration. @@ -134,20 +134,20 @@ The checkpoint journal tracks: - Per-organization completion status - Per-project progress across 11 migration phases - Upload deduplication via CE task ID verification (prevents re-uploading completed analyses) -- Session fingerprint validation (warns on SonarQube version changes between runs) +- Session fingerprint validation (warns on SonarQube Server version changes between runs) #### Layer 3: Post-Migration Verification -After migration completes, the verification pipeline runs **58+ automated checks** comparing SonarQube against SonarCloud: +After migration completes, the verification pipeline runs **58+ automated checks** comparing SonarQube Server against SonarQube Cloud: | Verification Area | Checks Performed | |-------------------|-----------------| | **Issues** | Count parity, status matching, status history (changelog sequence), assignments, comments, tags | | **Hotspots** | Count parity, status matching (Safe, Acknowledged, Fixed, To Review), comments | -| **Branches** | All SonarQube branches exist in SonarCloud | +| **Branches** | All SonarQube Server branches exist in SonarQube Cloud | | **Measures** | 18 key metrics compared (ncloc, complexity, coverage, violations, etc.) | | **Quality Gates** | Existence, condition definitions, project assignments | | **Quality Profiles** | Existence, active rule counts, project assignments | @@ -230,7 +230,7 @@ This parallelism means CloudVoyager fully utilizes available resources without r -Moving to SonarCloud should not mean rebuilding your entire quality governance framework from scratch. CloudVoyager migrates every governance artifact: +Moving to SonarQube Cloud should not mean rebuilding your entire quality governance framework from scratch. CloudVoyager migrates every governance artifact: | Governance Element | What Gets Migrated | |-------------------|-------------------| @@ -252,29 +252,29 @@ Quality gates are recreated with their exact condition logic: - **Metric + operator + threshold** preserved for every condition - **Gate permissions** migrated for custom gates - **Project assignments** applied per organization mapping -- Built-in gates (Sonar way) use SonarCloud's default since they are platform-managed +- Built-in gates (Sonar way) use SonarQube Cloud's default since they are platform-managed -This means your existing quality standards continue uninterrupted β€” projects that passed SonarQube's gate pass SonarCloud's gate with the same criteria. +This means your existing quality standards continue uninterrupted β€” projects that passed SonarQube Server's gate pass SonarQube Cloud's gate with the same criteria. ### Quality Profile Fidelity -Quality profiles use SonarQube's native backup/restore XML format, preserving: +Quality profiles use SonarQube Server's native backup/restore XML format, preserving: - **All rule activations and deactivations** - **Severity overrides** for individual rules - **Rule parameter values** (for rules with configurable parameters) - **Inheritance chains** β€” profiles restored in dependency order so parent profiles exist before children -**Built-in profile handling** is handled specially. SonarCloud's built-in profiles cannot be overwritten, so CloudVoyager extracts the built-in profile's rules from SonarQube and creates a custom profile with a `(SonarQube Migrated)` suffix. This ensures the exact same rules are active in SonarCloud as they were in SonarQube. +**Built-in profile handling** is handled specially. SonarQube Cloud's built-in profiles cannot be overwritten, so CloudVoyager extracts the built-in profile's rules from SonarQube Server and creates a custom profile with a `(SonarQube Server Migrated)` suffix. This ensures the exact same rules are active in SonarQube Cloud as they were in SonarQube Server. **Quality profile diff reports** are generated after migration, showing: -- **Missing rules** β€” Active in SonarQube but not available in SonarCloud -- **Added rules** β€” Available in SonarCloud but not in SonarQube +- **Missing rules** β€” Active in SonarQube Server but not available in SonarQube Cloud +- **Added rules** β€” Available in SonarQube Cloud but not in SonarQube Server -This enables governance teams to review rule parity before cutting over to SonarCloud. +This enables governance teams to review rule parity before cutting over to SonarQube Cloud. ### Project Settings and Bindings @@ -290,7 +290,7 @@ Per-project configuration that transfers includes: - **DevOps bindings** (GitHub, GitLab, Azure DevOps, Bitbucket) for pull request decoration and automatic branch analysis - **Non-inherited project-level settings** -This ensures each project arrives in SonarCloud fully configured and ready for your development teams to use immediately. +This ensures each project arrives in SonarQube Cloud fully configured and ready for your development teams to use immediately. --- @@ -307,7 +307,7 @@ CloudVoyager's business value is straightforward: | **Cost Efficiency** | No CI/CD runner consumption, no developer coordination overhead | | **Governance Continuity** | Quality gates, profiles, permissions, groups, and templates maintained across the migration | -CloudVoyager is purpose-built for enterprises that need to migrate to SonarCloud without disrupting development teams, sacrificing historical data, or rebuilding governance configuration from scratch. +CloudVoyager is purpose-built for enterprises that need to migrate to SonarQube Cloud without disrupting development teams, sacrificing historical data, or rebuilding governance configuration from scratch. --- diff --git a/docs/CHANGELOG.md b/docs/CHANGELOG.md index aca1b95..202b3a2 100644 --- a/docs/CHANGELOG.md +++ b/docs/CHANGELOG.md @@ -10,7 +10,7 @@ **Root causes identified and fixed:** -1. **POST body format + Content-Type** β€” Settings were sent as URL query parameters (`/api/settings/set?key=...&values=...`) with a `null` body and `Content-Type: application/json` (from axios defaults). Changed to send parameters as a form-encoded POST body string with an explicit `Content-Type: application/x-www-form-urlencoded` header, which is the correct format for SonarQube/SonarCloud web API POST endpoints. The explicit header is required because the axios client's default `Content-Type: application/json` would not be overridden by auto-detection (axios uses `rewrite=false` for URLSearchParams). +1. **POST body format + Content-Type** β€” Settings were sent as URL query parameters (`/api/settings/set?key=...&values=...`) with a `null` body and `Content-Type: application/json` (from axios defaults). Changed to send parameters as a form-encoded POST body string with an explicit `Content-Type: application/x-www-form-urlencoded` header, which is the correct format for SonarQube Server/SonarQube Cloud web API POST endpoints. The explicit header is required because the axios client's default `Content-Type: application/json` would not be overridden by auto-detection (axios uses `rewrite=false` for URLSearchParams). 2. **Truthy check bug in `dispatchSettingToApi`** β€” Used `if (setting.value)` which is a JavaScript truthy check. This silently skipped settings with empty string values (`value: ""`). Changed to `if (setting.value !== undefined && setting.value !== null)` for correctness. @@ -40,7 +40,7 @@ -Each supported SonarQube version range gets its own **complete, independent pipeline directory** under `src/pipelines/`. There are no runtime version checks within any pipeline -- each pipeline has its behavior hardcoded for its target version range. +Each supported SonarQube Server version range gets its own **complete, independent pipeline directory** under `src/pipelines/`. There are no runtime version checks within any pipeline -- each pipeline has its behavior hardcoded for its target version range. ``` src/pipelines/ -β”œβ”€β”€ sq-9.9/ # SonarQube 9.9 LTS -β”œβ”€β”€ sq-10.0/ # SonarQube 10.0–10.3 -β”œβ”€β”€ sq-10.4/ # SonarQube 10.4–10.8 -└── sq-2025/ # SonarQube 2025.1+ +β”œβ”€β”€ sq-9.9/ # SonarQube Server 9.9 LTS +β”œβ”€β”€ sq-10.0/ # SonarQube Server 10.0–10.3 +β”œβ”€β”€ sq-10.4/ # SonarQube Server 10.4–10.8 +└── sq-2025/ # SonarQube Server 2025.1+ ``` -The **version router** (`src/version-router.js`) detects the SonarQube server version at runtime and dynamically imports the correct pipeline. This means: +The **version router** (`src/version-router.js`) detects the SonarQube Server version at runtime and dynamically imports the correct pipeline. This means: - **Never add version-conditional logic inside a pipeline.** If behavior differs between versions, it belongs in a separate pipeline directory. - **Never import across pipeline boundaries.** Each pipeline is self-contained. @@ -35,10 +35,10 @@ src/ β”œβ”€β”€ version-router.js # Detects SQ version, loads correct pipeline β”œβ”€β”€ commands/ # CLI command handlers (transfer, migrate, sync-metadata, verify) β”œβ”€β”€ pipelines/ # Version-specific pipeline implementations -β”‚ β”œβ”€β”€ sq-9.9/ # SonarQube 9.9 LTS -β”‚ β”œβ”€β”€ sq-10.0/ # SonarQube 10.0–10.3 -β”‚ β”œβ”€β”€ sq-10.4/ # SonarQube 10.4–10.8 -β”‚ └── sq-2025/ # SonarQube 2025.1+ +β”‚ β”œβ”€β”€ sq-9.9/ # SonarQube Server 9.9 LTS +β”‚ β”œβ”€β”€ sq-10.0/ # SonarQube Server 10.0–10.3 +β”‚ β”œβ”€β”€ sq-10.4/ # SonarQube Server 10.4–10.8 +β”‚ └── sq-2025/ # SonarQube Server 2025.1+ └── shared/ # Version-independent shared code β”œβ”€β”€ config/ # Configuration loading and validation (Ajv-based) β”œβ”€β”€ mapping/ # Organization mapping and CSV tools @@ -54,7 +54,7 @@ Each pipeline directory (`src/pipelines/sq-{version}/`) contains a complete, sel sq-{version}/ β”œβ”€β”€ transfer-pipeline.js # Single-project transfer orchestrator β”œβ”€β”€ migrate-pipeline.js # Full multi-org migration orchestrator -β”œβ”€β”€ sonarqube/ # SonarQube integration +β”œβ”€β”€ sonarqube/ # SonarQube Server integration β”‚ β”œβ”€β”€ api-client.js # HTTP client with pagination, auth, SCM revision β”‚ β”œβ”€β”€ models.js # Data models and factory functions β”‚ β”œβ”€β”€ api/ # API method modules @@ -79,8 +79,8 @@ sq-{version}/ β”‚ └── schema/ # .proto definitions β”‚ β”œβ”€β”€ constants.proto β”‚ └── scanner-report.proto -β”œβ”€β”€ sonarcloud/ # SonarCloud integration -β”‚ β”œβ”€β”€ api-client.js # SonarCloud HTTP client (retry, throttle) +β”œβ”€β”€ sonarcloud/ # SonarQube Cloud integration +β”‚ β”œβ”€β”€ api-client.js # SonarQube Cloud HTTP client (retry, throttle) β”‚ β”œβ”€β”€ uploader.js # Report packaging and CE submission β”‚ β”œβ”€β”€ enterprise-client.js # Enterprise API client β”‚ β”œβ”€β”€ rule-enrichment.js # Rule enrichment for Clean Code attributes @@ -158,7 +158,7 @@ export async function getXxx(client, param) { ### 3. Migrator Pattern (`src/pipelines/sq-{version}/sonarcloud/migrators/`) -Transform extracted SonarQube data and apply it to SonarCloud: +Transform extracted SonarQube Server data and apply it to SonarQube Cloud: ```js export async function migrateXxx(extractedData, scClient, options) { @@ -191,8 +191,8 @@ Each pipeline has its own `SonarQubeClient` tailored to its version range: | Error Class | When to Use | |-------------|-------------| | `ConfigurationError` | Invalid config file or missing required fields | -| `SonarQubeAPIError` | SonarQube API failures (include endpoint in constructor) | -| `SonarCloudAPIError` | SonarCloud API failures (include endpoint in constructor) | +| `SonarQubeAPIError` | SonarQube Server API failures (include endpoint in constructor) | +| `SonarCloudAPIError` | SonarQube Cloud API failures (include endpoint in constructor) | | `AuthenticationError` | 401/403 responses (include service name) | | `ProtobufEncodingError` | Protobuf encoding failures | | `StateError` | State file I/O failures | @@ -205,7 +205,7 @@ Each pipeline has its own `SonarQubeClient` tailored to its version range: ### 6. Pagination -All paginated SonarQube APIs use `getPaginated()`: +All paginated SonarQube Server APIs use `getPaginated()`: ```js return await this.getPaginated('/api/xxx/search', { ps: 500 }, 'items'); @@ -213,7 +213,7 @@ return await this.getPaginated('/api/xxx/search', { ps: 500 }, 'items'); **Rules:** - Default page size: 500 -- Override to `ps: 100` for permission-related APIs (SonarQube limitation) +- Override to `ps: 100` for permission-related APIs (SonarQube Server limitation) - Handles both `data.paging.total` (10.x+) and `data.total` (9.x) response formats - Never implement manual pagination -- always use `getPaginated()` @@ -251,7 +251,7 @@ sqIssues (100K) β†’ fetchSqChangelogs() β†’ hasManualChanges() filter β†’ ~1-2K An issue is considered to have **manual changes** if any of: 1. Its changelog has at least one entry with a non-empty `user` field (human actor, not system) -2. It has comments not prefixed with `[Migrated from SonarQube]` (manual, not auto-generated) +2. It has comments not prefixed with `[Migrated from SonarQube Server]` (manual, not auto-generated) 3. It has custom tags (`tags` array is non-empty) **Shared utilities:** @@ -352,10 +352,10 @@ npm run lint # ESLint --- -## Adding New SonarQube Version Support +## Adding New SonarQube Server Version Support -When a new SonarQube version introduces API changes: +When a new SonarQube Server version introduces API changes: 1. **Research** the API differences (check release notes, deprecation logs) 2. **Create a new pipeline directory** under `src/pipelines/sq-{version}/` by copying the closest existing pipeline as a starting point diff --git a/docs/Core Technologies.md b/docs/Core Technologies.md index 7c1bdc1..ad8f1b7 100644 --- a/docs/Core Technologies.md +++ b/docs/Core Technologies.md @@ -2,7 +2,7 @@ -CloudVoyager is built on a carefully selected stack of battle-tested open-source technologies. Each technology was chosen to address specific requirements of the SonarQube-to-SonarCloud migration workflow. +CloudVoyager is built on a carefully selected stack of battle-tested open-source technologies. Each technology was chosen to address specific requirements of the SonarQube Server-to-SonarQube Cloud migration workflow. ## 1. Node.js @@ -49,12 +49,12 @@ This approach produces a distributable binary that does not require the end user -SonarCloud accepts scanner reports only in Protocol Buffer format. Protobuf was chosen because: +SonarQube Cloud accepts scanner reports only in Protocol Buffer format. Protobuf was chosen because: -- **SonarCloud API requirement** β€” The scanner report upload endpoint expects a binary-encoded Protobuf payload, not JSON +- **SonarQube Cloud API requirement** β€” The scanner report upload endpoint expects a binary-encoded Protobuf payload, not JSON - **Compact binary encoding** β€” Protobuf messages are significantly smaller than equivalent JSON, reducing upload time for large codebases - **Strict schema enforcement** β€” The `.proto` schema definitions (`scanner-report.proto`) catch structural errors at build time rather than at upload time -- **Language neutrality** β€” The same schema is used across all supported SonarQube versions (9.9, 10.0, 10.4, 2025.1) +- **Language neutrality** β€” The same schema is used across all supported SonarQube Server versions (9.9, 10.0, 10.4, 2025.1) ### Protobuf Schema @@ -151,7 +151,7 @@ src/commands/ reset.js β€” Reset migration state status.js β€” Show migration status validate.js β€” Validate configuration - test-connection.js β€” Test SonarQube/SonarCloud connectivity + test-connection.js β€” Test SonarQube Server/SonarQube Cloud connectivity ``` ## 5. Ajv @@ -249,7 +249,7 @@ npm run build:linux-arm64 # Linux ARM64 CI/CD is implemented entirely in GitHub Actions because: - **Native to the repository** β€” No external CI service configuration required -- **Matrix builds** β€” Test across Node.js versions, platforms, and SonarQube versions in parallel +- **Matrix builds** β€” Test across Node.js versions, platforms, and SonarQube Server versions in parallel - **Artifact sharing** β€” `actions/cache` (not `actions/upload-artifact`) passes build artifacts between jobs - **Workflow composition** β€” Reusable workflows (`workflow_call`) allow `build.yml` to be composed into release pipelines - **Secrets management** β€” `SONARCLOUD_TOKEN` and `SONARQUBE_TOKEN` stored as repository secrets @@ -263,7 +263,7 @@ CI/CD is implemented entirely in GitHub Actions because: | `unit-tests.yml` | Run AVA unit tests on every push | | `build.yml` | Build SEA binaries for all 6 platforms | | `build-desktop.yml` | Build Electron desktop app for all platforms | -| `regression.yml` | End-to-end regression tests with real SonarQube/SonarCloud | +| `regression.yml` | End-to-end regression tests with real SonarQube Server/SonarQube Cloud | | `release.yml` | Create GitHub releases with attached binaries | | `gh-release.yml` | Draft GitHub release notes | | `version-bump.yml` | Automated version bumping on main branch | @@ -287,4 +287,4 @@ Each platform is built in a dedicated job: -Regression workflows spin up real SonarQube and SonarCloud instances and run actual migration commands. The workflows use `actions/cache` to share state between jobs rather than artifact uploads, following the project's CI best practice of using cache for inter-job data transfer. +Regression workflows spin up real SonarQube Server and SonarQube Cloud instances and run actual migration commands. The workflows use `actions/cache` to share state between jobs rather than artifact uploads, following the project's CI best practice of using cache for inter-job data transfer. diff --git a/docs/Easy Local Development Guide.md b/docs/Easy Local Development Guide.md index e148a77..961cc39 100644 --- a/docs/Easy Local Development Guide.md +++ b/docs/Easy Local Development Guide.md @@ -2,7 +2,7 @@ -Welcome to CloudVoyager! This guide walks you through setting up and running CloudVoyager on your local machine. No prior experience with SonarQube or SonarCloud is required β€” just follow the steps below. +Welcome to CloudVoyager! This guide walks you through setting up and running CloudVoyager on your local machine. No prior experience with SonarQube Server or SonarQube Cloud is required β€” just follow the steps below. --- @@ -10,7 +10,7 @@ Welcome to CloudVoyager! This guide walks you through setting up and running Clo -CloudVoyager is a command-line tool that migrates projects from **SonarQube** (self-hosted) to **SonarCloud** (cloud-hosted). It transfers source code, quality gates, quality profiles, permissions, and more. +CloudVoyager is a command-line tool that migrates projects from **SonarQube Server** (self-hosted) to **SonarQube Cloud** (cloud-hosted). It transfers source code, quality gates, quality profiles, permissions, and more. **In plain English:** It copies your code analysis data from your own server to the cloud, saving you from doing it manually. @@ -155,8 +155,8 @@ CloudVoyager v1.x.x CloudVoyager needs a JSON configuration file that tells it: -- How to connect to your SonarQube server (source) -- How to connect to your SonarCloud organization (destination) +- How to connect to your SonarQube Server (source) +- How to connect to your SonarQube Cloud organization (destination) - Which projects to migrate Create a file called `config.json` at the project root: @@ -187,8 +187,8 @@ Create a file called `config.json` at the project root: ``` **Where to find your tokens:** -- **SonarQube token:** Log in to SonarQube β†’ My Account β†’ Security β†’ Generate Tokens -- **SonarCloud token:** Log in to SonarCloud β†’ Account β†’ Security β†’ Generate Tokens +- **SonarQube Server token:** Log in to SonarQube Server β†’ My Account β†’ Security β†’ Generate Tokens +- **SonarQube Cloud token:** Log in to SonarQube Cloud β†’ Account β†’ Security β†’ Generate Tokens --- @@ -220,7 +220,7 @@ ERROR: Missing required field 'source.sonarQube.url' -Test that CloudVoyager can connect to both SonarQube and SonarCloud: +Test that CloudVoyager can connect to both SonarQube Server and SonarQube Cloud: ```bash ./dist/bin/cloudvoyager-macos-arm64 test -c config.json --verbose @@ -229,8 +229,8 @@ Test that CloudVoyager can connect to both SonarQube and SonarCloud: **Expected output (success):** ``` Testing connections... -Source (SonarQube): Connected successfully -Target (SonarCloud): Connected successfully +Source (SonarQube Server): Connected successfully +Target (SonarQube Cloud): Connected successfully All connections verified. ``` @@ -356,7 +356,7 @@ npm run package ### "Connection refused" when testing Check that: -1. Your SonarQube/SonarCloud URLs are accessible from your network +1. Your SonarQube Server/SonarQube Cloud URLs are accessible from your network 2. Your tokens are valid and not expired 3. Your firewall allows outbound HTTPS (port 443) @@ -382,10 +382,10 @@ You can override config values using environment variables: |----------|-------------| | `LOG_LEVEL` | Set logging level: `debug`, `info`, `warn`, `error` | | `LOG_FILE` | Path to log file (logs to console by default) | -| `SONARQUBE_TOKEN` | Override SonarQube token | -| `SONARCLOUD_TOKEN` | Override SonarCloud token | -| `SONARQUBE_URL` | Override SonarQube URL | -| `SONARCLOUD_URL` | Override SonarCloud URL | +| `SONARQUBE_TOKEN` | Override SonarQube Server token | +| `SONARCLOUD_TOKEN` | Override SonarQube Cloud token | +| `SONARQUBE_URL` | Override SonarQube Server URL | +| `SONARCLOUD_URL` | Override SonarQube Cloud URL | **Example:** diff --git a/docs/architecture.md b/docs/architecture.md index da16ac6..6b8a4df 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -7,7 +7,7 @@ ## πŸ“ Project Structure -CloudVoyager uses a **pipeline-per-version** architecture. Each supported SonarQube version range has its own self-contained pipeline, while shared (version-independent) code lives in `src/shared/`. +CloudVoyager uses a **pipeline-per-version** architecture. Each supported SonarQube Server version range has its own self-contained pipeline, while shared (version-independent) code lives in `src/shared/`. ``` src/ @@ -19,10 +19,10 @@ src/ β”‚ β”œβ”€β”€ sync-metadata.js # Standalone metadata sync command β”‚ └── verify.js # Migration verification command β”œβ”€β”€ pipelines/ # Version-specific pipeline implementations -β”‚ β”œβ”€β”€ sq-9.9/ # SonarQube 9.9 LTS -β”‚ β”œβ”€β”€ sq-10.0/ # SonarQube 10.0–10.3 -β”‚ β”œβ”€β”€ sq-10.4/ # SonarQube 10.4–10.8 -β”‚ └── sq-2025/ # SonarQube 2025.1+ +β”‚ β”œβ”€β”€ sq-9.9/ # SonarQube Server 9.9 LTS +β”‚ β”œβ”€β”€ sq-10.0/ # SonarQube Server 10.0–10.3 +β”‚ β”œβ”€β”€ sq-10.4/ # SonarQube Server 10.4–10.8 +β”‚ └── sq-2025/ # SonarQube Server 2025.1+ └── shared/ # Version-independent shared code β”œβ”€β”€ config/ # Configuration loading and validation β”‚ β”œβ”€β”€ loader.js # Config loading (Ajv + ajv-formats) for transfer commands @@ -67,7 +67,7 @@ src/ β”‚ β”œβ”€β”€ shutdown.js # Graceful SIGINT/SIGTERM shutdown coordinator β”‚ β”œβ”€β”€ progress.js # Checkpoint progress display and ETA β”‚ β”œβ”€β”€ prompt.js # Interactive user prompts (confirmation dialogs) - β”‚ β”œβ”€β”€ version.js # SonarQube version parsing and comparison + β”‚ β”œβ”€β”€ version.js # SonarQube Server version parsing and comparison β”‚ β”œβ”€β”€ portfolio-skip.js # handleMissingEnterpriseKey β€” graceful portfolio skip when enterprise key absent β”‚ β”œβ”€β”€ search-slicer/ # Date-window slicing for 10K+ issue retrieval (SQ & SC) β”‚ β”‚ β”œβ”€β”€ index.js # fetchWithSlicing orchestrator @@ -90,7 +90,7 @@ src/ β”‚ β”‚ β”œβ”€β”€ apply-pre-filter.js # Applies hasManualChanges pre-filter; sets stats.filtered β”‚ β”‚ └── wait-for-sc-indexing.js # Retries SC fetch until analysis is indexed (Issue #91) β”‚ └── fallback-repos/ - β”‚ └── index.js # 44 known SonarCloud rule repositories (fallback set) + β”‚ └── index.js # 44 known SonarQube Cloud rule repositories (fallback set) └── verification/ # Migration verification β”œβ”€β”€ verify-pipeline.js # Verification orchestrator (read-only comparison) β”œβ”€β”€ checkers/ # Per-check verification modules @@ -141,7 +141,7 @@ sq-{version}/ β”‚ β”œβ”€β”€ index.js # Full multi-org migration orchestrator β”‚ └── helpers/ # 10 helper files β”‚ -β”œβ”€β”€ sonarqube/ # SonarQube integration +β”œβ”€β”€ sonarqube/ # SonarQube Server integration β”‚ β”œβ”€β”€ api-client.js # Re-export β†’ api-client/index.js β”‚ β”œβ”€β”€ api-client/ β”‚ β”‚ β”œβ”€β”€ index.js # HTTP client (factory function: createSonarQubeClient) @@ -194,7 +194,7 @@ sq-{version}/ β”‚ β”œβ”€β”€ scanner-report.proto β”‚ └── constants.proto β”‚ -β”œβ”€β”€ sonarcloud/ # SonarCloud integration +β”œβ”€β”€ sonarcloud/ # SonarQube Cloud integration β”‚ β”œβ”€β”€ api-client.js # Re-export β†’ api-client/index.js (factory: createSonarCloudClient) β”‚ β”œβ”€β”€ api-client/ β”‚ β”‚ β”œβ”€β”€ index.js @@ -207,7 +207,7 @@ sq-{version}/ β”‚ β”œβ”€β”€ enterprise-client/ β”‚ β”‚ β”œβ”€β”€ index.js β”‚ β”‚ └── helpers/ # 4 helper files -β”‚ β”œβ”€β”€ rule-enrichment.js # Rule enrichment from SonarCloud (sq-9.9 uses this) +β”‚ β”œβ”€β”€ rule-enrichment.js # Rule enrichment from SonarQube Cloud (sq-9.9 uses this) β”‚ β”œβ”€β”€ api/ β”‚ β”‚ β”œβ”€β”€ hotspots.js # Hotspot API methods (<50 lines) β”‚ β”‚ β”œβ”€β”€ issues.js # Re-export β†’ issues/helpers/ (3 helpers) @@ -256,7 +256,7 @@ sq-{version}/ ### Shared Utilities β€” SCM Date Backdating -The **batch-distributor** (`src/shared/utils/batch-distributor/`) preserves each issue's original SonarQube creation date in SonarCloud by rewriting SCM changeset blame dates in the protobuf report. +The **batch-distributor** (`src/shared/utils/batch-distributor/`) preserves each issue's original SonarQube Server creation date in SonarQube Cloud by rewriting SCM changeset blame dates in the protobuf report. The primary function is `backdateChangesets(extractedData)`, which: @@ -277,18 +277,18 @@ Legacy helpers (`computeBatchPlan`, `computeBatchDate`, `createBatchExtractedDat ## πŸ”„ Version Routing -`version-router.js` detects the SonarQube server version and dynamically imports the correct pipeline: +`version-router.js` detects the SonarQube Server version and dynamically imports the correct pipeline: 1. Makes a lightweight `GET /api/system/status` call to get the server version 2. Maps the version to a pipeline: `sq-9.9`, `sq-10.0`, `sq-10.4`, or `sq-2025` 3. Dynamically imports `transfer-pipeline.js` and `migrate-pipeline.js` from the selected pipeline 4. Returns the pipeline functions to the calling command -No runtime version checks exist within any pipeline β€” each pipeline has its behavior hardcoded for its target SonarQube version range. +No runtime version checks exist within any pipeline β€” each pipeline has its behavior hardcoded for its target SonarQube Server version range. | SQ Version | Pipeline | Key Differences | |------------|----------|-----------------| -| 9.9 LTS | sq-9.9 | Legacy `statuses` param, Clean Code enriched from SonarCloud | +| 9.9 LTS | sq-9.9 | Legacy `statuses` param, Clean Code enriched from SonarQube Cloud | | 10.0–10.3 | sq-10.0 | Legacy `statuses` param, native Clean Code | | 10.4–10.8 | sq-10.4 | Modern `issueStatuses` param | | 2025.1+ | sq-2025 | Modern `issueStatuses` param, Web API V2 with fallbacks | @@ -318,12 +318,12 @@ Uses `pipelines/sq-{version}/transfer-pipeline.js` (selected by version-router): 2. **Initialize state** β€” load previous state for incremental transfers 3. **Acquire lock file** β€” prevent concurrent runs on the same project 4. **Initialize checkpoint journal** β€” load or create checkpoint for pause/resume -5. **Test connections** β€” verify SonarQube and SonarCloud connectivity -6. **Extract data** β€” extract project data from SonarQube (issues, sources, measures, rules, hotspots, etc.) β€” 10+ extraction phases +5. **Test connections** β€” verify SonarQube Server and SonarQube Cloud connectivity +6. **Extract data** β€” extract project data from SonarQube Server (issues, sources, measures, rules, hotspots, etc.) β€” 10+ extraction phases 7. **Build messages** β€” transform extracted data into protobuf message structures (including external issues + ad-hoc rules for unsupported plugins) 8. **Encode** β€” encode messages to binary protobuf format 9. **Package** β€” create ZIP archive (metadata.pb, component-N.pb, issues-N.pb, externalissues-N.pb, adhocrules.pb, measures-N.pb, duplications-N.pb, source-N.txt, activerules.pb, changesets-N.pb) -10. **Upload** β€” submit scanner report ZIP to SonarCloud CE endpoint +10. **Upload** β€” submit scanner report ZIP to SonarQube Cloud CE endpoint 11. **Metadata sync** β€” sync issue statuses, comments, assignments, and tags from SQ to SC; sync hotspot statuses, comments, and source links (skippable via `skipIssueMetadataSync` / `skipHotspotMetadataSync`) 12. **Release lock** β€” release the advisory lock file 13. **Update state** β€” record successful transfer in state file @@ -348,7 +348,7 @@ Uses `pipelines/sq-{version}/migrate-pipeline.js` (selected by version-router): - Compare quality profiles and write diff report (`quality-profiles/quality-profile-diff.json`) - Create permission templates - For each project (11 steps): - - Resolve project key (use original SonarQube key; fall back to `{org}_{key}` if taken globally) + - Resolve project key (use original SonarQube Server key; fall back to `{org}_{key}` if taken globally) - Upload scanner report (via transfer pipeline) - Sync issue statuses, assignments, comments, tags - Sync hotspot statuses and comments @@ -368,8 +368,8 @@ On resume, completed organizations and projects are skipped based on the migrati Uses `shared/verification/verify-pipeline.js`: -1. **Connect to SonarQube** β€” verify connectivity -2. **Fetch project list** β€” get all projects from SonarQube +1. **Connect to SonarQube Server** β€” verify connectivity +2. **Fetch project list** β€” get all projects from SonarQube Server 3. **Build org mappings** β€” map projects to target orgs (same logic as `migrate`) 4. **For each target organization:** - Verify quality gates (existence + conditions) @@ -378,7 +378,7 @@ Uses `shared/verification/verify-pipeline.js`: - Verify global permissions - Verify permission templates - For each project: - - Verify project exists in SonarCloud + - Verify project exists in SonarQube Cloud - Verify branches (SQ vs SC) - Verify issues (matched by rule+file+line; compare statuses, status history via changelog, assignments, comments, tags) - Verify hotspots (matched by rule+file+line; compare statuses, comments) @@ -386,7 +386,7 @@ Uses `shared/verification/verify-pipeline.js`: - Verify quality gate and profile assignments - Verify project settings, tags, links, new code periods, DevOps bindings - Verify project permissions -5. **Portfolio check** β€” reference-only verification (always skipped; requires Enterprise API access that is not available). SonarQube portfolios are listed in the report for manual reference but no SonarCloud comparison is performed. +5. **Portfolio check** β€” reference-only verification (always skipped; requires Enterprise API access that is not available). SonarQube Server portfolios are listed in the report for manual reference but no SonarQube Cloud comparison is performed. 6. **Generate reports** β€” JSON, Markdown, PDF, and console summary @@ -394,7 +394,7 @@ Uses `shared/verification/verify-pipeline.js`: - **Extractor Pattern** β€” specialized modules for each data type with consistent interface -- **Migrator Pattern** β€” specialized modules for each SonarCloud migration target +- **Migrator Pattern** β€” specialized modules for each SonarQube Cloud migration target - **Client-Service Pattern** β€” API clients handle HTTP, services handle business logic - **Builder Pattern** β€” ProtobufBuilder constructs complex message structures - **State Pattern** β€” StateTracker manages transfer state for incremental sync @@ -408,7 +408,7 @@ Uses `shared/verification/verify-pipeline.js`: - Server-wide extraction steps run via `Promise.all` (quality gates, profiles, groups, permissions, templates, portfolios, server info, webhooks) - Org-wide resource migration uses two-batch parallelism: batch 1 (independent: groups, gates, profiles, templates) then batch 2 (dependent: global permissions, profile comparison) - Project-level steps run in parallel where possible (issue + hotspot sync concurrent, config steps concurrent, gate/profile/permission assignment concurrent) -- **Shared Throttler Pattern** β€” SonarCloud API clients within an org share a single POST throttler (`sharedThrottler`) to enforce `minRequestInterval` across all concurrent project migrations, preventing rate limit violations +- **Shared Throttler Pattern** β€” SonarQube Cloud API clients within an org share a single POST throttler (`sharedThrottler`) to enforce `minRequestInterval` across all concurrent project migrations, preventing rate limit violations ## ⚑ Concurrency and Performance @@ -502,10 +502,10 @@ setup ───────── β”œβ”€ sync-metadata (4 pa - **Integration Tests:** 30 parallel jobs testing every CLI flag combination via matrix strategy (`fail-fast: false`). Config files generated at runtime from GitHub Secrets. - **Summary:** Gate job that only passes when all 30 integration tests pass -### SonarCloud Analysis +### SonarQube Cloud Analysis -A standalone `SonarCloud Analysis` workflow (`sonarcloud.yml`) runs SAST and SCA scanning on every push to `main`. It does **not** run unit tests or ingest coverage β€” those are handled by the separate Unit Tests workflow. +A standalone `SonarQube Cloud Analysis` workflow (`sonarcloud.yml`) runs SAST and SCA scanning on every push to `main`. It does **not** run unit tests or ingest coverage β€” those are handled by the separate Unit Tests workflow. ### Auto Version Bump @@ -531,7 +531,7 @@ The `Build and Release` workflow (`release.yml`) builds binaries for 6 platforms | File | Purpose | |---|---| | `unit-tests.yml` | Standalone unit tests (push to main only) | -| `sonarcloud.yml` | SAST/SCA scanning via SonarCloud | +| `sonarcloud.yml` | SAST/SCA scanning via SonarQube Cloud | | `version-bump.yml` | Auto-bump version from PR milestone on merge | | `release.yml` | Orchestrator β€” install, build, desktop build, release | | `regression.yml` | Regression orchestrator β€” triggers, stage sequencing | diff --git a/docs/backward-compatibility.md b/docs/backward-compatibility.md index 81a1bad..8e1b650 100644 --- a/docs/backward-compatibility.md +++ b/docs/backward-compatibility.md @@ -1,45 +1,45 @@ -# Backward Compatibility: SonarQube Version Support +# Backward Compatibility: SonarQube Server Version Support -CloudVoyager supports migrating from **SonarQube 9.9 LTS** through **SonarQube 2025.1** (and newer) to SonarCloud. This document explains how the tool handles the differences between SonarQube versions. +CloudVoyager supports migrating from **SonarQube Server 9.9 LTS** through **SonarQube Server 2025.1** (and newer) to SonarQube Cloud. This document explains how the tool handles the differences between SonarQube Server versions. ## Background -Different SonarQube versions introduced significant API and taxonomy changes: +Different SonarQube Server versions introduced significant API and taxonomy changes: -**SonarQube 10.0** introduced the **Clean Code taxonomy**: +**SonarQube Server 10.0** introduced the **Clean Code taxonomy**: - **Clean Code Attributes** (e.g., `CONVENTIONAL`, `LOGICAL`, `TRUSTWORTHY`) β€” classify *why* code is problematic - **Software Qualities** (`MAINTAINABILITY`, `RELIABILITY`, `SECURITY`) β€” classify *what* is affected - **Impact Severities** (`LOW`, `MEDIUM`, `HIGH`, `BLOCKER`) β€” classify *how severe* the impact is -**SonarQube 10.4** introduced a **new issue lifecycle**: +**SonarQube Server 10.4** introduced a **new issue lifecycle**: - Replaced the `statuses` API parameter with `issueStatuses` - Removed `REOPENED`, `RESOLVED`, and `CLOSED` statuses - Added `ACCEPTED` (replacing "Won't Fix") and `FIXED` statuses -**SonarQube 2025.1** removed web services deprecated in 8.x/9.x, and continues the migration to Web API V2. +**SonarQube Server 2025.1** removed web services deprecated in 8.x/9.x, and continues the migration to Web API V2. -**SonarQube 9.9 LTS** uses the older taxonomy: +**SonarQube Server 9.9 LTS** uses the older taxonomy: - **Issue Types** (`CODE_SMELL`, `BUG`, `VULNERABILITY`, `SECURITY_HOTSPOT`) - **Severities** (`INFO`, `MINOR`, `MAJOR`, `CRITICAL`, `BLOCKER`) - Legacy `statuses` parameter with `OPEN`, `CONFIRMED`, `REOPENED`, `RESOLVED`, `CLOSED` -SonarCloud (always the latest version) uses the new Clean Code taxonomy. CloudVoyager bridges these gaps automatically. +SonarQube Cloud (always the latest version) uses the new Clean Code taxonomy. CloudVoyager bridges these gaps automatically. ## Architecture: Pipeline-Per-Version -Instead of a single codebase with runtime version checks, CloudVoyager uses a **pipeline-per-version** architecture. Each supported SonarQube version range has its own self-contained pipeline under `src/pipelines/`: +Instead of a single codebase with runtime version checks, CloudVoyager uses a **pipeline-per-version** architecture. Each supported SonarQube Server version range has its own self-contained pipeline under `src/pipelines/`: ``` src/ β”œβ”€β”€ version-router.js # Detects SQ version, loads correct pipeline β”œβ”€β”€ pipelines/ -β”‚ β”œβ”€β”€ sq-9.9/ # SonarQube 9.9 LTS -β”‚ β”œβ”€β”€ sq-10.0/ # SonarQube 10.0–10.3 -β”‚ β”œβ”€β”€ sq-10.4/ # SonarQube 10.4–10.8 -β”‚ └── sq-2025/ # SonarQube 2025.1+ +β”‚ β”œβ”€β”€ sq-9.9/ # SonarQube Server 9.9 LTS +β”‚ β”œβ”€β”€ sq-10.0/ # SonarQube Server 10.0–10.3 +β”‚ β”œβ”€β”€ sq-10.4/ # SonarQube Server 10.4–10.8 +β”‚ └── sq-2025/ # SonarQube Server 2025.1+ └── shared/ # Version-independent code (utils, state, config, etc.) ``` @@ -49,7 +49,7 @@ Each pipeline directory contains a complete set of modules (60–66 JS files eac sq-{version}/ β”œβ”€β”€ transfer-pipeline.js # Single-project transfer orchestrator β”œβ”€β”€ migrate-pipeline.js # Full multi-org migration orchestrator -β”œβ”€β”€ sonarqube/ # SonarQube API client, models, extractors +β”œβ”€β”€ sonarqube/ # SonarQube Server API client, models, extractors β”‚ β”œβ”€β”€ api-client.js β”‚ β”œβ”€β”€ models.js β”‚ β”œβ”€β”€ api/ # API method modules @@ -59,7 +59,7 @@ sq-{version}/ β”‚ β”œβ”€β”€ encoder.js β”‚ β”œβ”€β”€ build-*.js β”‚ └── schema/ # .proto definitions -β”œβ”€β”€ sonarcloud/ # SonarCloud API client, uploader, migrators +β”œβ”€β”€ sonarcloud/ # SonarQube Cloud API client, uploader, migrators β”‚ β”œβ”€β”€ api-client.js β”‚ β”œβ”€β”€ uploader.js β”‚ β”œβ”€β”€ rule-enrichment.js @@ -72,12 +72,12 @@ sq-{version}/ └── results.js ``` -**No runtime version checks exist within any pipeline.** Each pipeline has its behavior hardcoded for its target SonarQube version range. This eliminates branching logic and makes each pipeline easier to understand and maintain. +**No runtime version checks exist within any pipeline.** Each pipeline has its behavior hardcoded for its target SonarQube Server version range. This eliminates branching logic and makes each pipeline easier to understand and maintain. ## How Version Routing Works -1. `version-router.js` makes a lightweight `GET /api/system/status` call to detect the SonarQube server version +1. `version-router.js` makes a lightweight `GET /api/system/status` call to detect the SonarQube Server version 2. `resolvePipelineId()` maps the parsed version to a pipeline folder: - `major >= 2025` β†’ `sq-2025` - `(major === 10 && minor >= 4) || (major > 10 && major < 2025)` β†’ `sq-10.4` (covers 10.4–10.8 and any hypothetical 11.x–2024.x versions) @@ -98,14 +98,14 @@ If version detection fails (e.g., network error), the router falls back to the ` |----------|--------|---------|---------|---------| | Issue search param | `statuses` (legacy) | `statuses` (legacy) | `issueStatuses` (modern) | `issueStatuses` (modern) | | metricKeys limit | Batch at 15 | Batch at 15 | Batch at 15 | No batching | -| Clean Code source | SonarCloud enrichment map | Native from SQ | Native from SQ | Native from SQ | +| Clean Code source | SonarQube Cloud enrichment map | Native from SQ | Native from SQ | Native from SQ | | Rule enrichment | Always called | Not needed | Not needed | Not needed | | Groups API | `/api/user_groups/search` | Same | Same | Standard (with V2 API fallback) | ### Issue Lifecycle Differences -The issue search API changed significantly across SonarQube versions: +The issue search API changed significantly across SonarQube Server versions: | Pipeline | API Parameter | Valid Status Values | |----------|--------------|---------------------| @@ -124,32 +124,32 @@ Each pipeline uses the correct parameter directly β€” no conditional logic requi ### Metric Keys Batching -SonarQube 9.9 through 10.8 enforces a limit of **15 metric keys per request** to the measures API. Pipelines `sq-9.9`, `sq-10.0`, and `sq-10.4` batch metric key requests accordingly. +SonarQube Server 9.9 through 10.8 enforces a limit of **15 metric keys per request** to the measures API. Pipelines `sq-9.9`, `sq-10.0`, and `sq-10.4` batch metric key requests accordingly. -SonarQube 2025.1+ removed this limit, so the `sq-2025` pipeline sends all metric keys in a single request. +SonarQube Server 2025.1+ removed this limit, so the `sq-2025` pipeline sends all metric keys in a single request. ### Clean Code Taxonomy Enrichment -The `sq-9.9` pipeline fetches the Clean Code taxonomy from SonarCloud because SonarQube 9.9 does not provide it: +The `sq-9.9` pipeline fetches the Clean Code taxonomy from SonarQube Cloud because SonarQube Server 9.9 does not provide it: ``` -SonarQube 9.9.0.65466 does not support Clean Code taxonomy (requires 10.0+). Fetching enrichment from SonarCloud... +SonarQube Server 9.9.0.65466 does not support Clean Code taxonomy (requires 10.0+). Fetching enrichment from SonarQube Cloud... Rule enrichment map built: 1,247 rules with Clean Code data ``` -The `sq-10.0`, `sq-10.4`, and `sq-2025` pipelines read Clean Code data natively from SonarQube β€” no enrichment fetch needed. +The `sq-10.0`, `sq-10.4`, and `sq-2025` pipelines read Clean Code data natively from SonarQube Server β€” no enrichment fetch needed. -### Rule Enrichment from SonarCloud (sq-9.9 only) +### Rule Enrichment from SonarQube Cloud (sq-9.9 only) -For SonarQube 9.9, the `rule-enrichment.js` module in the `sq-9.9` pipeline: +For SonarQube Server 9.9, the `rule-enrichment.js` module in the `sq-9.9` pipeline: -1. For each SonarCloud quality profile, queries `/api/rules/search` with `f=cleanCodeAttribute,impacts` +1. For each SonarQube Cloud quality profile, queries `/api/rules/search` with `f=cleanCodeAttribute,impacts` 2. Builds a **rule enrichment map** β€” a lookup table mapping rule keys (e.g., `javascript:S1234`) to their Clean Code attributes and impacts -3. When building the scanner report, rules and issues are enriched with the correct Clean Code data from SonarCloud +3. When building the scanner report, rules and issues are enriched with the correct Clean Code data from SonarQube Cloud -This means the migrated data in SonarCloud will have the **exact same** Clean Code classification as if it had been scanned natively. +This means the migrated data in SonarQube Cloud will have the **exact same** Clean Code classification as if it had been scanned natively. ### Fallback Chain (sq-9.9 only) @@ -158,7 +158,7 @@ The enrichment follows a two-level fallback: | Priority | Source | When Used | |----------|--------|-----------| -| 1 | SonarCloud rule enrichment | Rule exists in SC quality profiles | +| 1 | SonarQube Cloud rule enrichment | Rule exists in SC quality profiles | | 2 | Type-based inference | Rule not in SC (external rules) | **Type-based inference** (last resort) maps the old taxonomy as follows: @@ -188,13 +188,13 @@ The enrichment follows a two-level fallback: | sq-10.4 | `/api/user_groups/search` (standard) | | sq-2025 | `/api/user_groups/search` (standard, with Web API V2 fallback) | -SonarQube 2025.1+ began deprecating some legacy Web API endpoints. The `sq-2025` pipeline uses the standard groups API but includes fallback to the V2 API (`/api/v2/authorizations/groups`) if the legacy endpoint is unavailable. +SonarQube Server 2025.1+ began deprecating some legacy Web API endpoints. The `sq-2025` pipeline uses the standard groups API but includes fallback to the V2 API (`/api/v2/authorizations/groups`) if the legacy endpoint is unavailable. ### Performance Optimization - **Transfer pipeline** (`transfer` command): Enrichment map is built once per project transfer (sq-9.9 only) -- **Migrate pipeline** (`migrate` command): Enrichment map is built **once per SonarCloud organization** and reused across all projects in that org (sq-9.9 only) +- **Migrate pipeline** (`migrate` command): Enrichment map is built **once per SonarQube Cloud organization** and reused across all projects in that org (sq-9.9 only) - **sq-10.0, sq-10.4, sq-2025**: Enrichment fetch is skipped entirely (no extra API calls) ## Common API Constraints (All Versions) @@ -220,11 +220,11 @@ These constraints apply across all four pipelines: All four pipelines support automatic detection and migration of external/plugin issues (e.g., MuleSoft, Checkstyle, PMD): -1. **Auto-detection**: Compares SonarQube rule repositories (`getRuleRepositories()`) against SonarCloud available repositories -2. **External encoding**: Rules not available in SonarCloud are encoded as `ExternalIssue` + `AdHocRule` protobuf messages in the scanner report -3. **Display in SonarCloud**: External rules appear as `external_{engineId}:{ruleId}` (e.g., `external_mulesoft:MS058`). Ad-hoc rules do not appear in SC rules search (expected behavior per SC docs). +1. **Auto-detection**: Compares SonarQube Server rule repositories (`getRuleRepositories()`) against SonarQube Cloud available repositories +2. **External encoding**: Rules not available in SonarQube Cloud are encoded as `ExternalIssue` + `AdHocRule` protobuf messages in the scanner report +3. **Display in SonarQube Cloud**: External rules appear as `external_{engineId}:{ruleId}` (e.g., `external_mulesoft:MS058`). Ad-hoc rules do not appear in SC rules search (expected behavior per SC docs). -**Critical implementation detail**: The `cleanCodeAttribute` field in `AdHocRule` must be encoded as a **protobuf enum (varint)**, NOT a string. Despite the GitHub proto definition showing `optional string`, the real SonarCloud scanner uses enum encoding. SonarCloud's Compute Engine silently ignores external issues if `cleanCodeAttribute` is string-encoded. +**Critical implementation detail**: The `cleanCodeAttribute` field in `AdHocRule` must be encoded as a **protobuf enum (varint)**, NOT a string. Despite the GitHub proto definition showing `optional string`, the real SonarQube Cloud scanner uses enum encoding. SonarQube Cloud's Compute Engine silently ignores external issues if `cleanCodeAttribute` is string-encoded. Clean Code attribute enum values: @@ -245,7 +245,7 @@ Clean Code attribute enum values: | `RESPECTFUL` | 13 | | `TRUSTWORTHY` | 14 | -The `impacts` and `defaultImpacts` (Impact message) fields are also required for external issues to be accepted by SonarCloud. +The `impacts` and `defaultImpacts` (Impact message) fields are also required for external issues to be accepted by SonarQube Cloud. ## Feature Support Matrix @@ -277,7 +277,7 @@ All features are supported across all four pipelines: Two resilience features apply identically across all four pipelines: - **Search slicing for 10K+ issues** β€” Each pipeline's `issues-hotspots.js` calls `fetchWithSlicing` from the shared `src/shared/utils/search-slicer/` utility. A `probe-total.js` helper in each pipeline's `api-client/helpers/` probes the total count with `ps=1`. If the total exceeds 10,000, the date-window bisection algorithm activates automatically. -- **Fallback rule repositories** β€” Each pipeline's `getRuleRepositories()` retries 3 times with exponential backoff and falls back to the shared `src/shared/utils/fallback-repos/index.js` (43 known SonarCloud repositories). The `isExternalIssue()` guard in each pipeline uses the fallback set when the live set is empty. +- **Fallback rule repositories** β€” Each pipeline's `getRuleRepositories()` retries 3 times with exponential backoff and falls back to the shared `src/shared/utils/fallback-repos/index.js` (43 known SonarQube Cloud repositories). The `isExternalIssue()` guard in each pipeline uses the fallback set when the live set is empty. ## What Works Identically Across Pipelines @@ -303,7 +303,7 @@ These components are shared across all version-specific pipelines (via `src/shar ## Usage -No special configuration is needed. CloudVoyager detects the SonarQube version automatically and loads the correct pipeline: +No special configuration is needed. CloudVoyager detects the SonarQube Server version automatically and loads the correct pipeline: ```bash # Works with SQ 9.9, 10.x, 2025.1, or any supported version @@ -314,7 +314,7 @@ node src/index.js migrate -c config.json Use `--verbose` to see detailed logs about version detection and pipeline selection: ``` -SonarQube server version: 9.9.0.65466 β†’ using pipeline: sq-9.9 +SonarQube Server version: 9.9.0.65466 β†’ using pipeline: sq-9.9 ``` Use the `test` command to verify connectivity and see which pipeline is selected: @@ -323,12 +323,12 @@ Use the `test` command to verify connectivity and see which pipeline is selected node src/index.js test -c config.json ``` -## Supported SonarQube Versions +## Supported SonarQube Server Versions | Version | Pipeline | Support Level | Notes | |---------|----------|--------------|-------| -| 9.9 LTS | sq-9.9 | Full | Clean Code enriched from SonarCloud; legacy `statuses` param | +| 9.9 LTS | sq-9.9 | Full | Clean Code enriched from SonarQube Cloud; legacy `statuses` param | | 10.0 - 10.3 | sq-10.0 | Full | Native Clean Code taxonomy; legacy `statuses` param with extended values | | 10.4 - 10.8 | sq-10.4 | Full | Modern `issueStatuses` param; metric batching at 15 | | 2025.1+ | sq-2025 | Full | Modern `issueStatuses` param; no metric batching; V2 API fallbacks | @@ -337,40 +337,40 @@ node src/index.js test -c config.json ## Troubleshooting -### "Failed to detect SonarQube version" +### "Failed to detect SonarQube Server version" If version detection fails, CloudVoyager falls back to the `sq-9.9` pipeline. This is non-fatal β€” the migration will proceed. Check: -- SonarQube URL is correct and reachable -- SonarQube token has sufficient permissions -- Network connectivity to the SonarQube server +- SonarQube Server URL is correct and reachable +- SonarQube Server token has sufficient permissions +- Network connectivity to the SonarQube Server ### "Failed to build rule enrichment map" If the enrichment fetch fails (sq-9.9 pipeline), CloudVoyager falls back to type-based inference. This is non-fatal β€” the migration will proceed, but Clean Code attributes may be less precise for active rules. Check: -- SonarCloud token has sufficient permissions -- Network connectivity to SonarCloud -- Quality profiles exist in SonarCloud for the relevant languages +- SonarQube Cloud token has sufficient permissions +- Network connectivity to SonarQube Cloud +- Quality profiles exist in SonarQube Cloud for the relevant languages ### Issues appear with generic Clean Code attributes If migrated issues show `CONVENTIONAL` instead of more specific attributes (like `LOGICAL` or `TRUSTWORTHY`), it means: -1. The rule wasn't found in the SonarCloud enrichment map +1. The rule wasn't found in the SonarQube Cloud enrichment map 2. Type-based inference was used as fallback -This is expected for external/plugin rules that don't exist in SonarCloud. For native rules, verify that SonarCloud quality profiles are properly configured for the relevant languages. +This is expected for external/plugin rules that don't exist in SonarQube Cloud. For native rules, verify that SonarQube Cloud quality profiles are properly configured for the relevant languages. -### External issues not appearing in SonarCloud +### External issues not appearing in SonarQube Cloud If external issues are missing after migration: 1. Verify that `cleanCodeAttribute` is encoded as a protobuf enum (varint), not a string 2. Verify that `impacts` and `defaultImpacts` fields are populated in `AdHocRule` messages -3. Check SonarCloud CE task logs for silent rejection of malformed external issue messages -4. Ad-hoc rules will not appear in the SonarCloud rules search β€” this is expected behavior +3. Check SonarQube Cloud CE task logs for silent rejection of malformed external issue messages +4. Ad-hoc rules will not appear in the SonarQube Cloud rules search β€” this is expected behavior diff --git a/docs/bespoke-algorithms.md b/docs/bespoke-algorithms.md index 21b8ae1..a84758b 100644 --- a/docs/bespoke-algorithms.md +++ b/docs/bespoke-algorithms.md @@ -20,7 +20,7 @@ This document describes custom, non-trivial algorithms implemented from scratch -SonarQube's `/api/issues/search` endpoint caps results at 10,000 items per query. Projects with large issue counts require the response window to be sliced by creation date and results stitched together. +SonarQube Server's `/api/issues/search` endpoint caps results at 10,000 items per query. Projects with large issue counts require the response window to be sliced by creation date and results stitched together. ### Algorithm @@ -197,7 +197,7 @@ CloudVoyager checkpoints migration progress so a run can be resumed after any in -SonarCloud's Compute Engine accepts analysis reports encoded as protobuf-over-zip, matching the format produced by SonarScanner. CloudVoyager reconstructs this format without running an actual scan. +SonarQube Cloud's Compute Engine accepts analysis reports encoded as protobuf-over-zip, matching the format produced by SonarScanner. CloudVoyager reconstructs this format without running an actual scan. ### Encoding Pipeline @@ -297,4 +297,4 @@ The protobuf schemas were reverse-engineered from the SonarScanner source and ar ### Implementation -### SonarQube Settings +### SonarQube Server Settings | Option | Required | Description | |--------|----------|-------------| -| `url` | Yes | SonarQube server URL | -| `token` | Yes | SonarQube API token (or set via `SONARQUBE_TOKEN` env var) | +| `url` | Yes | SonarQube Server URL | +| `token` | Yes | SonarQube Server API token (or set via `SONARQUBE_TOKEN` env var) | | `projectKey` | For `transfer` only | Project key to export (not needed for `migrate`) | -### SonarCloud Settings (Single Org) +### SonarQube Cloud Settings (Single Org) Used by `transfer`, `test`, `validate`, `status`, `reset`. | Option | Required | Description | |--------|----------|-------------| -| `url` | No | SonarCloud server URL (default: `https://sonarcloud.io`) | -| `token` | Yes | SonarCloud API token (or set via `SONARCLOUD_TOKEN` env var) | -| `organization` | Yes | SonarCloud organization key | -| `projectKey` | For `transfer` only | Destination project key. The display name is automatically carried over from SonarQube | +| `url` | No | SonarQube Cloud server URL (default: `https://sonarcloud.io`) | +| `token` | Yes | SonarQube Cloud API token (or set via `SONARCLOUD_TOKEN` env var) | +| `organization` | Yes | SonarQube Cloud organization key | +| `projectKey` | For `transfer` only | Destination project key. The display name is automatically carried over from SonarQube Server | -### SonarCloud Settings (Multi-Org) +### SonarQube Cloud Settings (Multi-Org) Used by `migrate`, `sync-metadata`. Instead of a single org, you provide an array of target organizations. | Option | Required | Description | |--------|----------|-------------| -| `organizations[].key` | Yes | SonarCloud organization key | -| `organizations[].token` | Yes | SonarCloud API token for this org | -| `organizations[].url` | No | SonarCloud server URL. Use `https://sonarcloud.io` (EU, default) or `https://sonarqube.us` (US). In the Desktop app an EU/US radio button sets this automatically. | -| `enterprise.key` | Optional | SonarCloud enterprise key. Required only for portfolio migration via V2 API. If absent, portfolio migration is gracefully skipped and the migration continues normally. | +| `organizations[].key` | Yes | SonarQube Cloud organization key | +| `organizations[].token` | Yes | SonarQube Cloud API token for this org | +| `organizations[].url` | No | SonarQube Cloud server URL. Use `https://sonarcloud.io` (EU, default) or `https://sonarqube.us` (US). In the Desktop app an EU/US radio button sets this automatically. | +| `enterprise.key` | Optional | SonarQube Cloud enterprise key. Required only for portfolio migration via V2 API. If absent, portfolio migration is gracefully skipped and the migration continues normally. | ### Transfer Settings @@ -172,7 +172,7 @@ The `transfer.checkpoint` block controls the pause/resume behavior. All settings | `enabled` | `true` | Enable checkpoint journal for pause/resume support | | `cacheExtractions` | `true` | Cache extraction results to disk for faster resume | | `cacheMaxAgeDays` | `7` | Maximum age of extraction cache files in days before auto-purge | -| `strictResume` | `false` | Fail on SonarQube version mismatch when resuming (default: warn only) | +| `strictResume` | `false` | Fail on SonarQube Server version mismatch when resuming (default: warn only) | ### Migrate Settings @@ -182,15 +182,15 @@ The `transfer.checkpoint` block controls the pause/resume behavior. All settings | `outputDir` | `./migration-output` | Directory for mapping CSVs and server info output | | `skipIssueMetadataSync` | `false` | Skip syncing issue metadata (statuses, assignments, comments, tags) | | `skipHotspotMetadataSync` | `false` | Skip syncing hotspot metadata (statuses, comments) | -| `skipQualityProfileSync` | `false` | Skip syncing quality profiles (projects use default SonarCloud profiles) | +| `skipQualityProfileSync` | `false` | Skip syncing quality profiles (projects use default SonarQube Cloud profiles) | | `dryRun` | `false` | Extract and generate mappings without migrating | -> **Project key behavior (migrate command):** By default, the `migrate` command uses the original SonarQube project key on SonarCloud. If the key is already taken by another SonarCloud organization, the tool falls back to a prefixed key (`{org}_{key}`) and logs a warning. Key conflicts are listed in the migration report. +> **Project key behavior (migrate command):** By default, the `migrate` command uses the original SonarQube Server project key on SonarQube Cloud. If the key is already taken by another SonarQube Cloud organization, the tool falls back to a prefixed key (`{org}_{key}`) and logs a warning. Key conflicts are listed in the migration report. ### Rate Limit Settings -Controls retry and throttling behavior for SonarCloud API requests. By default, retries are enabled (`maxRetries: 3`) but request throttling is off (`minRequestInterval: 0`). Add a `rateLimit` section to any config file to customize. +Controls retry and throttling behavior for SonarQube Cloud API requests. By default, retries are enabled (`maxRetries: 3`) but request throttling is off (`minRequestInterval: 0`). Add a `rateLimit` section to any config file to customize. ```json { @@ -204,7 +204,7 @@ Controls retry and throttling behavior for SonarCloud API requests. By default, | Option | Default | Description | |--------|---------|-------------| -| `maxRetries` | `3` | Max retry attempts when SonarCloud returns 503 or 429. Set to `0` to disable retries. Retries use exponential backoff (delay doubles each attempt). | +| `maxRetries` | `3` | Max retry attempts when SonarQube Cloud returns 503 or 429. Set to `0` to disable retries. Retries use exponential backoff (delay doubles each attempt). | | `baseDelay` | `1000` | Initial delay in ms before the first retry. Doubles each retry: 1000ms β†’ 2000ms β†’ 4000ms β†’ etc. Only applies when `maxRetries` > 0. | | `minRequestInterval` | `0` | Minimum ms to wait between POST (write) requests. Set to `0` to disable throttling. Values like `100`–`200` help avoid triggering rate limits during high-volume operations. | @@ -245,10 +245,10 @@ Controls CPU, memory, and concurrency tuning. Add a `performance` section to any | `autoTune` | `false` | Auto-detect CPU and RAM and set optimal values. When enabled, uses 75% of total RAM (max 16GB) and scales concurrency based on CPU cores. Explicit settings override auto-tuned values. | | `maxConcurrency` | `64` | General concurrency limit for parallel I/O operations (1–64) | | `maxMemoryMB` | `8192` | Max heap size in MB. Set to `0` for Node.js default. The tool auto-restarts with the increased heap size when needed. | -| `sourceExtraction.concurrency` | `50` | Max concurrent source file fetches from SonarQube (1–50) | -| `hotspotExtraction.concurrency` | `50` | Max concurrent hotspot detail fetches from SonarQube (1–50) | -| `issueSync.concurrency` | `20` | Max concurrent issue metadata sync operations to SonarCloud (1–20) | -| `hotspotSync.concurrency` | `20` | Max concurrent hotspot sync operations to SonarCloud (1–20) | +| `sourceExtraction.concurrency` | `50` | Max concurrent source file fetches from SonarQube Server (1–50) | +| `hotspotExtraction.concurrency` | `50` | Max concurrent hotspot detail fetches from SonarQube Server (1–50) | +| `issueSync.concurrency` | `20` | Max concurrent issue metadata sync operations to SonarQube Cloud (1–20) | +| `hotspotSync.concurrency` | `20` | Max concurrent hotspot sync operations to SonarQube Cloud (1–20) | | `projectMigration.concurrency` | `8` | Max concurrent project migrations (1–8) | **Auto-tune defaults:** When `autoTune` is enabled (or `--auto-tune` CLI flag is used), the following values are calculated based on your hardware: @@ -286,7 +286,7 @@ Explicit config values or CLI flags override auto-tuned values. | `--dry-run` | Extract data and generate mapping CSVs without migrating | `migrate` | | `--skip-issue-metadata-sync` | Skip syncing issue metadata (statuses, assignments, comments, tags) | `migrate`, `sync-metadata` | | `--skip-hotspot-metadata-sync` | Skip syncing hotspot metadata (statuses, comments) | `migrate`, `sync-metadata` | -| `--skip-quality-profile-sync` | Skip syncing quality profiles (projects use default SonarCloud profiles) | `migrate`, `sync-metadata` | +| `--skip-quality-profile-sync` | Skip syncing quality profiles (projects use default SonarQube Cloud profiles) | `migrate`, `sync-metadata` | **Selective migration flag:** @@ -335,14 +335,14 @@ Multiple components can be combined: `--only scan-data,quality-gates,permissions | Variable | Description | |----------|-------------| -| `SONARQUBE_TOKEN` | Override SonarQube token from config | -| `SONARCLOUD_TOKEN` | Override SonarCloud token from config | -| `SONARQUBE_URL` | Override SonarQube URL from config | -| `SONARCLOUD_URL` | Override SonarCloud URL from config | +| `SONARQUBE_TOKEN` | Override SonarQube Server token from config | +| `SONARCLOUD_TOKEN` | Override SonarQube Cloud token from config | +| `SONARQUBE_URL` | Override SonarQube Server URL from config | +| `SONARCLOUD_URL` | Override SonarQube Cloud URL from config | | `LOG_LEVEL` | Set logging level (`debug`, `info`, `warn`, `error`) | | `LOG_FILE` | Path to log file (optional) | | `MAX_SOURCE_FILES` | Limit number of source files to extract (0 = all) | -| `SONAR_TOKEN` | SonarCloud token used by the `sonarcloud.yml` GitHub Actions workflow for SAST/SCA scanning of the CloudVoyager repository itself (not used by the CLI) | +| `SONAR_TOKEN` | SonarQube Cloud token used by the `sonarcloud.yml` GitHub Actions workflow for SAST/SCA scanning of the CloudVoyager repository itself (not used by the CLI) | ## πŸ“œ npm Scripts vs Binary Commands @@ -411,7 +411,7 @@ For multi-project migrations (`migrate` command), we recommend the following 3-s ### Step 1: Dry run β€” verify everything -Run a dry run first to extract all data, generate mapping CSVs, and validate your config without touching SonarCloud: +Run a dry run first to extract all data, generate mapping CSVs, and validate your config without touching SonarQube Cloud: ```bash # npm @@ -436,7 +436,7 @@ npm run migrate:skip-all-metadata:auto-tune ./cloudvoyager migrate -c migrate-config.json --verbose --skip-issue-metadata-sync --skip-hotspot-metadata-sync --auto-tune ``` -Skipping metadata during the main migration avoids SonarCloud rate limiting (503 errors) that can occur during high-volume issue/hotspot sync. +Skipping metadata during the main migration avoids SonarQube Cloud rate limiting (503 errors) that can occur during high-volume issue/hotspot sync. ### Step 3: Sync metadata separately @@ -484,7 +484,7 @@ The journal records: - Which extraction phases have completed (metrics, issues, sources, etc.) - Per-branch transfer status (completed, in-progress, pending) - Upload deduplication data (prevents duplicate CE tasks after crashes) -- Session fingerprint (SonarQube version, URL, project key) for resume validation +- Session fingerprint (SonarQube Server version, URL, project key) for resume validation **Key behaviors:** - **Automatic resume**: Running the same `transfer` command after an interruption automatically resumes from the last checkpoint @@ -529,5 +529,5 @@ Additional files created during transfer: | 2026-02-19 | npm Scripts | Expanded script table with all commands | | 2026-02-18 | Performance Settings | Auto-tune feature added | | 2026-02-17 | Transfer-All, Migrate, Rate Limit, Workflow | Migration engine config options | -| 2026-02-16 | Single Project, SonarQube, Env Vars, State | Core configuration system | +| 2026-02-16 | Single Project, SonarQube Server, Env Vars, State | Core configuration system | --> diff --git a/docs/design-review-regression-testing.md b/docs/design-review-regression-testing.md index 9338c9a..7c0671f 100644 --- a/docs/design-review-regression-testing.md +++ b/docs/design-review-regression-testing.md @@ -51,5 +51,5 @@ PASS -- no YAGNI violations. | ID | Issue | Suggested Fix | |----|-------|---------------| -| 5.1 | `cv-test-malformed` requires Unicode keys and empty names that SonarQube API rejects | Specify DB-level seeding or descope to API-achievable scenarios only | +| 5.1 | `cv-test-malformed` requires Unicode keys and empty names that SonarQube Server API rejects | Specify DB-level seeding or descope to API-achievable scenarios only | | 5.2 | `max-parallel: 8` may still exceed SQC rate limits with no cross-job throttling | Lower to 4, add staggered startup delays, or use separate SQC orgs | diff --git a/docs/desktop-app.md b/docs/desktop-app.md index 808751d..f9303f7 100644 --- a/docs/desktop-app.md +++ b/docs/desktop-app.md @@ -33,7 +33,7 @@ Download the latest release from the [GitHub Releases](https://github.com/your-o On launch, the Welcome screen presents the available workflows: -- **Transfer One Project** β€” Migrate a single project from SonarQube to SonarCloud +- **Transfer One Project** β€” Migrate a single project from SonarQube Server to SonarQube Cloud - **Move All Projects & Settings** β€” Full organization migration (projects, quality gates, profiles, permissions, etc.) - **Verify Migration Results** β€” Compare data between source and destination after migration - **Sync Settings & Policies** β€” Update coding rules, policies, and permissions without re-migrating code data @@ -45,7 +45,7 @@ On launch, the Welcome screen presents the available workflows: Each workflow guides you through a series of screens: -1. **Connection setup** β€” Enter SonarQube and SonarCloud URLs and tokens +1. **Connection setup** β€” Enter SonarQube Server and SonarQube Cloud URLs and tokens 2. **Options** β€” Configure transfer mode, branch selection, and other settings 3. **Review** β€” Confirm all settings before starting 4. **Start** β€” Kick off the migration and watch live logs @@ -60,8 +60,8 @@ The live log viewer shows migration progress in real-time with a timer, cancel b | Step | Description | |------|-------------| -| 1. SonarQube Connection | Enter URL and token for the source SonarQube instance | -| 2. SonarCloud Connection | Enter URL, token, organization, and project key for the target. An **EU / US radio button** selects the SonarQube Cloud instance (`sonarcloud.io` vs `sonarqube.us`) | +| 1. SonarQube Server Connection | Enter URL and token for the source SonarQube Server instance | +| 2. SonarQube Cloud Connection | Enter URL, token, organization, and project key for the target. An **EU / US radio button** selects the SonarQube Cloud instance (`sonarcloud.io` vs `sonarqube.us`) | | 3. Transfer Settings | Choose transfer mode (full/incremental), batch size, branch options. The "More Settings (Advanced)" section is collapsed by default (click the gear icon to expand) | | 4. Review & Start | Review all settings and begin the transfer | @@ -70,8 +70,8 @@ The live log viewer shows migration progress in real-time with a timer, cancel b | Step | Description | |------|-------------| -| 1. SonarQube Connection | Enter URL and admin token for the source SonarQube instance | -| 2. SonarCloud Organizations | Add or remove target SonarCloud organizations with their tokens. Each organization entry includes an **EU / US radio button** to select the SonarQube Cloud instance (`sonarcloud.io` vs `sonarqube.us`) | +| 1. SonarQube Server Connection | Enter URL and admin token for the source SonarQube Server instance | +| 2. SonarQube Cloud Organizations | Add or remove target SonarQube Cloud organizations with their tokens. Each organization entry includes an **EU / US radio button** to select the SonarQube Cloud instance (`sonarcloud.io` vs `sonarqube.us`) | | 3. Migration Settings | Configure output directory, dry run, included/excluded projects. The "Choose What to Migrate" and "More Settings (Advanced)" sections are collapsed by default to reduce clutter (click the shield or gear icon to expand) | | 4. Review & Start | Review all settings and begin the full migration | @@ -82,8 +82,8 @@ The live log viewer shows migration progress in real-time with a timer, cancel b | Step | Description | |------|-------------| -| 1. SonarQube Connection | Enter URL and token for the source SonarQube instance | -| 2. SonarCloud Organizations | Add target SonarCloud organizations to verify against | +| 1. SonarQube Server Connection | Enter URL and token for the source SonarQube Server instance | +| 2. SonarQube Cloud Organizations | Add target SonarQube Cloud organizations to verify against | | 3. Review & Start | Review settings and begin verification | ### Sync Metadata Config (3 steps) @@ -91,14 +91,14 @@ The live log viewer shows migration progress in real-time with a timer, cancel b | Step | Description | |------|-------------| -| 1. SonarQube Connection | Enter URL and token for the source SonarQube instance | -| 2. SonarCloud Organizations | Add target SonarCloud organizations for metadata sync | +| 1. SonarQube Server Connection | Enter URL and token for the source SonarQube Server instance | +| 2. SonarQube Cloud Organizations | Add target SonarQube Cloud organizations for metadata sync | | 3. Review & Start | Review settings and begin metadata synchronization | ### Other Screens -- **Connection Test** β€” Runs the `test` command to verify connectivity to both SonarQube and SonarCloud +- **Connection Test** β€” Runs the `test` command to verify connectivity to both SonarQube Server and SonarQube Cloud - **Execution** β€” Live log viewer with elapsed timer, cancel button, and status badge (running/success/failed) ![Live Migration Progress](screenshots/migrate_progress.png) diff --git a/docs/dry-run-csv-reference.md b/docs/dry-run-csv-reference.md index 2ea4c2a..bd2bfc0 100644 --- a/docs/dry-run-csv-reference.md +++ b/docs/dry-run-csv-reference.md @@ -13,16 +13,16 @@ The `--dry-run` flag generates 9 exhaustive CSV files in `migration-output/mappi ``` 1. cloudvoyager migrate -c config.json --dry-run - -> Extracts all data from SonarQube + -> Extracts all data from SonarQube Server -> Generates exhaustive CSVs in migration-output/mappings/ -> Prints summary + instructions - -> Stops (no SonarCloud changes) + -> Stops (no SonarQube Cloud changes) 2. Review and edit CSVs (set Include=no to exclude resources) 3. cloudvoyager migrate -c config.json -> Detects existing CSVs, reads them into memory BEFORE wiping output dir - -> Re-extracts from SonarQube, applies CSV overrides + -> Re-extracts from SonarQube Server, applies CSV overrides -> Migrates using filtered data 4. If interrupted, re-run the same command to resume from the last checkpoint. @@ -53,7 +53,7 @@ Every CSV has an `Include` column as the first field. Values are case-insensitiv | 6 | `portfolio-mappings.csv` | Portfolios and member projects (parent/child) | Include | | 7 | `template-mappings.csv` | Permission templates and assignments (parent/child) | Include | | 8 | `global-permissions.csv` | Group-level permission assignments | Include | -| 9 | `user-mappings.csv` | SonarQube-to-SonarCloud user login mapping | Include, SonarCloud Login | +| 9 | `user-mappings.csv` | SonarQube Server-to-SonarQube Cloud user login mapping | Include, SonarQube Cloud Login | --- @@ -65,10 +65,10 @@ One row per branch per project. The main branch is listed first for each project | Column | Editable | Description | |--------|----------|-------------| | Include | Yes | Set to `no` to exclude this branch. Setting `no` on the main branch excludes the **entire project** | -| Project Key | Read-only | SonarQube project key | +| Project Key | Read-only | SonarQube Server project key | | Project Name | Read-only | Project display name | | Branch | Read-only | Branch name (main branch listed first) | -| Target Organization | Read-only | SonarCloud organization key | +| Target Organization | Read-only | SonarQube Cloud organization key | | ALM Platform | Read-only | DevOps binding (github, gitlab, etc.) | | Repository | Read-only | Repository identifier | | Monorepo | Read-only | Whether project is part of a monorepo | @@ -95,7 +95,7 @@ One row per organization binding group. | Column | Editable | Description | |--------|----------|-------------| | Include | Yes | Set to `no` to exclude | -| Target Organization | Read-only | SonarCloud organization key | +| Target Organization | Read-only | SonarQube Cloud organization key | | Binding Group | Read-only | DevOps binding group identifier | | ALM Platform | Read-only | DevOps platform | | Projects Count | Read-only | Number of projects in this group | @@ -112,7 +112,7 @@ One row per group. | Description | Read-only | Group description | | Members Count | Read-only | Number of members | | Is Default | Read-only | Whether this is a default group | -| Target Organization | Read-only | Target SonarCloud organization | +| Target Organization | Read-only | Target SonarQube Cloud organization | ### profile-mappings.csv @@ -128,12 +128,12 @@ One row per quality profile. | Is Built-In | Read-only | Whether this is a built-in profile | | Parent | Read-only | Parent profile name (if inherited) | | Active Rules | Read-only | Number of active rules | -| Target Organization | Read-only | Target SonarCloud organization | +| Target Organization | Read-only | Target SonarQube Cloud organization | ### gate-mappings.csv -One row per quality gate. Conditions are always migrated as-is from SonarQube β€” they cannot be modified via the CSV. +One row per quality gate. Conditions are always migrated as-is from SonarQube Server β€” they cannot be modified via the CSV. | Column | Editable | Description | |--------|----------|-------------| @@ -142,7 +142,7 @@ One row per quality gate. Conditions are always migrated as-is from SonarQube | Is Default | Read-only | Whether this is the default gate | | Is Built-In | Read-only | Whether this is a built-in gate | | Conditions Count | Read-only | Number of conditions on this gate | -| Target Organization | Read-only | Target SonarCloud organization | +| Target Organization | Read-only | Target SonarQube Cloud organization | **Example:** ```csv @@ -168,7 +168,7 @@ Uses a **parent/child row pattern**. Portfolio header rows have empty member fie | Visibility | Read-only | *(header row only)* Portfolio visibility | | Member Project Key | Read-only | *(member row)* Project key | | Member Project Name | Read-only | *(member row)* Project name | -| Target Organization | Read-only | Target SonarCloud organization | +| Target Organization | Read-only | Target SonarQube Cloud organization | ### template-mappings.csv (Parent/Child Pattern) @@ -183,7 +183,7 @@ Uses a **parent/child row pattern**. Template header rows have empty permission | Key Pattern | Read-only | *(header row only)* Project key pattern regex | | Permission Key | Read-only | *(permission row)* Permission key (e.g., `admin`, `codeviewer`) | | Group Name | Read-only | *(permission row)* Group that has this permission | -| Target Organization | Read-only | Target SonarCloud organization | +| Target Organization | Read-only | Target SonarQube Cloud organization | ### global-permissions.csv @@ -199,20 +199,20 @@ One row per group+permission combination. ### user-mappings.csv -One row per unique SonarQube issue assignee, sorted by issue count (descending). This CSV enables mapping SonarQube usernames to SonarCloud logins, which typically differ because SonarCloud uses SSO/GitHub authentication. +One row per unique SonarQube Server issue assignee, sorted by issue count (descending). This CSV enables mapping SonarQube Server usernames to SonarQube Cloud logins, which typically differ because SonarQube Cloud uses SSO/GitHub authentication. | Column | Editable | Description | |--------|----------|-------------| | Include | Yes | Set to `no` to skip assignment for this user entirely | -| SonarQube Login | Read-only | Username/login from SonarQube | -| SonarCloud Login | **Yes** | Fill in the corresponding SonarCloud login for this user | -| Display Name | Read-only | User's display name from SonarQube (for identification) | -| Email | Read-only | User's email from SonarQube (for identification) | +| SonarQube Server Login | Read-only | Username/login from SonarQube Server | +| SonarQube Cloud Login | **Yes** | Fill in the corresponding SonarQube Cloud login for this user | +| Display Name | Read-only | User's display name from SonarQube Server (for identification) | +| Email | Read-only | User's email from SonarQube Server (for identification) | | Issue Count | Read-only | Total number of issues assigned to this user across all projects | **Example:** ```csv -Include,SonarQube Login,SonarCloud Login,Display Name,Email,Issue Count +Include,SonarQube Server Login,SonarQube Cloud Login,Display Name,Email,Issue Count yes,john.doe,john-doe-github,John Doe,john@example.com,42 yes,jane.smith,jsmith,Jane Smith,jane@example.com,15 no,service-account,,Service Account,,3 @@ -220,10 +220,10 @@ yes,dev.user,,Dev User,dev@example.com,1 ``` In this example: -- `john.doe` will be mapped to `john-doe-github` in SonarCloud -- `jane.smith` will be mapped to `jsmith` in SonarCloud +- `john.doe` will be mapped to `john-doe-github` in SonarQube Cloud +- `jane.smith` will be mapped to `jsmith` in SonarQube Cloud - `service-account` is excluded β€” its issues will not be assigned to anyone -- `dev.user` has no SonarCloud Login filled in β€” assignment will be attempted with the original SQ login `dev.user` (current default behavior) +- `dev.user` has no SonarQube Cloud Login filled in β€” assignment will be attempted with the original SQ login `dev.user` (current default behavior) **Note:** The Display Name and Email columns are provided to help identify users β€” they are not used during migration. @@ -252,9 +252,9 @@ In `group-mappings.csv`, set `Include` to `no` for the group row. In `template-mappings.csv`, set `Include` to `no` on the specific permission row (not the template header). -### Map a SonarQube user to a SonarCloud user +### Map a SonarQube Server user to a SonarQube Cloud user -In `user-mappings.csv`, fill in the `SonarCloud Login` column with the user's SonarCloud login (e.g., their GitHub username). +In `user-mappings.csv`, fill in the `SonarQube Cloud Login` column with the user's SonarQube Cloud login (e.g., their GitHub username). ### Skip assignment for a service account @@ -274,7 +274,7 @@ The shared `csv-entity-filters.js` module (`src/shared/mapping/csv-entity-filter - CSVs are RFC 4180 compliant. Fields containing commas, quotes, or newlines are properly quoted. - Running `--dry-run` again will regenerate fresh CSVs, overwriting any edits. -- If CSV references entities that no longer exist in SonarQube (e.g., a project was deleted between dry-run and full run), a warning is logged and the reference is skipped. +- If CSV references entities that no longer exist in SonarQube Server (e.g., a project was deleted between dry-run and full run), a warning is logged and the reference is skipped. - Gate names, project keys, and group names are matched **case-sensitively**. - If a migration is interrupted mid-run, re-running the same command resumes from the last checkpoint. Already-migrated projects and resources are skipped. diff --git a/docs/key-capabilities.md b/docs/key-capabilities.md index 077e7c2..fc69f6e 100644 --- a/docs/key-capabilities.md +++ b/docs/key-capabilities.md @@ -9,9 +9,9 @@ 1. [Executive Summary](#1-executive-summary) 2. [The Core Innovation: Reverse-Engineered Scanner Protocol](#2-the-core-innovation-reverse-engineered-scanner-protocol) -3. [Exhaustive SonarQube Data Extraction](#3-exhaustive-sonarqube-data-extraction) +3. [Exhaustive SonarQube Server Data Extraction](#3-exhaustive-sonarqube-data-extraction) 4. [Protobuf Report Encoding Engine](#4-protobuf-report-encoding-engine) -5. [SonarCloud Integration and Upload](#5-sonarcloud-integration-and-upload) +5. [SonarQube Cloud Integration and Upload](#5-sonarcloud-integration-and-upload) 6. [Full Organization Migration Pipeline](#6-full-organization-migration-pipeline) 7. [Issue and Hotspot Metadata Synchronization](#7-issue-and-hotspot-metadata-synchronization) 8. [Quality Profile and Quality Gate Migration](#8-quality-profile-and-quality-gate-migration) @@ -40,9 +40,9 @@ + Use this guide to build and run CloudVoyager locally. All developers should **build the binary and run that** β€” do not run directly from source. This ensures consistent behavior across environments and eliminates "works on my machine" issues. @@ -183,7 +183,7 @@ chmod +x dist/bin/cloudvoyager-macos-arm64 # Validate a config file ./cloudvoyager validate -c config.json -# Test connections to SonarQube and SonarCloud +# Test connections to SonarQube Server and SonarQube Cloud ./cloudvoyager test -c config.json # Transfer a single project @@ -256,7 +256,7 @@ This section documents every command and flag available in CloudVoyager. The exa --- -### `test` β€” Test connections to SonarQube and SonarCloud +### `test` β€” Test connections to SonarQube Server and SonarQube Cloud | Flag | Short | Required | Argument | Description | |------|-------|----------|----------|-------------| @@ -318,19 +318,19 @@ This section documents every command and flag available in CloudVoyager. The exa --- -### `transfer` β€” Transfer a single project from SonarQube to SonarCloud +### `transfer` β€” Transfer a single project from SonarQube Server to SonarQube Cloud | Flag | Short | Required | Argument | Description | |------|-------|----------|----------|-------------| | `--config ` | `-c` | Yes | File path | Path to the configuration file | | `--verbose` | `-v` | No | β€” | Enable debug-level logging for detailed output | -| `--wait` | β€” | No | β€” | Wait for the SonarCloud analysis to complete before returning (blocks until the Compute Engine task finishes) | +| `--wait` | β€” | No | β€” | Wait for the SonarQube Cloud analysis to complete before returning (blocks until the Compute Engine task finishes) | | `--concurrency ` | β€” | No | Integer | Override the maximum concurrency for I/O operations (source file extraction, hotspot extraction, etc.) | | `--max-memory ` | β€” | No | Integer | Set the max heap size in MB; auto-restarts the process with increased heap if the current heap is too small | | `--auto-tune` | β€” | No | β€” | Auto-detect hardware (CPU cores, available memory) and set optimal concurrency and memory values | | `--skip-all-branch-sync` | β€” | No | β€” | Only sync the main branch (skip non-main branches). Equivalent to setting `transfer.syncAllBranches: false` in config | | `--force-restart` | β€” | No | β€” | Discard checkpoint journal and start a fresh transfer from scratch | -| `--force-fresh-extract` | β€” | No | β€” | Discard extraction caches and re-extract all data from SonarQube | +| `--force-fresh-extract` | β€” | No | β€” | Discard extraction caches and re-extract all data from SonarQube Server | | `--force-unlock` | β€” | No | β€” | Force release a stale lock file from a previous run | | `--show-progress` | β€” | No | β€” | Display checkpoint progress status table and exit | @@ -347,7 +347,7 @@ This section documents every command and flag available in CloudVoyager. The exa # Transfer with verbose logging (short flag) ./cloudvoyager transfer -c config.json -v -# Transfer and wait for SonarCloud analysis to finish +# Transfer and wait for SonarQube Cloud analysis to finish ./cloudvoyager transfer -c config.json --wait # Transfer with verbose logging and wait for analysis @@ -410,17 +410,17 @@ This section documents every command and flag available in CloudVoyager. The exa --- -### `migrate` β€” Full migration from SonarQube to one or more SonarCloud organizations +### `migrate` β€” Full migration from SonarQube Server to one or more SonarQube Cloud organizations | Flag | Short | Required | Argument | Description | |------|-------|----------|----------|-------------| | `--config ` | `-c` | Yes | File path | Path to the migration configuration file | | `--verbose` | `-v` | No | β€” | Enable debug-level logging for detailed output | -| `--wait` | β€” | No | β€” | Wait for each SonarCloud analysis to complete before proceeding | +| `--wait` | β€” | No | β€” | Wait for each SonarQube Cloud analysis to complete before proceeding | | `--dry-run` | β€” | No | β€” | Extract data and generate project/key mappings without actually migrating any data | | `--skip-issue-metadata-sync` | β€” | No | β€” | Skip syncing issue metadata (statuses, assignments, comments, tags) after transfer | | `--skip-hotspot-metadata-sync` | β€” | No | β€” | Skip syncing hotspot metadata (statuses, comments) after transfer | -| `--skip-quality-profile-sync` | β€” | No | β€” | Skip syncing quality profiles (projects use default SonarCloud profiles) | +| `--skip-quality-profile-sync` | β€” | No | β€” | Skip syncing quality profiles (projects use default SonarQube Cloud profiles) | | `--concurrency ` | β€” | No | Integer | Override the maximum concurrency for I/O operations (applies to source extraction, hotspot extraction, issue sync, and hotspot sync) | | `--max-memory ` | β€” | No | Integer | Set the max heap size in MB; auto-restarts with increased heap if needed | | `--project-concurrency ` | β€” | No | Integer | Maximum number of projects to migrate concurrently | @@ -527,7 +527,7 @@ This section documents every command and flag available in CloudVoyager. The exa # Migration with auto-tune, wait, and skip both metadata syncs ./cloudvoyager migrate -c migrate-config.json --verbose --wait --auto-tune --skip-issue-metadata-sync --skip-hotspot-metadata-sync -# Migration skipping quality profile sync (use default SonarCloud profiles) +# Migration skipping quality profile sync (use default SonarQube Cloud profiles) ./cloudvoyager migrate -c migrate-config.json --verbose --skip-quality-profile-sync # Migration skipping quality profile sync with auto-tune @@ -588,7 +588,7 @@ This section documents every command and flag available in CloudVoyager. The exa --- -### `verify` β€” Verify migration completeness by comparing SonarQube and SonarCloud data +### `verify` β€” Verify migration completeness by comparing SonarQube Server and SonarQube Cloud data | Flag | Short | Required | Argument | Description | |------|-------|----------|----------|-------------| @@ -658,7 +658,7 @@ This section documents every command and flag available in CloudVoyager. The exa | `--verbose` | `-v` | No | β€” | Enable debug-level logging for detailed output | | `--skip-issue-metadata-sync` | β€” | No | β€” | Skip syncing issue metadata (statuses, assignments, comments, tags); only sync hotspot metadata | | `--skip-hotspot-metadata-sync` | β€” | No | β€” | Skip syncing hotspot metadata (statuses, comments); only sync issue metadata | -| `--skip-quality-profile-sync` | β€” | No | β€” | Skip syncing quality profiles (projects use default SonarCloud profiles) | +| `--skip-quality-profile-sync` | β€” | No | β€” | Skip syncing quality profiles (projects use default SonarQube Cloud profiles) | | `--concurrency ` | β€” | No | Integer | Override the maximum concurrency for I/O operations (issue sync, hotspot sync, hotspot extraction) | | `--max-memory ` | β€” | No | Integer | Set the max heap size in MB; auto-restarts with increased heap if needed | | `--auto-tune` | β€” | No | β€” | Auto-detect hardware and set optimal concurrency and memory values | @@ -782,13 +782,13 @@ The project uses a multi-stage GitHub Actions pipeline: | `build.yml` | Build CLI binaries | 6 platform builds in parallel using Node.js 20 | | `build-desktop.yml` | Build Electron desktop apps | 6 Electron builds (depends on CLI build artifacts) | | `release.yml` | Orchestrator | Chains: install β†’ build β†’ build-desktop β†’ release | -| `sonarcloud.yml` | SonarCloud SAST/SCA | Automatic scanning on push to `main` and on PRs. Requires `SONAR_TOKEN` secret | +| `sonarcloud.yml` | SonarQube Cloud SAST/SCA | Automatic scanning on push to `main` and on PRs. Requires `SONAR_TOKEN` secret | | `gh-release.yml` | GitHub Releases | Creates releases with milestone links derived from version tags | -### SonarCloud Scanning +### SonarQube Cloud Scanning -The repository includes a `sonarcloud.yml` workflow and a `sonar-project.properties` file at the project root for automatic SAST/SCA scanning via SonarCloud. The workflow runs on every push to `main` and on pull requests. It requires a `SONAR_TOKEN` secret configured in the GitHub repository settings. +The repository includes a `sonarcloud.yml` workflow and a `sonar-project.properties` file at the project root for automatic SAST/SCA scanning via SonarQube Cloud. The workflow runs on every push to `main` and on pull requests. It requires a `SONAR_TOKEN` secret configured in the GitHub repository settings. ### Platform Build Matrix (6 targets) @@ -811,10 +811,10 @@ The repository includes a `sonarcloud.yml` workflow and a `sonar-project.propert |----------|-------------| | `LOG_LEVEL` | Set logging level: `debug`, `info`, `warn`, `error` | | `LOG_FILE` | Path to log file (optional, logs to console by default) | -| `SONARQUBE_TOKEN` | Override SonarQube token from config | -| `SONARCLOUD_TOKEN` | Override SonarCloud token from config | -| `SONARQUBE_URL` | Override SonarQube URL from config | -| `SONARCLOUD_URL` | Override SonarCloud URL from config | +| `SONARQUBE_TOKEN` | Override SonarQube Server token from config | +| `SONARCLOUD_TOKEN` | Override SonarQube Cloud token from config | +| `SONARQUBE_URL` | Override SonarQube Server URL from config | +| `SONARCLOUD_URL` | Override SonarQube Cloud URL from config | | `MAX_SOURCE_FILES` | Limit number of source files to extract (`0` = all) | @@ -909,7 +909,7 @@ The `.debugging/` folder contains convenience scripts for local testing: | `test-verify.sh` | Run `verify` command with `migrate-config.json` | | `build-desktop.sh` | Build the Electron desktop app | | `run-desktop.sh` | Launch the desktop app in dev mode | -| `delete-all-sonarcloud-projects.sh` | Delete all projects from a SonarCloud org (for cleanup) | +| `delete-all-sonarcloud-projects.sh` | Delete all projects from a SonarQube Cloud org (for cleanup) | These scripts expect the binary at `./dist/bin/cloudvoyager-macos-arm64` and config files at the repo root. Both `migrate-config.json` and `transfer-config.json` are gitignored since they contain credentials. @@ -931,7 +931,7 @@ These scripts expect the binary at `./dist/bin/cloudvoyager-macos-arm64` and con ## πŸ§ͺ Regression Testing -The regression testing system validates that bug fixes and behavioral changes in CloudVoyager remain correct across SonarQube versions. +The regression testing system validates that bug fixes and behavioral changes in CloudVoyager remain correct across SonarQube Server versions. ### Where the Code Lives @@ -944,16 +944,16 @@ The regression testing system validates that bug fixes and behavioral changes in Full regression tests require an environment that mirrors CI: -1. **Docker** β€” ephemeral SonarQube containers are spun up per test scenario. -2. **SonarQube Enterprise license key** β€” the containers need a valid Enterprise Edition license. -3. **SonarCloud organization access** β€” tests migrate data into SonarCloud and verify the results. +1. **Docker** β€” ephemeral SonarQube Server containers are spun up per test scenario. +2. **SonarQube Server Enterprise license key** β€” the containers need a valid Enterprise Edition license. +3. **SonarQube Cloud organization access** β€” tests migrate data into SonarQube Cloud and verify the results. > **Note:** These prerequisites make local runs heavyweight. Most day-to-day development does not require running the full regression suite locally β€” CI handles it on every PR. ### Meta-Tests (No Docker Required) -The helper utilities in `test/regression/helpers/` have their own unit tests (`test/regression/helpers/*.test.js`). These **meta-tests** run with the regular test command and do not require Docker or a running SonarQube instance: +The helper utilities in `test/regression/helpers/` have their own unit tests (`test/regression/helpers/*.test.js`). These **meta-tests** run with the regular test command and do not require Docker or a running SonarQube Server instance: ```bash npm test @@ -963,7 +963,7 @@ npm test 1. **Create an assertion script:** Add `test/regression/assert-{scenario-name}.js` in this repository. Follow the conventions of existing assertion scripts in the same directory. -2. **Add a matrix entry:** In the private repo `sonar-solutions/cloudvoyager-ci`, add a new entry to the matrix in `regression-bug-fixes.yml` that references your assertion script and the SonarQube version(s) it should run against. +2. **Add a matrix entry:** In the private repo `sonar-solutions/cloudvoyager-ci`, add a new entry to the matrix in `regression-bug-fixes.yml` that references your assertion script and the SonarQube Server version(s) it should run against. -**Problem:** SonarQube's `/api/issues/search` endpoint has a hard 10,000-result limit. Projects exceeding this silently lost data during migration. +**Problem:** SonarQube Server's `/api/issues/search` endpoint has a hard 10,000-result limit. Projects exceeding this silently lost data during migration. **Solution:** Implemented a date-window search-slicing algorithm that automatically activates when results hit the 10K limit. @@ -68,16 +68,16 @@ ## Issue #56 β€” Third-Party Issue Migration Bug -**Problem:** When `getRuleRepositories()` failed to fetch the SonarCloud rules API, it returned an empty Set. This caused `isExternalIssue()` to return `false` for ALL issues, silently dropping third-party analyzer issues (ruff, pylint, Trivy, etc.) during migration. +**Problem:** When `getRuleRepositories()` failed to fetch the SonarQube Cloud rules API, it returned an empty Set. This caused `isExternalIssue()` to return `false` for ALL issues, silently dropping third-party analyzer issues (ruff, pylint, Trivy, etc.) during migration. -**Solution:** Added a fallback set of 43 known SonarCloud rule repositories, retry logic with exponential backoff on `getRuleRepositories()`, and edge-case guards in `isExternalIssue()`. +**Solution:** Added a fallback set of 43 known SonarQube Cloud rule repositories, retry logic with exponential backoff on `getRuleRepositories()`, and edge-case guards in `isExternalIssue()`. ### Files Created | File | Purpose | |------|---------| -| `src/shared/utils/fallback-repos/index.js` | Exports `FALLBACK_SONARCLOUD_REPOS` β€” Set of 43 known built-in SonarCloud rule repository keys | +| `src/shared/utils/fallback-repos/index.js` | Exports `FALLBACK_SONARCLOUD_REPOS` β€” Set of 43 known built-in SonarQube Cloud rule repository keys | ### Files Modified @@ -111,20 +111,20 @@ --- -## Issue #66 β€” SonarCloud Public Scanning +## Issue #66 β€” SonarQube Cloud Public Scanning **Problem:** No automated SAST, unit test coverage, or SCA scanning was configured for the repository. -**Solution:** Added a standalone GitHub Actions workflow that runs SonarCloud analysis on every push to `main` and on pull requests. +**Solution:** Added a standalone GitHub Actions workflow that runs SonarQube Cloud analysis on every push to `main` and on pull requests. ### Files Created | File | Purpose | |------|---------| -| `.github/workflows/sonarcloud.yml` | Fully automatic SonarCloud scanning workflow | -| `sonar-project.properties` | SonarCloud project configuration | +| `.github/workflows/sonarcloud.yml` | Fully automatic SonarQube Cloud scanning workflow | +| `sonar-project.properties` | SonarQube Cloud project configuration | ### Verification @@ -193,6 +193,6 @@ Requires the `SONAR_TOKEN` secret to be configured in the GitHub repository sett |-------|--------|--------|--------| | #53 β€” 10K+ Search Slicing | 7 | 7 | 0 | | #56 β€” Third-Party Issue Fix | 9 | 9 | 0 | -| #66 β€” SonarCloud Scanning | 15 | 15 | 0 | +| #66 β€” SonarQube Cloud Scanning | 15 | 15 | 0 | | #75 β€” Release Milestones | 8 | 8 | 0 | | **Total** | **39** | **39** | **0** | diff --git a/docs/pseudocode-explanation.md b/docs/pseudocode-explanation.md index 3585025..528d430 100644 --- a/docs/pseudocode-explanation.md +++ b/docs/pseudocode-explanation.md @@ -14,7 +14,7 @@ This document describes each feature of the CloudVoyager migration tool in pseud 5. [Verification Pipeline](#5-verification-pipeline) 6. [Data Extraction](#6-data-extraction) 7. [Protobuf Building & Encoding](#7-protobuf-building--encoding) -8. [Report Upload to SonarCloud](#8-report-upload-to-sonarcloud) +8. [Report Upload to SonarQube Cloud](#8-report-upload-to-sonarcloud) 9. [External Issues & Plugin Migration](#9-external-issues--plugin-migration) 10. [Issue & Hotspot Metadata Sync](#10-issue--hotspot-metadata-sync) 11. [Quality Gates Migration](#11-quality-gates-migration) @@ -45,11 +45,11 @@ COMMAND: test INPUT: --config , [--verbose] STEPS: 1. Load config - 2. Detect SonarQube version via version-router (GET /api/system/status) + 2. Detect SonarQube Server version via version-router (GET /api/system/status) 3. Load the correct pipeline for the detected version (sq-9.9, sq-10.0, sq-10.4, or sq-2025) 4. Create SonarQubeClient from the selected pipeline - 5. Create SonarCloud client - 6. Test SonarCloud connection (GET /api/organizations/search) + 5. Create SonarQube Cloud client + 6. Test SonarQube Cloud connection (GET /api/organizations/search) 7. Report success or failure for each COMMAND: status @@ -90,7 +90,7 @@ COMMAND: verify ## 2. Transfer Pipeline -Transfers a single SonarQube project to SonarCloud. +Transfers a single SonarQube Server project to SonarQube Cloud. ``` FUNCTION transferProject(sonarqubeConfig, sonarcloudConfig, transferConfig, performanceConfig, wait): @@ -168,7 +168,7 @@ FUNCTION transferBranch(extractedData, branch, scConfig, scProfiles, scRepos, .. encoder.loadSchemas() encodedReport = encoder.encodeAll(messages) - // Step 3: Upload to SonarCloud + // Step 3: Upload to SonarQube Cloud uploader = new ReportUploader(scClient) ceTask = uploader.upload(encodedReport, metadata) @@ -198,7 +198,7 @@ FUNCTION transferBranchBatched(extractedData, branch, scConfig, ...): ## 3. Migration Pipeline -Migrates all projects across multiple SonarCloud organizations. +Migrates all projects across multiple SonarQube Cloud organizations. ``` FUNCTION migrateAll(sonarqubeConfig, sonarcloudOrgs, migrateConfig, transferConfig, ...): @@ -335,7 +335,7 @@ COMMAND sync-metadata: ## 5. Verification Pipeline -Compares SonarQube and SonarCloud data to verify migration completeness. +Compares SonarQube Server and SonarQube Cloud data to verify migration completeness. ``` FUNCTION verifyAll(sonarqubeConfig, sonarcloudOrgs, outputDir, onlyComponents, ...): @@ -398,7 +398,7 @@ FUNCTION verifyAll(sonarqubeConfig, sonarcloudOrgs, outputDir, onlyComponents, . ## 6. Data Extraction -Extracts all data from SonarQube for a single project. +Extracts all data from SonarQube Server for a single project. ``` FUNCTION DataExtractor.extractAll(): @@ -476,7 +476,7 @@ FUNCTION DataExtractor.extractBranch(branchName, mainData): ## 7. Protobuf Building & Encoding -Transforms extracted data into SonarCloud's scanner report protobuf format. +Transforms extracted data into SonarQube Cloud's scanner report protobuf format. ``` CLASS ProtobufBuilder: @@ -621,10 +621,10 @@ CLASS ProtobufEncoder: --- -## 8. Report Upload to SonarCloud +## 8. Report Upload to SonarQube Cloud -Packages the encoded protobuf report and submits it to SonarCloud's Compute Engine. +Packages the encoded protobuf report and submits it to SonarQube Cloud's Compute Engine. ``` CLASS ReportUploader: @@ -716,7 +716,7 @@ FUNCTION buildExternalIssues(): // --- Resolve Clean Code Attribute --- // CRITICAL: Must be a protobuf enum (varint int), NOT a string! - // SonarCloud CE silently ignores external issues if string-encoded. + // SonarQube Cloud CE silently ignores external issues if string-encoded. IF issue.cleanCodeAttribute: cleanCodeAttr = mapCleanCodeAttribute(issue.cleanCodeAttribute) // Maps: "CONVENTIONAL"->1, "FORMATTED"->2, ... "TRUSTWORTHY"->14 @@ -765,7 +765,7 @@ FUNCTION buildExternalIssues(): FUNCTION buildRuleEnrichmentMap(scClient, scProfiles): - // For SQ < 10.0: fetch Clean Code taxonomy from SonarCloud + // For SQ < 10.0: fetch Clean Code taxonomy from SonarQube Cloud enrichmentMap = new Map() FOR EACH profile IN scProfiles: @@ -784,7 +784,7 @@ FUNCTION buildRuleEnrichmentMap(scClient, scProfiles): ## 10. Issue & Hotspot Metadata Sync -Syncs issue/hotspot statuses, assignments, comments, and tags from SonarQube to SonarCloud after scanner report upload. +Syncs issue/hotspot statuses, assignments, comments, and tags from SonarQube Server to SonarQube Cloud after scanner report upload. ``` FUNCTION syncIssues(projectKey, sqIssues, scClient, options): @@ -834,8 +834,8 @@ FUNCTION syncIssues(projectKey, sqIssues, scClient, options): // 4e. Mark as metadata-synchronized scClient.setIssueTags(sc.key, [...existing, "metadata-synchronized"]) - // 4f. Add source link back to SonarQube - scClient.addIssueComment(sc.key, "SonarQube Source: {sonarqubeUrl}/issues?id=...") + // 4f. Add source link back to SonarQube Server + scClient.addIssueComment(sc.key, "SonarQube Server Source: {sonarqubeUrl}/issues?id=...") RETURN { matched, transitioned, assigned, commented, tagged, failed } @@ -860,7 +860,7 @@ FUNCTION syncHotspots(projectKey, sqHotspots, scClient, options): scClient.addHotspotComment(sc.key, formatComment(comment)) // Source link - scClient.addHotspotComment(sc.key, "SonarQube Source: {url}") + scClient.addHotspotComment(sc.key, "SonarQube Server Source: {url}") RETURN { matched, statusChanged, commentAdded, failed } ``` @@ -931,10 +931,10 @@ FUNCTION migrateQualityProfiles(extractedProfiles, scClient): // --- Restore built-in profiles as custom (with renamed suffix) --- FOR EACH profile IN builtInProfiles: renamedXml = modifyBackupXml(profile.backupXml, { - name: profile.name + " (SonarQube Migrated)" + name: profile.name + " (SonarQube Server Migrated)" }) scClient.restoreQualityProfile(renamedXml) - builtInProfileMapping.SET(profile.language, profile.name + " (SonarQube Migrated)") + builtInProfileMapping.SET(profile.language, profile.name + " (SonarQube Server Migrated)") // --- Set defaults --- FOR EACH profile IN customProfiles WHERE profile.isDefault: @@ -1013,7 +1013,7 @@ FUNCTION migrateProjectPermissions(project, scClient): ## 14. Organization Mapping & CSV Generation -Maps SonarQube projects to SonarCloud organizations and generates editable CSV files. +Maps SonarQube Server projects to SonarQube Cloud organizations and generates editable CSV files. ``` FUNCTION mapProjectsToOrganizations(allProjects, projectBindings, sonarcloudOrgs): @@ -1088,7 +1088,7 @@ FUNCTION applyCsvOverrides(csvData, extractedData, resourceMappings, orgAssignme ## 15. Version Router and Pipeline Selection -Detects SonarQube version at runtime and loads the correct version-specific pipeline. +Detects SonarQube Server version at runtime and loads the correct version-specific pipeline. ``` FUNCTION detectAndRoute(sonarqubeConfig): diff --git a/docs/regression-testing.md b/docs/regression-testing.md index 5504981..19a27e3 100644 --- a/docs/regression-testing.md +++ b/docs/regression-testing.md @@ -1,7 +1,7 @@ # Regression Testing Protocol -A reusable protocol for verifying bug fixes and features against a live SonarQube β†’ SonarCloud migration. This protocol is **issue-driven** β€” you MUST fully understand the GitHub issue before designing any test, and your test plan must be explicitly derived from the issue's symptoms, root cause, and fix. +A reusable protocol for verifying bug fixes and features against a live SonarQube Server β†’ SonarQube Cloud migration. This protocol is **issue-driven** β€” you MUST fully understand the GitHub issue before designing any test, and your test plan must be explicitly derived from the issue's symptoms, root cause, and fix. --- @@ -33,7 +33,7 @@ For the GitHub issue you are regression testing, you must be able to answer ALL From the issue understanding in 0.1, write a **specific regression test plan** BEFORE touching any code. This plan must include: -- **Which SonarQube project to use** β€” pick the project that best reproduces the issue (e.g., Angular Framework for hotspots because it has ~409 hotspots) +- **Which SonarQube Server project to use** β€” pick the project that best reproduces the issue (e.g., Angular Framework for hotspots because it has ~409 hotspots) - **What test data to create** β€” do you need to bulk-tag issues, add comments, transition statuses to trigger the pre-filter? How many? - **What to verify after migration** β€” specific API queries, specific counts, specific spot-checks - **What the pass criteria are** β€” concrete numbers, not vague "looks right" @@ -77,7 +77,7 @@ Find ALL code paths that touch the same data or API calls as the fix: 8. **Worker thread lifecycle**: Are workers properly joined before the process exits? Are they cleaned up on error? ### API Contract Compliance -9. **HTTP method**: Does the code use GET/POST/PUT/DELETE exactly as the SonarQube/SonarCloud API specifies? +9. **HTTP method**: Does the code use GET/POST/PUT/DELETE exactly as the SonarQube Server/SonarQube Cloud API specifies? 10. **Endpoint path**: Is the URL exactly right β€” no missing/misplaced path segments, no incorrect query params? 11. **Request body/params**: Are field names, types, and encoding exactly correct? Compare character-by-character against the real API docs or actual observed API traces. 12. **Auth**: Are tokens passed correctly (Authorization header vs query param)? Are tokens refreshed if expired? diff --git a/docs/scenario-multi-org.md b/docs/scenario-multi-org.md index 6f472f1..a2b278c 100644 --- a/docs/scenario-multi-org.md +++ b/docs/scenario-multi-org.md @@ -1,9 +1,9 @@ -# Migrate Everything to Multiple SonarCloud Organizations +# Migrate Everything to Multiple SonarQube Cloud Organizations -Use this when you want to migrate **all projects and configuration** from your SonarQube server to **multiple** SonarCloud organizations β€” for example, when different teams or business units each have their own SonarCloud org. +Use this when you want to migrate **all projects and configuration** from your SonarQube Server to **multiple** SonarQube Cloud organizations β€” for example, when different teams or business units each have their own SonarQube Cloud org. --- @@ -31,13 +31,13 @@ CloudVoyager uses DevOps platform bindings (GitHub, GitLab, etc.) to automatical ## Prerequisites -1. **Admin access** to your SonarQube server -2. **Admin access** to **each** target SonarCloud organization -3. **API tokens** for SonarQube and for each SonarCloud organization +1. **Admin access** to your SonarQube Server +2. **Admin access** to **each** target SonarQube Cloud organization +3. **API tokens** for SonarQube Server and for each SonarQube Cloud organization > **How to get your tokens:** -> - **SonarQube:** Go to `My Account > Security > Generate Tokens` in your SonarQube web UI -> - **SonarCloud:** Go to `My Account > Security > Generate Tokens` at [sonarcloud.io](https://sonarcloud.io) β€” you need a token with admin permissions for **each** target org +> - **SonarQube Server:** Go to `My Account > Security > Generate Tokens` in your SonarQube Server web UI +> - **SonarQube Cloud:** Go to `My Account > Security > Generate Tokens` at [sonarcloud.io](https://sonarcloud.io) β€” you need a token with admin permissions for **each** target org --- @@ -104,19 +104,19 @@ See [`examples/migrate-config.example.json`](../examples/migrate-config.example. | Field | Required | Description | |-------|----------|-------------| -| `sonarqube.url` | Yes | Full URL of your SonarQube server | -| `sonarqube.token` | Yes | SonarQube admin API token | +| `sonarqube.url` | Yes | Full URL of your SonarQube Server | +| `sonarqube.token` | Yes | SonarQube Server admin API token | | `sonarcloud.enterprise.key` | No | Enterprise key β€” required for portfolio migration (uses Enterprise V2 API) | -| `sonarcloud.organizations[].key` | Yes | SonarCloud organization key | -| `sonarcloud.organizations[].token` | Yes | SonarCloud admin API token for this org | -| `sonarcloud.organizations[].url` | No | SonarCloud URL (default: `https://sonarcloud.io`) | +| `sonarcloud.organizations[].key` | Yes | SonarQube Cloud organization key | +| `sonarcloud.organizations[].token` | Yes | SonarQube Cloud admin API token for this org | +| `sonarcloud.organizations[].url` | No | SonarQube Cloud URL (default: `https://sonarcloud.io`) | | `migrate.outputDir` | No | Directory for mapping CSVs and reports (default: `./migration-output`) | -> **Tip:** You can set the SonarQube token via the `SONARQUBE_TOKEN` environment variable. SonarCloud tokens must be specified per-org in the config. +> **Tip:** You can set the SonarQube Server token via the `SONARQUBE_TOKEN` environment variable. SonarQube Cloud tokens must be specified per-org in the config. -> **Project key conflict resolution:** By default, the tool uses the **original SonarQube project key** on SonarCloud. If the key is already taken by another SonarCloud organization, the tool automatically falls back to a prefixed key (`{org-key}_{sonarqube-project-key}`) and logs a warning. This is especially common in multi-org migrations where different orgs may have projects with overlapping keys. All key conflicts are listed in the migration report. +> **Project key conflict resolution:** By default, the tool uses the **original SonarQube Server project key** on SonarQube Cloud. If the key is already taken by another SonarQube Cloud organization, the tool automatically falls back to a prefixed key (`{org-key}_{sonarqube-project-key}`) and logs a warning. This is especially common in multi-org migrations where different orgs may have projects with overlapping keys. All key conflicts are listed in the migration report. -> **Enterprise key for portfolios:** Portfolios in SonarCloud are enterprise-wide, not per-organization. To migrate portfolios, add `sonarcloud.enterprise.key` to your config. Without it, portfolio migration is skipped. The tool uses the Enterprise V2 API to create portfolios after all organizations have been migrated. +> **Enterprise key for portfolios:** Portfolios in SonarQube Cloud are enterprise-wide, not per-organization. To migrate portfolios, add `sonarcloud.enterprise.key` to your config. Without it, portfolio migration is skipped. The tool uses the Enterprise V2 API to create portfolios after all organizations have been migrated. **Example config with enterprise key:** ```json @@ -139,7 +139,7 @@ We recommend a 3-step migration: dry run, migrate without metadata, then sync me ### Step 3a: Dry run β€” verify everything -A dry run extracts all data and generates mapping CSVs so you can review **which projects go to which org**, without changing anything in SonarCloud: +A dry run extracts all data and generates mapping CSVs so you can review **which projects go to which org**, without changing anything in SonarQube Cloud: ```bash ./cloudvoyager migrate -c migrate-config.json --dry-run @@ -156,9 +156,9 @@ Run the actual migration with metadata sync disabled and auto-tuned performance. ./cloudvoyager migrate -c migrate-config.json --verbose --skip-issue-metadata-sync --skip-hotspot-metadata-sync --auto-tune ``` -Skipping metadata during the main migration avoids SonarCloud rate limiting (503 errors) that can occur during high-volume issue/hotspot sync. +Skipping metadata during the main migration avoids SonarQube Cloud rate limiting (503 errors) that can occur during high-volume issue/hotspot sync. -> **Note:** By default, the tool does not wait for each project's analysis to complete on SonarCloud before moving on to the next project. This speeds up large migrations significantly. Add `--wait` if you need to block until each analysis finishes. +> **Note:** By default, the tool does not wait for each project's analysis to complete on SonarQube Cloud before moving on to the next project. This speeds up large migrations significantly. Add `--wait` if you need to block until each analysis finishes. ### Step 3c: Sync metadata separately @@ -188,7 +188,7 @@ After syncing metadata, run the verification command to confirm everything was m ./cloudvoyager verify -c migrate-config.json --verbose ``` -This performs read-only checks comparing SonarQube and SonarCloud data and generates a detailed pass/fail report in `./verification-output/`. You can also verify specific components: +This performs read-only checks comparing SonarQube Server and SonarQube Cloud data and generates a detailed pass/fail report in `./verification-output/`. You can also verify specific components: ```bash # Verify only issue and hotspot metadata @@ -226,7 +226,7 @@ Each project within an organization goes through 11 individually-checkpointed st ### Parallel project extraction -During the extraction phase, the tool fetches branches for all projects concurrently using bounded parallelism (`mapConcurrent`). The concurrency level is controlled by `performance.sourceExtraction.concurrency` in the config or by `--concurrency` on the CLI. This significantly speeds up the data-gathering phase for large SonarQube instances. +During the extraction phase, the tool fetches branches for all projects concurrently using bounded parallelism (`mapConcurrent`). The concurrency level is controlled by `performance.sourceExtraction.concurrency` in the config or by `--concurrency` on the CLI. This significantly speeds up the data-gathering phase for large SonarQube Server instances. --- @@ -269,7 +269,7 @@ Use defaults β€” no `performance` section needed. } ``` -Keep `hotspotSync.concurrency` low (3–5) to avoid SonarCloud rate limits. See the [Configuration Reference](configuration.md#performance-settings) for all options. +Keep `hotspotSync.concurrency` low (3–5) to avoid SonarQube Cloud rate limits. See the [Configuration Reference](configuration.md#performance-settings) for all options. --- @@ -291,7 +291,7 @@ Keep `hotspotSync.concurrency` low (3–5) to avoid SonarCloud rate limits. See | `reports/executive-summary.md` | High-level executive summary | | `reports/performance-report.md` | Performance metrics breakdown | | `reports/*.pdf` | PDF versions of the above reports (best-effort) | -| `quality-profiles/quality-profile-diff.json` | Per-language diff of active rules between SonarQube and SonarCloud | +| `quality-profiles/quality-profile-diff.json` | Per-language diff of active rules between SonarQube Server and SonarQube Cloud | Server info (version, plugins, settings, webhooks) is saved to `{outputDir}/server-info/` as JSON files. Per-project state files are saved to `{outputDir}/state/` for incremental transfer tracking. @@ -318,7 +318,7 @@ When you run `./cloudvoyager verify`, reports are written to `./verification-out | `--dry-run` | Extract data and generate mappings without migrating | | `--skip-issue-metadata-sync` | Skip syncing issue statuses, comments, assignments, tags | | `--skip-hotspot-metadata-sync` | Skip syncing hotspot statuses and comments | -| `--skip-quality-profile-sync` | Skip syncing quality profiles (projects use default SonarCloud profiles) | +| `--skip-quality-profile-sync` | Skip syncing quality profiles (projects use default SonarQube Cloud profiles) | | `--only ` | Only migrate specific components (comma-separated). Valid: `scan-data`, `scan-data-all-branches`, `portfolios`, `quality-gates`, `quality-profiles`, `permission-templates`, `permissions`, `issue-metadata`, `hotspot-metadata`, `project-settings` | | `--auto-tune` | Auto-detect CPU and RAM and set optimal performance values | | `--concurrency ` | Override max concurrency for I/O operations | @@ -334,7 +334,7 @@ When you run `./cloudvoyager verify`, reports are written to `./verification-out ## Limitations -- Historical metrics (the charts in each project's **Activity** tab in SonarQube) cannot be migrated. All actual issues and hotspots are migrated β€” only the historical trend data is lost. +- Historical metrics (the charts in each project's **Activity** tab in SonarQube Server) cannot be migrated. All actual issues and hotspots are migrated β€” only the historical trend data is lost. --- diff --git a/docs/scenario-single-org.md b/docs/scenario-single-org.md index d7e7bef..1ded075 100644 --- a/docs/scenario-single-org.md +++ b/docs/scenario-single-org.md @@ -1,9 +1,9 @@ -# Migrate Everything to One SonarCloud Organization +# Migrate Everything to One SonarQube Cloud Organization -Use this when you want to migrate **all projects and configuration** from your SonarQube server to a **single** SonarCloud organization. +Use this when you want to migrate **all projects and configuration** from your SonarQube Server to a **single** SonarQube Cloud organization. --- @@ -26,13 +26,13 @@ Use this when you want to migrate **all projects and configuration** from your S ## Prerequisites -1. **Admin access** to your SonarQube server -2. **Admin access** to your SonarCloud organization -3. **API tokens** for both SonarQube and SonarCloud +1. **Admin access** to your SonarQube Server +2. **Admin access** to your SonarQube Cloud organization +3. **API tokens** for both SonarQube Server and SonarQube Cloud > **How to get your tokens:** -> - **SonarQube:** Go to `My Account > Security > Generate Tokens` in your SonarQube web UI -> - **SonarCloud:** Go to `My Account > Security > Generate Tokens` at [sonarcloud.io](https://sonarcloud.io) +> - **SonarQube Server:** Go to `My Account > Security > Generate Tokens` in your SonarQube Server web UI +> - **SonarQube Cloud:** Go to `My Account > Security > Generate Tokens` at [sonarcloud.io](https://sonarcloud.io) --- @@ -84,18 +84,18 @@ Create a file called `migrate-config.json`: See [`examples/migrate-config.example.json`](../examples/migrate-config.example.json) for a ready-to-use template with all optional fields (rate limiting, performance tuning, etc.). -> **Project keys and names:** Each project's display name is automatically carried over from SonarQube. By default, the tool uses the **original SonarQube project key** on SonarCloud. If the key is already taken by another SonarCloud organization, the tool falls back to a prefixed key (`{org-key}_{sonarqube-project-key}`) and logs a warning. Any key conflicts are listed in the migration report. +> **Project keys and names:** Each project's display name is automatically carried over from SonarQube Server. By default, the tool uses the **original SonarQube Server project key** on SonarQube Cloud. If the key is already taken by another SonarQube Cloud organization, the tool falls back to a prefixed key (`{org-key}_{sonarqube-project-key}`) and logs a warning. Any key conflicts are listed in the migration report. ### Config fields | Field | Required | Description | |-------|----------|-------------| -| `sonarqube.url` | Yes | Full URL of your SonarQube server | -| `sonarqube.token` | Yes | SonarQube admin API token | -| `sonarcloud.organizations[].key` | Yes | SonarCloud organization key | -| `sonarcloud.organizations[].token` | Yes | SonarCloud admin API token | -| `sonarcloud.organizations[].url` | No | SonarCloud URL (default: `https://sonarcloud.io`) | +| `sonarqube.url` | Yes | Full URL of your SonarQube Server | +| `sonarqube.token` | Yes | SonarQube Server admin API token | +| `sonarcloud.organizations[].key` | Yes | SonarQube Cloud organization key | +| `sonarcloud.organizations[].token` | Yes | SonarQube Cloud admin API token | +| `sonarcloud.organizations[].url` | No | SonarQube Cloud URL (default: `https://sonarcloud.io`) | | `migrate.outputDir` | No | Directory for mapping CSVs and reports (default: `./migration-output`) | > **Tip:** You can set tokens via environment variables (`SONARQUBE_TOKEN` and `SONARCLOUD_TOKEN`) instead of putting them in the config file. @@ -108,7 +108,7 @@ We recommend a 3-step migration: dry run, migrate without metadata, then sync me ### Step 3a: Dry run β€” verify everything -A dry run extracts all data and generates mapping CSVs so you can review what will be migrated, without changing anything in SonarCloud: +A dry run extracts all data and generates mapping CSVs so you can review what will be migrated, without changing anything in SonarQube Cloud: ```bash ./cloudvoyager migrate -c migrate-config.json --dry-run @@ -125,9 +125,9 @@ Run the actual migration with metadata sync disabled and auto-tuned performance. ./cloudvoyager migrate -c migrate-config.json --verbose --skip-issue-metadata-sync --skip-hotspot-metadata-sync --auto-tune ``` -Skipping metadata during the main migration avoids SonarCloud rate limiting (503 errors) that can occur during high-volume issue/hotspot sync. +Skipping metadata during the main migration avoids SonarQube Cloud rate limiting (503 errors) that can occur during high-volume issue/hotspot sync. -> **Note:** By default, the tool does not wait for each project's analysis to complete on SonarCloud before moving on to the next project. This speeds up large migrations significantly. Add `--wait` if you need to block until each analysis finishes. +> **Note:** By default, the tool does not wait for each project's analysis to complete on SonarQube Cloud before moving on to the next project. This speeds up large migrations significantly. Add `--wait` if you need to block until each analysis finishes. ### Step 3c: Sync metadata separately @@ -157,7 +157,7 @@ After syncing metadata, run the verification command to confirm everything was m ./cloudvoyager verify -c migrate-config.json --verbose ``` -This performs read-only checks comparing SonarQube and SonarCloud data and generates a detailed pass/fail report in `./verification-output/`. You can also verify specific components: +This performs read-only checks comparing SonarQube Server and SonarQube Cloud data and generates a detailed pass/fail report in `./verification-output/`. You can also verify specific components: ```bash # Verify only issue and hotspot metadata @@ -209,7 +209,7 @@ The `--dry-run` flag generates 9 CSV files in `{outputDir}/mappings/`. Each CSV | `portfolio-mappings.csv` | Portfolios with member projects (parent/child rows) | | `template-mappings.csv` | Permission templates with per-permission rows (parent/child) | | `global-permissions.csv` | Group-to-permission assignments | -| `user-mappings.csv` | SonarQube-to-SonarCloud user login mapping for issue assignment | +| `user-mappings.csv` | SonarQube Server-to-SonarQube Cloud user login mapping for issue assignment | See the [Dry-Run CSV Reference](dry-run-csv-reference.md) for full column schemas and editing examples. @@ -254,7 +254,7 @@ Use defaults β€” no `performance` section needed. } ``` -Keep `hotspotSync.concurrency` low (3–5) to avoid SonarCloud rate limits. See the [Configuration Reference](configuration.md#performance-settings) for all options. +Keep `hotspotSync.concurrency` low (3–5) to avoid SonarQube Cloud rate limits. See the [Configuration Reference](configuration.md#performance-settings) for all options. --- @@ -276,7 +276,7 @@ Keep `hotspotSync.concurrency` low (3–5) to avoid SonarCloud rate limits. See | `reports/executive-summary.md` | High-level executive summary | | `reports/performance-report.md` | Performance metrics breakdown | | `reports/*.pdf` | PDF versions of the above reports (best-effort) | -| `quality-profiles/quality-profile-diff.json` | Per-language diff of active rules between SonarQube and SonarCloud | +| `quality-profiles/quality-profile-diff.json` | Per-language diff of active rules between SonarQube Server and SonarQube Cloud | Server info (version, plugins, settings, webhooks) is saved to `{outputDir}/server-info/` as JSON files. Per-project state files are saved to `{outputDir}/state/` for incremental transfer tracking. @@ -303,7 +303,7 @@ When you run `./cloudvoyager verify`, reports are written to `./verification-out | `--dry-run` | Extract data and generate mappings without migrating | | `--skip-issue-metadata-sync` | Skip syncing issue statuses, comments, assignments, tags | | `--skip-hotspot-metadata-sync` | Skip syncing hotspot statuses and comments | -| `--skip-quality-profile-sync` | Skip syncing quality profiles (projects use default SonarCloud profiles) | +| `--skip-quality-profile-sync` | Skip syncing quality profiles (projects use default SonarQube Cloud profiles) | | `--only ` | Only migrate specific components (comma-separated). Valid: `scan-data`, `scan-data-all-branches`, `portfolios`, `quality-gates`, `quality-profiles`, `permission-templates`, `permissions`, `issue-metadata`, `hotspot-metadata`, `project-settings` | | `--auto-tune` | Auto-detect CPU and RAM and set optimal performance values | | `--concurrency ` | Override max concurrency for I/O operations | @@ -319,7 +319,7 @@ When you run `./cloudvoyager verify`, reports are written to `./verification-out ## Limitations -- Historical metrics (the charts in each project's **Activity** tab in SonarQube) cannot be migrated. All actual issues and hotspots are migrated β€” only the historical trend data is lost. +- Historical metrics (the charts in each project's **Activity** tab in SonarQube Server) cannot be migrated. All actual issues and hotspots are migrated β€” only the historical trend data is lost. --- diff --git a/docs/scenario-single-project.md b/docs/scenario-single-project.md index fda78e4..207defc 100644 --- a/docs/scenario-single-project.md +++ b/docs/scenario-single-project.md @@ -3,7 +3,7 @@ -Use this when you want to migrate **one specific project** from SonarQube to SonarCloud. +Use this when you want to migrate **one specific project** from SonarQube Server to SonarQube Cloud. This does **not** migrate org-level settings like quality gates, quality profiles, groups, or permissions β€” for that, see [Migrate Everything to One Org](scenario-single-org.md). @@ -21,7 +21,7 @@ This does **not** migrate org-level settings like quality gates, quality profile | **Metrics & measures** | Project and component-level measures (coverage, complexity, etc.) | | **SCM changesets** | Per-file changeset info (author, date, revision) | | **Active rules** | Quality profile rules filtered by languages used in the project | -| **Issue metadata** | Status history (full changelog replay), comments, assignments, `metadata-synchronized` tag, and a link back to the original SonarQube issue URL | +| **Issue metadata** | Status history (full changelog replay), comments, assignments, `metadata-synchronized` tag, and a link back to the original SonarQube Server issue URL | | **Hotspot metadata** | Hotspot statuses, comments, and source links | > **Not included:** Quality gates, quality profiles, groups, permissions, portfolios, project settings, tags, links, DevOps bindings, and new code definitions. Use the [`migrate` command](scenario-single-org.md) to transfer these. @@ -31,13 +31,13 @@ This does **not** migrate org-level settings like quality gates, quality profile ## Prerequisites -1. **Admin access** to your SonarQube server -2. **Admin access** to your SonarCloud organization -3. **API tokens** for both SonarQube and SonarCloud +1. **Admin access** to your SonarQube Server +2. **Admin access** to your SonarQube Cloud organization +3. **API tokens** for both SonarQube Server and SonarQube Cloud > **How to get your tokens:** -> - **SonarQube:** Go to `My Account > Security > Generate Tokens` in your SonarQube web UI -> - **SonarCloud:** Go to `My Account > Security > Generate Tokens` at [sonarcloud.io](https://sonarcloud.io) +> - **SonarQube Server:** Go to `My Account > Security > Generate Tokens` in your SonarQube Server web UI +> - **SonarQube Cloud:** Go to `My Account > Security > Generate Tokens` at [sonarcloud.io](https://sonarcloud.io) --- @@ -82,7 +82,7 @@ Create a file called `config.json`: } ``` -> **Where to find your project key:** In SonarQube, go to your project's **Project Information** page β€” the key is shown there. You can use the same key for SonarCloud, or choose a new one. +> **Where to find your project key:** In SonarQube Server, go to your project's **Project Information** page β€” the key is shown there. You can use the same key for SonarQube Cloud, or choose a new one. See [`examples/config.example.json`](../examples/config.example.json) for a ready-to-use template with all optional fields (rate limiting, performance tuning, etc.). @@ -91,13 +91,13 @@ See [`examples/config.example.json`](../examples/config.example.json) for a read | Field | Required | Description | |-------|----------|-------------| -| `sonarqube.url` | Yes | Full URL of your SonarQube server | -| `sonarqube.token` | Yes | SonarQube API token (starts with `sqp_` on newer versions) | -| `sonarqube.projectKey` | Yes (for `transfer`) | Project key in SonarQube | -| `sonarcloud.url` | No | SonarCloud URL (default: `https://sonarcloud.io`) | -| `sonarcloud.token` | Yes | SonarCloud API token | -| `sonarcloud.organization` | Yes | SonarCloud organization key | -| `sonarcloud.projectKey` | Yes (for `transfer`) | Project key to use in SonarCloud | +| `sonarqube.url` | Yes | Full URL of your SonarQube Server | +| `sonarqube.token` | Yes | SonarQube Server API token (starts with `sqp_` on newer versions) | +| `sonarqube.projectKey` | Yes (for `transfer`) | Project key in SonarQube Server | +| `sonarcloud.url` | No | SonarQube Cloud URL (default: `https://sonarcloud.io`) | +| `sonarcloud.token` | Yes | SonarQube Cloud API token | +| `sonarcloud.organization` | Yes | SonarQube Cloud organization key | +| `sonarcloud.projectKey` | Yes (for `transfer`) | Project key to use in SonarQube Cloud | > **Tip:** You can set tokens via environment variables (`SONARQUBE_TOKEN` and `SONARCLOUD_TOKEN`) instead of putting them in the config file. @@ -142,7 +142,7 @@ Add a `transfer` section to control incremental mode, batch size, and checkpoint ./cloudvoyager test -c config.json ``` -You should see a success message for both SonarQube and SonarCloud. If not, double-check your URLs and tokens. +You should see a success message for both SonarQube Server and SonarQube Cloud. If not, double-check your URLs and tokens. ## Step 4: Run the transfer @@ -151,9 +151,9 @@ You should see a success message for both SonarQube and SonarCloud. If not, doub ./cloudvoyager transfer -c config.json --verbose ``` -That's it! The tool uploads the report and returns immediately β€” it does not wait for SonarCloud to finish processing. Your project data will appear in SonarCloud once the analysis completes in the background. +That's it! The tool uploads the report and returns immediately β€” it does not wait for SonarQube Cloud to finish processing. Your project data will appear in SonarQube Cloud once the analysis completes in the background. -> **Tip:** If you want the command to block until SonarCloud finishes processing, add `--wait`. +> **Tip:** If you want the command to block until SonarQube Cloud finishes processing, add `--wait`. --- @@ -217,7 +217,7 @@ To discard progress and start fresh, use `--force-restart`: ### Upload deduplication -On resume after a crash, the tool checks whether a Compute Engine (CE) task already exists for the current session before uploading. This prevents duplicate scanner reports from being submitted to SonarCloud. The check uses the `scm_revision_id` (git commit hash) included in each report. +On resume after a crash, the tool checks whether a Compute Engine (CE) task already exists for the current session before uploading. This prevents duplicate scanner reports from being submitted to SonarQube Cloud. The check uses the `scm_revision_id` (git commit hash) included in each report. ### Lock file handling @@ -247,7 +247,7 @@ An advisory lock file prevents multiple CloudVoyager instances from running agai ## Limitations -- Historical metrics (the charts in each project's **Activity** tab in SonarQube) cannot be migrated. All actual issues and hotspots are migrated β€” only the historical trend data is lost. +- Historical metrics (the charts in each project's **Activity** tab in SonarQube Server) cannot be migrated. All actual issues and hotspots are migrated β€” only the historical trend data is lost. --- diff --git a/docs/technical-details.md b/docs/technical-details.md index b78d4ef..813a68a 100644 --- a/docs/technical-details.md +++ b/docs/technical-details.md @@ -1,6 +1,6 @@ # πŸ”¬ Technical Details - + ## πŸ—ΊοΈ Main Flow Sequence Diagram @@ -13,8 +13,8 @@ sequenceDiagram actor User participant CLI as CLI (src/index.js) participant VR as VersionRouter - participant SQ as SonarQube - participant SC as SonarCloud + participant SQ as SonarQube Server + participant SC as SonarQube Cloud participant FS as File System rect rgb(30, 50, 80) @@ -137,7 +137,7 @@ sequenceDiagram | βž•B | Protobuf encoding | `Generate a mermaid sequence diagram showing how CloudVoyager builds and encodes a scanner report ZIP using protobuf messages` | | βž•C | CE submission retry | `Generate a mermaid sequence diagram showing the CE submission retry mechanism in CloudVoyager including fallback polling` | | βž•D | Issue sync | `Generate a mermaid sequence diagram showing the full issue sync pipeline in CloudVoyager including pre-filter, SC indexing wait, and changelog replay` | -| βž•E | Org mapping / CSVs | `Generate a mermaid sequence diagram showing how CloudVoyager maps SonarQube projects to SonarCloud organizations and generates dry-run CSV files` | +| βž•E | Org mapping / CSVs | `Generate a mermaid sequence diagram showing how CloudVoyager maps SonarQube Server projects to SonarQube Cloud organizations and generates dry-run CSV files` | | βž•F | Quality profiles | `Generate a mermaid sequence diagram for CloudVoyager quality profile migration including backup XML, rename of built-in profiles, and diff report` | | βž•G | Batch distribution | `Generate a mermaid sequence diagram showing how CloudVoyager batch-distributor splits large issue sets into ≀5K batches with backdated analysis dates and sequential CE uploads` | @@ -175,7 +175,7 @@ context-props.pb # Empty (matches real scanner) ## πŸ”„ CE Submission Retry Mechanism -Report submission to SonarCloud's Compute Engine (`/api/ce/submit`) uses a robust retry strategy (implemented in `ce-submitter.js` within each pipeline): +Report submission to SonarQube Cloud's Compute Engine (`/api/ce/submit`) uses a robust retry strategy (implemented in `ce-submitter.js` within each pipeline): 1. **Submit** the report ZIP via `POST /api/ce/submit` with a 60-second response timeout 2. **On timeout** (no server response): fall back to `/api/ce/activity` polling β€” check for a matching CE task 5 times at 3-second intervals @@ -196,7 +196,7 @@ The form data is buffered before sending (not streamed) to avoid runtime-specifi | **Severity** | UNSET=0, INFO=1, MINOR=2, MAJOR=3, CRITICAL=4, BLOCKER=5 | | **IssueType** | CODE_SMELL=1, BUG=2, VULNERABILITY=3, SECURITY_HOTSPOT=4 | -**CRITICAL**: `cleanCodeAttribute` in `ExternalIssue` and `AdHocRule` must be encoded as a protobuf enum (varint), NOT a string. Despite the `.proto` file showing `optional string`, the real scanner uses enum encoding. SonarCloud CE silently ignores external issues if `cleanCodeAttribute` is string-encoded. +**CRITICAL**: `cleanCodeAttribute` in `ExternalIssue` and `AdHocRule` must be encoded as a protobuf enum (varint), NOT a string. Despite the `.proto` file showing `optional string`, the real scanner uses enum encoding. SonarQube Cloud CE silently ignores external issues if `cleanCodeAttribute` is string-encoded. ### Field Name Convention @@ -221,26 +221,26 @@ Measures use typed value fields based on metric type: - Active rules are filtered by languages actually used in the project, resulting in ~84% reduction in payload size - Rule keys are stripped of the repository prefix (e.g., `S7788` not `jsarchitecture:S7788`) -- Quality profile keys are mapped to SonarCloud profile keys (not SonarQube keys) +- Quality profile keys are mapped to SonarQube Cloud profile keys (not SonarQube Server keys) ## 🧱 Component Structure -Components use a flat structure - all files are direct children of the project component (no directory components). Line counts are derived from actual source file content rather than SonarQube measures API values. +Components use a flat structure - all files are direct children of the project component (no directory components). Line counts are derived from actual source file content rather than SonarQube Server measures API values. ## πŸ”– SCM Revision Tracking -The tool includes `scm_revision_id` (git commit hash) in metadata. SonarCloud uses this to detect and reject duplicate reports, enabling proper analysis history tracking. +The tool includes `scm_revision_id` (git commit hash) in metadata. SonarQube Cloud uses this to detect and reject duplicate reports, enabling proper analysis history tracking. -**Batch uploads**: When issue batching is active (>5,000 issues), each batch receives a unique `scmRevisionId` generated via `randomBytes(20).toString('hex')` instead of the original git commit hash. This prevents the SonarCloud CE from deduplicating successive batch uploads as identical reports. See [Issue Batching for Upload](#-issue-batching-for-upload-5k-per-date-bucket) for details. +**Batch uploads**: When issue batching is active (>5,000 issues), each batch receives a unique `scmRevisionId` generated via `randomBytes(20).toString('hex')` instead of the original git commit hash. This prevents the SonarQube Cloud CE from deduplicating successive batch uploads as identical reports. See [Issue Batching for Upload](#-issue-batching-for-upload-5k-per-date-bucket) for details. ## 🌿 Branch Sync -By default, every branch discovered in SonarQube is transferred to SonarCloud (main branch first, then non-main branches). Each branch produces its own scanner report with branch-specific issues, measures, sources, and SCM data. +By default, every branch discovered in SonarQube Server is transferred to SonarQube Cloud (main branch first, then non-main branches). Each branch produces its own scanner report with branch-specific issues, measures, sources, and SCM data. -**Branch name resolution:** The main branch name is fetched from SonarCloud (via `getMainBranchName()` API) rather than using the SonarQube branch name. This avoids mismatches where SonarQube uses "main" but SonarCloud expects "master" (or vice versa). Non-main branches use their original SonarQube branch name and reference the main branch for new-code comparison. +**Branch name resolution:** The main branch name is fetched from SonarQube Cloud (via `getMainBranchName()` API) rather than using the SonarQube Server branch name. This avoids mismatches where SonarQube Server uses "main" but SonarQube Cloud expects "master" (or vice versa). Non-main branches use their original SonarQube Server branch name and reference the main branch for new-code comparison. **Incremental mode:** Completed branches are tracked in the state file. On subsequent runs, already-synced branches are skipped automatically. Use `reset` to re-transfer all branches. @@ -249,7 +249,7 @@ By default, every branch discovered in SonarQube is transferred to SonarCloud (m ## πŸ“„ API Pagination -SonarQube client handles pagination automatically via `getPaginated` method with a default page size of 500 items. All paginated results are concatenated into single arrays. +SonarQube Server client handles pagination automatically via `getPaginated` method with a default page size of 500 items. All paginated results are concatenated into single arrays. **APIs with max `ps=100`** (not the default 500): - `/api/permissions/groups` @@ -266,9 +266,9 @@ The extractors handle these lower limits automatically. ## πŸ“¦ Accurate Issue Creation Date Backdating -SonarCloud's CE assigns creation dates to NEW issues from SCM blame data. `backdateChangesets()` rewrites each file's changeset protobuf so that the CE assigns each issue its original SonarQube creation date. +SonarQube Cloud's CE assigns creation dates to NEW issues from SCM blame data. `backdateChangesets()` rewrites each file's changeset protobuf so that the CE assigns each issue its original SonarQube Server creation date. -**How it works:** The function reads each issue's `creationDate` field (preserved from SonarQube extraction) and maps it to the issue's `textRange` lines in the file's changeset data. The CE takes MAX(date) across an issue's line range, so using "oldest wins" for overlapping lines ensures accurate dates. A safety split handles days with >5K issues. +**How it works:** The function reads each issue's `creationDate` field (preserved from SonarQube Server extraction) and maps it to the issue's `textRange` lines in the file's changeset data. The CE takes MAX(date) across an issue's line range, so using "oldest wins" for overlapping lines ensures accurate dates. A safety split handles days with >5K issues. ### Algorithm (3 Phases) @@ -285,7 +285,7 @@ SonarCloud's CE assigns creation dates to NEW issues from SCM blame data. `backd | Per-line dating (not per-file) | Enables multiple issues with different creation dates in the same file | | Oldest date wins for overlapping lines | CE takes MAX across line range β€” older date on shared lines doesn't inflate newer issues that have their own non-overlapping lines | | Non-issue lines get oldest date | Prevents accidental MAX inflation for multi-line issues spanning non-issue lines | -| Safety split at 5,000/day | 50% margin under SonarCloud's 10K ES visualization cap per date bucket | +| Safety split at 5,000/day | 50% margin under SonarQube Cloud's 10K ES visualization cap per date bucket | | No file splitting in safety split | A file's issues stay together on the same synthetic date | | All projects backdated | No early return for small projects β€” every project gets accurate dates | @@ -307,14 +307,14 @@ All 6 pipeline `transfer-branch` entry points call `backdateChangesets(extracted ## πŸ” Search Slicing for 10K+ Issues -SonarQube's `/api/issues/search` endpoint returns a maximum of 10,000 results regardless of pagination. Projects with more than 10,000 issues would silently lose data during extraction. +SonarQube Server's `/api/issues/search` endpoint returns a maximum of 10,000 results regardless of pagination. Projects with more than 10,000 issues would silently lose data during extraction. **Solution:** `src/shared/utils/search-slicer/` implements a date-window bisection algorithm: 1. **Probe** β€” `probe-total.js` (in each pipeline's `api-client/helpers/`) sends a lightweight request (`ps=1, p=1`) to get the total issue count for a query without fetching data. -2. **Partition** β€” If the total exceeds 10,000, the full SonarQube era (`2006-01-01` β†’ now) is divided into **12 equal-width time windows** via `build-date-windows.js`. A fixed epoch is used instead of probing the date range β€” probing via `getPaginated(ps=1)` would loop page-by-page through all issues and itself hit the Elasticsearch 10K limit on large projects. +2. **Partition** β€” If the total exceeds 10,000, the full SonarQube Server era (`2006-01-01` β†’ now) is divided into **12 equal-width time windows** via `build-date-windows.js`. A fixed epoch is used instead of probing the date range β€” probing via `getPaginated(ps=1)` would loop page-by-page through all issues and itself hit the Elasticsearch 10K limit on large projects. 3. **Bisect** β€” Any window that still exceeds 10,000 is **recursively halved** at the midpoint timestamp (via `split-midpoint.js`) until every window is under the limit. This binary subdivision guarantees convergence for any realistic issue distribution. -4. **Guard** β€” If a window cannot be split further (both halves land on the same millisecond boundary, e.g. a mass-import scenario where all issues share the same creation timestamp), the window is fetched directly. This is an unavoidable SonarQube API limitation; it affects only projects where thousands of issues share an identical millisecond timestamp. +4. **Guard** β€” If a window cannot be split further (both halves land on the same millisecond boundary, e.g. a mass-import scenario where all issues share the same creation timestamp), the window is fetched directly. This is an unavoidable SonarQube Server API limitation; it affects only projects where thousands of issues share an identical millisecond timestamp. 5. **Fetch** β€” Each window is fetched independently using standard pagination (`ps=500`) with `createdAfter` / `createdBefore` API parameters scoping results to that window. 6. **Merge & deduplicate** β€” `deduplicate-results.js` merges results from all windows and removes duplicates by `item.key || item.id` to handle boundary overlaps where an issue's creation timestamp falls on a window edge. @@ -326,13 +326,13 @@ SonarQube's `/api/issues/search` endpoint returns a maximum of 10,000 results re | `fetch-window.js` | Fetches one window; recursively bisects if it exceeds the limit | | `slice-by-creation-date.js` | Partitions epochβ†’now into 12 windows, calls `fetchWindow` for each | | `build-date-windows.js` | Builds evenly-spaced `{ start, end }` window objects | -| `split-midpoint.js` | Computes the SonarQube-compatible datetime at the midpoint between two timestamps | -| `format-sonarqube-date.js` | Formats timestamps as `YYYY-MM-DDTHH:MM:SS+0000` (SonarQube rejects `.XXXZ` milliseconds) | +| `split-midpoint.js` | Computes the SonarQube Server-compatible datetime at the midpoint between two timestamps | +| `format-sonarqube-date.js` | Formats timestamps as `YYYY-MM-DDTHH:MM:SS+0000` (SonarQube Server rejects `.XXXZ` milliseconds) | | `deduplicate-results.js` | Deduplicates merged results by `item.key \|\| item.id` | The slicing is transparent to callers β€” `issues-hotspots.js` in each pipeline calls `fetchWithSlicing`, which falls back to a normal paginated fetch when the total is under 10,000. -**Applies to both SonarQube and SonarCloud.** The slicing workaround targets SonarQube's `/api/issues/search` and `/api/hotspots/search` endpoints, which enforce the 10K cap. The same `fetchWithSlicing` mechanism is also applied to SonarCloud's issue search during **issue sync** (i.e., when fetching SC issues to match against SQ issues). This prevents data loss on large SonarCloud organizations that have >10,000 issues per project. +**Applies to both SonarQube Server and SonarQube Cloud.** The slicing workaround targets SonarQube Server's `/api/issues/search` and `/api/hotspots/search` endpoints, which enforce the 10K cap. The same `fetchWithSlicing` mechanism is also applied to SonarQube Cloud's issue search during **issue sync** (i.e., when fetching SC issues to match against SQ issues). This prevents data loss on large SonarQube Cloud organizations that have >10,000 issues per project. **Both commands use it:** - `transfer` β€” calls `getIssuesWithComments()` via `fetch-and-sync-issues.js`, which invokes `fetchWithSlicing` under the hood. @@ -341,7 +341,7 @@ The slicing is transparent to callers β€” `issues-hotspots.js` in each pipeline ## πŸ”„ Fallback Rule Repositories -External-issue detection depends on knowing which rule repositories exist in SonarCloud. If the `/api/rules/repositories` call fails, the tool falls back to a built-in set of 44 known SonarCloud repositories (`src/shared/utils/fallback-repos/index.js`). +External-issue detection depends on knowing which rule repositories exist in SonarQube Cloud. If the `/api/rules/repositories` call fails, the tool falls back to a built-in set of 44 known SonarQube Cloud repositories (`src/shared/utils/fallback-repos/index.js`). The fallback set includes the `githubactions` IaC analyzer (previously omitted by accident; added back in the Apr 2026 release). Without it, GitHub Actions rules were incorrectly treated as external issues. @@ -358,28 +358,28 @@ The fallback set includes the `githubactions` IaC analyzer (previously omitted b ## 🚦 Rate Limit Handling -The SonarCloud API client supports a configurable two-layer strategy for rate limiting. Customize it via the `rateLimit` section in your config file. +The SonarQube Cloud API client supports a configurable two-layer strategy for rate limiting. Customize it via the `rateLimit` section in your config file. 1. **Exponential backoff retry** (`maxRetries`, `baseDelay`) β€” When a 503 or 429 response is received, the request is retried up to `maxRetries` times with exponentially increasing delays (baseDelay Γ— 2^attempt). If all retries are exhausted, the error is propagated to the caller. Default: `3` retries. -2. **Write request throttling** (`minRequestInterval`) β€” POST requests are spaced at least `minRequestInterval` ms apart via a request interceptor. This proactively reduces the chance of triggering SonarCloud's rate limits during high-volume operations like issue sync and hotspot sync. Default: `0` (no throttling). +2. **Write request throttling** (`minRequestInterval`) β€” POST requests are spaced at least `minRequestInterval` ms apart via a request interceptor. This proactively reduces the chance of triggering SonarQube Cloud's rate limits during high-volume operations like issue sync and hotspot sync. Default: `0` (no throttling). ## πŸ”Œ External Issues (Plugin Migration) -Issues from SonarQube plugins that are not available in SonarCloud (e.g., MuleSoft, ABAP) are automatically migrated as **external issues** using the `ExternalIssue` and `AdHocRule` protobuf messages. +Issues from SonarQube Server plugins that are not available in SonarQube Cloud (e.g., MuleSoft, ABAP) are automatically migrated as **external issues** using the `ExternalIssue` and `AdHocRule` protobuf messages. -**Auto-detection**: The tool compares SonarQube rule repositories against SonarCloud's available repositories via `getRuleRepositories()`. If a rule's repository is not present in SonarCloud, the issue is routed through the external issue path instead of the regular `Issue` path. +**Auto-detection**: The tool compares SonarQube Server rule repositories against SonarQube Cloud's available repositories via `getRuleRepositories()`. If a rule's repository is not present in SonarQube Cloud, the issue is routed through the external issue path instead of the regular `Issue` path. -**SonarQube 2025+ `external_` prefix handling**: SonarQube 2025+ returns external linter issues with an `external_` prefix in the rule key (e.g., `external_ruff:D200` instead of `ruff:D200`). The tool detects this prefix and always treats such rules as external, then strips the prefix before building the `ExternalIssue` protobuf message to avoid double-prefixing in SonarCloud. +**SonarQube Server 2025+ `external_` prefix handling**: SonarQube Server 2025+ returns external linter issues with an `external_` prefix in the rule key (e.g., `external_ruff:D200` instead of `ruff:D200`). The tool detects this prefix and always treats such rules as external, then strips the prefix before building the `ExternalIssue` protobuf message to avoid double-prefixing in SonarQube Cloud. **How it works**: 1. Rules with an `external_` prefix in SQ are always treated as external (regardless of SC repo list) 2. The `external_` prefix is stripped from the engineId before encoding (SQ `external_ruff:D200` β†’ engineId `ruff`, ruleId `D200`) 3. Issues with unsupported rule repos (no `external_` prefix) are detected by comparing against SC repositories 4. Each unique rule becomes an `AdHocRule` with name, description, severity, type, clean code attribute, and impacts -5. External issues appear in SonarCloud as `external_{engineId}:{ruleId}` (e.g., `external_ruff:D200`, `external_mulesoft:MS058`) -6. Ad-hoc rules do not appear in SonarCloud's rules search (expected behavior per SC docs) +5. External issues appear in SonarQube Cloud as `external_{engineId}:{ruleId}` (e.g., `external_ruff:D200`, `external_mulesoft:MS058`) +6. Ad-hoc rules do not appear in SonarQube Cloud's rules search (expected behavior per SC docs) **Requirements**: Each `ExternalIssue` and `AdHocRule` must include: - `cleanCodeAttribute` β€” encoded as protobuf enum (varint), not string (see Protobuf Encoding section) @@ -391,23 +391,23 @@ Issues from SonarQube plugins that are not available in SonarCloud (e.g., MuleSo The `migrate` command syncs issue metadata after the scanner report is uploaded. The sync pipeline runs in several stages: -1. **Pre-filter** β€” Batch-fetches SQ changelogs for all issues and discards any issue that has no human-authored changes (see below). This avoids redundant API calls to SonarCloud for issues that are still in their pristine `OPEN` state. -2. **Wait for SC indexing** β€” If SonarCloud returns 0 issues immediately after a first-time upload, the syncer retries with exponential backoff (initial delay 10 s, max 60 s, up to 10 attempts) until the CE analysis is fully indexed. This handles the race condition introduced by Issue #91 where the analysis report was uploaded but SonarCloud hadn't yet indexed its issues. -3. **Match** β€” Searches for a matching issue in SonarCloud by rule, component, and line number. -4. **Replay changelog** β€” Fetches the SonarQube issue changelog and replays all status transitions in order (Open β†’ Confirmed β†’ False Positive, etc.). +1. **Pre-filter** β€” Batch-fetches SQ changelogs for all issues and discards any issue that has no human-authored changes (see below). This avoids redundant API calls to SonarQube Cloud for issues that are still in their pristine `OPEN` state. +2. **Wait for SC indexing** β€” If SonarQube Cloud returns 0 issues immediately after a first-time upload, the syncer retries with exponential backoff (initial delay 10 s, max 60 s, up to 10 attempts) until the CE analysis is fully indexed. This handles the race condition introduced by Issue #91 where the analysis report was uploaded but SonarQube Cloud hadn't yet indexed its issues. +3. **Match** β€” Searches for a matching issue in SonarQube Cloud by rule, component, and line number. +4. **Replay changelog** β€” Fetches the SonarQube Server issue changelog and replays all status transitions in order (Open β†’ Confirmed β†’ False Positive, etc.). 5. **Assignee** β€” Sets the assignee (supports user mapping from SQ login to SC login). -6. **Comments** β€” Copies comments, skipping any that begin with `[Migrated from SonarQube]` to avoid double-migration. +6. **Comments** β€” Copies comments, skipping any that begin with `[Migrated from SonarQube Server]` to avoid double-migration. 7. **Tags** β€” Sets tags. ### Pre-filter: `hasManualChanges` (`src/shared/utils/issue-sync/`) -Before touching SonarCloud, the syncer applies a pre-filter that only keeps issues with human-authored changes: +Before touching SonarQube Cloud, the syncer applies a pre-filter that only keeps issues with human-authored changes: | Check | Condition | |-------|-----------| | Human changelog entry | At least one changelog entry with a non-empty `user` field | -| Manual comments | At least one comment not prefixed with `[Migrated from SonarQube]` | +| Manual comments | At least one comment not prefixed with `[Migrated from SonarQube Server]` | | Custom tags | `issue.tags` array is non-empty | | Manual assignee | `issue.assignee` is set | | Updated after creation | `updateDate !== creationDate` (catch-all safety net) | @@ -439,7 +439,7 @@ Total concurrent API calls: 20 workers Γ— 5 internal = **100** (vs 20 with singl - **Max attempts**: 10 (configurable via `options.maxRetries`) - **Transient errors**: fetch errors during a retry are caught and logged; the function returns `[]` after exhausting all retries -**SonarQube version differences**: +**SonarQube Server version differences**: - SQ 9.9 statuses: `OPEN`, `CONFIRMED`, `REOPENED`, `RESOLVED`, `CLOSED` - SQ 10.4+: `OPEN`, `CONFIRMED`, `FALSE_POSITIVE`, `ACCEPTED`, `FIXED` - Available transitions: `confirm`, `unconfirm`, `reopen`, `resolve`, `falsepositive`, `wontfix`, `accept` @@ -461,9 +461,9 @@ Hotspots are converted to `Issue` format for the scanner report (with `type=SECU ## πŸ”„ Issue Status Transition Mapping -Each pipeline includes an `issue-status-mapper.js` that maps SonarQube issue changelog entries to SonarCloud transitions: +Each pipeline includes an `issue-status-mapper.js` that maps SonarQube Server issue changelog entries to SonarQube Cloud transitions: -| SonarQube Status/Resolution | SonarCloud Transition | +| SonarQube Server Status/Resolution | SonarQube Cloud Transition | |-----------------------------|----------------------| | `FALSE-POSITIVE` (resolution or status) | `falsepositive` | | `WONTFIX` (resolution or status) | `wontfix` | @@ -481,10 +481,10 @@ The mapper handles both SQ < 10.4 (where `FALSE-POSITIVE` and `WONTFIX` appear a ## πŸ”‘ Project Key Resolution -SonarCloud requires globally unique project keys across all organizations. When migrating projects, the tool uses the following strategy: +SonarQube Cloud requires globally unique project keys across all organizations. When migrating projects, the tool uses the following strategy: -1. **Try the original SonarQube project key** β€” check if it's available globally on SonarCloud via `/api/components/show` -2. **If available** β€” use the original key as-is, so the SonarCloud project key matches SonarQube +1. **Try the original SonarQube Server project key** β€” check if it's available globally on SonarQube Cloud via `/api/components/show` +2. **If available** β€” use the original key as-is, so the SonarQube Cloud project key matches SonarQube Server 3. **If taken by another organization** β€” fall back to `{org}_{key}` (e.g., `my-org_my-project`) and log a warning 4. **If already owned by the target organization** β€” use the original key (the project was likely created in a previous migration run) @@ -493,7 +493,7 @@ Key conflicts are reported in the migration summary and in the `reports/migratio ## πŸ—ΊοΈ Organization Mapping -The `migrate` command maps projects to target SonarCloud organizations based on their DevOps platform bindings. Projects with the same ALM binding are grouped together. Mapping CSVs are generated for review before execution (via `--dry-run`). +The `migrate` command maps projects to target SonarQube Cloud organizations based on their DevOps platform bindings. Projects with the same ALM binding are grouped together. Mapping CSVs are generated for review before execution (via `--dry-run`). ## πŸ” Dry-Run & Editable CSV Workflow @@ -501,36 +501,36 @@ The `migrate` command maps projects to target SonarCloud organizations based on The `--dry-run` flag generates 8 exhaustive CSV files covering projects, organizations, groups, quality profiles, quality gates, portfolios, permission templates, and global permissions. Each CSV includes an `Include` column (defaulting to `yes`) that users can edit to filter what gets migrated. **Pipeline integration:** -1. `migrate --dry-run` extracts data from SonarQube, generates CSVs, then stops +1. `migrate --dry-run` extracts data from SonarQube Server, generates CSVs, then stops 2. User reviews/edits CSVs (set `Include=no` to exclude resources from migration) -3. `migrate` (without `--dry-run`) detects existing CSVs, reads them into memory **before** wiping the output directory, re-extracts from SonarQube, then applies CSV overrides via `applyCsvOverrides()` which returns filtered copies using `structuredClone` +3. `migrate` (without `--dry-run`) detects existing CSVs, reads them into memory **before** wiping the output directory, re-extracts from SonarQube Server, then applies CSV overrides via `applyCsvOverrides()` which returns filtered copies using `structuredClone` -Quality gate CSVs use a flat one-row-per-gate pattern β€” users can include or exclude entire gates, but conditions are always migrated as-is from SonarQube. Portfolio and permission template CSVs use a parent/child pattern for their member/permission rows. +Quality gate CSVs use a flat one-row-per-gate pattern β€” users can include or exclude entire gates, but conditions are always migrated as-is from SonarQube Server. Portfolio and permission template CSVs use a parent/child pattern for their member/permission rows. See [dry-run-csv-reference.md](dry-run-csv-reference.md) for full CSV schema documentation. ## πŸ“‹ Quality Profile Migration -Quality profiles are migrated using SonarQube's backup/restore XML format, which preserves all rule configurations, severity overrides, and parameter values. Profile permissions (user and group access) are migrated separately via the permissions API. +Quality profiles are migrated using SonarQube Server's backup/restore XML format, which preserves all rule configurations, severity overrides, and parameter values. Profile permissions (user and group access) are migrated separately via the permissions API. -Both **custom and built-in** profiles are migrated. Built-in profiles (e.g., "Sonar way") cannot be overwritten on SonarCloud, so they are restored as custom profiles with a "(SonarQube Migrated)" suffix (e.g., "Sonar way (SonarQube Migrated)"). These migrated profiles are automatically assigned to each project to ensure the same rules are active as in SonarQube. +Both **custom and built-in** profiles are migrated. Built-in profiles (e.g., "Sonar way") cannot be overwritten on SonarQube Cloud, so they are restored as custom profiles with a "(SonarQube Server Migrated)" suffix (e.g., "Sonar way (SonarQube Server Migrated)"). These migrated profiles are automatically assigned to each project to ensure the same rules are active as in SonarQube Server. -To skip quality profile migration entirely and use each language's existing default SonarCloud profile, pass `--skip-quality-profile-sync`. +To skip quality profile migration entirely and use each language's existing default SonarQube Cloud profile, pass `--skip-quality-profile-sync`. **API gotchas**: - `/api/qualityprofiles/backup` requires `language` + `qualityProfile` (name), not `profileKey` - `/api/qualityprofiles/search_users` requires `language` + `qualityProfile` (name) - Built-in profiles: permission APIs return 400 (expected, handle gracefully) -After profile migration, a **quality profile diff report** (`quality-profiles/quality-profile-diff.json`) is written to the output directory. This report compares active rules per language between SonarQube and SonarCloud, listing: -- **Missing rules** β€” rules active in SonarQube but not available in SonarCloud (may cause fewer issues) -- **Added rules** β€” rules available in SonarCloud but not in SonarQube (may create new issues) +After profile migration, a **quality profile diff report** (`quality-profiles/quality-profile-diff.json`) is written to the output directory. This report compares active rules per language between SonarQube Server and SonarQube Cloud, listing: +- **Missing rules** β€” rules active in SonarQube Server but not available in SonarQube Cloud (may cause fewer issues) +- **Added rules** β€” rules available in SonarQube Cloud but not in SonarQube Server (may create new issues) ## 🚧 Quality Gate Migration -Quality gates are created with their full condition definitions (metric, operator, threshold). The SonarQube API uses gate `name` (not `id`) for all operations. Built-in gates are skipped since they already exist in SonarCloud. +Quality gates are created with their full condition definitions (metric, operator, threshold). The SonarQube Server API uses gate `name` (not `id`) for all operations. Built-in gates are skipped since they already exist in SonarQube Cloud. **API gotchas**: - `/api/qualitygates/list` returns no `id` field, only `name` @@ -573,7 +573,7 @@ Each transfer creates a checkpoint journal file alongside the state file: - **Phase tracking**: Each extraction phase (project metadata, metrics, components, rules, issues, hotspots, measures, sources, etc.) is tracked individually - **Branch tracking**: Per-branch completion status with CE task IDs for upload deduplication -- **Session fingerprint**: SonarQube version, URL, and project key are recorded to detect environment changes between runs +- **Session fingerprint**: SonarQube Server version, URL, and project key are recorded to detect environment changes between runs ### State Management Components @@ -606,7 +606,7 @@ If the main state file is corrupted, the system falls back to the `.backup` file ### Extraction Caching -Completed extraction phases are cached as gzipped JSON in `/cache/extractions//`. On resume, cached phases are loaded from disk instead of re-fetching from SonarQube. Cache files include integrity metadata and auto-purge after 7 days (configurable via `transfer.checkpoint.cacheMaxAgeDays`). +Completed extraction phases are cached as gzipped JSON in `/cache/extractions//`. On resume, cached phases are loaded from disk instead of re-fetching from SonarQube Server. Cache files include integrity metadata and auto-purge after 7 days (configurable via `transfer.checkpoint.cacheMaxAgeDays`). ### Upload Deduplication @@ -658,7 +658,7 @@ The `csv-entity-filters.js` module in `src/shared/mapping/` provides dry-run CSV | `applyGlobalPermissionsCsv` | Global permissions | Group Name + Permission + Include | | `applyTemplateMappingsCsv` | Permission templates | Template Name + Permission Key + Include | | `applyPortfolioMappingsCsv` | Portfolios | Portfolio Key + Member Project Key + Include | -| `applyUserMappingsCsv` | User mappings | SonarQube Login + SonarCloud Login + Include | +| `applyUserMappingsCsv` | User mappings | SonarQube Server Login + SonarQube Cloud Login + Include | All filters use the `Include` column from the CSV (default: `yes`). Setting `Include=no` excludes the entity from migration. @@ -672,9 +672,9 @@ For non-standard environments (e.g. staging), the Desktop app's **Advanced Setti ## πŸ”‘ Enterprise Key & Portfolio Skipping -The SonarCloud enterprise key (`sonarcloud.enterprise.key`) is **optional** β€” its absence no longer aborts the migration. When the key is missing, portfolio migration is gracefully skipped via `handleMissingEnterpriseKey()` (`src/shared/utils/portfolio-skip.js`): +The SonarQube Cloud enterprise key (`sonarcloud.enterprise.key`) is **optional** β€” its absence no longer aborts the migration. When the key is missing, portfolio migration is gracefully skipped via `handleMissingEnterpriseKey()` (`src/shared/utils/portfolio-skip.js`): -- If portfolios were extracted from SonarQube, a `WARN` log is emitted and `results.portfoliosSkipped` is incremented. +- If portfolios were extracted from SonarQube Server, a `WARN` log is emitted and `results.portfoliosSkipped` is incremented. - If no portfolios exist, an `INFO` log is emitted and migration proceeds normally. The `portfoliosSkipped` count is included in the migration summary and migration reports. diff --git a/docs/troubleshooting.md b/docs/troubleshooting.md index 7aa031c..db83e0e 100644 --- a/docs/troubleshooting.md +++ b/docs/troubleshooting.md @@ -18,7 +18,7 @@ After every `migrate` run (whether it succeeds, partially succeeds, or crashes), 1. **Open `reports/migration-report.txt`** β€” it's structured top-down so you can quickly find problems: - **SUMMARY** β€” overall counts (succeeded / partial / failed) - - **SERVER-WIDE STEPS** β€” did extraction from SonarQube work? + - **SERVER-WIDE STEPS** β€” did extraction from SonarQube Server work? - **ORGANIZATION** β€” did org-level setup (groups, gates, profiles) work? - **FAILED / PARTIAL PROJECTS (DETAILED)** β€” step-by-step breakdown for every project that had issues, showing exactly which step failed and why - **ALL PROJECTS** β€” compact one-line-per-project list with failed step names @@ -125,7 +125,7 @@ grep "SKIP\|skipping" migration.log ### Re-running after failures -The migration can be re-run safely. Projects that already exist in SonarCloud will be updated (not duplicated). To fix specific failures: +The migration can be re-run safely. Projects that already exist in SonarQube Cloud will be updated (not duplicated). To fix specific failures: - **Rate limit errors on hotspot sync** β€” re-run with `--skip-issue-metadata-sync` (issues already synced) and increase rate limit config - **Report upload failures** β€” check the specific error, fix the root cause, and re-run @@ -156,10 +156,10 @@ If the checkpoint journal becomes corrupt (e.g., due to a system crash during a ./cloudvoyager transfer -c config.json --verbose --force-restart ``` -### SonarQube Version Mismatch on Resume +### SonarQube Server Version Mismatch on Resume -If you upgrade SonarQube between pause and resume, the tool warns about a version mismatch in the session fingerprint. By default, this is a warning only β€” the transfer continues. To enforce strict version matching: +If you upgrade SonarQube Server between pause and resume, the tool warns about a version mismatch in the session fingerprint. By default, this is a warning only β€” the transfer continues. To enforce strict version matching: ```json { @@ -176,7 +176,7 @@ With `strictResume: true`, a version mismatch will fail the transfer and require ### Source Code Changed Between Pause and Resume -If source code in SonarQube changes between pause and resume (e.g., new analysis uploaded), already-cached extraction phases will use stale data. Use `--force-fresh-extract` to re-extract all data while keeping the checkpoint journal: +If source code in SonarQube Server changes between pause and resume (e.g., new analysis uploaded), already-cached extraction phases will use stale data. Use `--force-fresh-extract` to re-extract all data while keeping the checkpoint journal: ```bash ./cloudvoyager transfer -c config.json --verbose --force-fresh-extract @@ -226,33 +226,33 @@ This only affects building the binary. Tests, linting, and `npm run build` work ## πŸ” Authentication Errors - Verify your tokens have the correct permissions - Check that tokens haven't expired -- Ensure the project key exists in SonarQube -- Verify the organization key is correct in SonarCloud +- Ensure the project key exists in SonarQube Server +- Verify the organization key is correct in SonarQube Cloud ## ⚠️ Generic "Issue whilst processing" Error -This vague SonarCloud error can be caused by: -- **Branch name mismatch** - SonarQube and SonarCloud have different main branch names. The tool handles this automatically via `getMainBranchName()`, but verify your SonarCloud project's branch configuration +This vague SonarQube Cloud error can be caused by: +- **Branch name mismatch** - SonarQube Server and SonarQube Cloud have different main branch names. The tool handles this automatically via `getMainBranchName()`, but verify your SonarQube Cloud project's branch configuration - **Line count mismatch** - Source file line counts don't match component metadata. The tool uses actual source content line counts to avoid this -## ❌ Report Rejected by SonarCloud +## ❌ Report Rejected by SonarQube Cloud - **Empty ScmInfo** - Ensure `changesetIndexByLine` is populated for ADDED files (array of zeros, one per line) -- **Issue gap field** - The `gap` field should not be included in issues (it's scanner-computed, not from SonarQube) -- **Duplicate report** - SonarCloud rejects reports with the same `scm_revision_id`. Use a different commit or update the source project +- **Issue gap field** - The `gap` field should not be included in issues (it's scanner-computed, not from SonarQube Server) +- **Duplicate report** - SonarQube Cloud rejects reports with the same `scm_revision_id`. Use a different commit or update the source project ## πŸ”‘ Project Key Conflicts -SonarCloud requires globally unique project keys across all organizations. By default, CloudVoyager uses the **original SonarQube project key** when creating projects on SonarCloud. If the key is already taken by another SonarCloud organization, the tool automatically falls back to a prefixed key (`{org}_{key}`) and logs a warning. +SonarQube Cloud requires globally unique project keys across all organizations. By default, CloudVoyager uses the **original SonarQube Server project key** when creating projects on SonarQube Cloud. If the key is already taken by another SonarQube Cloud organization, the tool automatically falls back to a prefixed key (`{org}_{key}`) and logs a warning. Key conflicts are reported in three places: - **Console logs** β€” a warning is logged immediately when a conflict is detected during migration - **Migration summary** β€” a "Project key conflicts" section at the end of the run lists all affected projects - **Migration report** β€” the `reports/migration-report.txt` includes a "PROJECT KEY CONFLICTS" section, and `reports/migration-report.json` includes a `projectKeyWarnings` array -If you see key conflicts, the affected projects were still migrated successfully β€” they just use a different key than the original SonarQube key. You can rename them later via the SonarCloud API (`/api/projects/update_key`) if the conflicting key becomes available. +If you see key conflicts, the affected projects were still migrated successfully β€” they just use a different key than the original SonarQube Server key. You can rename them later via the SonarQube Cloud API (`/api/projects/update_key`) if the conflicting key becomes available. ## 🌐 Connection Timeouts @@ -263,7 +263,7 @@ If you see key conflicts, the affected projects were still migrated successfully ## 🚦 Rate Limiting (503 / 429 Errors) -SonarCloud may return 503 or 429 errors when too many API requests are made in a short period, especially during issue and hotspot sync on large projects. +SonarQube Cloud may return 503 or 429 errors when too many API requests are made in a short period, especially during issue and hotspot sync on large projects. By default, CloudVoyager retries rate-limited requests up to 3 times with exponential backoff. You can tune this via the `rateLimit` section in your config file: @@ -289,9 +289,9 @@ If you still encounter rate limit errors after all retries are exhausted, consid ## 🏷️ Project Names Showing as Project Keys -If projects in SonarCloud show the project key as the display name instead of the original human-readable name from SonarQube, the project was likely created by an older version of CloudVoyager. The current version automatically carries over the original project name from SonarQube when creating projects in SonarCloud. +If projects in SonarQube Cloud show the project key as the display name instead of the original human-readable name from SonarQube Server, the project was likely created by an older version of CloudVoyager. The current version automatically carries over the original project name from SonarQube Server when creating projects in SonarQube Cloud. -To fix already-migrated projects, you can rename them manually in SonarCloud via **Project Settings > General Settings > Project Name**, or delete and re-migrate the project. +To fix already-migrated projects, you can rename them manually in SonarQube Cloud via **Project Settings > General Settings > Project Name**, or delete and re-migrate the project. ## 🚧 Quality Gate / Profile Permission Errors (400) @@ -299,18 +299,27 @@ To fix already-migrated projects, you can rename them manually in SonarCloud via When migrating quality gates or profiles, permission APIs may return 400 errors for built-in gates/profiles. This is expected β€” built-in resources don't support custom permissions. The migrators handle this gracefully and skip these entries. -## πŸ“Š Issue Counts Differ Between SonarQube and SonarCloud +## πŸ“Š Issue Counts Differ Between SonarQube Server and SonarQube Cloud -If you see different issue counts (Security, Reliability, Maintainability) after migration, this is usually caused by **different active rules** between the SonarQube and SonarCloud quality profiles. +If you see different issue counts (Security, Reliability, Maintainability) after migration, this is usually caused by **different active rules** between the SonarQube Server and SonarQube Cloud quality profiles. -The migrator now restores built-in profiles as custom profiles (e.g., "Sonar way (SonarQube Migrated)") and assigns them to projects. However, some rules may not exist on SonarCloud (e.g., rules from third-party plugins). Check `quality-profiles/quality-profile-diff.json` in the output directory to see which rules are missing or added per language. +The migrator now restores built-in profiles as custom profiles (e.g., "Sonar way (SonarQube Server Migrated)") and assigns them to projects. However, some rules may not exist on SonarQube Cloud (e.g., rules from third-party plugins). Check `quality-profiles/quality-profile-diff.json` in the output directory to see which rules are missing or added per language. -If you'd prefer to skip quality profile migration entirely and use each language's default SonarCloud profile instead, use `--skip-quality-profile-sync`. +If you'd prefer to skip quality profile migration entirely and use each language's default SonarQube Cloud profile instead, use `--skip-quality-profile-sync`. + + +## πŸ§ͺ Source Issues in `IN_SANDBOX` Are Migrated as `OPEN` (Issue #136) + +SonarQube Server 2025+ introduces an `IN_SANDBOX` issue status β€” used for issues raised by sandboxed AI rules that have not yet been promoted to the standard catalog. SonarQube Cloud does not have an equivalent state, so these issues are migrated as `OPEN` on the destination. + +This is a known limitation, not a bug. There is no SonarQube Cloud transition that maps to `IN_SANDBOX`, so the migrator leaves the migrated issue in its default `OPEN` state and the rest of the metadata sync (assignment, comments, tags) still applies. Once the underlying sandbox rule graduates and exists on both servers, you can re-evaluate the issues on SonarQube Cloud. + +If you want to filter these out of the destination project, you can manually triage them post-migration on SonarQube Cloud. -## πŸ“„ SonarQube API Pagination Limits +## πŸ“„ SonarQube Server API Pagination Limits -Some SonarQube APIs enforce a maximum page size of 100 (not 500): +Some SonarQube Server APIs enforce a maximum page size of 100 (not 500): - `/api/permissions/groups` - `/api/project_tags/search` - `/api/qualityprofiles/search_users` @@ -321,7 +330,7 @@ The extractors handle this automatically, but if you see pagination-related erro ## πŸ”€ Quality Gates Use Names, Not IDs -The SonarQube quality gates API uses `name` for all operations (`/api/qualitygates/show`, `/api/qualitygates/select`), not `id`. If you see "not found" errors related to quality gates, check that you're using the gate name. +The SonarQube Server quality gates API uses `name` for all operations (`/api/qualitygates/show`, `/api/qualitygates/select`), not `id`. If you see "not found" errors related to quality gates, check that you're using the gate name. ## πŸ’Ύ Out of Memory / Heap Allocation Errors @@ -366,7 +375,7 @@ Or manually increase concurrency via CLI flags: For persistent config, add a `performance` section to your config file. See the [Configuration Reference](configuration.md#performance-settings) for all options. -Keep `hotspotSync.concurrency` low (3–5) to avoid SonarCloud rate limits. +Keep `hotspotSync.concurrency` low (3–5) to avoid SonarQube Cloud rate limits. ## πŸ“¦ Large Reports @@ -427,7 +436,7 @@ You can also sync just one type of metadata at a time: > **Import bug fixes (v1.3):** Earlier versions had missing imports in `issue-details.js`, `hotspot-details.js`, and `format-pdf/index.js` that could cause report generation to fail with `ReferenceError` or produce incomplete output. These have been fixed in v1.3. If you encountered report generation errors on a prior version, upgrade and re-run `verify`. -After migration, use the `verify` command to generate a detailed pass/fail comparison of SonarQube vs SonarCloud data: +After migration, use the `verify` command to generate a detailed pass/fail comparison of SonarQube Server vs SonarQube Cloud data: ```bash ./cloudvoyager verify -c migrate-config.json --verbose @@ -447,7 +456,7 @@ The console also prints a summary with per-project breakdowns and overall pass/f | Status | Meaning | |--------|---------| -| **pass** | SonarQube and SonarCloud data match | +| **pass** | SonarQube Server and SonarQube Cloud data match | | **fail** | Differences detected that should have been migrated | | **warning** | Unsyncable differences (expected β€” see below) | | **skipped** | Check was skipped (e.g., project not found in SC) | @@ -456,12 +465,12 @@ The console also prints a summary with per-project breakdowns and overall pass/f ### Issue assignment failures -If issue assignments are failing during migration, the most likely cause is a **login mismatch** between SonarQube and SonarCloud. SonarQube uses local logins (e.g., `john.doe`) while SonarCloud typically uses SSO/GitHub logins (e.g., `john-doe-github`). +If issue assignments are failing during migration, the most likely cause is a **login mismatch** between SonarQube Server and SonarQube Cloud. SonarQube Server uses local logins (e.g., `john.doe`) while SonarQube Cloud typically uses SSO/GitHub logins (e.g., `john-doe-github`). **Fix:** Use the `user-mappings.csv` generated during `--dry-run` to map SQ logins to SC logins: 1. Run `--dry-run` to generate `migration-output/mappings/user-mappings.csv` -2. Fill in the `SonarCloud Login` column for each user +2. Fill in the `SonarQube Cloud Login` column for each user 3. Set `Include=no` for service accounts or users who should not have issues assigned 4. Run the actual migration β€” mappings are applied automatically @@ -472,7 +481,7 @@ See [Dry-Run CSV Reference β€” user-mappings.csv](dry-run-csv-reference.md#user- ### Unsyncable items (expected differences) -Some differences are expected because the SonarCloud API does not support syncing them: +Some differences are expected because the SonarQube Cloud API does not support syncing them: | Item | Why it's unsyncable | |------|-------------------| @@ -507,9 +516,9 @@ CloudVoyager uses a hierarchy of custom error classes (defined in `src/shared/ut |-------------|-----------|----------------| | **CloudVoyagerError** | 500 | Base class β€” all errors below extend this | | **ConfigurationError** | 400 | Invalid config file, missing required fields, schema validation failure | -| **SonarQubeAPIError** | varies | SonarQube API returned an error (includes `endpoint` for debugging) | -| **SonarCloudAPIError** | varies | SonarCloud API returned an error (includes `endpoint` for debugging) | -| **AuthenticationError** | 401 | Invalid or expired token for SonarQube or SonarCloud | +| **SonarQubeAPIError** | varies | SonarQube Server API returned an error (includes `endpoint` for debugging) | +| **SonarCloudAPIError** | varies | SonarQube Cloud API returned an error (includes `endpoint` for debugging) | +| **AuthenticationError** | 401 | Invalid or expired token for SonarQube Server or SonarQube Cloud | | **ProtobufEncodingError** | 500 | Failed to encode data into protobuf format (may include `data` payload) | | **StateError** | 500 | Corrupt state file, failed atomic write, or state inconsistency | | **ValidationError** | 400 | Data validation failure (includes `errors` array with details) | @@ -538,8 +547,8 @@ When resuming from a checkpoint journal, the tool validates a session fingerprin | Field Changed | Behavior | |---------------|----------| -| SonarQube version | **Warning** logged, resume continues | -| SonarQube URL | **Warning** logged, resume continues | +| SonarQube Server version | **Warning** logged, resume continues | +| SonarQube Server URL | **Warning** logged, resume continues | | CloudVoyager version | **Warning** logged, resume continues | | Project key | **Throws StaleResumeError** β€” hard fail, requires `--force-restart` | @@ -550,14 +559,14 @@ With `strictResume: true` in config, any fingerprint warning becomes a hard fail ## πŸ—ƒοΈ Extraction Cache TTL -Extraction caches are stored as **gzipped JSON** files in the `cache/` directory. Files older than **7 days** (configurable via `cacheMaxAgeDays`) are automatically purged. Use `--force-fresh-extract` to discard all caches and re-extract from SonarQube. +Extraction caches are stored as **gzipped JSON** files in the `cache/` directory. Files older than **7 days** (configurable via `cacheMaxAgeDays`) are automatically purged. Use `--force-fresh-extract` to discard all caches and re-extract from SonarQube Server. --- ## πŸ”„ Upload Deduplication on Resume -The checkpoint journal records successful CE task uploads per branch (task ID + timestamp). On resume, if a branch was already uploaded successfully, the upload is skipped. This prevents duplicate CE tasks in SonarCloud after a crash between upload and journal save. +The checkpoint journal records successful CE task uploads per branch (task ID + timestamp). On resume, if a branch was already uploaded successfully, the upload is skipped. This prevents duplicate CE tasks in SonarQube Cloud after a crash between upload and journal save. --- @@ -576,22 +585,22 @@ Between pipeline phases, the tool checks the shutdown flag and throws a `Gracefu ## πŸ”€ External Linter Issues Missing After Migration (SQ 2025+) -SonarQube 2025+ stores external linter rules (Ruff, Pylint, ESLint, Checkstyle, etc.) with an `external_` prefix in the rule key (e.g., `external_ruff:D200`). If you see zero Ruff/Pylint/etc. issues in SonarCloud after migration, ensure you are running CloudVoyager v1.2.0+ which correctly handles this prefix. Older versions misclassify these as native issues, causing SonarCloud to silently drop them. +SonarQube Server 2025+ stores external linter rules (Ruff, Pylint, ESLint, Checkstyle, etc.) with an `external_` prefix in the rule key (e.g., `external_ruff:D200`). If you see zero Ruff/Pylint/etc. issues in SonarQube Cloud after migration, ensure you are running CloudVoyager v1.2.0+ which correctly handles this prefix. Older versions misclassify these as native issues, causing SonarQube Cloud to silently drop them. --- ## πŸ”€ External Issue `cleanCodeAttribute` Must Be Enum -When migrating issues from SonarQube plugins not available in SonarCloud (e.g., MuleSoft), the tool creates external issues. A critical encoding detail: +When migrating issues from SonarQube Server plugins not available in SonarQube Cloud (e.g., MuleSoft), the tool creates external issues. A critical encoding detail: - The `cleanCodeAttribute` field in the protobuf `AdHocRule` message must be encoded as a **protobuf enum (varint)**, not a string. -- Despite the proto definition showing `optional string`, SonarCloud's CE silently ignores external issues if `cleanCodeAttribute` is string-encoded. +- Despite the proto definition showing `optional string`, SonarQube Cloud's CE silently ignores external issues if `cleanCodeAttribute` is string-encoded. - Valid enum values: `CONVENTIONAL=1`, `FORMATTED=2`, `IDENTIFIABLE=3`, `CLEAR=4`, `COMPLETE=5`, `EFFICIENT=6`, `LOGICAL=7`, `DISTINCT=8`, `FOCUSED=9`, `MODULAR=10`, `TESTED=11`, `LAWFUL=12`, `RESPECTFUL=13`, `TRUSTWORTHY=14`. --- -## πŸ•˜ SonarQube 9.9 Issue Statuses +## πŸ•˜ SonarQube Server 9.9 Issue Statuses SQ 9.9 LTS uses the legacy issue status model with only 5 statuses: @@ -605,23 +614,23 @@ The modern statuses (`FALSE_POSITIVE`, `ACCEPTED`, `FIXED`) do **not** exist in ## πŸ“Š Projects with 10,000+ Issues -SonarQube's `/api/issues/search` endpoint caps results at 10,000 due to an Elasticsearch hard limit. +SonarQube Server's `/api/issues/search` endpoint caps results at 10,000 due to an Elasticsearch hard limit. -> **Note (v1.3+):** Large-project issue handling now has two complementary mechanisms. The **search slicer** handles retrieval of >10K issues from SonarQube by splitting the date range into windows that each stay under the 10K API limit. The **SCM date backdating** (`backdateChangesets`) preserves each issue's original SonarQube creation date in SonarCloud by writing per-line blame dates into the changeset protobuf. A safety split ensures no single calendar day exceeds 5K issues (50% margin under the 10K ES visualization cap). Together, these two features ensure that projects of any size are fully extracted from SonarQube and fully visible in SonarCloud with accurate historical creation dates. +> **Note (v1.3+):** Large-project issue handling now has two complementary mechanisms. The **search slicer** handles retrieval of >10K issues from SonarQube Server by splitting the date range into windows that each stay under the 10K API limit. The **SCM date backdating** (`backdateChangesets`) preserves each issue's original SonarQube Server creation date in SonarQube Cloud by writing per-line blame dates into the changeset protobuf. A safety split ensures no single calendar day exceeds 5K issues (50% margin under the 10K ES visualization cap). Together, these two features ensure that projects of any size are fully extracted from SonarQube Server and fully visible in SonarQube Cloud with accurate historical creation dates. ### Error: `10001th result asked` -**Symptom:** Transfer or migration fails with `SonarQube API error (400): Can return only the first 10000 results. 10001th result asked.` +**Symptom:** Transfer or migration fails with `SonarQube Server API error (400): Can return only the first 10000 results. 10001th result asked.` **Fix (v1.2.1+):** The date-range probe that triggered this error has been replaced with a fixed epoch (`2006-01-01` β†’ now) range. No configuration needed β€” re-run the transfer or migration. ### Error: `Date cannot be parsed as either a date or date+time` -**Symptom:** Transfer fails with `SonarQube API error (400): Date '2007-09-08T21:21:02.125Z' cannot be parsed as either a date or date+time` when slicing activates. +**Symptom:** Transfer fails with `SonarQube Server API error (400): Date '2007-09-08T21:21:02.125Z' cannot be parsed as either a date or date+time` when slicing activates. -**Cause:** JavaScript's `toISOString()` produces milliseconds (`.125Z`). SonarQube's `createdAfter`/`createdBefore` API parameters require `+0000` format without milliseconds. +**Cause:** JavaScript's `toISOString()` produces milliseconds (`.125Z`). SonarQube Server's `createdAfter`/`createdBefore` API parameters require `+0000` format without milliseconds. **Fix (v1.2.1+):** Date window boundaries now use `+0000` format. Re-run the transfer or migration. @@ -630,22 +639,22 @@ SonarQube's `/api/issues/search` endpoint caps results at 10,000 due to an Elast The search slicer algorithm: 1. Detects when total issues exceed 10K -2. Divides the full SonarQube era into 12 equal-width time windows +2. Divides the full SonarQube Server era into 12 equal-width time windows 3. Recursively bisects any window that still exceeds 10K -4. Stops bisecting only if a window cannot be split further (same-millisecond boundary β€” an unavoidable SonarQube API limitation for mass-import scenarios where all issues share one identical timestamp) +4. Stops bisecting only if a window cannot be split further (same-millisecond boundary β€” an unavoidable SonarQube Server API limitation for mass-import scenarios where all issues share one identical timestamp) -**Verification:** Run `./cloudvoyager verify -c migrate-config.json --only issue-metadata` to compare issue counts between SonarQube and SonarCloud. +**Verification:** Run `./cloudvoyager verify -c migrate-config.json --only issue-metadata` to compare issue counts between SonarQube Server and SonarQube Cloud. --- ## πŸ“¦ Issue Creation Date Accuracy -CloudVoyager preserves each issue's original SonarQube creation date in SonarCloud by rewriting SCM changeset blame dates in the protobuf report. This is automatic and transparent β€” no user configuration needed. +CloudVoyager preserves each issue's original SonarQube Server creation date in SonarQube Cloud by rewriting SCM changeset blame dates in the protobuf report. This is automatic and transparent β€” no user configuration needed. -> **How it works (v1.3.1+):** `backdateChangesets()` reads each issue's `creationDate` field from SonarQube and maps it to the issue's `textRange` lines in the file's changeset data. The CE takes MAX(date) across an issue's line range to determine its creation date, so per-line dating with "oldest wins" for overlapping lines preserves accurate dates. A safety split handles calendar days with >5K issues. +> **How it works (v1.3.1+):** `backdateChangesets()` reads each issue's `creationDate` field from SonarQube Server and maps it to the issue's `textRange` lines in the file's changeset data. The CE takes MAX(date) across an issue's line range to determine its creation date, so per-line dating with "oldest wins" for overlapping lines preserves accurate dates. A safety split handles calendar days with >5K issues. -### Issue creation dates don't match SonarQube +### Issue creation dates don't match SonarQube Server Check that the migration was run with v1.3.1+. Earlier versions used arbitrary 30-day-spaced batch dates instead of original creation dates. Re-transfer affected projects to get accurate dates. @@ -653,12 +662,12 @@ Check that the migration was run with v1.3.1+. Earlier versions used arbitrary 3 ### Issue counts look correct in the API but not in the UI -SonarCloud's Elasticsearch caps visualization at 10K per date bucket. With accurate creation dates, issues are naturally distributed across their original dates. The safety split ensures no single day exceeds 5K issues. If you see this problem, verify the migration used v1.3.1+. +SonarQube Cloud's Elasticsearch caps visualization at 10K per date bucket. With accurate creation dates, issues are naturally distributed across their original dates. The safety split ensures no single day exceeds 5K issues. If you see this problem, verify the migration used v1.3.1+. ### Warning: "N issues on DATE exceed 5K cap" -**Expected behavior.** This means a single calendar day had more than 5,000 issues with the same creation date. The safety split automatically sub-groups them into ≀5K batches with 1-day-spaced synthetic dates. The sub-groups will appear as separate adjacent dates in SonarCloud's creation date facet. +**Expected behavior.** This means a single calendar day had more than 5,000 issues with the same creation date. The safety split automatically sub-groups them into ≀5K batches with 1-day-spaced synthetic dates. The sub-groups will appear as separate adjacent dates in SonarQube Cloud's creation date facet. ### Can I change the safety split threshold? @@ -667,19 +676,19 @@ The threshold is hardcoded at 5,000 (50% safety margin under the 10K ES visualiz --- -## πŸ”Œ Third-Party Issues Not Appearing in SonarCloud +## πŸ”Œ Third-Party Issues Not Appearing in SonarQube Cloud -External (third-party) issues from SonarQube plugins not available in SonarCloud (e.g., MuleSoft, ABAP) may be silently dropped if the rule-repository detection fails. +External (third-party) issues from SonarQube Server plugins not available in SonarQube Cloud (e.g., MuleSoft, ABAP) may be silently dropped if the rule-repository detection fails. -**Symptom:** Issues from third-party plugins are missing in SonarCloud after migration, even though they exist in SonarQube. +**Symptom:** Issues from third-party plugins are missing in SonarQube Cloud after migration, even though they exist in SonarQube Server. **Possible causes:** -1. The SonarCloud `/api/rules/repositories` endpoint was unreachable during migration (network issue, token scope). +1. The SonarQube Cloud `/api/rules/repositories` endpoint was unreachable during migration (network issue, token scope). 2. The rule key does not contain a colon separator (malformed rule). 3. The live repository set returned empty due to a transient API error. -**Fix (v1.2+):** CloudVoyager now retries the repository API 3 times with exponential backoff and falls back to a built-in set of 43 known SonarCloud repositories. Rules without a colon are handled gracefully. If you migrated before v1.2, re-run the migration for affected projects. +**Fix (v1.2+):** CloudVoyager now retries the repository API 3 times with exponential backoff and falls back to a built-in set of 43 known SonarQube Cloud repositories. Rules without a colon are handled gracefully. If you migrated before v1.2, re-run the migration for affected projects. **Debugging:** Enable `--verbose` and search the log for `FALLBACK_SONARCLOUD_REPOS` or `getRuleRepositories` to confirm whether the fallback was used. diff --git a/docs/verification.md b/docs/verification.md index 9291a3e..751cb7c 100644 --- a/docs/verification.md +++ b/docs/verification.md @@ -3,7 +3,7 @@ -After running a migration, use the `verify` command to compare your SonarQube instance against SonarCloud and confirm that all data was transferred correctly. The verification pipeline is implemented in `src/shared/verification/verify-pipeline.js` and is entirely **read-only** β€” it does not modify any data on either SonarQube or SonarCloud. +After running a migration, use the `verify` command to compare your SonarQube Server instance against SonarQube Cloud and confirm that all data was transferred correctly. The verification pipeline is implemented in `src/shared/verification/verify-pipeline.js` and is entirely **read-only** β€” it does not modify any data on either SonarQube Server or SonarQube Cloud. ```bash # Verify everything @@ -82,7 +82,7 @@ Compares quality profile **definitions** (not assignments β€” those are per-proj | What | How | |------|-----| | Profile existence | SQ profiles matched to SC by `language + name` | -| Migration suffix | SC profiles named `"X (SonarQube Migrated)"` are treated as equivalent to SQ `"X"` | +| Migration suffix | SC profiles named `"X (SonarQube Server Migrated)"` are treated as equivalent to SQ `"X"` | | Active rule count | Compared for **custom profiles only** | | Built-in profiles | Rule count differences **skipped** β€” platform versions have different rule sets | | Unsupported languages | Profiles for SQ-only languages (e.g., `mulesoft`) are **skipped** β€” SC doesn't have the language | @@ -110,7 +110,7 @@ Compares group-level permissions at the organization level. | What | How | |------|-----| | Permission sets | For each SQ group, checks that all SQ permissions exist in SC | -| SC-unsupported perms | `applicationcreator` and `portfoliocreator` are **excluded** β€” these are SonarQube Enterprise permissions that don't exist in SonarCloud | +| SC-unsupported perms | `applicationcreator` and `portfoliocreator` are **excluded** β€” these are SonarQube Server Enterprise permissions that don't exist in SonarQube Cloud | | Extra SC permissions | Allowed (e.g., SC may auto-grant `scan`) | **Pass** if all SQ group permissions (minus unsupported ones) exist in SC. @@ -166,7 +166,7 @@ The most detailed check. Compares all issues between SQ and SC. | **Status** | Normalized from `status + resolution` (OPEN, FIXED, FALSE-POSITIVE, WONTFIX) | | **Assignment** | Direct comparison of assignee | | **Status history** | Fetches changelogs from both SQ and SC, extracts status transitions, and verifies that all SQ transitions appear in SC in order. Issues with no status changes are skipped | -| **Comments** | Counts SC comments containing `[Migrated from SonarQube]` marker; flags if fewer than SQ comment count | +| **Comments** | Counts SC comments containing `[Migrated from SonarQube Server]` marker; flags if fewer than SQ comment count | | **Tags** | Only flags if SQ tags are **missing** from SC; SC adding extra tags (e.g., `type-dependent`) is expected | | **External issue tags** | **Skipped entirely** β€” SC external issues don't preserve tags | | **Rule not in SC** | If a rule has zero presence in SC (no matches at all), unmatched issues for that rule are **excluded** from the failure count β€” it's a platform difference, not a migration failure | @@ -189,7 +189,7 @@ Compares security hotspots between SQ and SC. |------|-----| | **Matching** | Hotspots matched by `ruleKey + filePath + lineNumber` | | **Status** | Normalized β€” `REVIEWED:SAFE`, `REVIEWED:ACKNOWLEDGED`, `REVIEWED:FIXED`, or `TO_REVIEW` | -| **Comments** | Fetches SC hotspot details and counts `[Migrated from SonarQube]` comments | +| **Comments** | Fetches SC hotspot details and counts `[Migrated from SonarQube Server]` comments | | **Assignments** | Tracked as "unsyncable" (hotspot API doesn't support assignment sync) | | **Rule not in SC** | Same as issues β€” if a rule has zero presence in SC hotspots (e.g., reclassified from hotspot to issue, or rule unavailable), unmatched hotspots are **excluded** from the failure count | @@ -227,7 +227,7 @@ Checks which quality profile is assigned per language. | What | How | |------|-----| | Profile assignment | Compared by language | -| Migration suffix | `"Sonar way (SonarQube Migrated)"` treated as equivalent to `"Sonar way"` | +| Migration suffix | `"Sonar way (SonarQube Server Migrated)"` treated as equivalent to `"Sonar way"` | | Unsupported languages | Languages not in SC (e.g., `mulesoft`) are **skipped** | **Pass** if all SC-supported language profiles match. @@ -338,7 +338,7 @@ Each check produces one of these statuses: ## Platform Differences (Gotchas) -These are known differences between SonarQube and SonarCloud that the verification system handles automatically. They are **not** migration failures. +These are known differences between SonarQube Server and SonarQube Cloud that the verification system handles automatically. They are **not** migration failures. ### Rule Availability @@ -358,14 +358,14 @@ When SQ has issues from plugins not available in SC (e.g., MuleSoft), the migrat | Platform | Rule key | |----------|----------| -| SonarQube | `mulesoft:MS058` | -| SonarCloud | `external_mulesoft:MS058` | +| SonarQube Server | `mulesoft:MS058` | +| SonarQube Cloud | `external_mulesoft:MS058` | The verifier normalizes rule keys by stripping the `external_` prefix before matching. **Important:** External issues in SC do **not** preserve tags from SQ. Tag comparison is skipped entirely for external issues. -**Improved in v1.2:** External issue detection is now more reliable. If the SonarCloud `/api/rules/repositories` API is unreachable during migration, the tool falls back to a built-in set of 43 known repositories (with 3 retries and exponential backoff). Rules without a colon separator and empty repository prefixes are also handled gracefully. If verification shows missing external issues from a pre-v1.2 migration, re-run the migration for affected projects. +**Improved in v1.2:** External issue detection is now more reliable. If the SonarQube Cloud `/api/rules/repositories` API is unreachable during migration, the tool falls back to a built-in set of 43 known repositories (with 3 retries and exponential backoff). Rules without a colon separator and empty repository prefixes are also handled gracefully. If verification shows missing external issues from a pre-v1.2 migration, re-run the migration for affected projects. ### Branch Naming @@ -375,11 +375,11 @@ SQ and SC may have different default branch names (e.g., SQ uses `main`, SC uses ### Quality Profile Naming -The migration creates profiles with a `(SonarQube Migrated)` suffix to avoid colliding with SC's built-in profiles: +The migration creates profiles with a `(SonarQube Server Migrated)` suffix to avoid colliding with SC's built-in profiles: -| SonarQube | SonarCloud | +| SonarQube Server | SonarQube Cloud | |-----------|------------| -| `Sonar way` | `Sonar way (SonarQube Migrated)` | +| `Sonar way` | `Sonar way (SonarQube Server Migrated)` | The verifier strips this suffix before comparing. @@ -388,10 +388,10 @@ The verifier strips this suffix before comparing. Built-in quality gates ("Sonar way", "Sonar way for AI Code") and built-in profiles ("Sonar way" for each language) are managed by the platform. Their conditions and rule counts naturally differ between SQ and SC versions. The verifier skips detailed comparison for these. -### SonarCloud-Unsupported Permissions +### SonarQube Cloud-Unsupported Permissions -These SonarQube Enterprise permissions don't exist in SonarCloud and are excluded from comparison: +These SonarQube Server Enterprise permissions don't exist in SonarQube Cloud and are excluded from comparison: - `applicationcreator` β€” create Applications (Enterprise feature) - `portfoliocreator` β€” create Portfolios (Enterprise feature) @@ -414,9 +414,9 @@ SQ may have plugins that add languages not available in SC (e.g., MuleSoft's `mu ### Batch-Distributed Issues and Multiple Analysis Dates -When a branch has more than 5,000 issues, the migration uses **batch distribution** to split those issues into chunks of up to 5,000 and assigns each chunk a separate `analysis_date` (computed by stepping backwards one day per batch from the original analysis date). This means a single branch that had one analysis in SonarQube may appear in SonarCloud with issues spread across multiple `analysis_date` values. +When a branch has more than 5,000 issues, the migration uses **batch distribution** to split those issues into chunks of up to 5,000 and assigns each chunk a separate `analysis_date` (computed by stepping backwards one day per batch from the original analysis date). This means a single branch that had one analysis in SonarQube Server may appear in SonarQube Cloud with issues spread across multiple `analysis_date` values. -**This is expected behavior, not a verification failure.** The batch distribution is a migration-time strategy to stay within SonarCloud's import limits and does not affect the correctness of the migrated data. +**This is expected behavior, not a verification failure.** The batch distribution is a migration-time strategy to stay within SonarQube Cloud's import limits and does not affect the correctness of the migrated data. The verification pipeline compares **total issue counts and attributes** (rule, file, line, status, comments, tags, etc.) across the entire branch β€” it does not compare per-date counts. As a result, batch distribution should never cause false verification failures. If verification reports an issue-count mismatch, the cause is not the batching itself but a genuine migration gap. @@ -446,7 +446,7 @@ SQ and SC may classify the same rule with different types (BUG vs CODE_SMELL) or --- -## SonarQube API Gotchas +## SonarQube Server API Gotchas These API quirks affect both migration and verification: