Skip to content

Commit 2671f01

Browse files
authored
Reject invalid index numeric options (#5134)
* Reject invalid index numeric options (#5097) * Address adversarial review findings (#5097) * Suppress irrelevant optimize warnings (#5097)
1 parent 84bff5e commit 2671f01

13 files changed

Lines changed: 513 additions & 168 deletions

DEVELOPER_GUIDE.md

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -269,6 +269,18 @@ Directory scan / shared path filter (built-in skip lists + `.gitignore` / `.cdid
269269
→ Populate FTS5 index
270270
```
271271

272+
Explicit numeric `index` options are validated before database setup or worker
273+
startup. The inclusive ranges are `--parallelism` 1..16,
274+
`--max-file-bytes` 1..2147483647 bytes, `--max-symbols-per-file` 1..50000,
275+
`--max-references-per-file` 1..1000000, `--dry-run-path-limit` 1..1000000,
276+
`--watch-pending-path-limit` 1..262144, and `--debounce` 0..60000 ms. Any
277+
explicit zero outside those ranges, negative, overflowed, non-numeric, or
278+
over-limit value returns structured `E010_USAGE_ERROR` output and exit code 1;
279+
an invalid occurrence is not hidden by a later duplicate. For compatibility,
280+
invalid `CDIDX_INDEX_PARALLELISM`, `CDIDX_MAX_FILE_BYTES`, and
281+
`CDIDX_INDEX_WATCH_PENDING_PATH_LIMIT` values still warn and fall back or clamp,
282+
but JSON warnings identify the environment source and the effective value.
283+
272284
Every loaded text file is normalized and analyzed in one UTF-16 walk. The
273285
resulting `NormalizedContentFacts` is the shared source for normalized line
274286
count, oversized-line and FTS-token diagnostics, conflict-marker detection,
@@ -4220,6 +4232,17 @@ query コマンドも JSON profile block 用の `--profile` と command-scoped p
42204232
→ FTS5インデックス反映
42214233
```
42224234

4235+
明示された数値 `index` オプションは、database setup や worker 起動より前に検証します。
4236+
許容範囲(両端を含む)は `--parallelism` が 1..16、`--max-file-bytes` が
4237+
1..2147483647 byte、`--max-symbols-per-file` が 1..50000、
4238+
`--max-references-per-file` が 1..1000000、`--dry-run-path-limit` が
4239+
1..1000000、`--watch-pending-path-limit` が 1..262144、`--debounce` が
4240+
0..60000 ms です。範囲外のゼロ、負数、overflow、非数値、上限超過を明示すると、
4241+
構造化された `E010_USAGE_ERROR` と exit code 1 を返し、後続の重複指定が有効でも先行する
4242+
不正値を隠しません。互換性のため、不正な `CDIDX_INDEX_PARALLELISM`、
4243+
`CDIDX_MAX_FILE_BYTES`、`CDIDX_INDEX_WATCH_PENDING_PATH_LIMIT` は引き続き警告して
4244+
fallback または clamp しますが、JSON warning には環境変数由来であることと実効値を含めます。
4245+
42234246
読み込んだ text file は UTF-16 上の1回の走査で正規化と解析を行います。得られた
42244247
`NormalizedContentFacts` を、正規化後の行数、長すぎる行 / FTS token の診断、conflict
42254248
marker、replacement character の件数 / 行、80行・10行 overlap の chunk slice に対する

TESTING_GUIDE.md

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -780,6 +780,8 @@ Candidate-ordered parallel-index recovery tests must prove that the fatal result
780780
- `.github/scripts/configure-windows-test-host.ps1`
781781
The `dotnet.yml` and `release.yml` Windows lanes share temp pinning and Defender exclusion setup here so both workflows keep the same test-host performance assumptions. General `TMP` / `TEMP` point to the runner's fast `RUNNER_TEMP\cdidx-temp` storage. Executable plugin, hook, and Git fixtures instead use `USERPROFILE\cdidx-trusted-test-temp`, whose protected current-user ACL and trusted ancestor chain satisfy the production executable-boundary contract; the script publishes this separate root as `CDIDX_TEST_TRUSTED_TEMP_ROOT`. Do not move ordinary SQLite or filesystem fixtures into that protected root, because placing the entire suite on the system drive materially increases Windows runtime. The script includes both roots in its normalized, de-duplicated Defender audit. When the `WinDefend` service is running, it submits the resulting string array in one `Add-MpPreference` invocation, reads Defender preferences back, and fails if any path is missing. When the service is stopped or unavailable, exclusions are irrelevant and the script emits a warning while continuing host setup; other Defender configuration or verification failures remain fatal. Update `CiWorkflowTests` when changing this split, batching, availability, audit, verification, or workflow call contract.
782782
- The `dotnet.yml` SDK setup has one conditional retry for transient SDK download failures. Keep the first attempt marked `continue-on-error` only while the retry is guarded by its failed outcome, so a second failure still fails the job.
783+
- `IndexCommandRunnerTests.cs`
784+
Numeric `index` option coverage owns inclusive boundaries, zero/negative/overflow/non-numeric forms, duplicate precedence, text and JSON `E010_USAGE_ERROR` output before database mutation, and structured environment fallback warnings with source/effective-value provenance. Keep every numeric option in the shared matrix when its range or parsing changes.
783785
- `DbRecoveryTests.cs`
784786
Database corruption recovery and graceful degradation behavior. Filesystem setup failures for `cdidx index` (read-only DB files and unwritable DB parent directories) are covered in `IndexCommandRunnerTests.cs` so they exercise the same CLI JSON/stderr boundary users see.
785787
- `Issue4857ManagedRestoreBackupTests.cs`
@@ -1867,6 +1869,8 @@ dotnet test --filter "FullyQualifiedName~GitHelperTests"
18671869
primary の `dotnet publish` 出力と `CodeIndex` build artifact は main branch への push または手動 dispatch でだけ materialize し、pull request では作成しません。coverage upload は `TestResults/**/coverage.cobertura.xml` が存在するときだけ起動し、そのfileをfailure向け`TestResults` artifactから除外して、TRX・text log・その他のXML blame evidenceを残しながらcoverageを1回だけ保存します。
18681870
- `.github/scripts/configure-windows-test-host.ps1`
18691871
`dotnet.yml` と `release.yml` の Windows lane は、temp 固定と Defender 除外 setup をこのスクリプトで共有します。通常の `TMP` / `TEMP` は runner の高速な `RUNNER_TEMP\cdidx-temp` を使います。実行可能な plugin / hook / Git fixture だけは `USERPROFILE\cdidx-trusted-test-temp` を使い、current-user 限定の protected ACL と trusted な祖先 chain で production の executable-boundary contract を満たします。この専用 root は `CDIDX_TEST_TRUSTED_TEMP_ROOT` として helper へ渡します。Windows の実行時間を大きく増やすため、通常の SQLite / filesystem fixture を protected root へ移してはいけません。スクリプトは両 root を含む候補 path を正規化・重複排除して監査表示します。`WinDefend` service が稼働中なら、生成した string array を1回の `Add-MpPreference` 呼び出しで登録し、Defender preference を読み戻して欠けた path があれば失敗します。service が停止中または利用不能なら除外は意味を持たないため、warning を出して host setup を継続します。それ以外の Defender 設定失敗や検証失敗は引き続き fatal です。この split、batching、availability、audit、verification、または workflow 呼び出し contract を変更するときは `CiWorkflowTests` も更新してください。
1872+
- `IndexCommandRunnerTests.cs`
1873+
数値 `index` オプションの coverage は、両端を含む境界値、ゼロ / 負数 / overflow / 非数値、重複指定の優先順位、database mutation 前の text / JSON `E010_USAGE_ERROR`、および source / 実効値の provenance を持つ構造化環境変数 fallback warning を扱います。範囲や parse を変更するときは、共有 matrix にすべての数値オプションを残してください。
18701874
- `DbRecoveryTests.cs`
18711875
DB破損からの復旧とグレースフル劣化のテスト。`cdidx index` の filesystem setup failure(read-only DB file や書き込み不可の DB 親ディレクトリ)は、ユーザーが見る CLI JSON/stderr 境界を通すため `IndexCommandRunnerTests.cs` で扱います。
18721876
- `Issue4857ManagedRestoreBackupTests.cs`
Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
---
2+
category: fixed
3+
issues:
4+
- 5097
5+
affected:
6+
- src/CodeIndex/Cli/IndexCommandRunner.Parse.cs
7+
- src/CodeIndex/Cli/IndexCommandRunner.cs
8+
- src/CodeIndex/Cli/IndexCommandRunner.DryRun.cs
9+
- src/CodeIndex/Cli/IndexCommandRunner.FullScan.cs
10+
- src/CodeIndex/Cli/IndexCommandRunner.Update.cs
11+
- src/CodeIndex/Cli/IndexWatchJsonContracts.cs
12+
- src/CodeIndex/Cli/IndexWatchRunner.Reporting.cs
13+
- src/CodeIndex/Cli/CliFlagSchema.cs
14+
- tests/CodeIndex.Tests/IndexCommandRunnerTests.cs
15+
- DEVELOPER_GUIDE.md
16+
- TESTING_GUIDE.md
17+
---
18+
19+
## English
20+
21+
- **Invalid explicit numeric `index` options now fail before database setup (#5097)** — zero outside an option's inclusive range, negative, overflowed, non-numeric, and over-limit CLI values return structured `E010_USAGE_ERROR` output instead of warning and silently using a fallback. Environment-variable compatibility fallbacks remain warnings and now report their source and effective value in JSON output.
22+
23+
## 日本語
24+
25+
- **明示された不正な数値 `index` オプションを database setup 前に拒否するようになりました (#5097)** — オプションの許容範囲外のゼロ、負数、overflow、非数値、上限超過の CLI 値は、警告後に暗黙の fallback を使わず、構造化された `E010_USAGE_ERROR` を返します。環境変数の互換 fallback は警告として維持し、JSON 出力に source と実効値を含めます。

src/CodeIndex/Cli/CliFlagSchema.cs

Lines changed: 7 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -649,7 +649,7 @@ private static IReadOnlyList<CliFlag> BuildAll()
649649
Safety = CliOptionSafety.Preview,
650650
},
651651
new() { Name = "--show-paths", Description = "Show resolved local paths in maintenance diagnostics, recovery commands, or status --config output", PrimaryCommands = Set(RecoveryPathCommands.Concat(["index", "backfill-fold", "optimize", "vacuum", "db", "status"]).ToArray()) },
652-
new() { Name = "--dry-run-path-limit", ValuePlaceholder = "<n>", Description = "Dry run only: candidate path processing limit before truncated lower-bound estimates", PrimaryCommands = Set("index") },
652+
new() { Name = "--dry-run-path-limit", ValuePlaceholder = "<n>", Description = "Dry run only: candidate path processing limit (1..1000000); explicit invalid values fail before database setup", PrimaryCommands = Set("index") },
653653
new() { Name = "--checkpoint", Description = "Create a DB checkpoint even when backfill preflight is already complete", PrimaryCommands = Set("backfill-fold") },
654654
new() { Name = "--no-checkpoint", Description = "Skip the automatic DB checkpoint before a required backfill mutation", PrimaryCommands = Set("backfill-fold") },
655655
new()
@@ -668,17 +668,17 @@ private static IReadOnlyList<CliFlag> BuildAll()
668668
Safety = CliOptionSafety.Override,
669669
},
670670
new() { Name = "--duration-format", ValueDomain = Values(["auto", "seconds", "hms"]), Description = "Index elapsed time display format", PrimaryCommands = Set("index") },
671-
new() { Name = "--max-file-bytes", ValuePlaceholder = "<bytes>", Description = "Override the per-file indexing size limit", PrimaryCommands = Set("index") },
672-
new() { Name = "--max-symbols-per-file", ValuePlaceholder = "<n>", Description = "Skip file content, symbols, and references when one file emits too many symbols (max 50000)", PrimaryCommands = Set("index") },
673-
new() { Name = "--max-references-per-file", ValuePlaceholder = "<n>", Description = "Skip references when one file emits too many references (max 1000000)", PrimaryCommands = Set("index") },
674-
new() { Name = "--parallelism", ValuePlaceholder = "<n>", Description = "Full-scan and authoritative C# update extraction worker count (default: CPU count capped at 8; explicit max: 16; also honors CDIDX_INDEX_PARALLELISM)", PrimaryCommands = Set("index") },
671+
new() { Name = "--max-file-bytes", ValuePlaceholder = "<bytes>", Description = "Per-file indexing size limit (1..2147483647 bytes; optional B/K/M/G suffix); invalid CDIDX_MAX_FILE_BYTES warns and uses the built-in default", PrimaryCommands = Set("index") },
672+
new() { Name = "--max-symbols-per-file", ValuePlaceholder = "<n>", Description = "Skip file content, symbols, and references when one file emits too many symbols (1..50000)", PrimaryCommands = Set("index") },
673+
new() { Name = "--max-references-per-file", ValuePlaceholder = "<n>", Description = "Skip references when one file emits too many references (1..1000000)", PrimaryCommands = Set("index") },
674+
new() { Name = "--parallelism", ValuePlaceholder = "<n>", Description = "Extraction worker count (1..16; default CPU count capped at 8); invalid CDIDX_INDEX_PARALLELISM warns and uses the documented effective value", PrimaryCommands = Set("index") },
675675
new() { Name = "--memory-trace", Description = "Include phase memory samples in index JSON output", PrimaryCommands = Set("index") },
676676
new() { Name = "--commits", ValuePlaceholder = "<commit-ref>", Description = "Update files changed in given git commits", PrimaryCommands = Set("index") },
677677
new() { Name = "--changed-between", ValuePlaceholder = "<old-ref> <new-ref>", Description = "Update files changed between two git refs", PrimaryCommands = Set("index") },
678678
new() { Name = "--files", ValuePlaceholder = "<path>", ValueKind = CliOptionValueKind.FilePath, Description = "Update explicit files after validating the full selection atomically; invalid or duplicate paths reject before writes, while indexed missing paths remain deletion targets", PrimaryCommands = Set("index") },
679679
new() { Name = "--watch", Description = "Continuous reindex on file changes (rejects --commits / --changed-between / --files / --dry-run)", PrimaryCommands = Set("index") },
680-
new() { Name = "--debounce", ValuePlaceholder = "<ms>", Description = "Watch only: coalesce file events into one update after <ms> of quiet (default 500)", PrimaryCommands = Set("index") },
681-
new() { Name = "--watch-pending-path-limit", ValuePlaceholder = "<n>", Description = "Watch only: changed-path queue limit before full-rescan fallback", PrimaryCommands = Set("index") },
680+
new() { Name = "--debounce", ValuePlaceholder = "<ms>", Description = "Watch only: coalesce file events after 0..60000 ms of quiet (default 500)", PrimaryCommands = Set("index") },
681+
new() { Name = "--watch-pending-path-limit", ValuePlaceholder = "<n>", Description = "Watch only: changed-path queue limit (1..262144); invalid CDIDX_INDEX_WATCH_PENDING_PATH_LIMIT warns and uses the documented effective value", PrimaryCommands = Set("index") },
682682
new() { Name = "--output", ShortName = "-o", ValuePlaceholder = "<path>", ValueKind = CliOptionValueKind.FilePath, Description = "Report bundle or suggestions export output path", PrimaryCommands = Set("report", "suggestions") },
683683
new() { Name = "--redact-paths", Description = "Redact machine-specific paths (the default for recovery/config JSON and reports)", PrimaryCommands = Set(RecoveryPathCommands.Concat(["status", "report"]).ToArray()) },
684684
new() { Name = "--no-log", Description = "Exclude global tool log from bundle", PrimaryCommands = Set("report") },

src/CodeIndex/Cli/IndexCommandRunner.DryRun.cs

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -46,8 +46,10 @@ private static int RunDryRun(
4646
bool authoritativeFullScan;
4747
var errorSamples = new List<CliJsonMessage>();
4848
var errorCount = 0;
49-
var warningSamples = new List<CliJsonMessage>();
50-
var warningCount = 0;
49+
var warningSamples = options.OptionWarnings
50+
.Take(DryRunWarningSampleLimit)
51+
.ToList();
52+
var warningCount = options.OptionWarnings.Count;
5153
var dryScanErrorKeys = new HashSet<string>(StringComparer.Ordinal);
5254
DryRunScanMetadata dryScanMetadata;
5355
DryRunDbSnapshot dbSnapshot;

src/CodeIndex/Cli/IndexCommandRunner.FullScan.cs

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -175,6 +175,7 @@ void ThrowIfFullScanCancelled(int filesProcessed, int? filesTotal)
175175
})
176176
.ToList();
177177
var warningList = discovery.WarningList;
178+
warningList.InsertRange(0, options.OptionWarnings);
178179
AddProjectMarkerFingerprintWarnings(currentHotspotFamilyMarkerFingerprints, warningList, options);
179180
var scanCheckpointPath = discovery.ScanCheckpointPath;
180181
if (options.MemoryTrace)

0 commit comments

Comments
 (0)