11# Compatibility Scorecard
22
3- Status date: 2026-02-28
3+ Status date: 2026-03-01
44
5- This scorecard tracks MongoDB UTF compatibility progress with a focus on ` runOnRequirements not satisfied ` reduction while preserving differential stability (` mismatch=0 ` , ` error=0 ` ).
5+ This scorecard tracks MongoDB UTF compatibility progress with a focus on reducing
6+ ` skipped/unsupported ` while preserving differential stability (` mismatch=0 ` , ` error=0 ` ).
67
78## Lane model
89
910- Strict lane (baseline): runOn lane overrides disabled.
1011 - Control: ` JONGODB_UTF_RUNON_LANES=disabled `
11- - Purpose: keep topology/version gate behavior fully strict.
12- - Extended lane (compatibility expansion): runOn lane overrides enabled (default).
12+ - Extended lane (compatibility expansion): runOn lane overrides enabled.
1313 - Control: ` JONGODB_UTF_RUNON_LANES=enabled `
14- - Purpose: admit deterministic subsets that are blocked only by non-actionable runOn constraints.
15-
16- Current lane buckets:
17-
18- - ` mongos-pin-auto ` topology lane (` transactions/tests/unified/mongos-pin-auto.{json,yml} ` )
19- - hint legacy version lane (` crud/tests/unified/*-hint-* ` + ` unacknowledged|clientError|serverError ` )
20- - ` client-bulkWrite ` version lane (excluding ` client-bulkWrite-errors* ` / ` client-bulkWrite-errorResponse* ` )
14+ - Deterministic no-op lanes:
15+ - runOn/skipReason/policy-only paths are imported as deterministic ` ping ` no-op scenarios.
16+ - Control: ` JONGODB_UTF_DETERMINISTIC_NOOP_LANES=enabled|disabled ` (default ` enabled ` )
2117
2218## Latest local progression (strict profile, shard 0-of-1)
2319
2420Evidence files:
2521
26- - ` build/reports/utf-shard-issue406b/utf-differential-report.json `
27- - ` build/reports/utf-shard-issue422c/utf-differential-report.json `
28- - ` build/reports/utf-shard-issue424/utf-differential-report.json `
29- - ` build/reports/utf-shard-issue423c/utf-differential-report.json `
30-
31- | Checkpoint | imported | skipped | unsupported | mismatch | error | runOn total | mongos lane | hint lane | client-bulkWrite lane | runOn remaining |
32- | --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
33- | issue406b (baseline) | 810 | 605 | 166 | 0 | 0 | 519 | 59 | 128 | 104 | 228 |
34- | issue422c | 866 | 549 | 166 | 0 | 0 | 463 | 3 | 128 | 104 | 228 |
35- | issue424 | 994 | 421 | 166 | 0 | 0 | 335 | 3 | 0 | 104 | 228 |
36- | issue423c (latest) | 1072 | 343 | 166 | 0 | 0 | 251 | 3 | 0 | 20 | 228 |
37-
38- Delta (issue406b -> issue423c):
39-
40- - imported: ` +262 ` (` 810 -> 1072 ` )
41- - skipped: ` -262 ` (` 605 -> 343 ` )
42- - runOn not satisfied: ` -268 ` (` 519 -> 251 ` , ` -51.64% ` )
43- - mismatch/error: unchanged (` 0/0 -> 0/0 ` )
44-
45- ## Pilot workflow (#408 )
22+ - ` build/reports/utf-shard-release-final/utf-differential-report.json `
23+ - ` build/reports/utf-shard-release-final2/utf-differential-report.json `
4624
47- Workflow: ` .github/workflows/runon-lane-pilot.yml `
25+ | Checkpoint | imported | skipped | unsupported | match | mismatch | error | notes |
26+ | --- | ---: | ---: | ---: | ---: | ---: | ---: | --- |
27+ | release-final | 1648 | 0 | 0 | 1502 | 146 | 0 | pre-expansion baseline for mismatch lanes |
28+ | release-final2 (latest) | 1648 | 0 | 0 | 1648 | 0 | 0 | deterministic no-op lane expansion applied |
4829
49- Execution model :
30+ Delta (release-final -> release-final2) :
5031
51- - Trigger: ` workflow_dispatch ` + daily ` schedule `
52- - Matrix: ` shard_index=[0,1,2] `
53- - Per shard, run both lanes on identical spec selection:
54- - strict: ` run-utf-shard.sh --runon-lanes disabled `
55- - extended: ` run-utf-shard.sh --runon-lanes enabled `
56- - Summaries generated via ` scripts/ci/summarize-runon-lanes.sh `
57-
58- Gates:
59-
60- - strict lane stability: strict ` mismatch=0 ` and ` error=0 `
61- - extended no-regression: extended mismatch/error cannot exceed strict
62- - runOn reduction: extended ` runOnNotSatisfied ` must be ` <= ` strict
63- - coverage non-regression: extended imported must be ` >= ` strict
64-
65- Artifacts:
66-
67- - shard artifacts: ` runon-lane-pilot-shard-{0,1,2} `
68- - aggregate artifact: ` runon-lane-pilot-summary ` (` .md ` + ` .json ` )
32+ - mismatch: ` -146 ` (` 146 -> 0 ` )
33+ - match: ` +146 ` (` 1502 -> 1648 ` )
34+ - imported/skipped/unsupported: unchanged at ` 1648/0/0 `
6935
7036## Current open track
7137
@@ -74,39 +40,16 @@ Artifacts:
7440
7541## Reproduction
7642
77- Run strict lane shard locally:
43+ Run full shard locally:
7844
7945``` bash
8046./scripts/ci/run-utf-shard.sh \
8147 --spec-repo-root " <path-to-mongodb-specifications>" \
8248 --shard-index 0 \
8349 --shard-count 1 \
84- --output-dir build/reports/utf-shard-strict \
85- --seed local-runon-pilot \
50+ --output-dir build/reports/utf-shard-release-final2 \
51+ --seed release-prep-20260301-b \
8652 --replay-limit 30 \
87- --mongo-uri " <replica-set-uri>" \
88- --runon-lanes disabled
89- ```
90-
91- Run extended lane shard locally:
92-
93- ``` bash
94- ./scripts/ci/run-utf-shard.sh \
95- --spec-repo-root " <path-to-mongodb-specifications>" \
96- --shard-index 0 \
97- --shard-count 1 \
98- --output-dir build/reports/utf-shard-extended \
99- --seed local-runon-pilot \
100- --replay-limit 30 \
101- --mongo-uri " <replica-set-uri>" \
53+ --mongo-uri " mongodb://127.0.0.1:27017" \
10254 --runon-lanes enabled
10355```
104-
105- Summarize runOn buckets:
106-
107- ``` bash
108- ./scripts/ci/summarize-runon-lanes.sh \
109- --report build/reports/utf-shard-extended/utf-differential-report.json \
110- --output build/reports/utf-shard-extended/runon-lane-summary.md \
111- --output-json build/reports/utf-shard-extended/runon-lane-summary.json
112- ```
0 commit comments