Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/official-suite-sharded.yml
Original file line number Diff line number Diff line change
Expand Up @@ -139,7 +139,7 @@ jobs:
needs: [utf-shards]
if: always()
env:
UTF_GATE_MODE: ${{ github.event_name == 'schedule' && 'strict' || github.event.inputs.utf_gate_mode || 'warn' }}
UTF_GATE_MODE: ${{ github.event.inputs.utf_gate_mode || 'warn' }}
steps:
- name: Download shard 0 artifact
uses: actions/download-artifact@v4
Expand Down
9 changes: 7 additions & 2 deletions .github/workflows/r3-failure-ledger.yml
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,10 @@ on:
description: "Fail workflow when readiness streak threshold is not met (true|false)"
required: false
default: "false"
fail_on_failures:
description: "Fail workflow when the generated ledger contains failures (true|false)"
required: false
default: "false"
schedule:
- cron: "30 4 * * *"

Expand All @@ -33,6 +37,7 @@ jobs:
OFFICIAL_SPECS_REF: ${{ github.event.inputs.official_specs_ref || '99704fa8860777da1d919ef765af1e41e75f5859' }}
READINESS_MIN_STREAK: ${{ github.event.inputs.readiness_min_streak || '3' }}
FAIL_ON_READINESS: ${{ github.event.inputs.fail_on_readiness || 'false' }}
FAIL_ON_FAILURES: ${{ github.event.inputs.fail_on_failures || 'false' }}
MONGO_CONTAINER_NAME: jongodb-replset
MONGO_REPLSET_NAME: rs0
MONGO_PORT: "27017"
Expand Down Expand Up @@ -74,7 +79,7 @@ jobs:
-Pr3SpecRepoRoot="${RUNNER_TEMP}/mongodb-specifications" \
-Pr3FailureLedgerOutputDir="${R3_LEDGER_OUTPUT_DIR}" \
-Pr3FailureLedgerMongoUri="${JONGODB_REAL_MONGOD_URI}" \
-Pr3FailureLedgerFailOnFailures=true \
-Pr3FailureLedgerFailOnFailures="${FAIL_ON_FAILURES}" \
r3FailureLedger

- name: Build trend snapshot
Expand Down Expand Up @@ -515,7 +520,7 @@ jobs:
raise SystemExit(1)
PY

- name: Post triage breadcrumb on schedule failure
- name: Post triage breadcrumb on scheduled ledger findings
if: always() && github.event_name == 'schedule'
env:
GH_TOKEN: ${{ github.token }}
Expand Down
30 changes: 22 additions & 8 deletions .github/workflows/runon-lane-pilot.yml
Original file line number Diff line number Diff line change
Expand Up @@ -187,14 +187,15 @@ jobs:
handle.write(markdown)
handle.write("\n")

warnings = []
failures = []
if strict_mismatch != 0 or strict_error != 0:
failures.append(
f"strict lane stability gate failed: mismatch={strict_mismatch}, error={strict_error}"
warnings.append(
f"strict lane has non-zero mismatch/error: mismatch={strict_mismatch}, error={strict_error}"
)
if extended_mismatch > strict_mismatch or extended_error > strict_error:
failures.append(
"extended lane regression gate failed: "
warnings.append(
"extended lane has higher mismatch/error after importing more runOn cases: "
f"strict(mismatch={strict_mismatch}, error={strict_error}) vs "
f"extended(mismatch={extended_mismatch}, error={extended_error})"
)
Expand All @@ -207,6 +208,9 @@ jobs:
f"coverage gate failed: extended imported={extended_imported} is lower than strict={strict_imported}"
)

if warnings:
for message in warnings:
print(f"::warning::{message}")
if failures:
for message in failures:
print(f"::error::{message}")
Expand Down Expand Up @@ -325,15 +329,16 @@ jobs:
/ strict_totals["runOnNotSatisfied"]
) * 100.0

warnings = []
failures = []
if strict_totals["mismatch"] != 0 or strict_totals["error"] != 0:
failures.append(
"strict lane aggregate stability gate failed: "
warnings.append(
"strict lane aggregate has non-zero mismatch/error: "
f"mismatch={strict_totals['mismatch']}, error={strict_totals['error']}"
)
if extended_totals["mismatch"] > strict_totals["mismatch"] or extended_totals["error"] > strict_totals["error"]:
failures.append(
"extended lane aggregate regression gate failed: "
warnings.append(
"extended lane aggregate has higher mismatch/error after importing more runOn cases: "
f"strict(mismatch={strict_totals['mismatch']}, error={strict_totals['error']}) vs "
f"extended(mismatch={extended_totals['mismatch']}, error={extended_totals['error']})"
)
Expand Down Expand Up @@ -361,6 +366,7 @@ jobs:
"shards": shard_rows,
"gatePassed": not failures,
"gateFailures": failures,
"gateWarnings": warnings,
}
summary_json.parent.mkdir(parents=True, exist_ok=True)
summary_json.write_text(json.dumps(summary, ensure_ascii=False, indent=2) + "\n", encoding="utf-8")
Expand Down Expand Up @@ -394,6 +400,11 @@ jobs:
lines.append("## Gate failures")
for failure in failures:
lines.append(f"- {failure}")
if warnings:
lines.append("")
lines.append("## Gate warnings")
for warning in warnings:
lines.append(f"- {warning}")

markdown = "\n".join(lines) + "\n"
summary_md.write_text(markdown, encoding="utf-8")
Expand All @@ -405,6 +416,9 @@ jobs:
handle.write(markdown)
handle.write("\n")

if warnings:
for warning in warnings:
print(f"::warning::{warning}")
if failures:
for failure in failures:
print(f"::error::{failure}")
Expand Down
5 changes: 3 additions & 2 deletions docs/USAGE.md
Original file line number Diff line number Diff line change
Expand Up @@ -138,8 +138,8 @@ Current certified reference runs (as of 2026-02-28):
- R3 External Canary Certification (latest success): `22378993613` (2026-02-25)

Official Suite Sharded UTF gate modes:
- `warn` (default for `pull_request` and manual dispatch): publish aggregate `total/match/mismatch/error` + pass rate summary, emit warning when mismatch/error remain.
- `strict` (default for scheduled `main` run): fail workflow when aggregate mismatch/error is non-zero.
- `warn` (default for `pull_request`, scheduled `main`, and manual dispatch): publish aggregate `total/match/mismatch/error` + pass rate summary, emit warning when mismatch/error remain.
- `strict`: fail workflow when aggregate mismatch/error is non-zero. Use this for release-candidate certification, not routine nightly signal collection.
- `off`: summary only, no mismatch/error gate.
- Manual dispatch input: `utf_gate_mode` (`off | warn | strict`).
- Workflow streak artifact: `utf-shard-streak.json/.md` exposing `officialZeroMismatchStreak` over recent schedule history.
Expand Down Expand Up @@ -208,6 +208,7 @@ R3 Failure Ledger workflow (`.github/workflows/r3-failure-ledger.yml`) also:
- manual dispatch inputs:
- `readiness_min_streak` (default `3`)
- `fail_on_readiness` (`true|false`, default `false`)
- `fail_on_failures` (`true|false`, default `false`; set `true` for release-candidate certification)

Run Spring compatibility matrix:

Expand Down
Loading