Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 3 additions & 14 deletions .github/dependabot.yml
Original file line number Diff line number Diff line change
Expand Up @@ -57,13 +57,6 @@ updates:
shipped-minor-and-patch:
patterns: ['fuse.js', 'mediabunny', 'sortablejs']
update-types: ['minor', 'patch']
# Stryker's packages come out together, and each requires the others at its own
# version exactly (@stryker-mutator/vitest-runner's peerDependencies): one pull
# request for all of them, majors included. Before tooling-minor-and-patch, which
# would take them otherwise. While the vitest runner carries a patch, both are
# ignored below and move together through the Patched libraries workflow.
stryker:
patterns: ['@stryker-mutator/*']
# WebdriverIO's packages come out together and only work at the same version: one pull
# request for all of them, majors included (10.0.0 came as four, each red alone,
# 2026-10-10). Before tooling-minor-and-patch, which would take them otherwise.
Expand All @@ -79,14 +72,10 @@ updates:
# from the Patched libraries workflow instead, which re-cuts the patch with
# tools/recut-patch.mjs and opens the pull request itself when that is clean
# (docs/updating-patched-libraries.md). Keep this list in step with
# patchedDependencies, and the packages that move with a patched one (LOCKSTEP in
# tools/check-patched-updates.mjs: @stryker-mutator/core with the vitest runner, which
# requires its exact version). When every change of a patch has landed upstream,
# tools/recut-patch.mjs drops the patch and takes the library and those packages off
# this list in the same pull request.
# patchedDependencies. When every change of a patch has landed upstream,
# tools/recut-patch.mjs drops the patch and takes the library off this list in the same
# pull request.
ignore:
- dependency-name: '@stryker-mutator/core'
- dependency-name: '@stryker-mutator/vitest-runner'
- dependency-name: 'Coloris'
- dependency-name: 'dashjs'
- dependency-name: 'gif.js'
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/firefox-beta.yml
Original file line number Diff line number Diff line change
Expand Up @@ -131,8 +131,8 @@ jobs:
fi
exit 0
fi
# A suite that runs out of its time ends cancelled, not failed (as in
# mutation-tests.yml): a Firefox that hangs the suites said nothing.
# A suite that runs out of its time ends cancelled, not failed: a Firefox that
# hangs the suites said nothing.
if [ "$RESULT" != failure ] && [ "$RESULT" != cancelled ]; then
echo "e2e result: $RESULT - nothing to report."
exit 0
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/firefox-stable.yml
Original file line number Diff line number Diff line change
Expand Up @@ -192,8 +192,8 @@ jobs:
done
exit 0
fi
# A suite that runs out of its time ends cancelled, not failed (as in
# mutation-tests.yml): a Firefox that hangs the suites said nothing.
# A suite that runs out of its time ends cancelled, not failed: a Firefox that
# hangs the suites said nothing.
if [ "$RESULT" != failure ] && [ "$RESULT" != cancelled ]; then
echo "e2e result: $RESULT - nothing to report."
exit 0
Expand Down
204 changes: 0 additions & 204 deletions .github/workflows/mutation-tests.yml

This file was deleted.

4 changes: 2 additions & 2 deletions .github/workflows/runner-images.yml
Original file line number Diff line number Diff line change
Expand Up @@ -226,8 +226,8 @@ jobs:
}
report() { # result, images, what the image is to -latest
local result=$1 images=$2 role=$3 title number
# A job that runs out of its time ends cancelled, not failed (as in
# mutation-tests.yml): an image that hangs CI said nothing.
# A job that runs out of its time ends cancelled, not failed: an image that
# hangs CI said nothing.
if [ "$result" != success ] && [ "$result" != failure ] && [ "$result" != cancelled ]; then
echo "$images: ${result:-not run} - nothing to report."
return
Expand Down
4 changes: 0 additions & 4 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -40,9 +40,5 @@ lint-*.json
# vitest --coverage output.
coverage/

# Stryker (pnpm run test:mutation): its reports and its sandboxes
reports/mutation/
.stryker-tmp/

# The helper's launch lock (faststream-mpv-host.mjs launchLockFile), next to it while mpv starts.
native-host/faststream-mpv-launch-*.lock
6 changes: 0 additions & 6 deletions docs/maintenance.md
Original file line number Diff line number Diff line change
Expand Up @@ -207,12 +207,6 @@ third attempt was cancelled too or GitHub refused the re-run: re-run it from the
tab, and the issue closes at the next green CI run on `main`. "Auto release failed" means
such a run was red or its release failed; the issue links the run.

Once a week, `mutation-tests.yml` changes the code the unit tests cover one thing at a
time and checks the tests notice. When they miss some, "Mutation testing: week to <date>"
lists them: each is a place a fix could ship without a test that fails without it. Nothing
waits on it; the next week's issue replaces it, and a week with every change caught closes
it.

## What update-prs.yml checks before it says "ready to merge"

Nothing merges itself (the owner's choice, 2026-10-02; from 2026-10-01 every kind of update
Expand Down
3 changes: 1 addition & 2 deletions docs/notes/mpv.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,8 +34,7 @@ the new hashes). The installer counts because `update-local.ps1` tells an instal
from the repository's by the host file alone; a raised `HostVersion` changes that file. The
extension changes with it, so such a push releases, Firefox updates the extension, and
the next hand-off on the owner's PC shows the "!" until `update-local.cmd` (or
`install.ps1`) has run. The hash check is skipped in Stryker's sandbox, whose copy of
the host holds every mutant.
`install.ps1`) has run.

**The thin helper (host 15, 2026-10-13).** The host changed 14 times in 10 days, each one a
reinstall by hand, because policy lived in it. The owner chose to move the policy into the
Expand Down
42 changes: 6 additions & 36 deletions docs/notes/workflows.md
Original file line number Diff line number Diff line change
Expand Up @@ -327,8 +327,7 @@
start: gh can give a log with every step "UNKNOWN STEP"). Without
E2E_SHARD a config runs all its spec files (local `pnpm run verify`). The private
browsing suite (one spec file) runs in the first extension group only. A run is 18 jobs
at once of the account's 20, so more groups would only queue; the weekly mutation run
(Mondays, 7 jobs for hours) leaves 13, and a push during it runs in two waves.
at once of the account's 20, so more groups would only queue.
`fail-fast: false`, so one failing group does not cancel the others and their lists of
retried specs. Per-job artifacts: `e2e-retried-<os>-<suite>-<group>`,
`e2e-logs-<os>-<suite>-<group>`; `e2e-moz-logs-windows-<suite>-<group>` only from the Windows
Expand Down Expand Up @@ -647,37 +646,8 @@
- All of it was dry-run with a stub `gh` against the real upstream history (sync: adopted
-> closed, push -> no rebuild, new release named, comment only on upstream movement;
patched libraries: open once, no duplicates, close when caught up or unpatched).
- **`mutation-tests.yml`** (Mondays 06:40 UTC), 2026-10-01, T4: Stryker
(`stryker.config.mjs`, `pnpm run test:mutation`) over 25 pure-logic modules with unit
tests (10 small ones added 2026-10-03, #253; the config says which wait, and why). A report, not a gate: the run summary gets a table per file
(`tools/mutation-report.mjs`), the HTML report is the `mutation-reports` artifact, and when
the unit tests missed mutants (survived, or no test reaches them), one issue "Mutation
testing: week to <date>" lists them, assigned + @mention; next week's replaces it, a week
with all caught closes it. **The command runner**, not the vitest one: every mutant runs the
whole unit suite (`__STRYKER_ACTIVE_MUTANT__`), about 1.2 s each with 4 workers locally - on
CI a mutant the tests miss runs the whole ~18 s suite, and one job per area ran out of its
240 minutes at 51-74% with no report (#349, 2026-10-05): since then the three areas are
seven shards (`PARTS` in the config, split by file size), each in a job of 350 minutes, and
a shard that runs out of time (it ends cancelled) is reported as a failure. **Since
2026-10-06 the vitest runner** with per-test coverage: a mutant runs only the tests that
reached its code, in a Vitest that stays up (143 mutants in 31 s locally, ~12.5 tests a
mutant). Its 10.0.0 release (August 2026) predates vitest 5, which joins a test's suite
chain with ' > ', so its test filter matched nothing and every mutant "survived"
(stryker-js #6210); FastStream carries the fix, stryker-js PR #6214, as a pnpm patch
(`patches/@stryker-mutator__vitest-runner@10.0.0.patch`, the compiled files). When a
release has it, the Patched libraries workflow opens the pull request that drops the patch,
moves @stryker-mutator/core along (LOCKSTEP in `tools/check-patched-updates.mjs`: the runner
requires its core's exact version) and hands both back to Dependabot, whose `stryker` group
then keeps all of Stryker's packages in one pull request. A test that runs a module as a
process of its own (mpvHostVersion's host; the Windows-only installer and update-local
tests) does not see a mutant there. Compared mutant by mutant (command-runner run
37384050152, 4.7 hours; vitest-runner run 37393654214, 10 minutes): 11,313 of 11,370 alike.
The rest: each tool's entry line (`if (process.argv[1] ...`) a crashed run where the command
runner caught, two swapped on one line of SubtitleUtils - and the mpv host's message loop
(`main()`, `sendMessage()`), 24 mutants with no test reaching them. So the host is an area of
its own on the command runner (`COMMAND_AREAS`, `host-1`, about an hour and a half), and core,
network and tools are one vitest-runner shard each: four jobs. The sandbox leaves out `tsconfig.json`: Stryker rewrites it through
TypeScript's JS API, which TypeScript 7 does not have. `tests/workflows/mutation-tests.test.sh`.
Baseline, 2026-10-01 (73 min locally): 71.8% of 3,614 mutants caught; the weakest are the
mpv host (47.7%), MpvBackend (58.4%), SubtitleUtils (63.2%) and TabTracker (68.4%), the
best SubtitleSyncUtils (96%), MultiRegexMatcher, UrlMatchList and StreamPick (92%).
- **`mutation-tests.yml`** (2026-10-01 to 2026-10-22): Stryker over the unit-tested modules,
weekly. Removed with Stryker: its reports (about 4,700 mutants not caught, 64.6% to 68%)
were closed by hand each week and no fix came from them, while it carried a 131-line patch
of its vitest runner, an override and 34 MB of packages. The fixes kept their own check: a
hand-made mutation of each one, which its test must catch.
Loading
Loading