Skip to content

Commit bcb99aa

Browse files
authored
test(fuzz): fuzz the guest-facing parsers, and fix what it found (#169)
Adds fuzzing for the parsers that read guest-controlled bytes — the control-channel frame decoder, the userspace vsock state machine, the split-virtqueue reader, and the 9P server together with the transport beneath it. Five `cargo fuzz` targets share their harness bodies with `tests/fuzz_corpus.rs`, which replays every committed seed and crash artifact on stable inside a plain `cargo test`. Only that replay blocks a merge: libFuzzer needs nightly and its search is non-deterministic, so a run that finds a bug says nothing about whether the change under review introduced it. Discovery runs weekly and on demand instead (ADR-0012). The fuzzing found seven guest-reachable defects in shipped code, all fixed here. A guest could kill the VMM process for every sandbox it hosts with two MMIO writes — activating a virtqueue it never sized, so every ring index divided by zero. It could escape the 9P shared directory, because `Tlcreate` and `Tmkdir` validated the parent fid and then joined an unfiltered guest name onto it. It could hang the host in a descriptor cycle, size 4 GiB host allocations from a descriptor length or a `Tread` count, overflow ring-address arithmetic under `overflow-checks`, and desynchronize a 9P mount with `Rread`, `Rreaddir`, or `Rwalk` replies exceeding the negotiated `msize`. The same allocation and overflow bugs sat in the userspace vsock TX path, and neither device clamped `QueueNum` to the size it advertises. **One wire-path behavior change:** a `Tversion` asking for less than 4 KiB is now refused with `Rerror`/`EINVAL` rather than served. The device derives its request-assembly budget from `msize`, and a budget that small starves every later request including the `Tversion` that would renegotiate. Raising the client's value instead would break the negotiation the other way, since the reply commits the server not to exceed what the client can receive. Linux's 9P client rejects `msize < 4096` at mount time, so a conforming mount never reaches it. The gate checks that it is still covering something. Each harness reports the work it performed, and every seed must clear a floor — a harness that reaches none of its parser returns cleanly and replays green, which is exactly how the `nine_p_transport` target shipped inert in an earlier revision of this branch and was caught by review rather than by CI. The floor applies where a harness drives its parser from a loop over the fuzzer's bytes, since that is what a consumption bug can starve; the other two hand the raw input straight in and cannot go inert that way. Discovered inputs go to a gitignored `fuzz/corpus-run/`, so `fuzz/corpus/` stays the curated set the floor polices. Every fix is verified by reverting it and requiring the gate to fail: nine of nine caught, five by the corpus and four by unit-test oracles, plus the floor's own check. Two fixes cannot be pinned by any input — a 4 GiB `alloc_zeroed` is mapped lazily and never faults, and an out-of-table descriptor index reads zeroed memory and ends the walk indistinguishably — so both carry oracles instead. Fuzzing pins crashes; everything else needs an assertion. Validated on Linux at rustc 1.98: fmt, clippy, workspace tests, corpus replay, and all five targets clean with no artifacts. Closes #155.
1 parent 43ac2ca commit bcb99aa

49 files changed

Lines changed: 1938 additions & 81 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.github/workflows/e2e.yml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -253,7 +253,7 @@ jobs:
253253
# --no-fail-fast collects every result instead of stopping at the first
254254
# failure.
255255
#
256-
# Two exclusions, both deliberate:
256+
# The exclusions, each deliberate:
257257
#
258258
# pty_command_not_allowed asserts the guest rejects a non-allowlisted
259259
# program, but the allowlist loads at guest boot from a file only the

‎.github/workflows/fuzz.yml‎

Lines changed: 104 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,104 @@
1+
name: Fuzz
2+
3+
# Coverage-guided fuzzing of the parsers that read guest-controlled bytes.
4+
#
5+
# Not a required status check, and deliberately not run on pull requests. A
6+
# fuzzer that finds a new bug on an unrelated change blocks that change, which
7+
# trains people to route around the gate; and libFuzzer needs a nightly
8+
# toolchain, which breaks on its own schedule and has no business on the path
9+
# that gates a merge. The pull-request gate instead replays every committed
10+
# corpus input and crash artifact through the same harnesses on stable, inside
11+
# `cargo test` (`tests/fuzz_corpus.rs`) — so anything found here is checked on
12+
# every change once its input is committed.
13+
#
14+
# When a run fails, download the `fuzz-artifacts` bundle, commit the crashing
15+
# input under `fuzz/artifacts/<target>/` together with the parser fix, and the
16+
# replay test keeps it fixed.
17+
18+
on:
19+
workflow_dispatch:
20+
inputs:
21+
max_total_time:
22+
description: 'Seconds to fuzz each target'
23+
required: false
24+
default: '300'
25+
schedule:
26+
# Sundays 04:00 UTC. Fuzzing costs only runner minutes, so unlike the
27+
# credentialed agent lane this one can run unattended; a weekly cadence is
28+
# enough for a parser surface that changes rarely.
29+
- cron: '0 4 * * 0'
30+
31+
env:
32+
CARGO_TERM_COLOR: always
33+
RUST_BACKTRACE: 1
34+
35+
permissions:
36+
contents: read
37+
38+
concurrency:
39+
group: fuzz
40+
cancel-in-progress: false
41+
42+
jobs:
43+
fuzz:
44+
name: Fuzz ${{ matrix.target }}
45+
runs-on: ubuntu-latest
46+
timeout-minutes: 45
47+
48+
strategy:
49+
fail-fast: false
50+
matrix:
51+
# One job per target so a crash in one still leaves the others' results
52+
# readable, and so each gets the full budget rather than a share of it.
53+
target: [vsock_frame, vsock_packet, virtqueue, nine_p, nine_p_transport]
54+
55+
steps:
56+
- uses: actions/checkout@v4
57+
58+
# libFuzzer's `fuzz_target!` and the sanitizer instrumentation
59+
# `cargo fuzz` passes are nightly-only. This is why fuzzing lives here and
60+
# not in the stable lanes.
61+
- name: Install Rust nightly
62+
uses: dtolnay/rust-toolchain@nightly
63+
64+
- name: Install cargo-fuzz
65+
uses: taiki-e/install-action@82cd3e7658a6f96c86c0234aeeda1748937cb0a1 # v2
66+
with:
67+
tool: cargo-fuzz
68+
69+
- name: Cache cargo registry + build
70+
uses: Swatinem/rust-cache@c19371144df3bb44fab255c43d04cbc2ab54d1c4 # v2.9.1
71+
with:
72+
workspaces: |
73+
.
74+
fuzz
75+
shared-key: fuzz-${{ matrix.target }}
76+
77+
- name: Fuzz ${{ matrix.target }}
78+
run: |
79+
# -rss_limit_mb bounds the harness's own footprint: an allocation
80+
# sized from a guest-supplied length is exactly the class of bug this
81+
# is looking for, so it must report rather than OOM the runner.
82+
# -max_len keeps inputs in the range the parsers actually see on the
83+
# wire, so the budget goes into structure rather than length.
84+
# Two corpus directories: libFuzzer writes what it discovers to the
85+
# first and reads the second without modifying it, so the committed
86+
# seed set stays exactly what `tests/fuzz_corpus.rs` polices.
87+
mkdir -p fuzz/corpus-run/${{ matrix.target }}
88+
cargo fuzz run ${{ matrix.target }} \
89+
fuzz/corpus-run/${{ matrix.target }} \
90+
fuzz/corpus/${{ matrix.target }} -- \
91+
-max_total_time=${{ github.event.inputs.max_total_time || '300' }} \
92+
-rss_limit_mb=2048 \
93+
-max_len=65536 \
94+
-print_final_stats=1
95+
96+
# `if: failure()` on purpose: a crash is the only case with an artifact to
97+
# collect, and it is the case where the reproducer must not be lost.
98+
- name: Upload crash artifacts
99+
if: failure()
100+
uses: actions/upload-artifact@v4
101+
with:
102+
name: fuzz-artifacts-${{ matrix.target }}
103+
path: fuzz/artifacts/
104+
if-no-files-found: ignore

‎.gitignore‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,15 @@
11
/target
2+
# cargo-fuzz build output. `fuzz/corpus/` and `fuzz/artifacts/` are tracked on
3+
# purpose: the corpus is the curated seed set and the artifacts are crash
4+
# regressions that `tests/fuzz_corpus.rs` replays on every run.
5+
/fuzz/target
6+
/fuzz/coverage
7+
# Where a fuzz run accumulates what it discovers. libFuzzer writes new units to
8+
# the first corpus directory it is given, so runs are pointed here and
9+
# `fuzz/corpus/` stays exactly the seed set the replay gate polices — a
10+
# discovered input is kept for reaching new coverage, which is not the same
11+
# property the gate's work floor checks.
12+
/fuzz/corpus-run
213
/artifacts
314
/tmp
415
package-lock.json

‎AGENTS.md‎

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -862,6 +862,40 @@ ANTHROPIC_API_KEY=... cargo test --test e2e_service_mode -- --ignored --test-thr
862862
ANTHROPIC_API_KEY=... cargo test --test e2e_agent_mcp -- --ignored --test-threads=1
863863
```
864864

865+
### Fuzzing the guest-facing parsers
866+
867+
Several host-side parsers read bytes a guest chooses: the control-channel frame decoder (`void-box-protocol` framing plus the multiplex request-id prefix), the userspace vsock connection state machine, the split-virtqueue reader that walks descriptor chains out of guest memory, and the 9P server together with the transport beneath it. Their harnesses live in `src/fuzz.rs`, and two callers drive the same bodies — `cargo fuzz` and the replay gate.
868+
869+
The merge gate replays them. `tests/fuzz_corpus.rs` runs every file under `fuzz/corpus/<target>/` and `fuzz/artifacts/<target>/` through its harness on stable, as part of a plain `cargo test`. It is deterministic and costs milliseconds, so it needs no special invocation.
870+
871+
Discovery is out of band. `cargo fuzz` needs a nightly toolchain, and a search that finds a new bug on an unrelated change would block that change, so it runs weekly and on demand in `.github/workflows/fuzz.yml` — never on a pull request (ADR-0012). To run it locally:
872+
873+
```bash
874+
rustup toolchain install nightly
875+
cargo install cargo-fuzz
876+
# libfuzzer-sys builds the libFuzzer runtime with cc, so a C++ compiler must be
877+
# on PATH as `c++` (Fedora: gcc-c++; Debian/Ubuntu: g++).
878+
# libFuzzer writes what it discovers to the first corpus directory and reads the
879+
# rest without modifying them, so runs go to `fuzz/corpus-run/` (gitignored) and
880+
# `fuzz/corpus/` stays the curated seed set.
881+
mkdir -p fuzz/corpus-run/vsock_frame
882+
cargo +nightly fuzz run vsock_frame fuzz/corpus-run/vsock_frame fuzz/corpus/vsock_frame \
883+
-- -max_total_time=60 -rss_limit_mb=2048
884+
# targets: vsock_frame, vsock_packet, virtqueue, nine_p, nine_p_transport
885+
```
886+
887+
Do not commit what a run accumulates in `fuzz/corpus-run/`. libFuzzer keeps an input because it reached new coverage, which is not the property the replay gate checks, so those inputs would be judged against a work floor written for curated seeds. Promote one deliberately when it covers a shape worth keeping: move it into `fuzz/corpus/<target>/` and give it a name that says what it covers.
888+
889+
`nine_p` drives the 9P message parser directly; `nine_p_transport` drives the device through its MMIO registers and guest memory, covering the queue programming and descriptor walk beneath it. The split matters: the transport parses guest data of its own — descriptor lengths size host allocations and `next` indices steer the walk — and a harness aimed at the message parser never reaches it.
890+
891+
When a run crashes, `cargo fuzz` writes the input under `fuzz/artifacts/<target>/`. Commit that file **together with the parser fix** — the replay test then keeps the bug fixed for everyone. Never commit a crashing input ahead of its fix: it reds the gate for every unrelated change.
892+
893+
Adding a target means adding a `[[bin]]` to `fuzz/Cargo.toml`, a harness body in `src/fuzz.rs`, an arm in `tests/fuzz_corpus.rs`, at least one seed under `fuzz/corpus/<target>/`, and a work floor in `work_floor`. `every_fuzz_target_is_replayed` fails if any of the middle three is missing, so a target cannot end up fuzzed but unguarded.
894+
895+
A harness returns the units of work it performed — frames decoded, packets routed, chains popped, requests dispatched, registers written — and the replay test holds every seed to that target's floor. Panic-freedom alone does not show a harness still reaches its parser: one that reaches nothing returns cleanly and replays green, so the count is what makes an inert target visible. A floor only bites where the harness drives its parser from a loop over the fuzzer's bytes, because that is what a consumption bug can starve; a harness that hands the raw input straight to its parser cannot go inert that way, and takes a floor of zero. Set a floor from what the seeds actually do, with enough margin that trimming one does not trip it, and check it against the negative-space seeds — an input written to prove the parser *rejects* something does zero units of successful work by construction. The floor applies to seeds only; a crash artifact may reach its bug before doing any countable work. Assert nothing about the count inside a harness body — under `cargo fuzz` an input that does no work is an ordinary outcome, and an assertion there would be reported as a crash.
896+
897+
A harness parameterizes itself from the raw input through a hand-rolled byte reader rather than the `arbitrary` crate. The committed corpus is tied to the exact byte-consumption order, and a crate upgrade that reorders its reads would silently repoint every seed at a different scenario.
898+
865899
### Test initramfs and BusyBox
866900

867901
`scripts/build_test_image.sh` builds a minimal initramfs with `guest-agent`

0 commit comments

Comments
 (0)