Bug: QEMU HTTP bridge returns 502 for buffered request bodies on Node.js >= 24.17 (invalid content-length header)

[geminixiang]

QEMU HTTP bridge returns 502 for buffered request bodies on Node.js >= 24.17 (invalid content-length header)

What I hit

I ran into this while doing a git clone inside a Gondolin VM. On Node.js 24.17.0 and later, any guest request whose body goes through Gondolin's buffered-body path comes back as 502 from the QEMU HTTP bridge. It's not just git — a plain small POST breaks too. GET requests and streamed bodies are fine.

I bisected Node.js and the break starts exactly at 24.17.0, which bumped the built-in undici from 7.25.0 to 7.28.0. Here's what's going on: Gondolin 0.12.0 depends on npm undici ^6.21.0 (resolved to 6.27.0 in my setup). For a buffered body, the undici 6 Fetch layer appends its own content-length on top of the one the bridge already set. It then dispatches through Node's built-in undici 7.28 Agent — because both sides share the global undici.globalDispatcher.1 symbol. 7.28 is stricter and rejects the duplicated value ("181, 181") with InvalidArgumentError: invalid content-length header, so the bridge gives up and returns 502.

The fix that worked for me: drop content-length before calling fetch() for buffered bodies, and let undici compute it from the body itself.

My environment

• Gondolin 0.12.0 (undici resolved to 6.27.0)
• QEMU backend, Linux/KVM (Debian 12, x86_64)
• Fails on Node.js 24.17.0 and 24.18.0; works on 24.16.0
• I also reproduced it on macOS arm64 with the QEMU backend, Node.js 24.18.0

How to reproduce

This only uses Gondolin's public API. I used mikan-sandbox:latest because it has curl and git — any image with those works.

import { VM, ensureImageSelector } from "@earendil-works/gondolin";

const { assetDir } = await ensureImageSelector("mikan-sandbox:latest");
const vm = await VM.create({
  sandbox: { imagePath: assetDir },
  sessionLabel: "content-length-repro",
});

await vm.start();
try {
  const run = async (label, command) => {
    const result = await vm.exec(["/bin/sh", "-c", command]);
    console.log(`--- ${label} (exit ${result.exitCode}) ---`);
    console.log(((result.stdout ?? "") + (result.stderr ?? "")).trim());
  };

  await run(
    "GET (no body)",
    "curl -sS -o /dev/null -w 'HTTP %{http_code}' https://github.com",
  );

  await run(
    "POST (buffered body)",
    "curl -sS -o /dev/null -w 'HTTP %{http_code}' " +
      "-X POST -H 'content-type: text/plain' --data-binary 'hello' " +
      "https://api.github.com/markdown/raw",
  );

  await run(
    "git clone",
    "cd /tmp && rm -rf Hello-World && " +
      "git clone --depth 1 https://github.com/octocat/Hello-World.git 2>&1; " +
      "rc=$?; echo rc=$rc; exit $rc",
  );
} finally {
  await vm.close();
}

What I got on Node.js 24.18.0:

--- GET (no body) (exit 0) ---
HTTP 200
--- POST (buffered body) (exit 0) ---
HTTP 502
--- git clone (exit 128) ---
Cloning into 'Hello-World'...
error: RPC failed; HTTP 502 curl 22 The requested URL returned error: 502
fatal: expected flush after ref listing
rc=128

On Node.js 24.16.0 the same POST comes back HTTP 200 and the clone works.

Bisecting Node.js

Same machine, same Gondolin 0.12.0, same guest image — I only swapped the Node.js binary:

Node.js	built-in undici	guest buffered POST / clone
24.13.0	7.18.2	✅ pass
24.13.1	7.18.2	✅ pass
24.14.0	7.21.0	✅ pass
24.14.1	7.24.4	✅ pass
24.15.0	7.24.4	✅ pass
24.16.0	7.25.0	✅ pass
24.17.0	7.28.0	❌ 502
24.18.0	7.28.0	❌ 502

I read the undici versions from process.versions.undici and cross-checked them against each nodejs/node tag's deps/undici/src/package.json. 24.17.0 is the release where the bundled undici jumps to 7.28.0 (cf44df3996).

It's only the buffered path

I want to be precise here — only buffered bodies break. On Node.js 24.18.0:

• A 5-byte POST reaches fetch() as a Uint8Array with content-length: 5 → 502.
• A 1 MiB fixed-length POST goes down the bodyStream path, keeps its single content-length: 1048576 → HTTP 200.

So whatever the fix is, it needs to keep the explicit length for streamed bodies and only drop it for buffered ones.

Digging in

What the bridge sends

I logged the request right before fetcher(...) for the failing git POST:

headers = {
  "accept": "application/x-git-upload-pack-result",
  "accept-encoding": "deflate, gzip, br, zstd",
  "content-length": "181",
  "content-type": "application/x-git-upload-pack-request",
  "git-protocol": "version=2",
  "host": "github.com",
  "user-agent": "git/2.52.0"
}
bodyInit = Uint8Array(181)
bodyStream = null

So the bridge isn't blindly forwarding the guest's length — for buffered bodies dist/src/qemu/http.js recomputes content-length from request.body.length. The value is correct; it just becomes redundant once the buffered body is handed to Fetch.

This is still on main at 29fa74d: it sets the measured length and then passes that header together with the buffered Uint8Array to Fetch.

Why npm undici 6 ends up dispatching through built-in undici 7.28

Gondolin imports fetch from npm undici 6.27.0, and its Fetch layer measures the buffered body and appends a second content-length before dispatch.

The thing that confused me at first: its global dispatcher lives under Symbol.for("undici.globalDispatcher.1"), and in the VM's module-loading order that symbol is already owned by Node's built-in undici Agent (getGlobalDispatcher() instanceof npmUndici.Agent is false). So undici 6's Fetch actually dispatches through the built-in 7.28 Agent. That's why the stack mixes both — the outer frame is npm undici 6.27.0, but the parser that throws is built-in 7.28.0.

The error

TypeError: fetch failed
    at fetch (.../node_modules/@earendil-works/gondolin/node_modules/undici/index.js:113:13)
    at async fetchHookRequestAndRespond (.../dist/src/qemu/http.js:925:24)
Caused by: InvalidArgumentError: invalid content-length header
    at processHeader (node:internal/deps/undici/undici:3190:17)
    at new Request (node:internal/deps/undici/undici:3018:15)
    at [dispatch] (node:internal/deps/undici/undici:9389:25)
    at Client.dispatch (node:internal/deps/undici/undici:2316:33)

7.28's parser rejects the non-digit value from the duplicate append ("181, 181"); older built-in versions let it through. The bridge treats the failure as a generic upstream error and returns 502, which git reports as fatal: expected flush after ref listing.

The fix I verified

For buffered bodies, strip content-length from a copy that's only used for fetch():

const fetchHeaders = { ...currentRequest.headers };
if (bodyInit && !bodyStream) {
  for (const key of Object.keys(fetchHeaders)) {
    if (key.toLowerCase() === "content-length") {
      delete fetchHeaders[key];
    }
  }
}

response = await fetcher(currentUrl.toString(), {
  method: currentRequest.method,
  headers: fetchHeaders,
  body: bodyInit,
  // ...existing options
});

I used a copy because currentRequest.headers gets reused on redirects and passed to response hooks.

With this on Node.js 24.18.0:

• The POST reproduction reaches upstream and returns HTTP 200.
• git clone --depth 1 https://github.com/octocat/Hello-World.git exits 0.
• The streamed-body path is untouched.
• onResponse still sees the original measured content-length (5 in the POST case).

What I'd suggest

Drop the explicit content-length for buffered bodies before calling Fetch, and keep it for bodyStream. Downgrading Node works around it, but it only hides the duplicate-header problem rather than fixing it.