Fix page.route changing request headers on the wire

[pratyush618]

Related Issue

Closes #428
Closes #271

Description

page.route("**/*", lambda r: r.continue_()) changed what every request looked like on the wire, so anti-bot services blocked pages that load fine without the route. Both reports are the same root cause, and the block lands on the document request, before any page script runs — this is an HTTP-level tell, not a JS leak.

real Firefox:  Host UA Accept Accept-Language Accept-Encoding | Connection | Cookie | Sec-Fetch-* | Priority
with route:    Host UA Accept Accept-Language Accept-Encoding | Sec-Fetch-* | Connection | Cookie | Priority | Pragma | Cache-Control

Two independent causes:

1. Pragma: no-cache + Cache-Control: no-cache on every request. Playwright pairs every setRequestInterception with setCacheDisabled(true) (ffNetworkManager.ts, ffBrowser.ts), and juggler implements that as LOAD_BYPASS_CACHE | INHIBIT_CACHING. LOAD_BYPASS_CACHE makes necko attach both headers. Firefox only sends them for a forced reload, so every request announced itself as automated.

Dropping LOAD_BYPASS_CACHE and keeping INHIBIT_CACHING gives identical cache behaviour (verified: 6 network hits for a max-age=99999 subresource either way — nothing is stored, so nothing is read) with no wire footprint. Page.setCacheDisabled is only ever called as a side effect of interception, so no public API changes.

2. Connection and Cookie displaced past Sec-Fetch-*. Resuming an intercepted request makes necko tear down the channel and build a new one (InterceptedHttpChannel::ResetInterception). The header copy skips connection and cookie — the replacement channel is expected to regenerate them — which it does, appended at the end. Header order feeds JA4H/Akamai-style fingerprints, and the resulting layout is one no real Firefox emits.

Both are fixable in additions/juggler/ — no Firefox C++ patch needed, since the headers are still mutable at http-on-modify-request on the resumed channel.

Type of Change

• Bug fix

Testing

Reproduced and verified against the prebuilt binary from the issue report (official/135.0.1-beta.24), driven with a raw-socket server that logs the exact request bytes in arrival order, with juggler patched inside omni.ja.

This is inherited from upstream juggler, not a Camoufox regression — stock Playwright Firefox 134 shows the identical diff. Worth reporting upstream too.

After the fix, a routed request is byte-identical to an unrouted one across the document, CSS and JS subresources, with and without cookies, and with set_extra_http_headers.

Playwright suite (tests/) against the patched binary: 203 passed, 0 regressions. The 12 failures are pre-existing (HTTPS/service-worker/asset-path) and fail identically on the pristine binary.

New tests/async/test_route_request_fingerprint.py — 4 tests that compare a routed page against an unrouted one rather than hardcoding a header layout, so they survive Firefox upgrades. Confirmed they fail on the unfixed binary and pass on the fixed one:

######## PRISTINE (unfixed) ########
test_page_route_should_not_change_request_headers FAILED
test_page_route_should_not_change_request_headers_with_extra_http_headers FAILED
test_page_route_should_not_advertise_a_disabled_cache FAILED
test_page_route_should_not_reorder_connection_and_cookie FAILED
========================= 4 failed =========================

######## PATCHED (fixed) ########
========================= 4 passed =========================

Reproduction script and the before/after captures are in the issue thread on request.

Fingerprint Report

I could not produce a meaningful build-tester report. The only binary I have is 135.0.1-beta.24; the current build-tester scores it 0/0, Grade F — and scores the pristine, unmodified binary identically, so the harness is simply incompatible with a binary that old (this branch targets 152). It needs a run on a real 152 build before merge.

Fingerprint report

Patched:  Grade: F   Score: 0/0   Profiles: 8   Status: ALL PASS
Pristine: Grade: F   Score: 0/0   Profiles: 8   Status: ALL PASS

Notes for review

• Verified on FF135, repo targets FF152. The edited _onInternalRedirectReady hunk is textually identical across both versions and the helper ports verbatim, but this deserves one confirmation run on a 152 build.
• route.continue_(headers=...) still reorders headers — clearRequestHeaders pins cookie near the front, and any explicitly-passed header list replaces the layout wholesale. Different code path, left alone here to keep this focused; happy to open a separate issue.

Checklist

• I have linked a related issue above
• My changes are focused on a single logical change
• I have added testing instructions which include the desired result
• Service tests pass temporarily out of service
• Build test passes — see Fingerprint Report above; not runnable against a 135 binary, needs a 152 build