feat(emulate): add fetch header ordering strategy - #105
Draft
XiaoYouChR wants to merge 2 commits into
Draft
Conversation
XiaoYouChR
force-pushed
the
feat
branch
2 times, most recently
from
July 24, 2026 03:41
0ccaeef to
b822c94
Compare
XiaoYouChR
marked this pull request as draft
July 24, 2026 03:44
Owner
|
Headers like |
Owner
|
I’ll probably move this auto-sorting feature over to |
XiaoYouChR
force-pushed
the
feat
branch
2 times, most recently
from
July 24, 2026 04:41
abad4cf to
bd3e747
Compare
Contributor
Author
|
Moving the sorting into The |
Contributor
Author
|
Header name constants added in the latest commit. |
XiaoYouChR
marked this pull request as ready for review
July 24, 2026 05:05
Contributor
Author
|
Chromium source references for the ① JS
② Blink
③ HashMap → network serialization
④ Network layer appends
|
Contributor
Author
|
Here's how Chromium assembles fetch headers, mapped to wreq's current responsibilities:
For fixed headers (①④–⑧), |
Owner
|
I’ve written a similar wrapper-style prototype abstraction before, but I haven’t had time to optimize it yet. Feel free to take a look. If it’s workable, you can integrate it into //! HTTP header types and wire-order policies.
use std::sync::Arc;
pub use http::header::*;
use rapidhash::v1::{DEFAULT_RAPID_SECRETS, rapidhash_v1};
/// The Chrome request path whose common header order should be modeled.
#[derive(Clone, Copy, Debug, Eq, PartialEq)]
#[non_exhaustive]
pub enum ChromeRequestType {
/// An address-bar, link, or document navigation.
Navigation,
/// A `fetch()`, XHR, or ordinary subresource request.
Fetch,
}
/// Controls the order in which request headers enter Chromium.
///
/// [`Self::chrome`] models the common Chromium 150 browser pipeline observed
/// for headers already present in the request and treats recognized browser
/// fields as browser-generated. Use [`Self::chrome_fetch_builder`] to identify
/// Fetch fields supplied by the application or script. Neither policy
/// synthesizes browser headers or their values. Ordering is per field name;
/// Chromium can still add or transform fields while encoding the request.
#[derive(Clone, Debug, Default, Eq, PartialEq)]
pub struct HeaderOrder {
kind: HeaderOrderKind,
}
/// Builds a Chrome Fetch/XHR header-order policy from existing script fields.
///
/// The builder copies only ambiguous field names from an application or script
/// [`HeaderMap`]. Header values stay in the original map and the final order is
/// calculated lazily after client defaults and request fields have been
/// merged. Ambiguous fields added after the policy is built are not inferred
/// as author fields.
#[derive(Debug, Default)]
#[must_use = "a Chrome header-order builder has no effect until build is called"]
pub struct ChromeFetchHeaderOrderBuilder {
author_headers: Vec<HeaderName>,
}
#[derive(Clone, Debug, Default, Eq, PartialEq)]
enum HeaderOrderKind {
#[default]
HeaderMap,
Chrome {
request_type: ChromeRequestType,
author_headers: Option<Arc<[HeaderName]>>,
},
Preferred(Arc<[HeaderName]>),
}
impl HeaderOrder {
/// Uses the order exposed by [`HeaderMap`].
///
/// `HeaderMap` does not guarantee a particular iteration order. Use
/// [`Self::chrome`] or [`Self::preferred`] when the wire order matters.
#[must_use]
pub const fn header_map() -> Self {
Self {
kind: HeaderOrderKind::HeaderMap,
}
}
/// Models Chrome 150's common order for one browser request path.
///
/// Recognized browser fields are placed in their browser-managed stages.
/// For Fetch/XHR fields explicitly supplied by JavaScript, use
/// [`Self::chrome_fetch_with_author_headers`].
#[must_use]
pub const fn chrome(request_type: ChromeRequestType) -> Self {
Self {
kind: HeaderOrderKind::Chrome {
request_type,
author_headers: None,
},
}
}
/// Starts a Chrome Fetch/XHR policy builder.
///
/// Use the builder when recognized browser fields, such as `Accept` or
/// `Cache-Control`, were instead supplied by the application or script.
pub const fn chrome_fetch_builder() -> ChromeFetchHeaderOrderBuilder {
ChromeFetchHeaderOrderBuilder::new()
}
/// Models a Chrome 150 Fetch/XHR request with named script headers.
///
/// Most custom fields, including `Content-Type`, are already recognized as
/// author headers. List fields here when the same name can instead be
/// generated by Chrome, such as `Accept`, `Accept-Language`,
/// `Cache-Control`, `Pragma`, legacy client hints, or `Priority`.
#[must_use]
pub fn chrome_fetch_with_author_headers(names: impl IntoIterator<Item = HeaderName>) -> Self {
Self::chrome_fetch_builder()
.author_header_names(names)
.build()
}
/// Places listed names first and unlisted names afterward in lexical order.
///
/// Duplicate names in `names` are ignored after their first occurrence.
#[must_use]
pub fn preferred(names: impl IntoIterator<Item = HeaderName>) -> Self {
let names = deduplicated_names(names);
Self {
kind: HeaderOrderKind::Preferred(names.into()),
}
}
pub(crate) fn ordered_names<'a>(&self, headers: &'a HeaderMap) -> Vec<&'a HeaderName> {
match &self.kind {
HeaderOrderKind::HeaderMap => headers.keys().collect(),
HeaderOrderKind::Chrome {
request_type: ChromeRequestType::Navigation,
..
} => chrome_navigation_order(headers),
HeaderOrderKind::Chrome {
request_type: ChromeRequestType::Fetch,
author_headers,
} => chrome_fetch_order(headers, author_headers.as_deref().unwrap_or_default()),
HeaderOrderKind::Preferred(names) => preferred_order(headers, names),
}
}
}
impl ChromeFetchHeaderOrderBuilder {
/// Creates a Chrome Fetch/XHR policy builder.
pub const fn new() -> Self {
Self {
author_headers: Vec::new(),
}
}
/// Marks one field as supplied by the application or script.
pub fn author_header(mut self, name: HeaderName) -> Self {
self.push_author_header(name);
self
}
/// Marks owned field names as supplied by the application or script.
pub fn author_header_names(mut self, names: impl IntoIterator<Item = HeaderName>) -> Self {
for name in names {
self.push_author_header(name);
}
self
}
/// Marks fields in `headers` as supplied by the application or script.
///
/// Pass the map before browser-managed fields are added. Only ambiguous
/// names are copied; custom fields already take the author-header path.
/// The caller can then move the same map into
/// [`crate::RequestBuilder::headers`] without cloning any values.
pub fn author_headers(mut self, headers: &HeaderMap) -> Self {
for name in headers.keys() {
if is_fetch_author_override_candidate(name) && !self.author_headers.contains(name) {
self.author_headers.push(name.clone());
}
}
self
}
/// Builds the ordering policy for the recorded author-field set.
#[must_use]
pub fn build(self) -> HeaderOrder {
let author_headers = if self.author_headers.is_empty() {
None
} else {
Some(self.author_headers.into())
};
HeaderOrder {
kind: HeaderOrderKind::Chrome {
request_type: ChromeRequestType::Fetch,
author_headers,
},
}
}
fn push_author_header(&mut self, name: HeaderName) {
if is_fetch_author_override_candidate(&name) && !self.author_headers.contains(&name) {
self.author_headers.push(name);
}
}
}
const NETWORK_FIXED_PREFIX: &[&str] = &["content-length", "transfer-encoding"];
const FETCH_BROWSER_PREFIX: &[&str] = &["pragma", "cache-control"];
const NAVIGATION_NETWORK_PREFIX: &[&str] = &[
"content-length",
"transfer-encoding",
"pragma",
"cache-control",
];
// This is the SetHttpHeaderField order in FrameFetchContext for Chromium 150.
const CLIENT_HINTS: &[&str] = &[
"device-memory",
"sec-ch-device-memory",
"rtt",
"downlink",
"ect",
"sec-ch-ua",
"sec-ch-ua-mobile",
"sec-ch-ua-arch",
"sec-ch-ua-platform",
"sec-ch-ua-platform-version",
"sec-ch-ua-model",
"sec-ch-ua-full-version",
"sec-ch-ua-full-version-list",
"sec-ch-ua-bitness",
"sec-ch-ua-wow64",
"sec-ch-ua-form-factors",
"save-data",
"sec-ch-prefers-reduced-transparency",
"sec-ch-prefers-reduced-motion",
"sec-ch-prefers-color-scheme",
"dpr",
"sec-ch-dpr",
"viewport-width",
"sec-ch-viewport-width",
"sec-ch-viewport-height",
"width",
"sec-ch-width",
];
const FETCH_SUFFIX: &[&str] = &[
"accept",
"origin",
"sec-fetch-site",
"sec-fetch-mode",
"sec-fetch-user",
"sec-fetch-dest",
"sec-fetch-storage-access",
"sec-fetch-frame-top",
"sec-fetch-frame-ancestors",
"referer",
"accept-encoding",
"accept-language",
"cookie",
"priority",
];
const NAVIGATION_BROWSER_HEADERS: &[&str] = &[
"dnt",
"upgrade-insecure-requests",
"user-agent",
"origin",
"accept",
];
const NAVIGATION_SUFFIX: &[&str] = &[
"sec-fetch-site",
"sec-fetch-mode",
"sec-fetch-user",
"sec-fetch-dest",
"sec-fetch-storage-access",
"sec-fetch-frame-top",
"sec-fetch-frame-ancestors",
"referer",
"accept-encoding",
"accept-language",
"cookie",
"priority",
];
fn chrome_fetch_order<'a>(
headers: &'a HeaderMap,
author_overrides: &[HeaderName],
) -> Vec<&'a HeaderName> {
let mut network_prefix = [None; NAVIGATION_NETWORK_PREFIX.len()];
let mut author_names = Vec::new();
let mut blink_managed = [None; CLIENT_HINTS.len() + 2];
let mut suffix = [None; FETCH_SUFFIX.len()];
// Classify each field once. Overrides take the Blink path even when
// Chrome can also generate the same field at a later stage.
for name in headers.keys() {
if let Some(index) = name_position(NETWORK_FIXED_PREFIX, name) {
network_prefix[index] = Some(name);
} else if contains_header(author_overrides, name) {
author_names.push(name);
} else if let Some(index) = name_position(FETCH_BROWSER_PREFIX, name) {
network_prefix[NETWORK_FIXED_PREFIX.len() + index] = Some(name);
} else if let Some(index) = name_position(CLIENT_HINTS, name) {
blink_managed[index] = Some(name);
} else if name.as_str() == "user-agent" {
blink_managed[CLIENT_HINTS.len()] = Some(name);
} else if name.as_str() == "dnt" {
// RenderFrameImpl finalizes DNT after Blink has installed the UA.
blink_managed[CLIENT_HINTS.len() + 1] = Some(name);
} else if let Some(index) = name_position(FETCH_SUFFIX, name) {
suffix[index] = Some(name);
} else {
author_names.push(name);
}
}
// FetchHeaderList is name-sorted before Blink copies it into HTTPHeaderMap.
author_names.sort_unstable_by(|left, right| left.as_str().cmp(right.as_str()));
let mut blink_input = author_names;
blink_input.extend(blink_managed.into_iter().flatten());
let mut output = Vec::with_capacity(headers.keys_len());
output.extend(network_prefix.into_iter().flatten());
output.extend(blink_hash_table_order(blink_input));
output.extend(suffix.into_iter().flatten());
output
}
fn chrome_navigation_order(headers: &HeaderMap) -> Vec<&HeaderName> {
let mut network_prefix = [None; NAVIGATION_NETWORK_PREFIX.len()];
let mut client_hints = [None; CLIENT_HINTS.len()];
let mut renderer_headers = Vec::new();
let mut browser_headers = [None; NAVIGATION_BROWSER_HEADERS.len()];
let mut suffix = [None; NAVIGATION_SUFFIX.len()];
for name in headers.keys() {
if let Some(index) = name_position(NAVIGATION_NETWORK_PREFIX, name) {
network_prefix[index] = Some(name);
} else if let Some(index) = name_position(CLIENT_HINTS, name) {
client_hints[index] = Some(name);
} else if let Some(index) = name_position(NAVIGATION_BROWSER_HEADERS, name) {
browser_headers[index] = Some(name);
} else if let Some(index) = name_position(NAVIGATION_SUFFIX, name) {
suffix[index] = Some(name);
} else {
renderer_headers.push(name);
}
}
// Renderer-provided navigation fields arrive as a serialized Blink map
// before the browser process appends navigation-specific fields.
renderer_headers.sort_unstable_by(|left, right| left.as_str().cmp(right.as_str()));
let mut output = Vec::with_capacity(headers.keys_len());
output.extend(network_prefix.into_iter().flatten());
output.extend(client_hints.into_iter().flatten());
output.extend(blink_hash_table_order(renderer_headers));
output.extend(browser_headers.into_iter().flatten());
output.extend(suffix.into_iter().flatten());
output
}
fn preferred_order<'a>(headers: &'a HeaderMap, names: &[HeaderName]) -> Vec<&'a HeaderName> {
let mut preferred = vec![None; names.len()];
let mut remaining = Vec::new();
for name in headers.keys() {
if let Some(index) = names.iter().position(|preferred| preferred == name) {
preferred[index] = Some(name);
} else {
remaining.push(name);
}
}
remaining.sort_unstable_by(|left, right| left.as_str().cmp(right.as_str()));
let mut output = Vec::with_capacity(headers.keys_len());
output.extend(preferred.into_iter().flatten());
output.extend(remaining);
output
}
fn name_position(names: &[&str], name: &HeaderName) -> Option<usize> {
names
.iter()
.position(|candidate| *candidate == name.as_str())
}
fn contains_header(names: &[HeaderName], name: &HeaderName) -> bool {
names.contains(name)
}
fn is_fetch_author_override_candidate(name: &HeaderName) -> bool {
name_position(FETCH_BROWSER_PREFIX, name).is_some()
|| name_position(CLIENT_HINTS, name).is_some()
|| matches!(name.as_str(), "user-agent" | "dnt")
|| name_position(FETCH_SUFFIX, name).is_some()
}
fn deduplicated_names(names: impl IntoIterator<Item = HeaderName>) -> Vec<HeaderName> {
let mut unique = Vec::new();
for name in names {
if !unique.contains(&name) {
unique.push(name);
}
}
unique
}
fn blink_hash_table_order(names: Vec<&HeaderName>) -> Vec<&HeaderName> {
if names.is_empty() {
return names;
}
let mut scratch = Vec::new();
let hashes = names
.iter()
.map(|name| blink_header_hash(name, &mut scratch))
.collect::<Vec<_>>();
let mut table = Vec::new();
let mut count = 0_usize;
for index in 0..names.len() {
if table.is_empty() {
table.resize(8, None);
}
insert_bucket(&mut table, &hashes, index);
count += 1;
if count.saturating_mul(2) >= table.len()
&& let Some(new_size) = table.len().checked_mul(2)
{
rehash(&mut table, &hashes, new_size);
}
}
table
.into_iter()
.flatten()
.map(|index| names[index])
.collect()
}
fn blink_header_hash(name: &HeaderName, scratch: &mut Vec<u8>) -> usize {
// Blink's case-folding reader expands Latin-1 to UTF-16 while hashing. A
// one-character name takes its specialized two-byte ReadSmall path, which
// is not equivalent to hashing materialized UTF-16LE bytes. All longer
// valid header names use reads that are byte-for-byte equivalent.
let hash = if name.as_str().len() == 1 {
blink_single_character_hash(name.as_str().as_bytes()[0].to_ascii_lowercase())
} else {
scratch.clear();
scratch.reserve(name.as_str().len().saturating_mul(2));
for byte in name.as_str().bytes() {
scratch.push(byte.to_ascii_lowercase());
scratch.push(0);
}
rapidhash_v1(scratch)
};
let hash = (hash & 0x00ff_ffff) as usize;
if hash == 0 { 0x0080_0000 } else { hash }
}
fn blink_single_character_hash(character: u8) -> u64 {
const UTF16_BYTE_LENGTH: u64 = 2;
let secrets = DEFAULT_RAPID_SECRETS;
let seed = secrets.seed ^ UTF16_BYTE_LENGTH;
let a = u64::from(character) ^ secrets.secrets[1];
let b = seed;
let product = u128::from(a).wrapping_mul(u128::from(b));
let (low, high) = split_u128(product);
rapid_mix(
low ^ secrets.secrets[0] ^ UTF16_BYTE_LENGTH,
high ^ secrets.secrets[1],
)
}
fn rapid_mix(a: u64, b: u64) -> u64 {
let product = u128::from(a).wrapping_mul(u128::from(b));
let (low, high) = split_u128(product);
low ^ high
}
fn split_u128(value: u128) -> (u64, u64) {
let bytes = value.to_le_bytes();
(
u64::from_le_bytes([
bytes[0], bytes[1], bytes[2], bytes[3], bytes[4], bytes[5], bytes[6], bytes[7],
]),
u64::from_le_bytes([
bytes[8], bytes[9], bytes[10], bytes[11], bytes[12], bytes[13], bytes[14], bytes[15],
]),
)
}
fn insert_bucket(table: &mut [Option<usize>], hashes: &[usize], index: usize) {
let mask = table.len() - 1;
let mut bucket = hashes[index] & mask;
let mut probe_count = 0_usize;
while table[bucket].is_some() {
probe_count += 1;
bucket = bucket.wrapping_add(probe_count) & mask;
}
table[bucket] = Some(index);
}
fn rehash(table: &mut Vec<Option<usize>>, hashes: &[usize], new_size: usize) {
let old_table = std::mem::replace(table, vec![None; new_size]);
for index in old_table.into_iter().flatten() {
insert_bucket(table, hashes, index);
}
}
#[cfg(test)]
mod tests {
use super::{ChromeRequestType, HeaderMap, HeaderName, HeaderOrder, HeaderValue};
fn headers(names: &[&'static str]) -> HeaderMap {
names
.iter()
.map(|name| {
(
HeaderName::from_static(name),
HeaderValue::from_static("value"),
)
})
.collect()
}
fn ordered(order: &HeaderOrder, headers: &HeaderMap) -> Vec<String> {
order
.ordered_names(headers)
.into_iter()
.map(ToString::to_string)
.collect()
}
#[test]
fn chromium_150_header_hashes_match_blink() {
let mut scratch = Vec::new();
for (name, expected) in [
("a", 6_579_038),
("x", 11_791_085),
("z", 12_880_985),
("x-alpha", 11_224_226),
("x-middle", 15_388_390),
("x-zeta", 3_286_528),
("sec-ch-ua", 3_403_785),
("sec-ch-ua-mobile", 10_176_653),
("sec-ch-ua-platform", 16_760_515),
("user-agent", 12_718_456),
] {
assert_eq!(
super::blink_header_hash(&HeaderName::from_static(name), &mut scratch),
expected
);
}
}
#[test]
fn fetch_order_matches_chrome_150_custom_header_capture() {
let headers = headers(&[
"x-zeta",
"x-alpha",
"sec-ch-ua-platform",
"x-middle",
"user-agent",
"sec-ch-ua",
"sec-ch-ua-mobile",
"accept",
"sec-fetch-site",
"sec-fetch-mode",
"sec-fetch-dest",
"referer",
"accept-encoding",
"accept-language",
"priority",
]);
assert_eq!(
ordered(&HeaderOrder::chrome(ChromeRequestType::Fetch), &headers),
[
"x-zeta",
"x-alpha",
"sec-ch-ua-platform",
"x-middle",
"user-agent",
"sec-ch-ua",
"sec-ch-ua-mobile",
"accept",
"sec-fetch-site",
"sec-fetch-mode",
"sec-fetch-dest",
"referer",
"accept-encoding",
"accept-language",
"priority",
]
);
}
#[test]
fn fetch_order_matches_chrome_150_single_character_capture() {
let headers = headers(&[
"pragma",
"cache-control",
"a",
"x",
"z",
"x-alpha",
"sec-ch-ua-platform",
"sec-ch-ua",
"sec-ch-ua-mobile",
"user-agent",
"accept",
"sec-fetch-site",
"sec-fetch-mode",
"sec-fetch-dest",
"referer",
"accept-encoding",
"accept-language",
"cookie",
"priority",
]);
assert_eq!(
ordered(&HeaderOrder::chrome(ChromeRequestType::Fetch), &headers),
[
"pragma",
"cache-control",
"x-alpha",
"sec-ch-ua-platform",
"sec-ch-ua",
"sec-ch-ua-mobile",
"x",
"user-agent",
"z",
"a",
"accept",
"sec-fetch-site",
"sec-fetch-mode",
"sec-fetch-dest",
"referer",
"accept-encoding",
"accept-language",
"cookie",
"priority",
]
);
}
#[test]
fn fetch_author_header_sources_match_chrome_150_capture() {
let request_headers = headers(&[
"accept",
"accept-language",
"cache-control",
"pragma",
"x-probe",
"sec-ch-ua-platform",
"sec-ch-ua",
"sec-ch-ua-mobile",
"user-agent",
"sec-fetch-site",
"sec-fetch-mode",
"sec-fetch-dest",
"referer",
"accept-encoding",
"cookie",
"priority",
]);
let author_headers = headers(&[
"accept",
"accept-language",
"cache-control",
"pragma",
"x-probe",
]);
let order = HeaderOrder::chrome_fetch_builder()
.author_headers(&author_headers)
.build();
assert_eq!(
ordered(&order, &request_headers),
[
"sec-ch-ua-platform",
"cache-control",
"accept-language",
"pragma",
"x-probe",
"sec-ch-ua",
"sec-ch-ua-mobile",
"user-agent",
"accept",
"sec-fetch-site",
"sec-fetch-mode",
"sec-fetch-dest",
"referer",
"accept-encoding",
"cookie",
"priority",
]
);
}
#[test]
fn fetch_dnt_order_matches_chrome_150_capture() {
let headers = headers(&[
"sec-ch-ua-platform",
"sec-ch-ua",
"sec-ch-ua-mobile",
"user-agent",
"dnt",
"accept",
"sec-fetch-site",
"sec-fetch-mode",
"sec-fetch-dest",
"referer",
"accept-encoding",
"accept-language",
"cookie",
"priority",
]);
assert_eq!(
ordered(&HeaderOrder::chrome(ChromeRequestType::Fetch), &headers),
[
"sec-ch-ua-platform",
"user-agent",
"sec-ch-ua",
"dnt",
"sec-ch-ua-mobile",
"accept",
"sec-fetch-site",
"sec-fetch-mode",
"sec-fetch-dest",
"referer",
"accept-encoding",
"accept-language",
"cookie",
"priority",
]
);
}
#[test]
fn fetch_author_overrides_cover_client_hints_and_priority() {
let headers = headers(&[
"sec-ch-ua-platform",
"ect",
"device-memory",
"sec-ch-ua",
"sec-ch-ua-mobile",
"save-data",
"user-agent",
"accept",
"rtt",
"priority",
"downlink",
"sec-fetch-site",
"sec-fetch-mode",
"sec-fetch-dest",
"referer",
"accept-encoding",
"accept-language",
"cookie",
]);
let order = HeaderOrder::chrome_fetch_builder()
.author_header_names(
[
"ect",
"device-memory",
"save-data",
"accept",
"rtt",
"priority",
"downlink",
]
.map(HeaderName::from_static),
)
.build();
assert_eq!(
ordered(&order, &headers),
[
"sec-ch-ua-platform",
"ect",
"device-memory",
"sec-ch-ua",
"sec-ch-ua-mobile",
"save-data",
"user-agent",
"accept",
"rtt",
"priority",
"downlink",
"sec-fetch-site",
"sec-fetch-mode",
"sec-fetch-dest",
"referer",
"accept-encoding",
"accept-language",
"cookie",
]
);
}
#[test]
fn fetch_order_matches_chrome_150_after_multiple_rehashes() {
let mut names = (0..20)
.map(|index| format!("x-probe-{index:02}"))
.collect::<Vec<_>>();
names.extend(
[
"sec-ch-ua",
"sec-ch-ua-mobile",
"sec-ch-ua-platform",
"user-agent",
]
.map(str::to_owned),
);
let mut headers = HeaderMap::new();
for name in &names {
headers.insert(
HeaderName::from_bytes(name.as_bytes()).expect("valid test header"),
HeaderValue::from_static("value"),
);
}
assert_eq!(
ordered(&HeaderOrder::chrome(ChromeRequestType::Fetch), &headers),
[
"sec-ch-ua-platform",
"sec-ch-ua",
"x-probe-08",
"sec-ch-ua-mobile",
"x-probe-13",
"x-probe-05",
"x-probe-02",
"x-probe-00",
"x-probe-15",
"x-probe-01",
"x-probe-07",
"x-probe-14",
"x-probe-17",
"x-probe-10",
"x-probe-09",
"x-probe-16",
"x-probe-06",
"x-probe-12",
"x-probe-19",
"x-probe-18",
"user-agent",
"x-probe-11",
"x-probe-04",
"x-probe-03",
]
);
}
#[test]
fn fetch_builder_deduplicates_ambiguous_names_and_ignores_custom_names() {
let author_headers = headers(&["accept", "x-probe"]);
let built = HeaderOrder::chrome_fetch_builder()
.author_headers(&author_headers)
.author_header(HeaderName::from_static("accept"))
.author_header(HeaderName::from_static("x-probe"))
.build();
assert_eq!(
built,
HeaderOrder::chrome_fetch_with_author_headers([HeaderName::from_static("accept")])
);
assert_eq!(
HeaderOrder::chrome_fetch_builder().build(),
HeaderOrder::chrome(ChromeRequestType::Fetch)
);
}
#[test]
fn navigation_order_matches_chrome_150_capture() {
let headers = headers(&[
"sec-ch-ua",
"sec-ch-ua-mobile",
"sec-ch-ua-platform",
"dnt",
"upgrade-insecure-requests",
"user-agent",
"accept",
"sec-fetch-site",
"sec-fetch-mode",
"sec-fetch-user",
"sec-fetch-dest",
"referer",
"accept-encoding",
"accept-language",
"cookie",
"priority",
]);
assert_eq!(
ordered(
&HeaderOrder::chrome(ChromeRequestType::Navigation),
&headers
),
[
"sec-ch-ua",
"sec-ch-ua-mobile",
"sec-ch-ua-platform",
"dnt",
"upgrade-insecure-requests",
"user-agent",
"accept",
"sec-fetch-site",
"sec-fetch-mode",
"sec-fetch-user",
"sec-fetch-dest",
"referer",
"accept-encoding",
"accept-language",
"cookie",
"priority",
]
);
}
#[test]
fn preferred_order_is_deduplicated_and_appends_unknown_names_stably() {
let headers = headers(&["x-zeta", "accept", "user-agent", "x-alpha"]);
let order = HeaderOrder::preferred([
HeaderName::from_static("user-agent"),
HeaderName::from_static("accept"),
HeaderName::from_static("user-agent"),
]);
assert_eq!(
ordered(&order, &headers),
["user-agent", "accept", "x-alpha", "x-zeta"]
);
}
} |
Contributor
Author
|
Thanks for sharing the prototype — the author-header classification and single-character hash handling are much more thorough than my approach. I'll look into integrating it into wreq. |
Owner
|
Let’s just see what can be optimized first. I feel integrating it into |
XiaoYouChR
marked this pull request as draft
July 24, 2026 12:00
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Implements the header-ordering strategy discussed in #104.
Strategyenum (Navigate/Fetch) toEmulationwtf_hashmodule: Chromium's CaseFolding rapidhash + quadratic-probing hash table simulationstrategyto produce correct order and valuesDefault is
Navigate, existing behavior unchanged.Navigate vs Fetch
std::vector)WTF::HashMap)text/html,...*/*navigatecors?1documentempty1u=0, iu=1, iHash function version
Chromium switched
WTF::StringHasherfrom SuperFastHash to rapidhash at Chrome 129 (string_hasher.h). Thewtf_hashmodule implements rapidhash, soStrategy::Fetchproduces exact header order for Chrome ≥ 129. Profiles for Chrome 100–128 use rapidhash as an approximation — header values are correct, only the Blink-layer order may differ slightly.Verified against
Chrome 150 + pingly.us.kg, 4 scenarios: default fetch, custom headers, 5 custom headers, POST with content-type.