Use SWAR in cow_http1 header and path parsers

[eagle-head]

Applies the SWAR (SIMD Within A Register) technique from erlang/otp#10938 to four byte-by-byte parsers in cow_http1:

• parse_hd_name (header name, delimiter :, lowercases ASCII A–Z)
• parse_hd_value (header value, delimiter \r)
• parse_fullpath (URL path, delimiters # and ?)
• parse_fullpath_query (delimiter #)

Each scan now consumes 7 bytes per iteration on the plain-byte fast path and appends them in a single bit-string operation, replacing the previous byte-by-byte append pattern. The byte-by-byte clauses remain as a correct fallback for tails shorter than 7 bytes, for non-ASCII bytes, and for any chunk that contains a delimiter.

For parse_hd_name the bulk-append is not raw passthrough: the 7 bytes are ASCII-lowercased in parallel via a SWAR formula (add 0x3F, add 0x25, isolate the high-bit AND result, shift to bit 5, bor with the original word) so mixed-case header names still produce the same lowercase output as the per-byte ?LOWER slow path.

A new src/cow_swar.hrl holds the primitives (SWAR_MASK80, SWAR_MASK01, no_zero_byte, swar_lower), mirroring OTP's swar_ascii.hrl.

Performance — 20 interleaved runs of horse:mod_perf(cow_http1), medians in ms

Functions touched by SWAR:

Benchmark	Before (master)	After (PR)	Delta
parse_headers	81.3	60.7	-25.3%
parse_headers_lower *	78.5	62.2	-20.8%
parse_headers_mixed_case *	94.1	63.5	-32.6%
parse_fullpath_short	25.4	15.9	-37.2%
parse_fullpath_long	47.4	23.0	-51.5%
parse_fullpath_query	62.6	32.4	-48.3%
parse_fullpath_fragment	26.3	17.4	-34.1%

* new benchmarks introduced by this PR.

Functions outside the changed code (noise envelope):

Benchmark	Before (master)	After (PR)	Delta
parse_request_line_get_path	46.8	46.2	-1.1%
parse_status_line_200	2.3	2.2	-2.4%
parse_status_line_404	2.5	2.5	+0.9%
parse_status_line_500	2.5	2.4	-0.8%
parse_status_line_other	20.0	19.5	-2.2%

All untouched benchmarks move within ±2.5%, confirming the added predicates introduce no lateral regression.

Validation

• make eunit — 3192/3192 pass
• make proper — all properties pass
• make dialyze — clean

Six new horse benches added (two for parse_hd_name coverage, four for parse_fullpath).

Notes

parse_hd_name is included on @essen's suggestion in the review thread. The predicate checks all-ASCII, no :, no \s, no \t; on a match, the 7 bytes are bulk-lowercased and appended, otherwise the existing ?LOWER byte-by-byte path runs.

I tried cow_qs:urldecode/parse_qs first; that regressed 6 of 9 benchmarks because those loops already had a 4-byte AND-chain on a cheap 2-disjunction guard, where SWAR's heavier failed-guard cost outweighed the stride gain. The four sites in this PR differ in that the originals were either pure byte-by-byte with a <<>> accumulator (path/value/query) or used a 26-arm ?LOWER case dispatch (name), so SWAR delivers both a 7× stride and a 7-bytes-per-append win.

Refs: ninenines/cowboy#1711

[essen]

Thanks but not that the impact in Cowboy will be limited. Most of the parsing in Cowboy happens inline in cowboy_http.

[essen]

Why not in parse_hd_name?

I was also thinking SWAR may be a good enabler to be stricter about the allowed characters in header name and values (strictly follow ABNF).

[eagle-head]

Thanks — both points addressed below. The PR description and commit message are updated to reflect the changes.

Thanks but not that the impact in Cowboy will be limited. Most of the parsing in Cowboy happens inline in cowboy_http.

Agreed. After tracing cowboy_http.erl, the production hot paths are the inline parse_hd_name/parse_hd_value/parse_uri_path/parse_uri_query, not the cow_http1 exports (which serve Gun and offline use). I'll open a follow-up PR against ninenines/cowboy that ports the same SWAR predicates from cow_swar.hrl to those four sites once this PR is in a mergeable state — same fast/slow split, same fallback discipline.

Why not in parse_hd_name?

Done — parse_hd_name is now in this PR. The reason it was skipped initially: ?LOWER does a per-byte transform, not just a predicate test, so a naive bulk-append of W would leave A–Z uppercase. The fix uses the predicate to check the 7-byte window for no :, no \s, no \t, then bulk-lowercases the word in parallel via the standard ASCII lowercase formula (add 0x3F, add 0x25, isolate the high-bit AND result, shift to bit 5, bor with W). On a match the 7 bytes are appended already lowercase; otherwise the existing ?LOWER slow path runs.

Measured impact on 20 interleaved horse:mod_perf(cow_http1) runs (medians, ms):

Benchmark	Before (master)	After (PR)	Delta
parse_headers	81.3	60.7	-25.3%
parse_headers_lower	78.5	62.2	-20.8%
parse_headers_mixed_case	94.1	63.5	-32.6%

make eunit 3192/3192, make proper and make dialyze remain clean. parse_headers itself went from −16.7% (value-only) to −25.3% with parse_hd_name added.

I was also thinking SWAR may be a good enabler to be stricter about the allowed characters in header name and values (strictly follow ABNF).

This is the natural next step. The relevant grammar lives in RFC 9110 — field-name = token, token = 1*tchar (§5.1, §5.6.2), and field-value = *field-content over field-vchar = VCHAR / obs-text (§5.5) — and the SWAR predicate is the right enforcement point. Once it tests "all bytes are valid tchar" (for names) or "all bytes are VCHAR / SP / HTAB" (for values), strict ABNF compliance happens for free on the fast path, and the slow path becomes the place to emit a 400 on invalid bytes. I'd rather sequence that as a separate PR after this one, because it's a behavior change — rejecting bytes that current Cowboy/cowlib accepts — and deserves its own test surface and changelog entry. If you'd prefer it bundled here instead, say so and I'll restructure.

Plan:

1. This PR (Use SWAR in cow_http1 header and path parsers #156) — SWAR for parse_hd_name/parse_hd_value/parse_fullpath/parse_fullpath_query, no behavior change. (now)
2. Follow-up PR in ninenines/cowboy — SWAR in the four inline parsers in cowboy_http.erl (parse_hd_name/parse_hd_value/parse_uri_path/parse_uri_query).
3. Separate PR — strict ABNF (RFC 9110 §5.1/§5.5/§5.6.2) enforcement via the SWAR predicate.

Let me know which direction you'd like — happy to restructure scope here (bundle the strict-ABNF work into this PR, or anything else you'd prefer differently) or hold and proceed with the sequencing above. Waiting on your call before I open the cowboy follow-up.

[eagle-head]

Gentle nudge on the scope question above — no rush if you're mid-release. Unless you'd prefer the strict-ABNF enforcement bundled in here, I'll keep this PR scoped as the no-behavior-change optimization (now including parse_hd_name) and do strict ABNF as a separate PR afterwards. I'll hold the ninenines/cowboy follow-up until this one is merged, as planned — just confirming the sequencing works for you.

[essen]

I will review this in details some time after today's release, haven't had time before.

[essen]

Yeah but this means there will be false positives.

[essen]

It is more of a naming issue than anything else though. It's fine to reject non-ascii most of the time.

[eagle-head]

Agreed, it's a naming issue. I renamed the predicate macros so the all-ASCII requirement is explicit: ?are_ascii_hd_name_swar, ?are_ascii_no_cr_swar, ?are_ascii_no_hash_qmark_swar, ?are_ascii_no_hash_swar. I also rewrote the ?no_zero_byte comment to be honest that dropping bnot V is safe only because every caller gates the macro behind W band ?SWAR_MASK80 =:= 0 andalso ..., so it only ever runs on all-ASCII words. The comment now states explicitly that a window with any non-ASCII byte (>= 0x80) fails the MASK80 guard and intentionally falls to the byte-by-byte slow path, even when it has no delimiter — a deliberate fast-path miss, not a defect. No behavior change; eunit (3192) and proper green.

[essen]

You do not need to do the respective change in Cowboy because I will do it as part of related changes.

[eagle-head]

Thanks, sounds good. This PR stays scoped as the no-behavior-change SWAR optimization (now also covering parse_hd_name). The naming + comment fix from your review is pushed: the four predicate macros are renamed to ?are_ascii_*_swar and the ?no_zero_byte comment now explains the all-ASCII guard and the intentional slow-path fallback for non-ASCII windows. The Cowboy port and the strict-ABNF/obs-text decision are yours to fold into the related changes.

[essen]

I think we can generalize. For example

-define(are_ascii_hd_name_swar(W),
	(W) band ?SWAR_MASK80 =:= 0 andalso
	?no_zero_byte((W) bxor 16#3A3A3A3A3A3A3A) andalso
	?no_zero_byte((W) bxor 16#20202020202020) andalso
	?no_zero_byte((W) bxor 16#09090909090909)
).

Can be

-define(is_safe_ascii_swar(W, A, B, C),
	(W) band ?SWAR_MASK80 =:= 0 andalso
	?no_zero_byte((W) bxor (?SWAR_MASK01 * A)) andalso
	?no_zero_byte((W) bxor (?SWAR_MASK01 * B)) andalso
	?no_zero_byte((W) bxor (?SWAR_MASK01 * C))
).

And called when ?is_safe_ascii_swar(W, $:, $\s, $\t) (name could be improved).

That would make the code a lot clearer.

Need to double check that the compiler does the calculation at compile time (it should) and that it works as intended. Also should double check that it works fine for \0. Then it's just a matter of having swar macros for 1 to N characters.

This kind of general SWAR macro could be used across Cowboy, Gun and Cowlib easily.

[eagle-head]

Done — pushed as an update to this PR (kept as a single, force-pushed commit). I implemented the generalization you proposed: the four delimiter-specific predicate macros are gone, replaced by one arity-overloaded ?is_safe_ascii_swar/2,3,4 in cow_swar.hrl, so the all-ASCII gate lives in one place and the delimiters are visible at each call site.

-define(is_safe_ascii_swar(W, A),
	(W) band ?SWAR_MASK80 =:= 0 andalso
	?no_zero_byte((W) bxor (?SWAR_MASK01 * (A)))).
-define(is_safe_ascii_swar(W, A, B),
	(W) band ?SWAR_MASK80 =:= 0 andalso
	?no_zero_byte((W) bxor (?SWAR_MASK01 * (A))) andalso
	?no_zero_byte((W) bxor (?SWAR_MASK01 * (B)))).
-define(is_safe_ascii_swar(W, A, B, C),
	(W) band ?SWAR_MASK80 =:= 0 andalso
	?no_zero_byte((W) bxor (?SWAR_MASK01 * (A))) andalso
	?no_zero_byte((W) bxor (?SWAR_MASK01 * (B))) andalso
	?no_zero_byte((W) bxor (?SWAR_MASK01 * (C)))).

Call sites:

parse_hd_name(...)        when ?is_safe_ascii_swar(W, $:, $\s, $\t) -> ...
parse_hd_value(...)       when ?is_safe_ascii_swar(W, $\r) -> ...
parse_fullpath(...)       when ?is_safe_ascii_swar(W, $#, $?) -> ...
parse_fullpath_query(...) when ?is_safe_ascii_swar(W, $#) -> ...

On your three checks:

(a) Compile-time multiply — confirmed. ?SWAR_MASK01 * $C is constant-folded. The generated BEAM assembly contains no * BIF for the broadcast (the only two gc_bif '*' in the module are the pre-existing status-line *100/*10 digit multiplies), and each folded 56-bit constant appears directly as the bxor operand — e.g. $: → 16#3A3A3A3A3A3A3A (16389570408626746).

(b) Works as intended — confirmed by bytecode identity. Compiling cow_http1 before vs. after, the assembly is byte-for-byte identical once {line,...}/file metadata is stripped (same md5; the four target function bodies are instruction-identical), so the generalized macro emits exactly the same code as the old hex literals — a pure readability/reuse change with no behavioral or typing impact. make eunit is 3192/3192. I also ran a differential check that defines the old and new macros side by side and compares them over all same-byte words, every delimiter at every position, boundary bytes, and millions of random words — 0 mismatches.

(c) \0 — fine. A NUL byte in the input is < 0x80, so it passes the MASK80 gate, and 0x00 bxor (broadcast of a non-zero delimiter) is non-zero, so NUL is consumed by the fast path exactly as the byte-by-byte slow path would (old =:= new on every NUL case). If 0 were ever passed as a delimiter, ?SWAR_MASK01 * 0 = 0 and the term degenerates to ?no_zero_byte(W), i.e. "reject any window containing a NUL" — correct by construction, though none of the four sites use it.

Naming is your call — I went with is_safe_ascii_swar since it reads as a predicate with the delimiters inline, but are_ascii_without_swar / is_ascii_excluding_swar work just as well; whatever you prefer applies identically to all three arities and the four call sites. The macro now lives in cow_swar.hrl, so it's ready to reuse across Cowboy and Gun as you mentioned. The Cowboy port and the strict-ABNF/obs-text decision remain yours.

[essen]

It does well while parsing, but I'm wondering how it compares to e.g binary:match(B, [<<$\r>>, <<$\n>>]). Perhaps in some cases a binary:match can be replaced with SWAR, or binary:match itself can be specialised when looking specifically for characters like this.

[essen]

Slop. Should have asked binary:match against a pure scan. For this case we don't need to accumulate data or anything just need to know there's a match.

[eagle-head]

You're right — the first benchmark was bogus. It had SWAR scanning and accumulating into a new binary while binary:match only located the match, so it measured copy cost, not the scan. Redone as a pure scan: every variant only returns the offset of the first delimiter, nothing is built. OTP 28, single scheduler, 21 reps, medians in ns/op, outputs verified identical.

Locate first CR (single delimiter — the parse_hd_value case):

len (B)	SWAR	binary:match (raw)	binary:match (precompiled)	byte
8	14.9	71.8	27.7	25.4
16	20.9	96.5	27.7	42.4
32	31.3	83.0	28.2	64.4
256	105.9	88.7	30.7	384
2048	661	87.9	40.5	2928

Locate first CR or LF (two delimiters — your literal example; same regime as parse_fullpath on #/?):

len (B)	SWAR	binary:match (raw)	binary:match (precompiled)	byte
8	16.0	336.8	38.8	25.0
64	42.6	374.5	79.7	115.8
256	123.8	487.4	190.3	416
2048	764.8	1528.5	1223.2	3268

Reading:

• Multi-delimiter scan: SWAR wins at every size (1.5–2.5x over precompiled binary:match). Multi-pattern binary:match (Aho-Corasick) pays more per byte, while SWAR tests N delimiters in one word op. This is the parse_fullpath (#/?) and parse_hd_name (:/SP/HT) regime — and parse_hd_name also needs the ?swar_lower transform, which binary:match can't do at all.
• Single-delimiter scan: crossover. SWAR is faster for short values (≲ ~24 bytes: 15 vs 28 ns); precompiled binary:match is near-flat (~28–40 ns) and wins on long values (256 B → 3.4x, 2 KB → 16x). That's the parse_hd_value case, so it depends on typical value length — short values favor SWAR, large ones (dates, User-Agent, big cookies) favor binary:match.
• Raw, non-precompiled binary:match is never competitive for the multi-pattern case (~340 ns fixed per-call compile).

Corrected picture: SWAR is the better tool for the multi-delimiter sites and for short single-delimiter scans; precompiled binary:match only clearly wins for parse_hd_value on long values. So one option is to keep SWAR for parse_fullpath/parse_hd_name and switch only parse_hd_value to a precompiled binary:match (with a length guard if the short-value case matters), or leave the PR as-is — your call. Happy to share the standalone benchmark module if you want to reproduce.