Skip to content

fix(vgs/http): preserve duplicate response headers such as Set-Cookie - #703

Open
ekluev wants to merge 1 commit into
verygoodsecurity:masterfrom
ekluev:master
Open

fix(vgs/http): preserve duplicate response headers such as Set-Cookie#703
ekluev wants to merge 1 commit into
verygoodsecurity:masterfrom
ekluev:master

Conversation

@ekluev

@ekluev ekluev commented Aug 25, 2026

Copy link
Copy Markdown

Problem

VGSHttpResponse keeps headers in a plain Starlark dict:

# larky/src/main/resources/vgs/http/response.star
def add_header(key, val):
    self.headers[key] = val          # L26
...
    self.headers = {}                # L37
    for key, value in headers.items():
        self.add_header(key, value)  # L39  <- duplicates are dropped here

Any header that legitimately appears more than once in a response is collapsed to a single
value before a Larky script is ever invoked.

Set-Cookie is the case that matters in practice. RFC 6265 §3 requires one Set-Cookie
header per cookie and explicitly forbids folding them into one comma-separated line, so an
origin that sets N cookies emits N headers. A RESPONSE-phase filter that reads and writes
input.headers — the normal way to rewrite Domain= / Path= / Secure when proxying an
application onto a different hostname — silently drops N-1 of them and the end user is
logged out.

VGSHttpRequest does not have this problem: it stores headers in VGSCIMultiDict
(vgs/http/request.star L52-58). So type(input.headers) is VGSCIMultiDict in the
REQUEST phase and dict in the RESPONSE phase, and only the request phase can round-trip
repeated headers.

This is #292. #497 fixed the request side; response.star has not been touched since it was
introduced in #244, so the issue is still open for responses.

Reproducer

load("@vgs//http/response", "VGSHttpResponse")

r = VGSHttpResponse(headers=[
    ("Set-Cookie", "a=1; Path=/"),
    ("Set-Cookie", "b=2; Path=/"),
])
print(r.headers)   # before: {"Set-Cookie": "b=2; Path=/"}  -- a=1 is gone

Fix

Use VGSCIMultiDict for VGSHttpResponse.headers too, behind a property, exactly as
VGSHttpRequest does. The container accepts a mapping, another multidict, or a sequence of
(key, value) pairs, so the caller can hand over repeated headers as pairs without any
further change.

Two follow-on changes come with it:

  • remove_header() is added. _set_body() has always called it to drop a stale
    Content-length (issue 16464, inherited from urllib), but the method was never defined
    on the response object — the call would have raised. It went unnoticed because the guard
    ran against a case-sensitive dict and the literal key "Content-length" practically
    never matches what an origin actually sends (Content-Length). With a case-insensitive
    container the branch becomes reachable, so the method has to exist. Side effect: rewriting
    input.body now really does drop the stale Content-Length, which is what the comment
    always said it should do.
  • append_header(), has_header(), get_header(), get_all_headers() are added so a
    filter can read and write multi-valued headers without reaching into the multidict
    directly.

add_header() keeps replace-all semantics — same as before this change, and same as
VGSHttpRequest.add_header().

Compatibility

  • input.headers = {...} still works: the setter wraps whatever it is given.
  • input.headers.items() / .keys() / .values() / in / [key] all keep working.
  • str(input.headers) now produces the same JSON-ish shape as the request phase.
  • One visible difference: for key in input.headers now yields a repeated key once per
    value, and input.headers[key] returns the first value. A filter that rebuilds headers
    with the naive for key in headers: new[key] = headers[key] loop will now emit N copies of
    the first value instead of one copy of the last. That loop was already lossy; it is now
    lossy in a way that is visible and fixable from the script (getall / items), and it
    behaves identically to what the REQUEST phase has done since Implement the Starlark index expression assignment operation #497.

Tests

larky/src/test/resources/vgs_tests/http/test_default_response.star gains coverage for:
duplicate Set-Cookie preservation and ordering, case-insensitive lookup, append_header,
add_header replace-all, remove_header (including the missing-key no-op), and
_set_body() dropping a stale Content-Length.

Run with:

mvn -pl larky -am test -Dtest=VGSLibTests -Dlarky.vgs_test=test_default_response.star

Note for the proxy side

This fixes the loss that happens inside Starlark. If the component that constructs
VGSHttpResponse already flattens the origin's headers into a single-valued map before
handing them over, it needs a matching one-line change to pass a list of (key, value)
pairs instead — which VGSCIMultiDict accepts as-is.

Closes #292 for the RESPONSE phase.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

When multiple headers have the same key, only one is preserved

1 participant