fix(client): handle multi-value default headers without overriding requests

[0x676e67]

Story Background

When sending multiple cookies, people often combine all cookies into a single header. But some servers might flag this as bot-like behavior, because browsers usually split multiple cookies into separate headers.

Solution

This PR fixes the issue by letting reqwest add headers using append instead of just insert (which would overwrite). Now, you can send requests with multiple cookie headers, just like browsers do. The change also keeps the original design logic.

Test Cases

Default Client Configuration

• The default request headers include: USER_AGENT: "default-agent", cookie: a=b, and cookie: c=d
• If you don’t set these headers in your request, the client will automatically add the default values.

Test Branches and Request Conditions

/1 Branch

• The request explicitly sets USER_AGENT: "my-custom-agent" and cookie: a=b, cookie: c=d
• Checks:
  • USER_AGENT is "my-custom-agent" (overrides the default)
  • There are two cookies: a=b and c=d
  • No other cookies

/2 Branch

• The request explicitly sets USER_AGENT: "my-custom-agent" and cookie: e=f, cookie: g=h
• Checks:
  • USER_AGENT is "my-custom-agent" (overrides the default)
  • There are two cookies: e=f and g=h (overrides the default)
  • No other cookies

/3 Branch

• The request doesn’t set USER_AGENT or cookie
• Checks:
  • USER_AGENT is "default-agent" (uses the default)
  • There are two cookies: a=b and c=d (uses the default)
  • No other cookies

/4 Branch

• The request doesn’t set USER_AGENT, but sets cookie: e=f, cookie: g=h
• Checks:
  • USER_AGENT is "default-agent" (uses the default)
  • There are two cookies: e=f and g=h (overrides the default)
  • No other cookies

---

Summary:
This test case checks how the client merges and overrides default headers. It makes sure:

• If you set a header in your request, it will override the default (like USER_AGENT).
• Cookie headers can be added multiple times, and custom cookies will override the default ones.
• If you don’t set a header, the client will add the default value for you.
• There are no duplicate or missing header values.

[seanmonstar]

Would you mind explaining briefly the problem in the description of the PR?

[0x676e67]

OK, I have summarized the test results of the repaired "replace headers" function.

[seanmonstar]

I think this changes the behavior (is that what you're trying to do?), because before this would prevent the default header from overwriting a header the user had set, by only checking if Vacant.

Hm, would this mean that if the default had several values for a key, only the first one is added, and the second iteration will see Occupied and not add it?

[0x676e67]

I think this changes the behavior (is that what you're trying to do?), because before this would prevent the default header from overwriting a header the user had set, by only checking if Vacant.

Hm, would this mean that if the default had several values for a key, only the first one is added, and the second iteration will see Occupied and not add it?

I ran some tests, and it does indeed change the original behavior (it only adds support for handling multiple values under the same header key). Default headers are filled in only when the user hasn’t defined that header key.

Based on the test cases, if a default header contains multiple values for the same key, all of those values will be added. I achieved this by cleverly reusing util::replace_headers.

reqwest/src/util.rs

Line 50 in 74e6f84

pub(crate) fn replace_headers(dst: &mut HeaderMap, src: HeaderMap) {

[0x676e67]

Hm, would this mean that if the default had several values for a key, only the first one is added, and the second iteration will see Occupied and not add it?

The behavior before the change was indeed like that.