Identify DuckDB in AWS API User-Agent via ClientConfiguration appId#171
Merged
Conversation
Tmonster
reviewed
Jul 16, 2026
Tmonster
left a comment
Tmonster
left a comment
Member
There was a problem hiding this comment.
Just wondering why we do not user GetDefaultUserAgent()?
| //! identifies this as an SDK call, so no extension-specific suffix is needed. The SDK | ||
| //! appends the app id verbatim. See aws-sdk-cpp ClientConfiguration::ComputeUserAgentString. | ||
| static string DuckDBAwsUserAgentAppId() { | ||
| return "duckdb/" + string(DuckDB::LibraryVersion()); |
Member
Author
There was a problem hiding this comment.
That is a non-static method on the config, so it would need config to be wired in the callstack. I think that might be appropriate with a rework, but for a bug fix this is I think the proper solution.
Member
Author
There was a problem hiding this comment.
Now I got the question, we can't use DBConfig::UserAgent, see above, and we could use GetDefaultUserAgent but: requires changes duckdb side, and more importantly '(' and ')' are not whitelisted, so depending on details they might become rendered as --, and then we can as well just skip them, and solve this properly.
Every AWS API call the extension makes (STS AssumeRole, SSO portal, RDS) previously went out with the bare aws-sdk-cpp default User-Agent, with nothing identifying DuckDB. Stamp ClientConfiguration::appId with "duckdb/<DuckDB::LibraryVersion()>" in BuildClientConfigWithCa(), and route the SSO client config through the same helper, so the SDK appends " app/duckdb/<version>" to the User-Agent of every client we build. The token matches DuckDB's own User-Agent product token (duckdb/<version>, GetDefaultUserAgent) that httpfs sends on regular HTTP/S3 calls. A user-configured app id (AWS_SDK_UA_APP_ID env var or sdk_ua_app_id profile key, which the ClientConfiguration constructor loads into appId) still wins. This is a purely programmatic per-client approach: no process-global env mutation, and no custom HttpClientFactory. It covers the clients this extension constructs and hands a config (STS AssumeRole, SSO, RDS). The credential-discovery providers that build their own default-config client internally (IMDS/instance-profile, STS web-identity, the default provider chain) are not covered, as the SDK exposes no per-config app id for them short of the env var or an HTTP-layer interceptor.
Member
|
Thanks! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Every AWS API call the extension makes (STS AssumeRole, SSO portal, RDS) previously went out with the bare aws-sdk-cpp default User-Agent, with nothing identifying DuckDB. Stamp ClientConfiguration::appId with "duckdb/DuckDB::LibraryVersion()" in BuildClientConfigWithCa(), and route the SSO client config through the same helper, so the SDK appends " app/duckdb/" to the User-Agent of every client we build.
The token matches DuckDB's own User-Agent product token (duckdb/, GetDefaultUserAgent) that httpfs sends on regular HTTP/S3 calls.
A user-configured app id (AWS_SDK_UA_APP_ID env
var or sdk_ua_app_id profile key, which the ClientConfiguration constructor loads into appId) still wins.
This is a purely programmatic per-client approach: no process-global env mutation, and no custom HttpClientFactory. It covers the clients this extension constructs and hands a config (STS AssumeRole, SSO, RDS). The credential-discovery providers that build their own default-config client internally (IMDS/instance-profile, STS web-identity, the default provider chain) are not covered, as the SDK exposes no per-config app id for them short of the env var or an HTTP-layer interceptor.
Example, before:
after:
(see
app/duckdb/v1.5.4appended)