Skip to content

feat(s3): identify Metaflow in the S3 client user agent #3323

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
goanpeca wants to merge 1 commit into
base: master
Choose a base branch
from
+14 −0
Draft

feat(s3): identify Metaflow in the S3 client user agent #3323

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 14 additions & 0 deletions metaflow/plugins/aws/aws_client.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,8 @@ def get_client(
"Could not import module 'boto3'. Install boto3 first."
)

from metaflow.version import metaflow_version

# Convert dictionary config to Config object if needed
if "config" in client_params and not isinstance(
client_params["config"], Config
Expand All @@ -49,6 +51,18 @@ def get_client(
config.retries = S3_CLIENT_RETRY_CONFIG
client_params["config"] = config

if module == "s3":
# Identify Metaflow to any S3-compatible endpoint, appending to (not
# replacing) any user agent the caller already configured.
config = client_params.get("config", Config())
ua = "metaflow/%s" % metaflow_version
existing_ua = config.user_agent_extra
if not existing_ua:
config.user_agent_extra = ua
elif ua not in existing_ua.split():
config.user_agent_extra = "%s %s" % (existing_ua, ua)
client_params["config"] = config

if AWS_SANDBOX_ENABLED:
# role is ignored in the sandbox
global cached_aws_sandbox_creds
Expand Down