Skip to content

Additional object fields - #1

Merged
jschlyter merged 19 commits into
mainfrom
additional_object_fields
May 12, 2026
Merged

Additional object fields#1
jschlyter merged 19 commits into
mainfrom
additional_object_fields

Conversation

@jschlyter

@jschlyter jschlyter commented Apr 19, 2026

Copy link
Copy Markdown
Member

Replace hostname argument with a generic list of key/value pairs for S3 object names, e.g.

--fields provider=xyzzy,site=got,hostname=xyzzy.example.com

Keep support for environment variable S3PUSHER_HOSTNAME as hostname for backwards compability.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR generalizes S3 object key generation by replacing the single hostname component with a configurable set of key/value fields, while retaining backwards compatibility via S3PUSHER_HOSTNAME.

Changes:

  • Replace --hostname/hostname usage with --fields and an object_kvs dict used when building the S3 object key.
  • Add parsing for S3PUSHER_FIELDS and map S3PUSHER_HOSTNAME into object_kvs["hostname"].
  • Update documentation and bump tooling/dependencies in uv.lock (plus a small VS Code Ruff setting tweak).

Reviewed changes

Copilot reviewed 3 out of 4 changed files in this pull request and generated 7 comments.

File Description
s3pusher.py Switches object key construction to include arbitrary fields and updates CLI/env parsing.
README.md Updates usage/docs to describe --fields and new object key format.
uv.lock Updates locked versions for dependencies/tools (e.g., boto3/botocore/ruff).
.vscode/settings.json Enables Ruff formatter preview setting.
Comments suppressed due to low confidence (1)

s3pusher.py:65

  • boto3.client("s3") is created twice inside the same upload attempt (once before the contextvars block and again inside it). This is redundant and adds overhead; create it once and reuse the same client for the upload.
                s3_client = boto3.client("s3")
                with structlog.contextvars.bound_contextvars(s3_bucket=self.bucket, s3_object_key=s3_object_key):
                    self.logger.debug("Uploading file")
                    s3_client = boto3.client("s3")
                    t1 = time.time()

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread s3pusher.py
Comment thread s3pusher.py Outdated
Comment thread s3pusher.py Outdated
Comment thread s3pusher.py Outdated
Comment thread s3pusher.py
Comment thread README.md
Comment thread README.md Outdated

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 3 out of 4 changed files in this pull request and generated 6 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread s3pusher.py
Comment thread s3pusher.py Outdated
Comment thread s3pusher.py Outdated
Comment thread README.md
Comment thread s3pusher.py Outdated
Comment thread s3pusher.py Outdated

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 3 out of 4 changed files in this pull request and generated 2 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread s3pusher.py
Comment thread s3pusher.py Outdated
jschlyter and others added 3 commits April 19, 2026 13:34
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Signed-off-by: Jakob Schlyter <jakob@kirei.se>
@jschlyter
jschlyter merged commit 048f469 into main May 12, 2026
1 check passed
@jschlyter
jschlyter deleted the additional_object_fields branch May 12, 2026 14:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants