ATTACH 'arn:aws:s3tables:...' and infra for expanding to other resources#165
Open
carlopi wants to merge 2 commits into
Open
ATTACH 'arn:aws:s3tables:...' and infra for expanding to other resources#165carlopi wants to merge 2 commits into
carlopi wants to merge 2 commits into
Conversation
carlopi
force-pushed
the
attach-arn
branch
2 times, most recently
from
July 11, 2026 12:07
6effd43 to
c473277
Compare
Member
Author
|
Question from @Tmonster: what happens if I do: ? How are those treaded / conflicts & co. |
Tmonster
reviewed
Jul 16, 2026
Parse an AWS ARN from the verbatim ATTACH path (AttachedDatabase::
GetOriginalPath()) and dispatch by service to the backing storage backend
(s3tables -> iceberg). Lives in its own arn_storage.cpp; aws_extension.cpp
just calls ArnStorage::Register().
Registered under two storage type names, both backed by the same extension
instance:
- 'aws', for the explicit `ATTACH '<arn>' (TYPE aws)` form.
- 'arn', because core derives the db type from the 'arn:' path prefix, so a
bare `ATTACH '<arn>'` looks for a storage type of that name. Registering it
here serves that form without needing a core arn->aws extension alias.
Also include duckdb/logging/logger.hpp explicitly in aws_secret.cpp; it is no
longer pulled in transitively via duckdb.hpp.
Requires a duckdb with AttachedDatabase::GetOriginalPath().
Adapted from arn_attach/.github/patches/extensions/aws/0002-arn-storage-dispatch.patch.
… still pass through
Member
Author
|
Options are passed down correctly, with a throw on conflicts. One question I had is whether there was a better than hand parsing the arn, and there is none in the AWS's SDK and non API to do so. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Registers a pass-through:
that will recognize paths-like strings like
arn:aws:<service>:<region>:<AMI>:<type/<name>, and then either refuse them as not supported or redirect to the relevant attach.Redirect is implemented only for
arn:aws:s3tables:..., where it remaps to theiceberg-type attach on an ARN, but it's somewhat generalizable to other resources.This PR will allow for:
and allow a path for other Catalog-shaped resources that have an ARN to be attached referencing the ARN. Once #162 lands, that would also be a prime addition to this surface.
A follow up
duckdb/ducdkbside would be registering arn->aws remapping, allowing autoloading to happen, that makes so that if credentials are already on the user machine, the single statement version to also work:This is on top of #164, bumped to include the required duckdb/duckdb#23759