Summary
The session duration is effectively hardcoded to 15 minutes, which cannot be overridden.
This limitation causes failures for long-running operations such as exports.
Problem
When an export process exceeds 15 minutes:
- The STS credentials expire
- DuckDB does not refresh them
- The final upload (e.g. via dbt) fails with a
400 Bad Request due to expired credentials
Expected behavior
It should be possible to configure the STS session duration when using AssumeRole, for example:
- via a parameter in the profile (e.g.
session_duration)
- or via environment variables
- or by exposing this setting through DuckDB configuration
Current behavior
- Session duration is fixed (~15 minutes)
- No way to override or extend it
- No automatic credential refresh
Proposed solution
- Expose a configuration option to control
AssumeRole session duration
- Pass this value to the underlying AWS SDK credential provider
- Optionally support automatic credential refresh for long-running queries
Possible implementation hints
- Modify usage of
STSAssumeRoleCredentialsProvider
- Allow setting
DurationSeconds when calling AssumeRole
- Ensure compatibility with existing authentication flows
Workaround
Currently, the only workaround is to:
- Perform
AssumeRole outside DuckDB
- Inject temporary credentials before running dbt/DuckDB commands
Impact
This issue impacts any workflow involving:
- Long-running exports
- Large datasets
- dbt pipelines using DuckDB + S3
Additional context
This limitation also adds complexity for setups already working around missing IRSA support in DuckDB.
Summary
The session duration is effectively hardcoded to 15 minutes, which cannot be overridden.
This limitation causes failures for long-running operations such as exports.
Problem
When an export process exceeds 15 minutes:
400 Bad Requestdue to expired credentialsExpected behavior
It should be possible to configure the STS session duration when using
AssumeRole, for example:session_duration)Current behavior
Proposed solution
AssumeRolesession durationPossible implementation hints
STSAssumeRoleCredentialsProviderDurationSecondswhen callingAssumeRoleWorkaround
Currently, the only workaround is to:
AssumeRoleoutside DuckDBImpact
This issue impacts any workflow involving:
Additional context
This limitation also adds complexity for setups already working around missing IRSA support in DuckDB.