Skip to content

Allow configurable STS session duration instead of hardcoded 15 minutes #143

Description

@Serpan07

Summary

The session duration is effectively hardcoded to 15 minutes, which cannot be overridden.

This limitation causes failures for long-running operations such as exports.

Problem

When an export process exceeds 15 minutes:

  • The STS credentials expire
  • DuckDB does not refresh them
  • The final upload (e.g. via dbt) fails with a 400 Bad Request due to expired credentials

Expected behavior

It should be possible to configure the STS session duration when using AssumeRole, for example:

  • via a parameter in the profile (e.g. session_duration)
  • or via environment variables
  • or by exposing this setting through DuckDB configuration

Current behavior

  • Session duration is fixed (~15 minutes)
  • No way to override or extend it
  • No automatic credential refresh

Proposed solution

  • Expose a configuration option to control AssumeRole session duration
  • Pass this value to the underlying AWS SDK credential provider
  • Optionally support automatic credential refresh for long-running queries

Possible implementation hints

  • Modify usage of STSAssumeRoleCredentialsProvider
  • Allow setting DurationSeconds when calling AssumeRole
  • Ensure compatibility with existing authentication flows

Workaround

Currently, the only workaround is to:

  • Perform AssumeRole outside DuckDB
  • Inject temporary credentials before running dbt/DuckDB commands

Impact

This issue impacts any workflow involving:

  • Long-running exports
  • Large datasets
  • dbt pipelines using DuckDB + S3

Additional context

This limitation also adds complexity for setups already working around missing IRSA support in DuckDB.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions