Skip to content

Docs: document all internal storage backend configuration options (with required markers) — example: Azure Blob Storage #5120

Description

@xdatanomad

Summary

The Internal Storage section of the configuration docs is missing a full reference of the properties each storage backend accepts. Today users have to read the plugin source to discover which fields exist, which are required, and which credential modes are supported (connection string, access key, SAS token, managed identity, workload identity, etc.).

Requested change

For every internal storage backend (Local, AWS S3, GCS, Azure Blob, MinIO, S3-compatible, etc.), add a reference table under kestra.storage.<backend> that lists:

  • Property name
  • Type
  • Whether it is required or optional
  • Default value (if any)
  • Short description
  • Which auth mode the property belongs to (e.g. static credentials vs. managed identity vs. workload identity)

Also document:

  • The value to set for kestra.storage.type to select the backend.
  • The minimum config needed to boot the backend.
  • The credential-resolution order (which property short-circuits which fallback).

Example — Azure Blob Storage

Selected via kestra.storage.type: azure. Config prefix: kestra.storage.azure

Property Type Required Description
endpoint string required Blob service endpoint (e.g. https://<account>.blob.core.windows.net).
container string required Container name used for internal storage.
connectionString string (secret) optional Full Azure Storage connection string (highest-priority auth).
sharedKeyAccountName string optional Storage account name (used with access key).
sharedKeyAccountAccessKey string (secret) optional Storage account access key.
sasToken string (secret) optional SAS token.
managedIdentityClientId string optional User/system-assigned MI client ID (falls back to AZURE_CLIENT_ID).
managedIdentityResourceId string optional MI resource ID.
workloadIdentityClientId string optional Entra app client ID for AKS workload identity.
additionallyAllowedTenants list optional Extra tenants for multi-tenant apps ("*" for any).

Minimum config:

kestra:
  storage:
    type: azure
    azure:
      endpoint: https://myaccount.blob.core.windows.net
      container: storage

Credential resolution order:

  1. connectionString
  2. sharedKeyAccountName + sharedKeyAccountAccessKey
  3. sasToken
  4. DefaultAzureCredential chain (env vars, managed identity, workload identity, Azure CLI…), honoring managedIdentityClientId / managedIdentityResourceId / workloadIdentityClientId / additionallyAllowedTenants when set.

Managed identity example (AKS workload identity):

kestra:
  storage:
    type: azure
    azure:
      endpoint: https://myaccount.blob.core.windows.net
      container: storage
      workloadIdentityClientId: <entra-app-client-id>

Why this matters

Enterprise users evaluating Kestra for production repeatedly ask "does this support managed identity?" and "what's the minimum config?" — both are answerable from source but not from the docs today.

Metadata

Metadata

Assignees

Labels

area/docsIssues related to documentation, plugin examples, blueprints, and guides

Type

Fields

No fields configured for Task.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions