Skip to content

feat: select a storage provider at runtime - #12

Merged
kauandotnet merged 1 commit into
mainfrom
agent/provider-agnostic-drivers
Aug 8, 2026
Merged

feat: select a storage provider at runtime#12
kauandotnet merged 1 commit into
mainfrom
agent/provider-agnostic-drivers

Conversation

@kauandotnet

Copy link
Copy Markdown
Contributor

Why

@nestm/storage shipped one provider factory: createS3StorageDriver. Any application that wanted to run on GCS, Azure, R2, MinIO, or a plain filesystem had to import files-sdk/<provider> itself and wrap the adapter through the bridge — which puts provider SDK types back in the consumer, the exact leak this package exists to prevent.

That also means a provider cannot be named at runtime. An application shipping to more than one environment has to hard-code a driver per backend and choose between them with its own switch.

What

@nestm/storage/files-sdk/providercreateProviderStorageDriver({ provider, config, prefix }) takes the slug as data and resolves that provider's adapter, and only that one, through a lazy import. A deployment selects its store with an environment variable and installs a single native SDK.

config is one flat bag of provider settings (bucket/region for an object store, root for the filesystem, accountName/container for Azure); each provider reads what it needs and ignores the rest, so the shape survives a provider change. The caller's own driver options — prefix, hooks, plugins, readonly, retries — still apply, because the client loadFiles builds is discarded and the bridge builds its own over the resolved adapter.

The same entry point exposes the provider catalog as pure data — listStorageProviders, getStorageProvider, listStorageProviderEnvVars, listStorageProviderSecretEnvVars, isStorageProvider — so startup validation, health checks, and config UIs can read a provider's env contract without loading an adapter. An unknown slug fails closed with INVALID_ARGUMENT before anything is imported.

@nestm/storage/files-sdk/fscreateFsStorageDriver, mirroring the S3 factory. The adapter reaches only node:fs, so unlike every object-store provider it adds no native SDK to an install.

@nestm/storage/files-sdk/s3 — the S3 capability set is split out of createS3StorageDriver as withS3Capabilities(base, options), so the provider factory can keep conditional-copy promotion and the signed policies for the s3 slug without re-deriving S3AdapterOptions from flat config. The S3-compatible wrappers keep only what they declare themselves. EnhancedS3Adapter is renamed S3StorageAdapter and exported; it was previously private.

Notes

  • No behavior change to createS3StorageDriver — it now composes the extracted helper. Its existing spec passes untouched.
  • files-sdk types stay inside the files-sdk bridge subtree, matching how s3/index.ts already re-exports S3AdapterOptions.

Verification

pnpm run check (oxlint, prettier, tsc), pnpm run test — 49 unit + 22 e2e, pnpm run verify:pack — build, publint --strict, packed-consumer check. All green.

New coverage: filesystem round-trip and prefix scoping, provider resolution by name, driver options reaching the resolved adapter, readonly honored for any provider, the s3 slug keeping its capabilities, a provider that declares no conditional copy not claiming one, unknown-slug rejection, and the catalog surface.

Add `@nestm/storage/files-sdk/provider`, which builds a driver from a provider
slug carried as data and imports that provider's adapter — and only that one —
on demand. A deployment picks its store with an environment variable and
installs a single native SDK instead of hard-coding a driver per backend. The
same entry point exposes the provider catalog as pure data so config validation
and health checks can read a provider's env contract without loading an adapter.

Add `@nestm/storage/files-sdk/fs` for local filesystem storage, mirroring the S3
factory. Its adapter reaches only `node:fs`, so it adds no native SDK.

Split the S3 capability set out of `createS3StorageDriver` as
`withS3Capabilities`, so the provider factory can keep conditional-copy
promotion and the signed policies for the `s3` slug without re-deriving
`S3AdapterOptions` from flat provider config. `EnhancedS3Adapter` becomes
`S3StorageAdapter` and is now exported.
@kauandotnet
kauandotnet merged commit 79ad3af into main Aug 8, 2026
6 checks passed
@kauandotnet
kauandotnet deleted the agent/provider-agnostic-drivers branch August 8, 2026 06:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant