full-loading page does not say which replace strategies each destination supports (test, ignore)

[ShreyasGS]

Documentation for #4334: full-loading page does not say which replace strategies each destination supports (test, ignore)

Closes #4334

What changed

Edited the existing page general-usage/full-loading.md. The sidebar is unchanged.

Its runnable code is in full-loading-snippets.py, which pytest executed during verification.

Sections: Performing a full load, Choosing the correct replace strategy for your full load.

How this was checked

All claims checked against the checkout (dlt installed from source in docs/.venv).

Source read:

• dlt/common/destination/utils.py::resolve_replace_strategy — applies capabilities.replace_strategies_selector (if any) to capabilities.supported_replace_strategies, returns None when the list is empty or when required_strategy is not in it, otherwise required_strategy or supported_replace_strategies[0]. So the default is the first entry of the (possibly narrowed) list, not a hardcoded truncate-and-insert.
• dlt/destinations/utils.py::verify_schema_replace_disposition raises SchemaCorruptedException("Requested replace strategy ... not available for table ..."); dlt/destinations/job_client_impl.py::prepare_load_table asserts on the same condition. Either way the load step fails — there is no fallback.
• Per-table narrowing exists only in dlt/destinations/impl/filesystem/factory.py::filesystem_replace_strategies_selector and dlt/destinations/impl/athena/factory.py::athena_replace_strategies_selector: delta/iceberg tables → ["insert-from-staging"], all other tables → ["truncate-and-insert"].
• Runtime-only restriction quoted for ClickHouse: dlt/destinations/impl/clickhouse/clickhouse.py::_verify_database_supports_exchange raises DestinationTerminalException unless the database engine is Atomic or Shared.

Capabilities dumped from the branch (<factory>().capabilities().supported_replace_strategies), showing truncate-and-insert is always first where present: postgres/bigquery/snowflake/mssql/databricks/clickhouse have all three; duckdb, motherduck, redshift, synapse, dremio, sqlalchemy, athena, filesystem have truncate-and-insert + insert-from-staging; qdrant, weaviate, lancedb only truncate-and-insert.

Snippet run under the docs pytest setup → 1 passed, printing

['truncate-and-insert', 'insert-from-staging']
truncate-and-insert
insert-from-staging
None

black --check, mypy --config-file ../pyproject.toml and flake8 --max-line-length=200 on the snippet file all pass.

Failure behaviour reproduced: with DESTINATION__REPLACE_STRATEGY=staging-optimized on duckdb, pipeline.run([{"id": 1}], table_name="items", write_disposition="replace") raised PipelineStepFailed at step=load and the items table did not exist afterwards (Catalog Error: Table with name items does not exist!). The same request on filesystem with insert-from-staging for a regular table also failed at the load step.

What I could not confirm

The user-visible exception for an unsupported strategy differs by code path: in my duckdb and filesystem runs the loader hit the AssertionError in prepare_load_table (via the truncate filter in dlt/load/utils.py::init_client, which runs before job_client.verify_schema) rather than the SchemaCorruptedException from verify_schema_replace_disposition that the issue quotes. Because the message a user sees is not stable between those two, the page says only that the load step fails with an error naming the table. That ordering may be worth a look on the engineering side.

The snippet uses dlt.common.destination.utils.resolve_replace_strategy and PreparedTableSchema, which are not part of the documented public API (the loader and tests/load/test_job_client.py use the same function). If you would rather not point users at them, the destination-level caps.supported_replace_strategies line alone still answers most of the question.

I did not verify the staging-optimized behaviour of any warehouse destination end to end (no credentials), so nothing in the existing per-destination bullet list was changed.

---

Draft opened by agentic-docs from the labelled issue.

Verification ran dlt's linters, the link and anchor checker and the style rules over this page. Its examples are in full-loading-snippets.py, which pytest executed here and runs again in dlt's CI.

[cloudflare-workers-and-pages]

Deploying with    Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status	Name	Latest Commit	Preview URL	Updated (UTC)
✅ Deployment successful! View logs	docs	bf70e43	Commit Preview URL Branch Preview URL	Aug 10 2026, 10:11 PM

[ShreyasGS]

name the exception this raises, and say which destinations support only
truncate-and-insert — lance is the one I know of

[ShreyasGS]

Named the exception and added the destinations restricted to truncate-and-insert, in the last two paragraphs of "Which strategies your destination supports".

On the exception: I ran it rather than reading it off the source, because the code has two error paths and only one of them is reachable. verify_schema_replace_disposition in dlt/destinations/utils.py builds a SchemaCorruptedException ("Requested replace strategy ... not available for table ..."), but init_client computes the truncate list through should_truncate_table_before_load → prepare_load_table before it calls verify_schema, and prepare_load_table asserts on the resolved strategy. So what a user actually sees is PipelineStepFailed at step=load wrapping AssertionError: Must be able to get replace strategy for <table>. Verified on duckdb (replace_strategy = "staging-optimized"), on filesystem, and on filesystem with table_format="delta" — the same AssertionError in all three; I never saw SchemaCorruptedException surface. That is what the page now says; I did not mention SchemaCorruptedException since it is unreachable in these runs.

On the destinations: truncate-and-insert only is lance, lancedb, qdrant, and weaviate — I enumerated capabilities().supported_replace_strategies for every destination in dlt.destinations.__all__. dummy and the @dlt.destination reverse-ETL destination declare None (no replace support at all), which I left out as it is not the same case.

[ShreyasGS]

add a line showing how to set replace_strategy per resource or per pipeline, if
that is possible at all — the toml block only shows the global setting

[ShreyasGS]

Per pipeline is possible, per resource is not. Added a second toml block with [destination.duckdb] and [github_issues.destination], a snippet passing replace_strategy to the destination factory, and a sentence saying there is no per-resource setting. Confirmed in the checkout: replace_strategy lives on DestinationClientDwhConfiguration (dlt/common/destination/client.py:313); dlt.resource(replace_strategy=...) raises TypeError: resource() got an unexpected keyword argument 'replace_strategy'. With that config.toml, a pipeline named github_issues on duckdb resolved to staging-optimized while another pipeline resolved to insert-from-staging from the [destination.duckdb] section.

[ShreyasGS]

Added the missing scoping to the "Choosing the correct replace strategy" section: a config.toml block showing [destination.duckdb] (per destination type) and [github_issues.destination] (per pipeline), a new runnable snippet passing replace_strategy to the destination factory, and one sentence stating that there is no per-resource setting.

To answer the question directly: per pipeline is possible, per resource is not. replace_strategy is a field of DestinationClientDwhConfiguration (dlt/common/destination/client.py:313), so it is resolved from the destination config sections or from an explicit factory argument. dlt.resource() takes no replace_strategy argument, and only merge has a per-resource strategy (TMergeDispositionDict in dlt/common/schema/typing.py); the only per-table variation for replace is the capability narrowing already documented below.

[ShreyasGS]

/revise the new section names the destinations that support truncate-and-insert
only. Also name the ones that support truncate-and-insert and insert-from-staging
but not staging-optimized — I think filesystem, dremio, redshift and sqlalchemy,
but check.

[ShreyasGS]

Added the grouping asked for, as a three-item list at the end of Which strategies your destination supports. Your guess was right as far as it went but the group is larger: I enumerated capabilities().supported_replace_strategies for every factory in dlt.destinations on this branch, and the destinations that declare truncate-and-insert and insert-from-staging but not staging-optimized are Athena, Dremio, DuckDB, DuckLake, Fabric, filesystem, MotherDuck, Redshift, SQLAlchemy and Synapse. The truncate-and-insert-only list (lance, LanceDB, Qdrant, Weaviate) is unchanged, and I added the complement — BigQuery, ClickHouse, Databricks, MS SQL, Postgres, Snowflake — so the three groups cover every built-in destination and a reader does not have to infer the third from the other two. Note MS SQL, ClickHouse and Databricks do declare staging-optimized even though the staging-optimized section only describes Postgres, BigQuery and Snowflake behaviour; I left that section as it is, pointing at the destination pages.