data_type contract semantic change: update schema contracts documentation

[ShreyasGS]

Documentation for #4194: data_type contract semantic change: update schema contracts documentation

Closes #4194

What changed

• general-usage/data-quality-lifecycle.md — edited
• general-usage/schema-evolution.md — edited
• general-usage/schema.md — edited
• tutorial/advanced-course.md — edited (3 attempts)
• intro.md — edited

The sidebar is unchanged: every page here already existed.

Named by the issue, and judged to need no change

The agent read these and decided they were already correct. That is a judgement, not a check — worth confirming:

• general-usage/resource.md — Also touched by overhaul support for Pydantic data validation #3572. Its only contract text is the 'Put a contract on tables, columns, and data' section (line 71), which describes freeze generically and links to schema-contracts.md#setting-up-the-contract; the other flagged lines (83, 113, 120, 166) are data_type column hints in examples, not statements about the contract. Nothing there is made wrong by the change.
• general-usage/schema-contracts.md — Already updated by overhaul support for Pydantic data validation #3572 and correct. Line 25 reads: data_type "is applied when a data type property of an existing column changes. This includes variant columns ... as well as explicit changes to data_type, nullable, precision, scale, or timezone on a column that is already complete", and the Arrow/Pandas/Polars section (line 217) repeats the same property list. That matches Schema.apply_schema_contract in dlt/common/schema/schema.py, including timezone. Nothing on the page still says the contract is variant-only.

How this was checked

Third attempt on tutorial/advanced-course.md only: the sentence is now split in two, with no em dash and no semicolon. Content unchanged from the verified claim.

What I could not confirm

• general-usage/schema-evolution.md line 134 (Refer to our **[documentation]...**) contains two U+00A0 non-breaking spaces in the current file. I cannot emit that character, so they come back as ordinary spaces in the submitted text. Renders identically, but it shows up as a whitespace-only change on a line I was not asked to touch.
• While rewriting the last paragraph of schema-evolution.md I also corrected the entity and mode names in the same sentence: it said entities data_types and modes discard_rows/discard_columns, none of which exist. The real names are data_type and discard_row/discard_value (verified from TSchemaContractDict / TSchemaEvolutionMode).
• I left the neighbouring row in data-quality-lifecycle.md ("Set your schema to frozen") alone; frozen is not a real mode name (freeze is), but that row is about the columns contract and outside this change.
• tutorial/advanced-course.md only links out to the Lesson 7 notebook (docs/education/dlt-advanced-course/lesson_7_data_contracts.*). I updated the page's description; the notebook content itself is not a documentation page and I did not check or change it — worth a look if it demonstrates data_type: freeze.

---

Draft opened by agentic-docs from the labelled issue.

Verification ran dlt's linters, the link and anchor checker and the style rules over all 5 pages. The code is in fenced blocks: type-checked, but never executed by CI. The agent ran it while writing — see above.

Note

Revised since this description was written. Everything above describes
the first draft — in particular How this was checked and What I could not
confirm, which may refer to text that has changed since. The reply on each
round says what it did and why.

• Round 1 — general-usage/schema.md

[cloudflare-workers-and-pages]

Deploying with    Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status	Name	Latest Commit	Preview URL	Updated (UTC)
✅ Deployment successful! View logs	docs	9617b6b	Commit Preview URL Branch Preview URL	Aug 13 2026, 12:22 PM

[ShreyasGS]

Reviewed against dlt/common/schema/typing.py and Schema.apply_schema_contract. The vocabulary is right, including timezone, and the corrections to data_types -> data_type and discard_rows/discard_columns -> discard_row/discard_value are real fixes — those were wrong before. Three comments inline: two changes and one question.

[ShreyasGS]

"on a column that already has a data_type" is narrower than the code. The condition in apply_schema_contract is not is_new_column and not is_variant and existing_col — so a pre-existing column that had hints but no data_type and now gains one also trips the contract, because existing_col.get(prop) is None and differs.

Please reword to describe the rule as it is: the contract governs a type-property change on any column that already existed and is not a variant. Do not overstate it either — if the precise boundary is awkward to phrase, say what holds and leave the corner case out rather than guessing.

[ShreyasGS]

Reworded: the note now says the contract governs a change to one of those properties on a column that already exists in the table and is not a variant, without the "already has a data_type" qualifier.

On the specific case you raised — I ran it and a pre-existing column with hints but no data_type does not trip the data_type contract. is_new_column = not existing_col or not is_complete_column(existing_col), and is_complete_column requires a name and a data_type (dlt/common/schema/utils.py:403), so such a column is treated as new and falls under the columns contract instead. With {"columns": "evolve", "data_type": "freeze"}, {'name':'c','nullable':False} gaining data_type: bigint passed without raising. What does raise is a property change on an already complete column, including a property that was absent before (precision, timezone) or a flipped nullable — all DataValidationError. I left that boundary out of the page as the awkward corner, per your last sentence.

[ShreyasGS]

This makes Lesson 7's description about twice as long as every other lesson's, which breaks the rhythm of the index. The page is a course contents list, not a reference.

Please cut it to one short clause pointing at the contract rules, and let schema-contracts.md carry the property list.

[ShreyasGS]

Cut. Lesson 7 is now the original two sentences plus one clause: "See schema contracts for what each contract mode covers." The property list stays in schema-contracts.md.

[ShreyasGS]

Question, not a change request: why is this line in the diff at all? It looks byte-identical to me and nothing in the issue asks for it. What actually differs here?

[ShreyasGS]

Not byte-identical — the old line used non-breaking spaces (U+00A0) around the bold link: Refer to our\xa0**[documentation](./schema-contracts)**\xa0for details. The branch has ordinary U+0020 there. It was incidental, from the line being retyped rather than edited, and nothing in the issue asks for it. It renders the same and no line wrapping depends on it, but say the word and I will restore the two nbsp characters so the line drops out of the diff.

[ShreyasGS]

Applied the two changes and answered the question.

• schema.md — reworded the note so the rule is stated as it holds: with data_type set to freeze, dlt raises when a variant column is created, and when one of the type properties changes on a column that already exists in the table and is not a variant. Dropped the "already has a data_type" phrasing.
• advanced-course.md — Lesson 7 is back to its original two sentences plus one short pointer: "See schema contracts for what each contract mode covers." The property list now lives only in schema-contracts.md.
• schema-evolution.md line 134 — left unchanged; explanation in the thread.

One thing worth flagging on the first point. I checked the case you described against the branch and it does not trip the contract: is_new_column = not existing_col or not is_complete_column(existing_col), and is_complete_column requires both a name and a data_type (dlt/common/schema/utils.py:403). So a pre-existing column that has hints but no data_type counts as new, the not is_new_column guard short-circuits, and it falls under the columns contract rather than data_type. Ran it to be sure:

existing {'name': 'c', 'nullable': False} -> incoming {'name': 'c', 'data_type': 'bigint', 'nullable': False}
with {"columns": "evolve", "data_type": "freeze"} -> no raise

What does trip it, and what I could not have said with the old wording, is a type property appearing or changing on an already complete column — nullable flipping, or precision/timezone being absent before and present now, since existing_col.get(prop) is None and differs. All of those raise DataValidationError. The new wording covers them without claiming the incomplete-column case, which I left out as the awkward corner.

[ShreyasGS]

/revise My earlier comment on general-usage/schema.md was wrong and your reply was right. I read the not is_new_column guard without reading how is_new_column is computed, so I asked you to loosen a sentence that was already correct. Please put the precision back.

The note now ends "on a column that already exists in the table and is not a variant". That is looser than the rule: is_new_column = not existing_col or not is_complete_column(existing_col), and is_complete_column requires both a name and a data_type. So a pre-existing column without a data_type counts as new and falls under the columns contract, exactly as you demonstrated. Your original phrasing was right.

Restore that sense. Word it however reads best — the constraint is that the sentence must not imply the data_type contract covers a column that has no data_type yet. Change nothing else on that page, and nothing on the other four.

[ShreyasGS]

Restored the precision on the data_type contract note in general-usage/schema.md. It now reads: the contract governs a change to those properties "on a non-variant column that already has a data_type in the table", followed by one sentence saying that a column present in the schema but without a data_type counts as new and falls under the columns contract instead. Nothing else on that page changed, and the other four pages are untouched.

Re-confirmed against the branch before rewording: apply_schema_contract computes is_new_column = not existing_col or not is_complete_column(existing_col) (dlt/common/schema/schema.py:254) and is_complete_column requires both a name and a data_type (dlt/common/schema/utils.py:403), so the data_type branch at line 283 is only reached for a column that already carries a data_type.

Ran it on the branch to check both sides of that boundary. A resource declaring columns={"b": {"nullable": True, "partition": True}} and never yielding b leaves {'nullable': True, 'partition': True, 'name': 'b'} in the schema — no data_type. Loading b afterwards with {"columns": "freeze", "data_type": "evolve"} raises DataValidationError with schema_entity == "columns"; with {"columns": "evolve", "data_type": "freeze"} the same load succeeds and b gains data_type: text. So the incomplete column is governed by columns, not data_type, as the note now says.