Restructure example notebooks and add cross-table merge support - #45
Merged
Conversation
Split the monolithic examples/vowl_usage_patterns_demo.ipynb (91 cells) into three focused, self-contained notebooks under their own folders: - 1_core_tutorial/ setup, running a validation, understanding results - 2_multiple_sources/ validating one contract across multiple sources - 3_real_databases/ server-side validation with Testcontainers Each notebook resolves the shared dataset paths on its own (repo-root walk-up), imports what it needs, and writes generated artifacts to a local outputs/ folder. Section numbering/titles are scoped per notebook, and doc links in README.md and docs/usage-patterns.md are updated to the new paths. Also lands the cross-table-merge annotated-output work: a subquery-projected referential check now merges onto its anchor table instead of becoming a residue (src/vowl/validation/result.py), with expanded tests and expected outputs. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Apply `prettier --write` to the three files the lint CI job flagged: table column-padding and `*emphasis*` → `_emphasis_` normalization. No content changes. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Mark get_consolidated_output_dfs() and output_mode="failed_rows"/"both" as deprecated, steering users to get_annotated_output() / output_mode="annotated". - get_consolidated_output_dfs() now emits a DeprecationWarning and delegates to a private _get_consolidated_output_dfs() so internal callers (save() in failed_rows/both mode) reuse the grouping without warning. - save() warns on the deprecated paths: implicit default (upcoming flip to "annotated"), explicit "failed_rows", and the failed-rows half of "both". "annotated" stays silent. All warnings use stacklevel=2. - ValidationConfig.output_mode docstring notes the default will change. - Tests assert the warnings fire and that the private helper stays silent; internal/golden callers switched to the private helper. - CHANGELOG Deprecated entry; README/getting-started/known-issues updated. - Notebook TOC anchor-link fixes. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Rename 1_core_tutorial/core_tutorial.ipynb to 1_basic_tutorial/basic_tutorial.ipynb and move the two denser sections (Explicitly Defined Adapter incl. PooledAdapter, and Filtering Rows Before Validation) into a new 4_advanced_usage/advanced_usage.ipynb so the basic tutorial stays focused on the everyday workflow. - Renumber the basic tutorial sections and fix its Contents/anchors - Link the "cross-table check that merges" note out to known-issues instead of duplicating the mechanics - Update cross-references in README, docs, examples/README, and the multiple-sources / real-databases notebooks - Replace non-ASCII typographic symbols (em-dashes, arrows, ellipses) with ASCII equivalents in both notebooks - Re-execute both notebooks top-to-bottom so outputs and execution counts are consistent Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Clarify which non-mergeable checks produce residues vs. appear only in summary.json, and replace em-dash asides with plainer punctuation. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Delete and regenerate the outputs/ artifacts for the basic tutorial and multiple-sources notebooks so nothing is stale. Re-executing both notebooks top-to-bottom also removes an orphaned join-output CSV that the current multi-source run no longer produces. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Remove the "Consolidated Failed Rows (per table)" section, which demonstrates the deprecated get_consolidated_output_dfs() accessor. The annotated output is now the recommended per-table view. Re-execute the notebook so execution counts stay contiguous. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Remove the get_consolidated_output_dfs() row from the ValidationResult method table and trim it from the save() deprecation note, matching its removal from the basic tutorial. The docs (getting-started, known-issues) keep their deprecation notices for users still on the legacy accessor. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Three related changes to the examples and annotated-output feature:
1. Example notebook restructuring
The monolithic
examples/vowl_usage_patterns_demo.ipynb(91 cells, ~357 KB) grew unwieldy. Split it into four focused, self-contained notebooks, each in its own folder:1_basic_tutorial/basic_tutorial.ipynbValidationResult, annotated output)2_multiple_sources/multiple_sources.ipynb3_real_databases/real_databases.ipynb4_advanced_usage/advanced_usage.ipynbPooledAdapter) and filtering rows before validation1_basic_tutorial/basic_tutorial.ipynb). Its two denser sections — explicit adapters (incl.PooledAdapter) and row filtering — moved into the new4_advanced_usage/notebook so the basic tutorial stays focused on the everyday workflow.save()calls write to a per-notebook localoutputs/folder; pre-generated reference artifacts moved there viagit mv(history preserved). A clean delete-and-regenerate flushed one stale orphan CSV the current multi-source run no longer produces.../docs→../../docsnow that notebooks sit one level deeper) and repaired 3 corrupted source cells from the original notebook.README.md,docs/usage-patterns.md, and rewroteexamples/README.md.2. Cross-table merge for annotated output
A subquery-projected referential check now merges onto its anchor table's
check_infocolumn instead of becoming a residue (src/vowl/validation/result.py), since its failed-rows query projects only the anchor table's columns. Bare-JOIN referential checks still correctly become residues. Includes expanded tests and expected outputs, and a clarifyingdocs/known-issues.mdupdate. The basic tutorial links out to the known-issues explanation rather than duplicating the mechanics.3. Deprecate consolidated failed-rows output
get_consolidated_output_dfs()and theoutput_mode="failed_rows"/"both"save modes (the legacy consolidated failed-rows CSVs) are now deprecated in favour of annotated output (get_annotated_output()/output_mode="annotated").get_consolidated_output_dfs()emits aDeprecationWarningand delegates to a private_get_consolidated_output_dfs(), so internal callers (save()infailed_rows/bothmode) reuse the grouping without warning.save()warns on the deprecated paths: the implicit default (which will flip to"annotated"in a future minor release), explicit"failed_rows", and the failed-rows half of"both"."annotated"stays silent. All warnings usestacklevel=2so they point at the caller.ValidationConfig.output_modedocstring notes the upcoming default change. No behaviour changes yet — existing calls still work, they just warn.get_consolidated_output_dfs()reference fromREADME.md, since annotated output is the recommended per-table view.docs/getting-started.mdanddocs/known-issues.mdkeep their deprecation notices for users still on the legacy accessor.CHANGELOG.mdgains aDeprecatedentry.Test plan
pytest tests/test_annotated_output.py tests/test_check_reference_variations.py tests/test_usage_patterns.py tests/test_readme_examples.py→ 237 passed, 17 skipped (skips are Docker/Java-gated)DeprecationWarningfires for each deprecated entry point and that the private_get_consolidated_output_dfs()helper stays silentbasic_tutorial.ipynb,advanced_usage.ipynb, andmultiple_sources.ipynbend-to-end via nbconvert — no error cells, contiguous execution counts; outputs regenerate into the correctoutputs/foldersruff check,ruff format --check,prettier --check "**/*.md")real_databases.ipynbrequires Docker (not run in this environment)🤖 Generated with Claude Code