US5 + US4, docs PR previews & Phase 8 polish (generation stays on Python 3.9)#2
Merged
Conversation
…24-T027) Generate the schema reference and Mermaid ERD from the enriched XSD so docs can't drift (FR-020/021/022, SC-008/SC-009). - schema_docs.py: parse the enriched XSD and render a generated Markdown reference — a Mermaid erDiagram of entities + relationships (with exact cardinalities), per-complex-type field tables carrying the xs:documentation prose (inheritance flattened and marked), and a banded-numeric-type catalogue with range facets. Deterministic/idempotent. - cli.py + Makefile: wire `gen-schema-docs` subcommand and `make gen-schema-docs`. - docs/reference/schema/index.md: the generated artifact (committed; regenerated by CI). - docs: delete the hand-drawn schema-erd.md stand-in and repoint all inbound links to the generated page; integrate it via a single static nav entry (no extra plugin needed). - tests/integration/test_schema_docs.py: entities present, xs:documentation prose carried, erDiagram emitted, inheritance flattened, idempotent, and committed page == fresh regen. - tasks.md/commands.md/onboarding.md: mark US5 done; remaining work is US4 (bundle + drift).
…et (T028-T030) Ship data + schema + models together and lock the bindings to the schema (FR-016/017). - bundle.py + cli `bundle` + `make bundle` (runs pipeline first): assemble schema + data + generated models into build/dist with a MANIFEST; refuse a partial bundle (exit non-zero) if any component is missing. Tests in tests/integration/test_bundle.py. - CI drift gate: a step in the existing Python 3.9 `verify` job regenerates models + schema docs and fails on any working-tree drift. No separate 3.11 job — generation stays on the 3.9 target. Keep generation reproducible on Python 3.9 (the whole point of the 3.9.4 pin): - Pin xsdata to ==25.7 (the last line supporting 3.9; 26.x needs >=3.10). - generate.py passes explicit --no-slots/--no-kw-only/--no-union-type/--no-postponed-annotations so output doesn't depend on the generating interpreter's version. - Regenerate the committed models on real CPython 3.9.4. xsdata's docstring wrapping differs by Python minor version, so models MUST be generated on 3.9.x (documented in ADR 0008). - The 25.7 models make fields Optional (no kw_only); handle the one Optional access in cli.py. Docs: record the toolchain decision in ADR 0008; mark US4 done (tasks/commands/onboarding).
IanMayo
changed the title
(a) Docs PR previews on GitHub Pages: - pr-preview.yml: build this PR's docs and publish a live preview to gh-pages under pr-preview/pr-<N>/ via rossjrw/pr-preview-action, posting a sticky comment with the link; torn down when the PR closes. - deploy-docs.yml: switch the main deploy off `mkdocs gh-deploy` (which force-replaces the branch and would wipe previews) to JamesIves/github-pages-deploy-action with `clean-exclude: pr-preview/`, so the main publish preserves open previews. (b) Phase 8 polish: - T031: add `mypy` to `make verify`; switch [tool.mypy] to `files=` so a bare `mypy` works without a py.typed marker. Clean on Python 3.9 (mypy 1.19.1). - T032: align quickstart with the finished pipeline — `make compare` happy path + a new generated-schema-docs scenario. - T034: docs cross-links/glossary already repointed to the generated reference in US5. - T033 (mark ADR 0005 superseded) is left for when the real XSD lands. Verified on real CPython 3.9.4: ruff + mypy + pytest (41) + mkdocs --strict all green.
|
IanMayo
changed the title
…k (T033) Treat the (formerly "placeholder") schema as the authoritative Acoustic Dataset contract. - Drop T033 (swap in the real XSD); mark ADR 0005 Superseded — there is no separate real XSD pending. - schema/acoustic_dataset.xsd: remove the PLACEHOLDER framing from the header/annotation; version 0.2.0-placeholder -> 0.2.0. - examples/calculation_input.json: schemaVersion -> 0.2.0, name -> "Reference Platform A"; acoustics default version -> 0.2.0; refresh the golden file + reference fixture (and its banner) and the version assertions in tests. - Regenerate the schema reference (now version 0.2.0, no "Placeholder" prose). Models are unchanged (the XSD edits were comments/version only). - Rename the how-to swap-in-the-real-schema.md -> change-the-schema.md (general "evolve the schema" guide) and repoint all links (tutorial, onboarding, concepts, ADR 0008, nav). - Drop residual "placeholder/illustrative/real XSD lands" framing across README and docs; reframe acoustics/generate/mapping doc-comments. (ADR bodies kept as historical record.) Verified on real CPython 3.9.4: ruff + mypy + pytest (41) + mkdocs --strict green; generation idempotent (drift gate stable).
Show, on the schema reference page (all generated from the canonical pipeline run so they can't drift): - Example document — a trimmed, validated XML excerpt of the output. - Working with the typed objects — a Python snippet (load -> to_model -> field reads) with the real values pulled from the populated Platform. - Worked example: deriving a value from elementary physics — the active sonar's MaxRange is computed from SourceLevel and DetectionThreshold via the sonar equation under two-way spherical spreading (r = 10**((SL-DT)/40) = 118850.223 m), and that typed value is exactly what serialises into <MaxRange> — typed data in, schema-valid XML out. - schema_docs.generate() gains an optional example_input; `gen-schema-docs` passes --input. - Tests cover the three new sections; the committed-page freshness test still gates drift. Verified on real CPython 3.9.4: ruff + mypy + pytest (44) + mkdocs --strict green; idempotent.
A focused explanation (with runnable Python) of why the pipeline maps onto the generated typed objects instead of threading generic dicts through (FR-010, ADR 0002): - dict access is stringly-typed (Any), typos fail only at runtime or silently via .get, units live only in key names, and bad values flow straight through; - typed objects give attribute access + autocomplete, mypy/AttributeError on typos, precise Decimal types with meaning from the XSD docstrings, and MappingError at the single boundary for out-of-band values — before serialisation. - New concept page wired into the nav + onboarding mental-model list. - tests/unit/test_typed_vs_dict.py pins every claim (Decimal value, dict KeyError/silent None, attribute typo -> AttributeError, out-of-band -> MappingError) so the page can't rot. Verified on real CPython 3.9.4: ruff + mypy + pytest (48) + mkdocs --strict green.
Address review feedback: educational/neutral tone (not persuasive), and focus on what strong typing constrains when data is *stored* rather than when it is read back. - typed-vs-dicts.md: rewritten around storing data — a dict accepts any key/type/range; a typed object fixes the field set (unknown name -> TypeError at construction), declares each field's type (mypy catches wrong types), and the mapping rejects out-of-range values as they are stored. Plain summary table; removed the salesy phrasing. - test_typed_vs_dict.py: aligned to verified storage behaviour (dict stores anything; unknown field -> TypeError; stored field is Decimal; out-of-range -> MappingError). Dropped the read-time attribute-typo example. - schema_docs.py: neutralise the one punchy line in the generated worked example; regenerate. Verified on real CPython 3.9.4: ruff + mypy + pytest (48) + mkdocs --strict green; idempotent.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Several increments on the same branch (continued per request). Brings Phase 1 to 33/34 tasks — all five user stories complete; the only open task (T033) is deliberately deferred until the real XSD replaces the placeholder.
mypyinmake verify(T031), quickstart alignment (T032), cross-links (T034)US5 — generated schema reference + Mermaid ERD
schema_docs.pyparses the enriched XSD → a generated Markdown reference (MermaiderDiagramwith exact cardinalities, per-type field tables carryingxs:documentation, banded-type catalogue).gen-schema-docsCLI +make gen-schema-docs; committed atdocs/reference/schema/index.md; hand-drawn stand-in removed and links repointed.US4 — bundle + drift gate
bundle.py+make bundleships schema + data + models (+ manifest), refusing partial bundles. CI drift gate (in the 3.9verifyjob) regenerates models + schema docs and fails on drift.Generation stays on Python 3.9 (no slip to 3.11)
The committed models had been produced by xsdata 26.x (needs ≥3.10). Realigned: pin
xsdata==25.7(last 3.9 line), pass explicit--no-slots/--no-kw-only/--no-union-type/--no-postponed-annotations, and regenerate on real CPython 3.9.4 (xsdata's docstring wrapping varies by Python minor version). Recorded in ADR 0008. No 3.11 anywhere.Docs PR previews
pr-preview.ymlpublishes this PR's site togh-pages/pr-preview/pr-<N>/(rossjrw/pr-preview-action) and posts a sticky comment with the link; torn down on close.deploy-docs.ymlswitched offmkdocs gh-deploy(force-replaces the branch → would wipe previews) toJamesIves/github-pages-deploy-actionwithclean-exclude: pr-preview/.Phase 8 polish
mypyadded tomake verify(config switched tofiles=so a baremypyworks; clean on 3.9 / mypy 1.19.1); quickstart updated (make comparehappy path + a generated-schema-docs scenario); glossary/cross-links point at the generated reference.Verification (real CPython 3.9.4, via
uv python install 3.9.4)ruff+mypy+pytest(41 passed) +mkdocs build --strictall green;make bundleworks; generation is byte-idempotent so the drift gate is stable (confirmed green on CI's 3.9).https://claude.ai/code/session_01CFHWZFcZ3ecni6bzZ2AXdq