[Docs] Create code to generate discoverable agent facing markdown and update docs to inform agents

[stichbury]

Note for bots and automated tooling: We do not accept pull requests opened automatically by bots or other automated tooling without a human in the loop. To request permission to submit such a PR, the human owner of the bot must open an issue for approval in advance of the PR's submission.
See CONTRIBUTING.md.

@petar-qb This is now ready for review.

You previously reviewed #1841 which improved the way we advertise and use Cloudflare's Markdown for Agents but this is inherently flawed, and not all agents will even ask CF to deliver docs in that format. We are also going to offer a markdown version of our docs that is published on the RtD website alongside HTML. The tooling for this (and changes to announce the fact to passing agents) are what this PR delivers, along with addition of llms-full.txt.

Description

Closes https://github.com/McK-Internal/vizro-internal/issues/2938 and https://github.com/McK-Internal/vizro-internal/issues/2939 and https://github.com/McK-Internal/vizro-internal/issues/2940

This PR does the following for vizro-core only:

• It generates clean Markdown versions of every Vizro documentation page during the docs build. This is better for agents to consumer (fewer tokens) and removes complications like tabbed code and admonition markdown. In Claude's words "it preserves code-block languages and removes website-only controls and navigation noise."
  • For example, look at the markdown version of the Explore Vizro tutorial.
  • This code will be usable by Kedro when they migrate to Zensical too.
• It publishes the markdown to both common Markdown URL formats, such as page.md and page/index.md as RtD mangles HTML pages it publishes into folders.
• It creates llms-full.txt, containing all narrative documentation without the large API reference (again to avoid wasting agent's tokens)
• It splits the Models API reference into smaller, per-model Markdown files to enable progressive discovery.
• It makes the Markdown discoverable through llms.txt, page metadata, and guidance for AI agents.
• It is versioned both for published and pull-request documentation builds.
• It validates generated files and links during the build to catch missing, stale, or malformed output.

Notice

• I acknowledge and agree that, by checking this box and clicking "Submit Pull Request":

  • I submit this contribution under the Apache 2.0 license and represent that I am entitled to do so on behalf of myself, my employer, or relevant third parties, as applicable.
  • I certify that (a) this contribution is my original creation and / or (b) to the extent it is not my original creation, I am authorized to submit this contribution on behalf of the original creator(s) or their licensees.
  • I certify that the use of this contribution as authorized by the Apache 2.0 license does not violate the intellectual property rights of anyone else.
  • I have not referenced individuals, products or companies in any commits, directly or indirectly.
  • I have not added data or restricted code in any commits, directly or indirectly.

[github-actions]

View the example dashboards of the current commit live on PyCafe ☕ 🚀

Updated on: 2026-09-10 14:18:18 UTC
Commit: c49aff5

Compare the examples using the commit's wheel file vs the latest released version:

vizro-core/examples/scratch_dev

View with commit's wheel vs View with latest release

vizro-core/examples/dev/

View with commit's wheel vs View with latest release

vizro-core/examples/visual-vocabulary/

View with commit's wheel vs View with latest release

vizro-core/examples/tutorial/

View with commit's wheel vs View with latest release

[Copilot]

🟡 Changes recommended

tools/stamp_llms_txt.py currently refuses to stamp when READTHEDOCS_VERSION is unset even if READTHEDOCS_CANONICAL_URL is present, which can break stamping in RTD-like environments that only provide the canonical URL.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

This PR adds tooling and documentation updates in vizro-core to publish and validate agent-friendly Markdown outputs (per-page .md, llms-full.txt, and split per-model API reference files) alongside the existing HTML docs, and updates guidance/metadata to make these artifacts discoverable.

Changes:

• Add tools/emit_agent_markdown.py to generate clean Markdown twins from built HTML, plus llms-full.txt and per-model Markdown files with build-time validation.
• Update docs build pipeline (Hatch scripts) and link validation to account for generated artifacts, and add unit tests for the new/updated tooling.
• Refresh agent-facing docs pointers (llms.txt, READMEs, CLAUDE.md, for-llms.md, and HTML <link> metadata) to advertise the Markdown/bundle/model-file access patterns.

File summaries

File	Description
vizro-core/zensical.toml	Ensures code blocks emit language-* classes to preserve fence languages in generated Markdown.
vizro-core/tools/check_llms_txt.py	Extends llms.txt validation to include generated llms-full.txt and per-model files.
vizro-core/tests/unit/tools/test_stamp_llms_txt.py	Adds unit coverage for stamping multiple files and RTD canonical URL behavior.
vizro-core/tests/unit/tools/test_emit_agent_markdown.py	Adds unit coverage for HTML→Markdown conversion, bundling, model splitting, and validation rules.
vizro-core/tests/unit/tools/test_check_llms_txt.py	Adds unit coverage for validating authored + generated links and per-model index completeness.
vizro-core/README.md	Updates agent guidance to prefer per-page Markdown, bundle, and per-model references.
vizro-core/hatch.toml	Wires generation + checks into docs:build and adds a serve-markdown workflow.
vizro-core/docs/pages/for-llms.md	Adds “Fetch documentation as Markdown” instructions for agents.
vizro-core/docs/overrides/main.html	Adds page-level <link rel="alternate" type="text/markdown"> metadata for Markdown twins.
vizro-core/docs/llms.txt	Switches index links to .md pages, adds llms-full.txt, and includes per-model API links.
vizro-core/CLAUDE.md	Updates agent authoring guidance to reference per-page Markdown, bundle, and per-model docs.
tools/stamp_llms_txt.py	Extends stamping to multiple files and supports RTD canonical URL usage.
tools/emit_agent_markdown.py	New standalone generator/validator for agent-facing Markdown outputs.
README.md	Updates repository-level agent guidance similarly to vizro-core/README.md.
.github/workflows/docs-link-checker.yml	Removes standalone llms.txt check step (now covered by docs:build).

Review details

• Files reviewed: 15/15 changed files
• Comments generated: 1
• Review effort level: Lite

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.