Add llms.txt and llms-full.txt#612
Merged
sboldyreva merged 8 commits intoJul 7, 2026
Merged
Conversation
Publish an llms.txt at the site root (served from .vuepress/public) following the llmstxt.org format: H1, blockquote summary, and H2 link sections covering every top-level docs.tuxcare.com product area. Links are aligned to the actual published routes (from documents.ts / sidebar.ts) rather than the ticket draft, which referenced several paths that do not resolve: - enterprise-support -> enterprise-support-for-almalinux - kernelcare -> live-patching-services - kernelcare-iot -> kernelcare-for-iot - securechain/machine-readable-security-data -> securechain/managing-securechain-repository - subscription-management -> tuxcare-cln Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AJYGjkHmgKthwWLbyWZ8gy
Optional companion to llms.txt: concatenates all 156 documentation pages into a single file served at the site root, each preceded by its source URL. Generated from the docs source with cleaning applied — YAML frontmatter, HTML comments, <script>/<style> blocks and VuePress custom component tags are stripped, while literal code snippets from CodeTabs/TableTabs and ALL-CAPS placeholders are preserved. Interactive content is flattened; the source URL points to the rendered page. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AJYGjkHmgKthwWLbyWZ8gy
- Add robots.txt at the site root that explicitly allows major AI/LLM crawlers and training bots (GPTBot, ClaudeBot, Google-Extended, PerplexityBot, CCBot, etc.) and points to the llms.txt index. No Sitemap directive since the site does not generate a sitemap. - Reference llms-full.txt from the llms.txt summary for discoverability. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AJYGjkHmgKthwWLbyWZ8gy
The server args path (mcp-docs-server/server.mjs) is relative to the repo root, so it remains valid after the move. .mcp.json at the root is the standard location read by Claude Code and other MCP clients. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AJYGjkHmgKthwWLbyWZ8gy
Reverts the earlier robots.txt addition per request. Absence means crawlers are allowed by default (nothing blocked); llms.txt/llms-full.txt remain in place. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AJYGjkHmgKthwWLbyWZ8gy
Adds scripts/generate-llms-full.mjs and runs it before `vuepress build` via the docs:build script, so llms-full.txt is regenerated from the current docs on every deploy and copied to the site root by VuePress (public/ -> dist/). This removes the stale 1 MB snapshot from the repo. The generator ports the cleaning applied in the original file and also strips VuePress container markers (:::tip / :::warning / :::), which the committed snapshot still contained. The curated llms.txt stays hand-maintained and tracked; only llms-full.txt is git-ignored. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AJYGjkHmgKthwWLbyWZ8gy
scripts/check-llms-links.mjs compares the product links in docs/.vuepress/config-client/documents.ts (the site product-menu source of truth) against llms.txt and fails if any product is missing, so a newly added or renamed product can't silently leave llms.txt stale. It does not require the reverse, so llms.txt keeps its extra SecureChain subsection and Optional links. A dedicated .github/workflows/check-llms.yml runs the check on pull requests (and pushes to master) touching the relevant files. It uses only Node built-ins — no install step — and is kept separate from the deploy workflow so it never blocks a production deploy. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AJYGjkHmgKthwWLbyWZ8gy
generate-llms-full.mjs stripped <script setup> blocks before salvaging CodeTabs/TableTabs content, so tabs whose `content:` is a script variable (the dominant pattern across ELS library/runtime pages — ~93 tabs on 49 pages) emitted nothing. Capture the variable bodies before stripping and resolve each tab's content — inline literal or variable — under its title. check-llms-links.mjs: also verify every docs.tuxcare.com page link in llms.txt resolves to a real README.md on disk, catching typos and links left stale when a product is renamed or removed in documents.ts. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
yel-hadd
reviewed
Jul 7, 2026
Collaborator
There was a problem hiding this comment.
@sboldyreva Took a look and pushed the fixes straight to the branch (209b826) to save a round-trip.
generate-llms-full.mjswas silently dropping CodeTabs content. If a tab'scontent:pointed at a<script setup>variable, it came out empty, because the script block got stripped before the salvage step ran. That's the common pattern on the ELS library and runtime pages, so it hit ~93 tabs across 49 pages. It now reads the variable bodies first and emits each tab under its own title. On the Angular page,angular-corewent from 26 hits to 38, which matches the source.check-llms-links.mjsnow also checks that every docs.tuxcare.com link in llms.txt resolves to a real README on disk. Catches the rename case you flagged and plain typos too, and it does it off the filesystem so there are no network calls in CI.
On your two questions: I'd leave llms.txt hand-curated and llms-full.txt git-ignored. No reason to change either.
Please have a look when you get a chance.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.