Serve full page content to AI page-action buttons on long pages#680
Serve full page content to AI page-action buttons on long pages#680jon-neher wants to merge 2 commits into
Conversation
The "Open in Claude" (and other AI page-action) buttons embed the page's Markdown into the tool URL. On long pages that exceeds MAX_AI_URL_LENGTH, so the content was binary-search-truncated and the assistant got an incomplete page. Instead of truncating, long pages now link the assistant to a complete, same-origin raw-Markdown file: - New docusaurus/plugins/raw-markdown plugin emits <permalink>.md for every doc at build time (frontmatter stripped, title reinstated as H1). - nginx serves .md as text/plain with X-Robots-Tag: noindex. - PageActions.buildAIUrl falls back to the <permalink>.md URL (derived from window.location) when inline embedding won't fit; short pages are unchanged. The fallback stays on the reader's own domain by design — it must not link the GitHub source repo, which would leak the internal org/repo name and break gray-label. Documented in the plugin README and CLAUDE.md. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AewtfT1KkdDjjByXcmfbmN
c88f8a1 to
18fcd4f
Compare
maryam6samadi
left a comment
There was a problem hiding this comment.
Thanks for catching this bug. Long pages were sending silently truncated content to LLMs. No good, and that needs a fix. However, this solution seems to be overly complicated for the problem it's solving. To go from "truncated content" to "clean Markdown," it adds a build plugin, an nginx rule, ~430 emitted .md files, and a permalink-to-filename mapping duplicated in two files that must stay in sync with nothing enforcing it. A custom slug or the WordPress Hosting index pattern can point the button at a .md that was never written, which can cause a silent 404 nobody would notice.
Simpler fix: instead of building a parallel .md delivery pipeline, we can just stop truncating and point the assistant at the page's own URL:
"The full page is at . Please read it before helping me."
That's a few lines in buildAIUrl. The only thing we give up versus the full PR is serving pre-parsed Markdown instead of letting the assistant parse the HTML itself, which is a modest quality difference.
|
@maryam6samadi that works for me! Previously, agents were not able to grab the actual content from these pages. That's why I came up with the markdown serving pipeline. It seems like agents can fetch the comment easily now so it's not a problem anymore! |
@jon-neher That's fair. Things change fast. Would you rather trim it yourself or shall I make the change? |
Please go-ahead and make the change! I can review later if you need a reviewer. |
I committed the change. It's now ready for your approval and a possible merge. |
What & why
The AI page-action buttons ("Open in Claude", "Open in ChatGPT", etc. in
PageActions.tsx) prefill an assistant with the current page's Markdown by embedding it into the tool URL. On long pages that embedded prompt exceedsMAX_AI_URL_LENGTH(8000), so the content was binary-search-truncated — the assistant received an incomplete page.This change stops truncating. Long pages now link the assistant to a complete, same-origin raw-Markdown file instead. Short pages are unchanged.
An alternative would be to link the assistant to the GitHub source file, but that would expose the internal org/repo name to end users and break the gray-label rule (
.cursor/rules/gray-label-business-owner-docs.mdc). Keeping the fallback on the reader's own domain avoids any brand leak.How it works — three coupled pieces
docusaurus/plugins/raw-markdown/(new local plugin) — apostBuildstep that writes each doc's complete source tobuild/<permalink>.md(frontmatter stripped,titlereinstated as an H1). No new npm dependency. Runs once after HTML generation; cost is negligible.nginx.conf— alocation ~ \.md$block that serves those files astext/plain; charset=utf-8(render inline, not download) withX-Robots-Tag: noindex(never search-indexed). Placed before the SPA catch-all so.mdrequests hit the real file instead of theindex.htmlfallback.docusaurus/src/components/PageActions.tsx—buildAIUrlnow falls back to the<permalink>.mdURL (derived fromwindow.locationvia the newtoMdUrlhelper) when the fully-embedded prompt won't fit, instead of truncating. Shared by all five AI buttons, so each gains the complete-content fallback.Docs added:
docusaurus/plugins/raw-markdown/README.mdand an "AI Page Actions" section inCLAUDE.md, both flagging the gray-label constraint so a future edit doesn't swap the same-origin fallback for a GitHub-source link.Net behavior change: only long pages change — from truncated inline to complete via same-origin
.md. Short pages are byte-for-byte identical to today.How to test
1. Build emits the
.mdfilesConfirm build time is essentially unchanged. Note the index-page case does not collide:
ai-chat-receptionist.md(file) coexists with theai-chat-receptionist/directory.2. nginx serves
.mdcorrectly (this is the part not yet verified locally — no Docker daemon in the authoring env)Build the image and run it (mirrors the Dockerfile: build →
/usr/share/nginx/html,nginx.conf→/etc/nginx/conf.d/default.conf), then:Expect:
200,Content-Type: text/plain; charset=utf-8,X-Robots-Tag: noindex. Open the URL in a browser and confirm it renders inline as text (no download prompt). Confirm normal HTML routes still load.3. Button behavior in the running site (
npm run startor the built image)…/<page>.mdinstead of a truncated slice. Open it and confirm the assistant can read the full page. Decode theq=param to inspect the prompt.4. Gray-label check
Decode a long-page prompt and grep it for
githuband your internal org/repo name — neither should appear. The.mdURL host must equal the page host (whatever partner/gray-label domain is serving it).Out of scope
No changes to
docs/content or.github/workflows/.