This file is for maintainers. It describes how the docs site (docs.1claw.xyz) is made discoverable and how to submit it to search engines.
- Static HTML (SSG): Docusaurus builds static HTML for every page (including the index). Crawlers receive full content without executing JavaScript.
- Index page: The root
/page is a real landing page with title, description, and links to docs (not a client-side redirect), so Google and other crawlers can index it. - Meta and Open Graph: Site-wide meta description, keywords,
og:*andtwitter:*tags are set indocusaurus.config.ts. - Sitemap: The build produces
build/sitemap.xml(via@docusaurus/plugin-sitemap). The sitemap is linked fromstatic/robots.txt. - robots.txt:
static/robots.txtallows all crawlers and points tohttps://docs.1claw.xyz/sitemap.xml.
- Go to Google Search Console.
- Add a property: URL prefix
https://docs.1claw.xyz. - Verify ownership using one of:
- HTML tag: Add a meta tag to the docs. We can add a custom
metadataentry indocusaurus.config.tswith the value Google gives you (e.g.google-site-verification). - DNS: Add the TXT record Google provides to your domain’s DNS for
docs.1claw.xyz(or the root if you use a CNAME).
- HTML tag: Add a meta tag to the docs. We can add a custom
- After verification, open Sitemaps and submit:
https://docs.1claw.xyz/sitemap.xml. - Optionally use URL Inspection to request indexing of the homepage and a few key docs.
After deployment, the sitemap is available at https://docs.1claw.xyz/sitemap.xml. Re-submit the sitemap in GSC after large doc changes if you want to speed up re-crawling.
Open Graph currently uses the logo SVG. Some platforms prefer a PNG (e.g. 1200×630). To improve link previews:
- Add an image (e.g.
static/img/og-image.png, 1200×630). - In
docusaurus.config.ts, setthemeConfig.imageto"img/og-image.png"and/or update theog:imagemetadata tohttps://docs.1claw.xyz/img/og-image.png.
After the basics above are in place, these can improve visibility and rich results.
| Priority | Action | Why |
|---|---|---|
| High | Submit sitemap to Bing Webmaster Tools | Bing Webmaster Tools — add property https://docs.1claw.xyz, verify (HTML tag or DNS), then submit https://docs.1claw.xyz/sitemap.xml. Bing powers DuckDuckGo and parts of other search/UIs. |
| High | Audit doc frontmatter | Key doc pages should have a unique title and description in frontmatter so search results show good snippets. Many docs already do; spot-check high-traffic pages (intro, quickstart, MCP overview, SDK pages). |
| Medium | Add JSON-LD structured data | Optional: add WebSite and/or Organization schema so search can show site links. Docusaurus doesn’t ship this by default; you can add a custom plugin or inject script in docusaurus.config.ts (e.g. headTags or a layout component). |
| Medium | Use a PNG for og:image | As above; 1200×630 PNG improves how links look when shared (Twitter, Slack, etc.). |
| Low | Core Web Vitals | Static Docusaurus builds are usually fast. If you use Vercel/Netlify, they handle compression and CDN. Check PageSpeed Insights after deploy if you want to tune. |
| Low | Backlinks and content | General SEO: links from 1claw.xyz, GitHub README, and npm package pages to the docs help discovery. A short “Documentation” link in the main site header/footer is useful. |
Nothing in this checklist is required for basic discoverability; the in-repo setup (SSG, meta, sitemap, robots.txt) plus GSC (and optionally Bing) submission is enough to get indexed.