DEVOPS-5014: Cloudflare Workers Static Assets (no Worker script)

.gitignore

@@ -18,4 +18,6 @@ docs/*/images/
 
 # Python bytecode cache (from scripts/)
 __pycache__/
-*.pyc
+*.pyc
+# Wrangler local state
+/.wrangler

.python-version

@@ -0,0 +1 @@
+3.14

CLAUDE.md

@@ -29,10 +29,35 @@ docker build -t wallarm-docs . && docker run -p 8080:80 wallarm-docs
 
 ### Deployment pipeline
 
-* There are no tests or linters. Validation happens via **Netlify preview builds**.
-* Creating a **PR to `master`** triggers a Netlify test build — the preview link appears in the PR checks (takes 3-5 minutes).
-* **Merging to `master`** triggers production deployment via Netlify.
-* Build configuration is in `netlify.toml`.
+The site is **migrating from Netlify to Cloudflare** (DEVOPS-5014). Both run in
+parallel until the cutover, so a PR gets a preview from each.
+
+* There are no tests or linters. Validation happens via **preview builds**.
+* Creating a **PR to `master`** triggers a build — the preview link appears in
+  the PR checks (takes 3-5 minutes).
+* **Merging to `master`** deploys production.
+* Build configuration: `scripts/build.sh` (Cloudflare) and `netlify.toml`
+  (Netlify). **They must be kept in sync until Netlify is retired.**
+
+#### Cloudflare specifics
+
+* Hosting is **Workers Static Assets** — `wrangler.jsonc`, publish dir `site/`.
+  There is no Worker script; it is a plain static deployment.
+* Redirects, headers, and `Accept: text/markdown` negotiation are split between
+  this repo (`_redirects`, `_headers`) and zone rules in
+  [infra/cloudflare-iac](https://gl.wallarm.com/infra/cloudflare-iac).
+* **Preview URLs do not exercise zone rules.** Previews are served from
+  `*.workers.dev`, where redirects, markdown negotiation and WAF do not apply.
+  A preview shows *content* accurately but not *routing* — verify any redirect
+  or negotiation change on `docs-staging.wallarm.com` instead.
+* `_redirects` has a hard limit: Cloudflare rejects the whole deployment past
+  **100 dynamic rules**, and counts every rule appearing *after the first
+  wildcard rule* toward that budget. Keep plain rules above the wildcard block
+  at the bottom of the file — the file says so too.
+* Every page must have a `.md` companion, enforced by
+  `scripts/check_markdown_companions.py` in the build. A page without one is
+  not a fallback to HTML: it returns the 404 page labelled
+  `Content-Type: text/markdown`.
 
 ## Repository architecture
 
@@ -98,14 +123,19 @@ The `_redirects` file lives in the root version's `docs_dir` (currently `docs/6.
 /old/path/page  /new/path/page
 ```
 
-This prevents 404 errors for users with bookmarked URLs. Redirect syntax supports wildcards (`/*`). See [Netlify redirect docs](https://docs.netlify.com/routing/redirects/).
+This prevents 404 errors for users with bookmarked URLs. Redirect syntax supports wildcards (`/*`).
+
+**Add plain path-to-path redirects ABOVE the wildcard block at the bottom of the file.** Cloudflare counts every rule after the first wildcard against a 100-rule budget and rejects the entire deployment past it. See the header comment in `_redirects`, plus the [Cloudflare](https://developers.cloudflare.com/workers/static-assets/redirects/) and [Netlify](https://docs.netlify.com/routing/redirects/) references.
 
 ### Key platform files
 
 | File | Purpose |
 |------|---------|
 | `mkdocs-base.yml` | Shared config: plugins, extensions, theme |
-| `netlify.toml` | Build commands and deploy config (edit only for version management) |
+| `netlify.toml` | Netlify build/deploy config (edit only for version management) |
+| `scripts/build.sh` | Cloudflare build; mirrors `netlify.toml`, keep in sync |
+| `wrangler.jsonc` | Cloudflare Workers Static Assets deploy config |
+| `docs/6.x/_headers` | Response headers, Cloudflare |
 | `stylesheets/extra.js` | Custom JS: image zoom, version selector logic, `rootVersion` variable |
 | `stylesheets/partials/` | Custom HTML overrides: `nav.html` (version selector), `header.html`, `footer.html`, `feedback.html`, `actions.html` (edit actions), `toc.html`, `integrations/` |
 | `Dockerfile` | Full multi-version build for local testing |
@@ -134,7 +164,7 @@ Follow these guides before writing or editing any content:
 ## What NOT to do
 
 * Do NOT edit `mkdocs-base.yml` unless explicitly required
-* Do NOT edit `netlify.toml` or files in `stylesheets/` except for version management tasks (adding/deprecating a guide version)
+* Do NOT edit `netlify.toml`, `scripts/build.sh`, `wrangler.jsonc` or files in `stylesheets/` except for version management tasks (adding/deprecating a guide version)
 * Do NOT edit files in `docs/6.x/` or `docs/7.x/` directly — they are include wrappers (exception: creating new wrappers for new pages)
 * Do NOT invent features not described in the source material
 * Do NOT add content in languages other than English

DEPLOY-CLOUDFLARE.md

@@ -0,0 +1,110 @@
+# Cloudflare deployment
+
+Hosting for docs.wallarm.com on **Workers Static Assets**. Tracked in
+[DEVOPS-5014](https://wallarm.atlassian.net/browse/DEVOPS-5014).
+
+Netlify still builds and serves production. Nothing here is live until the
+cutover, and `netlify.toml` is deliberately untouched so rollback stays trivial.
+
+## What lives where
+
+| Concern | Here | infra/cloudflare-iac |
+|---|---|---|
+| Build | `scripts/build.sh` | — |
+| Deploy config | `wrangler.jsonc` | — |
+| Response headers | `docs/6.x/_headers` | `Vary`, `Link` for negotiable pages |
+| Redirects | `docs/6.x/_redirects` | legacy `/2.x`–`/4.10` catch-all |
+| `Accept: text/markdown` | `.md` companions from the build | the rewrite rules |
+| DNS, WAF, bot management | — | all of it |
+
+There is **no Worker script**. Static Assets is served by a code-less Worker
+entry, which is why asset requests are free and never invoke anything.
+
+## Workers Builds settings
+
+Workers Builds is configured in the Cloudflare dashboard, not in this repo, so
+the settings are recorded here.
+
+| Setting | Value |
+|---|---|
+| Repository | `wallarm/product-documentation` |
+| Production branch | `master` |
+| Build command | `git submodule update --init --recursive && scripts/build.sh` |
+| Deploy command | `npx wrangler deploy` |
+| Non-production deploy command | `npx wrangler versions upload` |
+| Build variable | `CONTEXT=production` |
+| Build variable | `PUPPETEER_SKIP_CHROMIUM_DOWNLOAD=true` |
+
+Notes on each of those:
+
+* **Submodules are not cloned automatically.** The build fails without them —
+  terraform module snippets are included from `docs/latest/`. Both submodules
+  are public, so anonymous HTTPS is enough.
+* **Python** is pinned by `.python-version` (3.14). The build image defaults to
+  3.13.
+* **`CONTEXT=production`** is what enables image optimisation, the `.md`
+  companions, OG images and feeds. Without it the companions are not generated
+  and markdown negotiation returns 404s. `scripts/build.sh` mirrors the variable
+  Netlify sets, so the two stay comparable.
+* **`PUPPETEER_SKIP_CHROMIUM_DOWNLOAD`** — `package.json` carries puppeteer for
+  the PDF tooling, which the docs build does not use. Workers Builds installs
+  dependencies automatically, and without this it downloads ~170 MB of Chromium
+  on every build.
+
+Build takes roughly 3-6 minutes on a 4-vCPU runner; measured at 1m35s wall
+clock locally with 518s of CPU across 6 cores. The platform timeout is 20
+minutes, so there is comfortable headroom.
+
+## Preview builds do not test routing
+
+Previews are served from `*.workers.dev`, and **zone rules do not apply there**.
+Verified against staging:
+
+| | `*.workers.dev` | `docs-staging.wallarm.com` |
+|---|---|---|
+| `Accept: text/markdown` | `text/html` | `text/markdown` |
+| `/4.8/admin-en/foo/` | `404` | `301` |
+| `Vary` | absent | present |
+
+So a preview shows **content** accurately but not **routing**. Any change to
+redirects, headers or markdown negotiation must be checked on
+`docs-staging.wallarm.com`, which is a real hostname on the zone and behaves
+exactly like production.
+
+For the same reason `workers_dev` is `false` on production: it would publish the
+whole site a second time with no redirects, no negotiation, no WAF, and
+indexable by search engines.
+
+## Two things that will break the build
+
+**Redirect ordering.** Cloudflare rejects the entire deployment past 100
+dynamic `_redirects` rules, and counts every rule appearing *after the first
+wildcard rule* toward that budget regardless of its own shape. Plain rules go
+above the wildcard block at the bottom of the file.
+
+**Missing markdown companions.** `scripts/check_markdown_companions.py` fails
+the build if a page has no `.md`. This is deliberate: the negotiation is a blind
+URL rewrite, so a missing companion does not fall back to HTML — it returns the
+404 page labelled `Content-Type: text/markdown`. Its exemption list must stay in
+sync with `local.md_no_companion` in the IaC repo.
+
+## Cutover
+
+1. Connect Workers Builds with the settings above; confirm a `master` build
+   deploys and a PR gets a preview.
+2. Widen the zone rules in `infra/cloudflare-iac` from
+   `docs-staging.wallarm.com` to include `docs.wallarm.com`.
+3. Point Cloudflare's record for the proxied hostname at the Worker. **This is a
+   Cloudflare-side DNS change, not a Route53 one** — `wallarm.com` is a partial
+   zone, and Route53 already delegates `docs` to Cloudflare. Today that record
+   reads `CNAME docs.wallarm.com -> pensive-dubinsky-5f7a00.netlify.app`.
+4. Verify, then ask Anastasiia Popova to check.
+5. Once stable, delete `netlify.toml`, `netlify/`, and the duplication between
+   `scripts/build.sh` and the Netlify build command.
+
+**Rollback** is restoring that one CNAME to the Netlify value. Netlify keeps
+building from the same branch throughout, so its copy stays current.
+
+`cloudflare_workers_custom_domain` does **not** work here: Custom Domains refuse
+a hostname that already carries a CNAME and are documented against full zones.
+Production takes the same route-based binding staging uses.

docs/6.x/_headers

@@ -0,0 +1,74 @@
+# Static response headers for docs.wallarm.com. Port of the netlify.toml
+# [[headers]] blocks (DEVOPS-5014). Lives in the root version's docs_dir so
+# zensical copies it to the publish dir root, same as _redirects.
+#
+# Cloudflare's `*` is greedy and matches `/`, so one `/*.md` rule covers both
+# `/foo.md` and `/7.x/a/b.md` — no `/**/*.md` twin needed as on Netlify.
+#
+# NOTE: these rules apply only to assets served directly by the asset server.
+# The Worker (src/worker.ts) runs first for extension-less page URLs and sets
+# the site-wide security headers itself, because `_headers` is not applied to
+# responses returned from Worker code.
+
+/*
+  Content-Security-Policy: frame-ancestors *
+  X-Content-Type-Options: nosniff
+  Strict-Transport-Security: max-age=63072000; includeSubDomains
+
+# Raw .md / llms*.txt companions, served as plain UTF-8 text. CORS is open so
+# AI tools and browser-side fetchers can read them cross-origin (Cloudflare /
+# Anthropic / Stripe do the same on their .md docs); X-Robots-Tag tells search
+# engines not to index these as duplicates of the HTML pages (the markdown has
+# no <head>, so a robots meta cannot live in the file itself); the 1-hour
+# Cache-Control prevents CDN/browser thrash.
+/*.md
+  Content-Type: text/markdown; charset=utf-8
+  Access-Control-Allow-Origin: *
+  Access-Control-Allow-Methods: GET, OPTIONS
+  Cache-Control: public, max-age=3600, must-revalidate
+  X-Robots-Tag: noindex, follow
+
+/llms*.txt
+  Content-Type: text/plain; charset=utf-8
+  Access-Control-Allow-Origin: *
+  Access-Control-Allow-Methods: GET, OPTIONS
+  Cache-Control: public, max-age=3600, must-revalidate
+  X-Robots-Tag: noindex, follow
+
+# Aggregated changelog feeds (scripts/generate_feeds.py). Atom is served as
+# application/atom+xml so RSS readers and Slack's /feed app recognise it. CORS
+# is open so scripts, CI, and browser-side agents can fetch the JSON
+# cross-origin (same policy as the .md companions). Cloudflare adds
+# ETag/Last-Modified, so conditional polling still works.
+/feeds/*.xml
+  Content-Type: application/atom+xml; charset=utf-8
+  Access-Control-Allow-Origin: *
+  Access-Control-Allow-Methods: GET, OPTIONS
+  Cache-Control: public, max-age=3600, must-revalidate
+
+/feeds/*.rss
+  Content-Type: application/rss+xml; charset=utf-8
+  Access-Control-Allow-Origin: *
+  Access-Control-Allow-Methods: GET, OPTIONS
+  Cache-Control: public, max-age=3600, must-revalidate
+
+/feeds/*.json
+  Content-Type: application/json; charset=utf-8
+  Access-Control-Allow-Origin: *
+  Access-Control-Allow-Methods: GET, OPTIONS
+  Cache-Control: public, max-age=3600, must-revalidate
+
+# Screenshots and diagrams. Without an explicit header these are revalidated on
+# every page view, and in analytics the top bandwidth consumers are all
+# /images/*.png fetched tens of thousands of times each. A week-long browser
+# cache stops the bleeding. NOT `immutable`: image filenames are not
+# content-hashed, so a replaced screenshot reuses its name — max-age bounds how
+# long a stale copy can linger, and the deploy purges the edge cache anyway.
+/images/*
+  Cache-Control: public, max-age=604800, stale-while-revalidate=86400
+
+# Theme bundles and fonts under /assets/ ARE content-hashed by the build
+# (e.g. main.f62f0af6.min.css, bundle.d7f30b55.min.js), so a new build ships a
+# new filename — the old URL can be cached forever safely.
+/assets/*
+  Cache-Control: public, max-age=31536000, immutable