From 00f822caf9832c88178315822e9225fb679d1cc7 Mon Sep 17 00:00:00 2001 From: Tom Cobb Date: Thu, 11 Jun 2026 12:39:13 +0000 Subject: [PATCH 01/28] docs: Stage A scaffold Instantiate the DLS python-copier-template MyST docs skeleton into PandABlocks-server as a one-off scaffold (Stage A, runbook Prompt A). - Move legacy Sphinx/RST tree to docs/_legacy_rst/ (ignored by myst build) so Stage D/E conversion can read sources in-tree; add docs/images/ with the logo. - New myst.yml: title/github/logo swapped, server TOC + stubs for every v4 target page, empty tutorials.md landing for parity, Release Notes url. - index.md README-include + how-to/contribute CONTRIBUTING-include resolve; add root README.md and .github/CONTRIBUTING.md. - xref/intersphinx prototype: probe in reference/capture into PandABlocks-client resolves in built output; other repos commented until published. - CI (.github/workflows/docs.yml) aligned with python-copier-template-example: npm+mystmd build, upload-artifact (docs/_build minus templates cache), versioned move, make_switcher.py, peaceiris v4; publish on main/tags; redirect -> main. - docs/PROGRESS.md tracks Stage-A status. myst build: green (exit 0, no warnings/errors), 18 pages, probe resolves. Co-Authored-By: Claude Opus 4.8 --- .github/CONTRIBUTING.md | 17 + .github/pages/index.html | 12 +- .github/pages/make_switcher.py | 96 +++++ .github/workflows/docs.yml | 90 +++-- .gitignore | 3 + README.md | 24 ++ docs/PROGRESS.md | 53 +++ .../PandA-logo-for-black-background.svg | 0 .../_static/theme_overrides.css | 0 docs/{ => _legacy_rst}/_templates/layout.html | 0 docs/{ => _legacy_rst}/building.rst | 0 docs/{ => _legacy_rst}/capture.rst | 0 docs/{ => _legacy_rst}/commands.rst | 0 docs/{ => _legacy_rst}/conf.py | 0 docs/{ => _legacy_rst}/config.rst | 0 docs/{ => _legacy_rst}/extension.rst | 0 docs/{ => _legacy_rst}/favicon.ico | Bin docs/{ => _legacy_rst}/fields.rst | 0 docs/{ => _legacy_rst}/index.rst | 0 .../presentation-2016-05/.gitignore | 0 .../presentation-2016-05/Makefile | 0 .../presentation-2016-05/building.rst | 0 .../presentation-2016-05/docutils.conf | 0 .../presentation-2016-05/images/admin.png | Bin .../images/architecture.png | Bin .../presentation-2016-05/images/software.png | Bin .../presentation-2016-05/images/system.png | Bin .../presentation-2016-05/panda.rst | 0 .../presentation-2016-05/server.rst | 0 .../presentation-2016-05/styles/blank.gif | Bin .../styles/dlsfooterpad.png | Bin .../presentation-2016-05/styles/framing.css | 0 .../presentation-2016-05/styles/html4css1.css | 0 .../presentation-2016-05/styles/iepngfix.htc | 0 .../presentation-2016-05/styles/opera.css | 0 .../presentation-2016-05/styles/outline.css | 0 .../presentation-2016-05/styles/pretty.css | 0 .../presentation-2016-05/styles/print.css | 0 .../presentation-2016-05/styles/pygments.css | 0 .../presentation-2016-05/styles/s5-core.css | 0 .../presentation-2016-05/styles/slides.css | 0 .../presentation-2016-05/styles/slides.js | 0 docs/{ => _legacy_rst}/startup.rst | 0 docs/{ => _legacy_rst}/support.rst | 0 docs/explanations.md | 7 + docs/explanations/architecture.md | 11 + docs/how-to.md | 7 + docs/how-to/building.md | 11 + docs/how-to/contribute.md | 2 + docs/how-to/startup.md | 11 + .../PandA-logo-for-black-background.svg | 331 ++++++++++++++++++ docs/index.md | 42 +++ docs/myst.yml | 54 +++ docs/reference.md | 7 + docs/reference/c-standard.md | 13 + docs/reference/capture-options.md | 11 + docs/reference/capture.md | 16 + docs/reference/commands.md | 11 + docs/reference/config.md | 11 + docs/reference/extension.md | 13 + docs/reference/fields.md | 11 + docs/reference/streaming-tables.md | 11 + docs/reference/support.md | 11 + docs/tutorials.md | 6 + 64 files changed, 854 insertions(+), 38 deletions(-) create mode 100644 .github/CONTRIBUTING.md create mode 100755 .github/pages/make_switcher.py create mode 100644 README.md create mode 100644 docs/PROGRESS.md rename docs/{ => _legacy_rst}/PandA-logo-for-black-background.svg (100%) rename docs/{ => _legacy_rst}/_static/theme_overrides.css (100%) rename docs/{ => _legacy_rst}/_templates/layout.html (100%) rename docs/{ => _legacy_rst}/building.rst (100%) rename docs/{ => _legacy_rst}/capture.rst (100%) rename docs/{ => _legacy_rst}/commands.rst (100%) rename docs/{ => _legacy_rst}/conf.py (100%) rename docs/{ => _legacy_rst}/config.rst (100%) rename docs/{ => _legacy_rst}/extension.rst (100%) rename docs/{ => _legacy_rst}/favicon.ico (100%) rename docs/{ => _legacy_rst}/fields.rst (100%) rename docs/{ => _legacy_rst}/index.rst (100%) rename docs/{ => _legacy_rst}/presentation-2016-05/.gitignore (100%) rename docs/{ => _legacy_rst}/presentation-2016-05/Makefile (100%) rename docs/{ => _legacy_rst}/presentation-2016-05/building.rst (100%) rename docs/{ => _legacy_rst}/presentation-2016-05/docutils.conf (100%) rename docs/{ => _legacy_rst}/presentation-2016-05/images/admin.png (100%) rename docs/{ => _legacy_rst}/presentation-2016-05/images/architecture.png (100%) rename docs/{ => _legacy_rst}/presentation-2016-05/images/software.png (100%) rename docs/{ => _legacy_rst}/presentation-2016-05/images/system.png (100%) rename docs/{ => _legacy_rst}/presentation-2016-05/panda.rst (100%) rename docs/{ => _legacy_rst}/presentation-2016-05/server.rst (100%) rename docs/{ => _legacy_rst}/presentation-2016-05/styles/blank.gif (100%) rename docs/{ => _legacy_rst}/presentation-2016-05/styles/dlsfooterpad.png (100%) rename docs/{ => _legacy_rst}/presentation-2016-05/styles/framing.css (100%) rename docs/{ => _legacy_rst}/presentation-2016-05/styles/html4css1.css (100%) rename docs/{ => _legacy_rst}/presentation-2016-05/styles/iepngfix.htc (100%) rename docs/{ => _legacy_rst}/presentation-2016-05/styles/opera.css (100%) rename docs/{ => _legacy_rst}/presentation-2016-05/styles/outline.css (100%) rename docs/{ => _legacy_rst}/presentation-2016-05/styles/pretty.css (100%) rename docs/{ => _legacy_rst}/presentation-2016-05/styles/print.css (100%) rename docs/{ => _legacy_rst}/presentation-2016-05/styles/pygments.css (100%) rename docs/{ => _legacy_rst}/presentation-2016-05/styles/s5-core.css (100%) rename docs/{ => _legacy_rst}/presentation-2016-05/styles/slides.css (100%) rename docs/{ => _legacy_rst}/presentation-2016-05/styles/slides.js (100%) rename docs/{ => _legacy_rst}/startup.rst (100%) rename docs/{ => _legacy_rst}/support.rst (100%) create mode 100644 docs/explanations.md create mode 100644 docs/explanations/architecture.md create mode 100644 docs/how-to.md create mode 100644 docs/how-to/building.md create mode 100644 docs/how-to/contribute.md create mode 100644 docs/how-to/startup.md create mode 100644 docs/images/PandA-logo-for-black-background.svg create mode 100644 docs/index.md create mode 100644 docs/myst.yml create mode 100644 docs/reference.md create mode 100644 docs/reference/c-standard.md create mode 100644 docs/reference/capture-options.md create mode 100644 docs/reference/capture.md create mode 100644 docs/reference/commands.md create mode 100644 docs/reference/config.md create mode 100644 docs/reference/extension.md create mode 100644 docs/reference/fields.md create mode 100644 docs/reference/streaming-tables.md create mode 100644 docs/reference/support.md create mode 100644 docs/tutorials.md diff --git a/.github/CONTRIBUTING.md b/.github/CONTRIBUTING.md new file mode 100644 index 00000000..6eaa9eee --- /dev/null +++ b/.github/CONTRIBUTING.md @@ -0,0 +1,17 @@ +# Contributing + +Contributions and issues are most welcome! All issues and pull requests are handled +through [GitHub](https://github.com/PandABlocks/PandABlocks-server/issues). Please check +for any existing issues before filing a new one. + +## Issue or Pull Request? + +Raise an issue if you want to report a bug or request a feature. Open a pull request if +you already have a change ready to propose. If you are unsure, raise an issue first to +discuss the change. + +## Documentation + +Documentation is built with [MyST](https://mystmd.org) and published to GitHub Pages. +The docs follow the [Diátaxis](https://diataxis.fr) framework — tutorials, how-to +guides, reference and explanations. diff --git a/.github/pages/index.html b/.github/pages/index.html index cc33127d..c495f39f 100644 --- a/.github/pages/index.html +++ b/.github/pages/index.html @@ -1,9 +1,11 @@ - - Redirecting to master branch + + + Redirecting to main branch - - - + + + + diff --git a/.github/pages/make_switcher.py b/.github/pages/make_switcher.py new file mode 100755 index 00000000..c06813af --- /dev/null +++ b/.github/pages/make_switcher.py @@ -0,0 +1,96 @@ +"""Make switcher.json to allow docs to switch between different versions.""" + +import json +import logging +from argparse import ArgumentParser +from pathlib import Path +from subprocess import CalledProcessError, check_output + + +def report_output(stdout: bytes, label: str) -> list[str]: + """Print and return something received frm stdout.""" + ret = stdout.decode().strip().split("\n") + print(f"{label}: {ret}") + return ret + + +def get_branch_contents(ref: str) -> list[str]: + """Get the list of directories in a branch.""" + stdout = check_output(["git", "ls-tree", "-d", "--name-only", ref]) + return report_output(stdout, "Branch contents") + + +def get_sorted_tags_list() -> list[str]: + """Get a list of sorted tags in descending order from the repository.""" + stdout = check_output(["git", "tag", "-l", "--sort=-v:refname"]) + return report_output(stdout, "Tags list") + + +def get_versions(ref: str, add: str | None) -> list[str]: + """Generate the file containing the list of all GitHub Pages builds.""" + # Get the directories (i.e. builds) from the GitHub Pages branch + try: + builds = set(get_branch_contents(ref)) + except CalledProcessError: + builds = set() + logging.warning(f"Cannot get {ref} contents") + + # Add and remove from the list of builds + if add: + builds.add(add) + + # Get a sorted list of tags + tags = get_sorted_tags_list() + + # Make the sorted versions list from main branches and tags + versions: list[str] = [] + for version in ["master", "main"] + tags: + if version in builds: + versions.append(version) + builds.remove(version) + + # Add in anything that is left to the bottom + versions += sorted(builds) + print(f"Sorted versions: {versions}") + return versions + + +def write_json(path: Path, repository: str, versions: list[str]): + """Write the JSON switcher to path.""" + org, repo_name = repository.split("/") + struct = [ + {"version": version, "url": f"https://{org}.github.io/{repo_name}/{version}/"} + for version in versions + ] + text = json.dumps(struct, indent=2) + print(f"JSON switcher:\n{text}") + path.write_text(text, encoding="utf-8") + + +def main(args=None): + """Parse args and write switcher.""" + parser = ArgumentParser( + description="Make a versions.json file from gh-pages directories" + ) + parser.add_argument( + "--add", + help="Add this directory to the list of existing directories", + ) + parser.add_argument( + "repository", + help="The GitHub org and repository name: ORG/REPO", + ) + parser.add_argument( + "output", + type=Path, + help="Path of write switcher.json to", + ) + args = parser.parse_args(args) + + # Write the versions file + versions = get_versions("origin/gh-pages", args.add) + write_json(args.output, args.repository, versions) + + +if __name__ == "__main__": + main() diff --git a/.github/workflows/docs.yml b/.github/workflows/docs.yml index c5227cd3..9fd92ce4 100644 --- a/.github/workflows/docs.yml +++ b/.github/workflows/docs.yml @@ -2,53 +2,77 @@ name: Docs CI on: push: + branches: + - main + tags: + - '*' pull_request: +permissions: + contents: write + jobs: build: runs-on: ubuntu-latest - # Use container from ghcr - container: - image: ghcr.io/pandablocks/pandablocks-dev-container:latest - credentials: - username: ${{ github.actor }} - password: ${{ secrets.GITHUB_TOKEN }} - volumes: - - ${{ github.workspace }}:/repos - - ${{ github.workspace }}/build:/build steps: - # Git repositories - - name: Checkout Source - uses: actions/checkout@v2 + - name: Avoid git conflicts when tag and branch pushed at same time + if: github.ref_type == 'tag' + run: sleep 60 + + - name: Checkout + uses: actions/checkout@v5 with: - path: PandABlocks-server - # require history to get back to last tag for version number of branches + # Need this to get version number from last tag fetch-depth: 0 - # Build - - name: Build Docs + # PandABlocks-server is a C project, not a Python/uv project, so we install + # mystmd directly with npm rather than via `uv run tox -e docs`. The build + # command itself (`cd docs && myst build --html`) matches the template. + - name: Setup Node + uses: actions/setup-node@v4 + with: + node-version: 20 + + - name: Install MyST + run: npm install -g mystmd@1.10.1 + + - name: Sanitize ref name for docs version + run: echo "DOCS_VERSION=${GITHUB_REF_NAME//[^A-Za-z0-9._-]/_}" >> $GITHUB_ENV + + # BASE_URL is required so assets/links resolve under the versioned Pages + # sub-path (https://pandablocks.github.io/PandABlocks-server/$DOCS_VERSION/). + - name: Build docs + env: + BASE_URL: /PandABlocks-server/${{ env.DOCS_VERSION }} run: | - cd PandABlocks-server - ln -s CONFIG.example CONFIG - make docs - - # Verification of build files - #- name: List build files - # run: ls ${{ github.workspace }}/build/html - - # Publish + cd docs + myst build --html + + # Drop the ~135 MB templates/ dir (downloaded book-theme node sources, a + # build-time cache) before uploading. We keep docs/_build as the artifact + # root so the archive retains the intermediate html/ directory. + - name: Remove build cache from artifact + run: rm -rf docs/_build/templates + + - name: Upload built docs artifact + uses: actions/upload-artifact@v4 + with: + name: docs + path: docs/_build + - name: Move to versioned directory - # e.g. master or 0.1.2 - run: mv build/build-server/html "PandABlocks-server/.github/pages/${GITHUB_REF##*/}" + run: mv docs/_build/html .github/pages/$DOCS_VERSION + + - name: Write switcher.json + run: python3 .github/pages/make_switcher.py --add $DOCS_VERSION ${{ github.repository }} .github/pages/switcher.json - name: Publish Docs to gh-pages - # Only master and tags are published - if: "${{ github.repository_owner == 'PandABlocks' && (github.ref == 'refs/heads/master' || startsWith(github.ref, 'refs/tags')) }}" + if: github.ref_type == 'tag' || github.ref_name == 'main' # We pin to the SHA, not the tag, for security reasons. - # https://docs.github.com/en/free-pro-team@latest/actions/learn-github-actions/security-hardening-for-github-actions#using-third-party-actions - uses: peaceiris/actions-gh-pages@bbdfb200618d235585ad98e965f4aafc39b4c501 # v3.7.3 + # https://docs.github.com/en/actions/learn-github-actions/security-hardening-for-github-actions#using-third-party-actions + uses: peaceiris/actions-gh-pages@4f9cc6602d3f66b9c108549d475ec49e8ef4d45e # v4.0.0 with: github_token: ${{ secrets.GITHUB_TOKEN }} - publish_dir: PandABlocks-server/.github/pages - keep_files: true \ No newline at end of file + publish_dir: .github/pages + keep_files: true diff --git a/.gitignore b/.gitignore index 6677454f..39686957 100644 --- a/.gitignore +++ b/.gitignore @@ -4,3 +4,6 @@ *.pyc tests/fpga_sequences /venv + +# MyST build output +docs/_build/ diff --git a/README.md b/README.md new file mode 100644 index 00000000..6506d048 --- /dev/null +++ b/README.md @@ -0,0 +1,24 @@ +# PandABlocks-server + +The PandA socket server provides a bridge between the register interface to the +FPGA firmware controlling the PandA hardware and users and other software. The +interface is designed to be simple and robust. + +The PandA firmware is structured into numerous functional blocks, each +configured via a number of fields. This structure is directly reflected in the +functional interface provided by this server: most commands read or write +specific fields. + +The server publishes two socket end points: one for configuration control +(simple ASCII commands, ASCII responses) and one for streamed data capture +(no commands, a lightly structured binary stream). + + + +## Where to find things + +| | | +|---|---| +| Source code | | +| Documentation | | +| Releases | | diff --git a/docs/PROGRESS.md b/docs/PROGRESS.md new file mode 100644 index 00000000..8d6a3b0d --- /dev/null +++ b/docs/PROGRESS.md @@ -0,0 +1,53 @@ +# Docs Rewrite Progress — PandABlocks-server + +Tracks every target page for this repo. Update the relevant line **in the same commit** as the file +it refers to. Statuses: ☐ todo · ◐ stub · ✅ converted · 🔍 needs-review · ⛔ blocked (→ issue #). +Page list expanded from `06-source-provenance-map.md`. + +## Stage A — scaffold +- ✅ skeleton instantiated (title/github/logo swapped, apidoc + pip-tutorial + run-container removed) +- ✅ TOC + stubs for every target page present (12 stubs + 4 landing + index) +- ✅ `index.md` README-include + `how-to/contribute` CONTRIBUTING-include resolve +- ✅ xref/intersphinx wired + cross-link probe resolves in built output (reference/capture → PandABlocks-client) +- ☐ (FPGA only) — N/A +- ✅ `myst build` green (exit 0, no warnings/errors) +- ✅ Pages deploy wired — `.github/workflows/docs.yml` switched to `myst build` (copier-aligned) + +## Tutorials +- (none in v4 — empty `tutorials.md` landing kept for parity) + +## How-to +- ◐ how-to/startup — source: startup.rst — writable-now +- ◐ how-to/building — source: building.rst — writable-now (CONFIG syntax = verify) +- ✅ how-to/contribute — `{include} .github/CONTRIBUTING.md` — scaffold include resolves + +## Explanations +- ◐ explanations/architecture — source: NEW (server code + docs) — writable-now (skeleton; depth = author) + +## Reference +- ◐ reference/commands — source: commands.rst — writable-now [Prompt E] +- ◐ reference/fields — source: fields.rst — writable-now [Prompt E] +- ◐ reference/capture — source: capture.rst — writable-now (60MB/s = verify) [Prompt E]. Holds the Stage A xref probe. +- ◐ reference/capture-options — source: capture.rst ⊃ + Interview5 §4 — partial (superset = verify) [Prompt E split] +- ◐ reference/config — source: config.rst — writable-now +- ⛔ reference/extension — source: extension.rst — blocked: verify +- ◐ reference/streaming-tables — source: NEW (Interview5 §8) — writable-now +- ◐ reference/support — source: support.rst — writable-now +- ⛔ reference/c-standard — source: NEW (Interview1 §4) — blocked: author + +## Blocked (issues raised) +Issues not yet created — Stage B (Prompt B) will create and link them. +- ⛔ reference/extension — issue #TBD — verify +- ⛔ reference/c-standard — issue #TBD — author + +## Notes +- **Legacy sources preserved.** Original Sphinx/RST tree moved to `docs/_legacy_rst/` (not in TOC, + ignored by `myst build`) so Stage D/E conversion can read sources in-tree; also on `origin/main`. + `presentation-2016-05/*` is dropped per 06 (kept under `_legacy_rst/` for now; delete with the rest + of `_legacy_rst/` once conversion is complete). +- **Default branch is `main`** — publish gated on `main`/tags; redirect to `./main/index.html`. +- **xref/intersphinx prototype.** Only `PandABlocks-client` (deployed) is active; the probe in + `reference/capture` resolves in built output. meta-panda + FPGA + devcontainer + fastcs kept + commented until published; uncomment in Stage F and upstream into python-copier-template. +- **CI** mirrors python-copier-template-example `_docs.yml` (npm+mystmd build, upload-artifact of + `docs/_build` minus the templates cache, versioned move, `make_switcher.py`, peaceiris v4). diff --git a/docs/PandA-logo-for-black-background.svg b/docs/_legacy_rst/PandA-logo-for-black-background.svg similarity index 100% rename from docs/PandA-logo-for-black-background.svg rename to docs/_legacy_rst/PandA-logo-for-black-background.svg diff --git a/docs/_static/theme_overrides.css b/docs/_legacy_rst/_static/theme_overrides.css similarity index 100% rename from docs/_static/theme_overrides.css rename to docs/_legacy_rst/_static/theme_overrides.css diff --git a/docs/_templates/layout.html b/docs/_legacy_rst/_templates/layout.html similarity index 100% rename from docs/_templates/layout.html rename to docs/_legacy_rst/_templates/layout.html diff --git a/docs/building.rst b/docs/_legacy_rst/building.rst similarity index 100% rename from docs/building.rst rename to docs/_legacy_rst/building.rst diff --git a/docs/capture.rst b/docs/_legacy_rst/capture.rst similarity index 100% rename from docs/capture.rst rename to docs/_legacy_rst/capture.rst diff --git a/docs/commands.rst b/docs/_legacy_rst/commands.rst similarity index 100% rename from docs/commands.rst rename to docs/_legacy_rst/commands.rst diff --git a/docs/conf.py b/docs/_legacy_rst/conf.py similarity index 100% rename from docs/conf.py rename to docs/_legacy_rst/conf.py diff --git a/docs/config.rst b/docs/_legacy_rst/config.rst similarity index 100% rename from docs/config.rst rename to docs/_legacy_rst/config.rst diff --git a/docs/extension.rst b/docs/_legacy_rst/extension.rst similarity index 100% rename from docs/extension.rst rename to docs/_legacy_rst/extension.rst diff --git a/docs/favicon.ico b/docs/_legacy_rst/favicon.ico similarity index 100% rename from docs/favicon.ico rename to docs/_legacy_rst/favicon.ico diff --git a/docs/fields.rst b/docs/_legacy_rst/fields.rst similarity index 100% rename from docs/fields.rst rename to docs/_legacy_rst/fields.rst diff --git a/docs/index.rst b/docs/_legacy_rst/index.rst similarity index 100% rename from docs/index.rst rename to docs/_legacy_rst/index.rst diff --git a/docs/presentation-2016-05/.gitignore b/docs/_legacy_rst/presentation-2016-05/.gitignore similarity index 100% rename from docs/presentation-2016-05/.gitignore rename to docs/_legacy_rst/presentation-2016-05/.gitignore diff --git a/docs/presentation-2016-05/Makefile b/docs/_legacy_rst/presentation-2016-05/Makefile similarity index 100% rename from docs/presentation-2016-05/Makefile rename to docs/_legacy_rst/presentation-2016-05/Makefile diff --git a/docs/presentation-2016-05/building.rst b/docs/_legacy_rst/presentation-2016-05/building.rst similarity index 100% rename from docs/presentation-2016-05/building.rst rename to docs/_legacy_rst/presentation-2016-05/building.rst diff --git a/docs/presentation-2016-05/docutils.conf b/docs/_legacy_rst/presentation-2016-05/docutils.conf similarity index 100% rename from docs/presentation-2016-05/docutils.conf rename to docs/_legacy_rst/presentation-2016-05/docutils.conf diff --git a/docs/presentation-2016-05/images/admin.png b/docs/_legacy_rst/presentation-2016-05/images/admin.png similarity index 100% rename from docs/presentation-2016-05/images/admin.png rename to docs/_legacy_rst/presentation-2016-05/images/admin.png diff --git a/docs/presentation-2016-05/images/architecture.png b/docs/_legacy_rst/presentation-2016-05/images/architecture.png similarity index 100% rename from docs/presentation-2016-05/images/architecture.png rename to docs/_legacy_rst/presentation-2016-05/images/architecture.png diff --git a/docs/presentation-2016-05/images/software.png b/docs/_legacy_rst/presentation-2016-05/images/software.png similarity index 100% rename from docs/presentation-2016-05/images/software.png rename to docs/_legacy_rst/presentation-2016-05/images/software.png diff --git a/docs/presentation-2016-05/images/system.png b/docs/_legacy_rst/presentation-2016-05/images/system.png similarity index 100% rename from docs/presentation-2016-05/images/system.png rename to docs/_legacy_rst/presentation-2016-05/images/system.png diff --git a/docs/presentation-2016-05/panda.rst b/docs/_legacy_rst/presentation-2016-05/panda.rst similarity index 100% rename from docs/presentation-2016-05/panda.rst rename to docs/_legacy_rst/presentation-2016-05/panda.rst diff --git a/docs/presentation-2016-05/server.rst b/docs/_legacy_rst/presentation-2016-05/server.rst similarity index 100% rename from docs/presentation-2016-05/server.rst rename to docs/_legacy_rst/presentation-2016-05/server.rst diff --git a/docs/presentation-2016-05/styles/blank.gif b/docs/_legacy_rst/presentation-2016-05/styles/blank.gif similarity index 100% rename from docs/presentation-2016-05/styles/blank.gif rename to docs/_legacy_rst/presentation-2016-05/styles/blank.gif diff --git a/docs/presentation-2016-05/styles/dlsfooterpad.png b/docs/_legacy_rst/presentation-2016-05/styles/dlsfooterpad.png similarity index 100% rename from docs/presentation-2016-05/styles/dlsfooterpad.png rename to docs/_legacy_rst/presentation-2016-05/styles/dlsfooterpad.png diff --git a/docs/presentation-2016-05/styles/framing.css b/docs/_legacy_rst/presentation-2016-05/styles/framing.css similarity index 100% rename from docs/presentation-2016-05/styles/framing.css rename to docs/_legacy_rst/presentation-2016-05/styles/framing.css diff --git a/docs/presentation-2016-05/styles/html4css1.css b/docs/_legacy_rst/presentation-2016-05/styles/html4css1.css similarity index 100% rename from docs/presentation-2016-05/styles/html4css1.css rename to docs/_legacy_rst/presentation-2016-05/styles/html4css1.css diff --git a/docs/presentation-2016-05/styles/iepngfix.htc b/docs/_legacy_rst/presentation-2016-05/styles/iepngfix.htc similarity index 100% rename from docs/presentation-2016-05/styles/iepngfix.htc rename to docs/_legacy_rst/presentation-2016-05/styles/iepngfix.htc diff --git a/docs/presentation-2016-05/styles/opera.css b/docs/_legacy_rst/presentation-2016-05/styles/opera.css similarity index 100% rename from docs/presentation-2016-05/styles/opera.css rename to docs/_legacy_rst/presentation-2016-05/styles/opera.css diff --git a/docs/presentation-2016-05/styles/outline.css b/docs/_legacy_rst/presentation-2016-05/styles/outline.css similarity index 100% rename from docs/presentation-2016-05/styles/outline.css rename to docs/_legacy_rst/presentation-2016-05/styles/outline.css diff --git a/docs/presentation-2016-05/styles/pretty.css b/docs/_legacy_rst/presentation-2016-05/styles/pretty.css similarity index 100% rename from docs/presentation-2016-05/styles/pretty.css rename to docs/_legacy_rst/presentation-2016-05/styles/pretty.css diff --git a/docs/presentation-2016-05/styles/print.css b/docs/_legacy_rst/presentation-2016-05/styles/print.css similarity index 100% rename from docs/presentation-2016-05/styles/print.css rename to docs/_legacy_rst/presentation-2016-05/styles/print.css diff --git a/docs/presentation-2016-05/styles/pygments.css b/docs/_legacy_rst/presentation-2016-05/styles/pygments.css similarity index 100% rename from docs/presentation-2016-05/styles/pygments.css rename to docs/_legacy_rst/presentation-2016-05/styles/pygments.css diff --git a/docs/presentation-2016-05/styles/s5-core.css b/docs/_legacy_rst/presentation-2016-05/styles/s5-core.css similarity index 100% rename from docs/presentation-2016-05/styles/s5-core.css rename to docs/_legacy_rst/presentation-2016-05/styles/s5-core.css diff --git a/docs/presentation-2016-05/styles/slides.css b/docs/_legacy_rst/presentation-2016-05/styles/slides.css similarity index 100% rename from docs/presentation-2016-05/styles/slides.css rename to docs/_legacy_rst/presentation-2016-05/styles/slides.css diff --git a/docs/presentation-2016-05/styles/slides.js b/docs/_legacy_rst/presentation-2016-05/styles/slides.js similarity index 100% rename from docs/presentation-2016-05/styles/slides.js rename to docs/_legacy_rst/presentation-2016-05/styles/slides.js diff --git a/docs/startup.rst b/docs/_legacy_rst/startup.rst similarity index 100% rename from docs/startup.rst rename to docs/_legacy_rst/startup.rst diff --git a/docs/support.rst b/docs/_legacy_rst/support.rst similarity index 100% rename from docs/support.rst rename to docs/_legacy_rst/support.rst diff --git a/docs/explanations.md b/docs/explanations.md new file mode 100644 index 00000000..a1dca5ac --- /dev/null +++ b/docs/explanations.md @@ -0,0 +1,7 @@ +# Explanations + +Explanations of how it works and why it works that way. + +```{toc} +:context: children +``` diff --git a/docs/explanations/architecture.md b/docs/explanations/architecture.md new file mode 100644 index 00000000..4bd5fa9d --- /dev/null +++ b/docs/explanations/architecture.md @@ -0,0 +1,11 @@ +# Server architecture + +:::{admonition} 🚧 TODO — documentation stub +:class: note + +This page is a Stage A scaffold stub and has not yet been written. + +**Status:** writable-now (depth = blocked: author) + +**Source:** NEW (server code + docs) — TCP/config/data threads, block model +::: diff --git a/docs/how-to.md b/docs/how-to.md new file mode 100644 index 00000000..ba801fd7 --- /dev/null +++ b/docs/how-to.md @@ -0,0 +1,7 @@ +# How-to Guides + +Practical step-by-step guides for the more experienced user. + +```{toc} +:context: children +``` diff --git a/docs/how-to/building.md b/docs/how-to/building.md new file mode 100644 index 00000000..b704741e --- /dev/null +++ b/docs/how-to/building.md @@ -0,0 +1,11 @@ +# Build and test the server + +:::{admonition} 🚧 TODO — documentation stub +:class: note + +This page is a Stage A scaffold stub and has not yet been written. + +**Status:** writable-now (CONFIG syntax = blocked: verify) + +**Source:** building.rst (493w: Dependencies, CONFIG file, Build Targets, Generated Files) — point to PandABlocks-devcontainer for local dev +::: diff --git a/docs/how-to/contribute.md b/docs/how-to/contribute.md new file mode 100644 index 00000000..6e419797 --- /dev/null +++ b/docs/how-to/contribute.md @@ -0,0 +1,2 @@ +```{include} ../../.github/CONTRIBUTING.md +``` diff --git a/docs/how-to/startup.md b/docs/how-to/startup.md new file mode 100644 index 00000000..f45f741e --- /dev/null +++ b/docs/how-to/startup.md @@ -0,0 +1,11 @@ +# Start the PandA server + +:::{admonition} 🚧 TODO — documentation stub +:class: note + +This page is a Stage A scaffold stub and has not yet been written. + +**Status:** writable-now + +**Source:** startup.rst (501w: Starting Panda Server) +::: diff --git a/docs/images/PandA-logo-for-black-background.svg b/docs/images/PandA-logo-for-black-background.svg new file mode 100644 index 00000000..f52449b4 --- /dev/null +++ b/docs/images/PandA-logo-for-black-background.svg @@ -0,0 +1,331 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + image/svg+xml + + + + + + + + + + + + + + + + + diff --git a/docs/index.md b/docs/index.md new file mode 100644 index 00000000..6c00c2f8 --- /dev/null +++ b/docs/index.md @@ -0,0 +1,42 @@ +--- +site: + hide_outline: true +--- + +```{include} ../README.md +:end-before: +Captured data can be read out programmatically with the Python client's +[`BlockingClient`](xref:PandABlocks-client#pandablocks.blocking.BlockingClient). diff --git a/docs/reference/commands.md b/docs/reference/commands.md new file mode 100644 index 00000000..eb8bb3f9 --- /dev/null +++ b/docs/reference/commands.md @@ -0,0 +1,11 @@ +# Command interface + +:::{admonition} 🚧 TODO — documentation stub +:class: note + +This page is a Stage A scaffold stub and has not yet been written. + +**Status:** writable-now + +**Source:** commands.rst (1984w: Configuration Commands, System Commands) — keep *IDN? +::: diff --git a/docs/reference/config.md b/docs/reference/config.md new file mode 100644 index 00000000..4ad008a9 --- /dev/null +++ b/docs/reference/config.md @@ -0,0 +1,11 @@ +# Config, registers and description files + +:::{admonition} 🚧 TODO — documentation stub +:class: note + +This page is a Stage A scaffold stub and has not yet been written. + +**Status:** writable-now + +**Source:** config.rst (1198w: config, registers [+ extension syntax], description) +::: diff --git a/docs/reference/extension.md b/docs/reference/extension.md new file mode 100644 index 00000000..0ecf431c --- /dev/null +++ b/docs/reference/extension.md @@ -0,0 +1,13 @@ +# Extension server + +:::{admonition} 🚧 TODO — documentation stub +:class: warning + +This page is a Stage A scaffold stub and has not yet been written. + +**Status:** blocked: verify + +**Source:** extension.rst (739w: Extension Modules, Injected Values) — verify vs implementation + +**Blocked:** a GitHub issue will be created in Stage B (Prompt B) and linked here. +::: diff --git a/docs/reference/fields.md b/docs/reference/fields.md new file mode 100644 index 00000000..0bc42d18 --- /dev/null +++ b/docs/reference/fields.md @@ -0,0 +1,11 @@ +# Field types and attributes + +:::{admonition} 🚧 TODO — documentation stub +:class: note + +This page is a Stage A scaffold stub and has not yet been written. + +**Status:** writable-now + +**Source:** fields.rst (2671w: Field Types, Field Sub-Types, Summary of Sub-Types, Summary of Attributes) — keep MODE matrix; point to streaming-tables +::: diff --git a/docs/reference/streaming-tables.md b/docs/reference/streaming-tables.md new file mode 100644 index 00000000..58c9228a --- /dev/null +++ b/docs/reference/streaming-tables.md @@ -0,0 +1,11 @@ +# Streaming tables + +:::{admonition} 🚧 TODO — documentation stub +:class: note + +This page is a Stage A scaffold stub and has not yet been written. + +**Status:** writable-now + +**Source:** NEW (Interview5 §8) — << / <<|, buffer sizes +::: diff --git a/docs/reference/support.md b/docs/reference/support.md new file mode 100644 index 00000000..06c9b9aa --- /dev/null +++ b/docs/reference/support.md @@ -0,0 +1,11 @@ +# Supporting tools and LEDs + +:::{admonition} 🚧 TODO — documentation stub +:class: note + +This page is a Stage A scaffold stub and has not yet been written. + +**Status:** writable-now + +**Source:** support.rst (391w: Useful Tools, Panda Status LEDs) +::: diff --git a/docs/tutorials.md b/docs/tutorials.md new file mode 100644 index 00000000..66c75886 --- /dev/null +++ b/docs/tutorials.md @@ -0,0 +1,6 @@ +# Tutorials + +```{note} +The PandABlocks-server has no tutorials yet. New users should start with the +[meta-panda tutorials](https://PandABlocks.github.io/meta-panda). +``` From 00e7a4520af5ced10cfca5e19caa197280a1e231 Mon Sep 17 00:00:00 2001 From: Tom Cobb Date: Thu, 11 Jun 2026 13:23:24 +0000 Subject: [PATCH 02/28] docs: point commented FPGA xref at /main (was /master) Co-Authored-By: Claude Opus 4.8 --- docs/myst.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/myst.yml b/docs/myst.yml index 387613e7..485022d3 100644 --- a/docs/myst.yml +++ b/docs/myst.yml @@ -11,7 +11,7 @@ project: # Not deployed yet — kept (commented) as the complete prototype wiring to # uncomment in Stage F and upstream into python-copier-template. # meta-panda: https://pandablocks.github.io/meta-panda/main - # PandABlocks-FPGA: https://pandablocks.github.io/PandABlocks-FPGA/master + # PandABlocks-FPGA: https://pandablocks.github.io/PandABlocks-FPGA/main # PandABlocks-devcontainer: https://pandablocks.github.io/PandABlocks-devcontainer # fastcs-PandABlocks: https://diamondlightsource.github.io/fastcs-PandABlocks/main toc: From b81f6bcc508740eddfeabe9180cb0cd8c7fb1d0f Mon Sep 17 00:00:00 2001 From: Tom Cobb Date: Thu, 11 Jun 2026 13:56:55 +0000 Subject: [PATCH 03/28] docs: convert reference/support.md from RST (Prompt D) Co-Authored-By: Claude Sonnet 4.6 --- docs/reference/support.md | 62 +++++++++++++++++++++++++++++++++++---- 1 file changed, 56 insertions(+), 6 deletions(-) diff --git a/docs/reference/support.md b/docs/reference/support.md index 06c9b9aa..4c39fdd4 100644 --- a/docs/reference/support.md +++ b/docs/reference/support.md @@ -1,11 +1,61 @@ # Supporting tools and LEDs -:::{admonition} 🚧 TODO — documentation stub -:class: note +## Useful tools -This page is a Stage A scaffold stub and has not yet been written. +Several supporting tools are found in the `python/` directory of the server source. -**Status:** writable-now +`sim_server` +: Runs as part of the top-level `simserver` script to provide emulation of the + PandA hardware. The version supplied with the server is minimal; for a more + functional emulation use the corresponding tool in the PandABlocks-FPGA project. -**Source:** support.rst (391w: Useful Tools, Panda Status LEDs) -::: +`tcp_client [server [port]]` +: Connects to the PandA server configuration port and helps with sending and + receiving configuration commands. + +`save-state server file` +: Saves the entire configuration state for the given PandA to the given file. + +`load-state server file` +: Writes the given configuration file to the PandA. + +## PandA status LEDs + +Two LEDs on PandA give a rough indication of its current status: **STA** (status, +green) indicates normal activity, and **DIA** (diagnostic, red) indicates fault +conditions. + +| Mnemonic | DIA | STA | Meaning | +|------------|-------|-------|------------------------------------| +| — | Off | Off | System not running | +| BOOTING | Off | Blink | PandA booting | +| SYSTEM_OK | Off | On | PandA running OK | +| ATTENTION | Blink | Off | User attention required | +| NW_ERR | Blink | Blink | Network problem detected | +| — | Blink | On | (not used, should not occur) | +| ZPKG_ERR | On | Off | Problem loading installed package | +| SYSTEM_ERR | On | Blink | System error | +| — | On | On | (not used, should not occur) | + +`BOOTING` +: The system is currently booting. Unless a new image is being configured this + should only take a few seconds; during image installation it can take a few + minutes. + +`SYSTEM_OK` +: Booting has completed and the system is running normally. + +`ATTENTION` +: User attention is required. Either a fresh installation is prompting for a MAC + address, or no system packages have been installed. Connect a serial port in the + first case, or connect to the administration web page on port 8080 in the second. + +`NW_ERR` +: A network error has been detected. (Not currently implemented.) + +`ZPKG_ERR` +: An installed package has failed to start. Try power-cycling PandA first; if that + fails, check the logs and serial port for relevant diagnostic messages. + +`SYSTEM_ERR` +: An internal system error has been detected. (Not currently implemented.) From 3c5f76ebb67df8e329f59f9b72b040f7a75f860e Mon Sep 17 00:00:00 2001 From: Tom Cobb Date: Thu, 11 Jun 2026 14:14:42 +0000 Subject: [PATCH 04/28] =?UTF-8?q?docs:=20Prompt=20D=20bulk=20conversion=20?= =?UTF-8?q?=E2=80=94=20all=20quadrants=20(PandABlocks-server)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Converts all writable-now Prompt D pages for the server repo: - how-to/startup (from startup.rst) - how-to/building (from building.rst; devcontainer pointer added; CONFIG syntax verify noted) - reference/config (from config.rst) - reference/streaming-tables (new; authored from Interview5 §8 + fields.rst) - explanations/architecture (skeleton from server code) Skipped per spec: commands/fields/capture/capture-options (Prompt E), extension (blocked:verify), c-standard (blocked:author). myst build green after each file. Co-Authored-By: Claude Sonnet 4.6 --- docs/PROGRESS.md | 12 +-- docs/explanations/architecture.md | 54 ++++++++++- docs/how-to/building.md | 88 ++++++++++++++++- docs/how-to/startup.md | 61 ++++++++++-- docs/reference/config.md | 146 +++++++++++++++++++++++++++-- docs/reference/streaming-tables.md | 82 ++++++++++++++-- 6 files changed, 409 insertions(+), 34 deletions(-) diff --git a/docs/PROGRESS.md b/docs/PROGRESS.md index 8d6a3b0d..cefd8af4 100644 --- a/docs/PROGRESS.md +++ b/docs/PROGRESS.md @@ -17,22 +17,22 @@ Page list expanded from `06-source-provenance-map.md`. - (none in v4 — empty `tutorials.md` landing kept for parity) ## How-to -- ◐ how-to/startup — source: startup.rst — writable-now -- ◐ how-to/building — source: building.rst — writable-now (CONFIG syntax = verify) +- ✅ how-to/startup — converted from startup.rst (Prompt D) +- ✅ how-to/building — converted from building.rst; devcontainer pointer added; CONFIG syntax note = verify (Prompt D) - ✅ how-to/contribute — `{include} .github/CONTRIBUTING.md` — scaffold include resolves ## Explanations -- ◐ explanations/architecture — source: NEW (server code + docs) — writable-now (skeleton; depth = author) +- ✅ explanations/architecture — skeleton authored from server code (Prompt D); depth = blocked: author ## Reference - ◐ reference/commands — source: commands.rst — writable-now [Prompt E] - ◐ reference/fields — source: fields.rst — writable-now [Prompt E] - ◐ reference/capture — source: capture.rst — writable-now (60MB/s = verify) [Prompt E]. Holds the Stage A xref probe. - ◐ reference/capture-options — source: capture.rst ⊃ + Interview5 §4 — partial (superset = verify) [Prompt E split] -- ◐ reference/config — source: config.rst — writable-now +- ✅ reference/config — converted from config.rst (Prompt D) - ⛔ reference/extension — source: extension.rst — blocked: verify -- ◐ reference/streaming-tables — source: NEW (Interview5 §8) — writable-now -- ◐ reference/support — source: support.rst — writable-now +- ✅ reference/streaming-tables — authored from Interview5 §8 + fields.rst streaming section (Prompt D) +- ✅ reference/support — converted from support.rst (Prompt D) - ⛔ reference/c-standard — source: NEW (Interview1 §4) — blocked: author ## Blocked (issues raised) diff --git a/docs/explanations/architecture.md b/docs/explanations/architecture.md index 4bd5fa9d..0b0a2d99 100644 --- a/docs/explanations/architecture.md +++ b/docs/explanations/architecture.md @@ -1,11 +1,55 @@ # Server architecture -:::{admonition} 🚧 TODO — documentation stub +:::{admonition} Skeleton — depth coming later :class: note -This page is a Stage A scaffold stub and has not yet been written. +This page provides an orientation skeleton. Detailed internals (locking +strategy, DMA pipeline, persistence state machine) are tracked in the issue +backlog as **blocked: author**. +::: -**Status:** writable-now (depth = blocked: author) +The PandABlocks server is a C daemon that presents two TCP sockets and bridges +them to the PandA FPGA hardware (or a software simulation of it). -**Source:** NEW (server code + docs) — TCP/config/data threads, block model -::: +## Sockets + +**Configuration port (default 8888)** +: An ASCII command/response interface used to read and write block fields and + execute system commands. Described in {doc}`/reference/commands`. + +**Data port (default 8889)** +: A binary streaming interface for captured experiment data. Described in + {doc}`/reference/capture`. + +## Internal structure + +The server is organised around three cooperating layers. + +**Configuration thread** (`config_server.c`) +: Accepts connections on the configuration port. Parses incoming ASCII commands, + dispatches them to the block/field database (`config_command.c`, `fields.c`), + and formats responses. One thread per client connection. + +**Data thread** (`data_server.c`) +: Accepts connections on the data port. Reads captured data from the central + circular buffer and streams it to connected clients. Handles framing, + base-64 encoding, and metadata headers. + +**Hardware layer** (`hardware.c` / `sim_hardware.c`) +: Abstracts register reads/writes and DMA transfers to the FPGA. In simulation + mode (`sim_hardware.c`) the hardware layer is replaced by an in-process + emulator. + +## Block model + +The server loads its block and field topology from the `config_d` configuration +files at startup (see {doc}`/reference/config`). Each block is a named, +potentially multi-instance hardware component (e.g. `PULSE[4]`). Fields within +a block map to registers and expose typed operations (read, write, capture, etc.) +as documented in {doc}`/reference/fields`. + +## Persistence + +Field values that survive a server restart are written to a persistence file +(see the `-f` and `-t` options in {doc}`/how-to/startup`). The file is updated +on a poll/holdoff/backoff schedule to reduce write pressure. diff --git a/docs/how-to/building.md b/docs/how-to/building.md index b704741e..cb0f817f 100644 --- a/docs/how-to/building.md +++ b/docs/how-to/building.md @@ -1,11 +1,89 @@ # Build and test the server -:::{admonition} 🚧 TODO — documentation stub -:class: note +:::{tip} +For local development the [PandABlocks-devcontainer](https://github.com/PandABlocks/PandABlocks-devcontainer) +provides a pre-configured environment and is the recommended starting point. +The instructions below cover a native build for those who need it. +::: + +## Dependencies + +The following must be available before building the server. + +**Zynq cross-compiler toolchain** +: Available as part of the Xilinx Vivado build environment, or any ARMv7-A + cross-compiler. Required for all applications running on PandA. + +**PandABlocks-FPGA** +: Must be available before building the server; it contains the configuration + file defining the low-level register interface to the PandA firmware. + +**PandABlocks-rootfs** +: Provides the kernel build tree and the `zpkg` build tool. -This page is a Stage A scaffold stub and has not yet been written. +## Setting up the `CONFIG` file -**Status:** writable-now (CONFIG syntax = blocked: verify) +Copy `CONFIG.example` to `CONFIG` in the base directory and edit as needed, +commenting out lines that are not required. -**Source:** building.rst (493w: Dependencies, CONFIG file, Build Targets, Generated Files) — point to PandABlocks-devcontainer for local dev +:::{note} +The exact syntax accepted by the server Makefile for `CONFIG` values is pending +verification (tracked in the issue backlog). Use the form shown in +`CONFIG.example`. ::: + +The following symbols must point to the appropriate dependencies: + +`BINUTILS_DIR` +: Path to the Zynq cross-compiler toolchain if it is not on `PATH`. Not required + for the simulation server or documentation builds. + +`KERNEL_DIR` +: Path to the kernel build tree from the PandABlocks-rootfs build. Required to + build the kernel module. + +`PANDA_ROOTFS` +: Path to the PandABlocks-rootfs checkout where the `zpkg` build tool is found. + +The following symbols can be left at their defaults: + +`BUILD_DIR` +: Destination for built files. + +`PYTHON` +: Python interpreter used during the build. + +`SPHINX_BUILD` +: `sphinx-build` script for documentation builds. + +`DEFAULT_TARGETS` +: Makefile targets built by `make` or `make default`. Default list: `driver`, + `server`, `sim_server`, `docs`, `zpkg`. + +## Build targets + +| Target | Description | +|--------------|----------------------------------------------------------| +| `default` | Builds all targets listed in `$(DEFAULT_TARGETS)` | +| `driver` | Kernel driver module | +| `server` | Server binary to run on PandA | +| `sim_server` | Simulation server to run on the local PC | +| `docs` | HTML documentation | +| `zpkg` | `panda-server` zpkg package | +| `clean` | Removes the entire `$(BUILD_DIR)` directory | + +## Generated files + +After a successful build, `$(BUILD_DIR)` contains: + +`driver/` +: Kernel module for hardware access. + +`server/` and `sim_server/` +: The on-PandA server and the local simulation server respectively. + +`html/` +: HTML documentation. + +`panda-server@`*version*`.zpg` and `zpkg-panda-server/` +: The zpkg package for installation onto PandA. diff --git a/docs/how-to/startup.md b/docs/how-to/startup.md index f45f741e..8a9873d5 100644 --- a/docs/how-to/startup.md +++ b/docs/how-to/startup.md @@ -1,11 +1,60 @@ # Start the PandA server -:::{admonition} 🚧 TODO — documentation stub -:class: note +The PandA socket server normally starts automatically at boot time or when the +`zpkg-daemon` script is run. It is started and stopped by the script +`etc/panda-server` installed at `/opt/etc/init.d`. -This page is a Stage A scaffold stub and has not yet been written. +The server can also be started from the command line with the following options. -**Status:** writable-now +`-h` +: Display help text listing all available command-line options. -**Source:** startup.rst (501w: Starting Panda Server) -::: +`-p` *port* +: TCP port for configuration commands. Default: `8888`. + +`-d` *port* +: TCP port for data capture. Default: `8889`. + +`-R` +: Allow socket reuse via the `SO_REUSEADDR` socket option. + +`-c` *config-dir* +: Directory from which the `config`, `registers`, and `description` files are + loaded. **This argument must be specified.** + +`-f` *persistence-file* +: File from which persistence state is loaded on startup and to which it is saved + during operation. If not specified, persistence state is not saved. + +`-t` *[poll][:holdoff[:backoff]]* +: Three parameters (in seconds) controlling writes to the persistence file. + + Every *poll* seconds the server checks for configuration changes. If a change + is found, it waits a further *holdoff* seconds before writing. It then waits + *backoff* seconds before resuming polling. + + Defaults: `poll` = 2, `holdoff` = 10, `backoff` = 60. With these defaults all + parameters reach the persistence file within 72 seconds. + + The colon syntax lets you set any subset: for example `-t:20` sets *holdoff* = 20 + while leaving *poll* and *backoff* unchanged. + +`-D` +: Run the server as a daemon. This is the normal production mode; omit for + debugging. + +`-p` *pid-file* +: Write the server process ID to the given file; the file is deleted on exit. + +`-T` +: Config-file validation mode: the server exits immediately after loading + configuration files without accepting connections. + +`-M` *MAC-list* +: Initialise up to four MAC address registers from the given file. The file may + contain comment lines (starting with `#`) and up to four MAC address lines, + each either blank or a six-octet address in `XX:XX:XX:XX:XX:XX` format. + +`-X` *port* +: Connect to an extension server running locally on the specified port. See + {doc}`/reference/extension` for details. diff --git a/docs/reference/config.md b/docs/reference/config.md index 4ad008a9..c443d869 100644 --- a/docs/reference/config.md +++ b/docs/reference/config.md @@ -1,11 +1,145 @@ # Config, registers and description files -:::{admonition} 🚧 TODO — documentation stub -:class: note +On startup the PandA server loads its configuration from three files: `config`, +`registers`, and `description`. When running the simulation these are loaded from +the `config_d` directory in the build directory; on PandA they are loaded from +`/opt/share/panda/config_d`. -This page is a Stage A scaffold stub and has not yet been written. +The syntax of each file mirrors the field definitions described in +{doc}`/reference/fields`. All three files share a common structure: indentation +indicates hierarchy, `#` starts a comment, block names appear in the first +column, and fields are indented one level. -**Status:** writable-now +| File | Purpose | +|---------------|--------------------------------------------------------------| +| `config` | Defines all blocks, their fields, and each field's behaviour | +| `registers` | Maps each block and field to hardware register offsets | +| `description` | Optionally provides human-readable descriptions | -**Source:** config.rst (1198w: config, registers [+ extension syntax], description) -::: +## Configuration file `config` + +This file defines all blocks and fields available to this PandA instance and is +processed first. + +**Block definition syntax:** + +``` +block-name [ "[" count "]" ] + [ field-definition ]* +``` + +A block definition gives the block a name and optionally a repeat *count* (the +number of hardware instances). It is followed by indented field definitions. + +**Field definition syntax:** + +``` +field-name field-type [ field-type-data ] +``` + +The *field-type* determines the basic function of the field, what operations are +permitted on it, and how it maps to hardware registers. + +### Field types + +| Field type | Description | +|-----------------------------------------|-----------------------------------------------------| +| `param` *subtype* [`=` *value*] | Single 32-bit value written to a register; optional initial value | +| `read` *subtype* | Read-only register | +| `write` *subtype* | Write-only register (triggers an immediate action) | +| `time` | Like `param` but 64-bit, spanning two registers | +| `bit_out` | Output bit | +| `pos_out` [*scale* [*offset* [*units*]]]| Position bus output; optional default scale/offset/units | +| `ext_out` (`timestamp` \| `samples` \| `bits` *group*) | Extension bus entry needing special treatment | +| `bit_mux` [`=` *value*] | Bit bus input selector; optionally defaults to `ONE` (129) | +| `pos_mux` | Position bus input selector | +| `table` [*row-words*] | Long table of numbers with special access methods | + +`ext_out` *ext-extra* values: + +| Value | Description | +|-----------------|----------------------------------------------------------| +| `timestamp` | Captures a 64-bit timestamp | +| `samples` | Captures the sample count for data capture | +| `bits` *group* | Captures 32 bits of the 128-bit bit bus; *group* selects which 32-bit slice | + +### Field subtypes + +`param`, `read`, and `write` fields require one of the following subtypes: + +| Subtype | Description | +|-----------------------------------------|-----------------------------------------------------| +| `uint` [*max-value*] | Unsigned integer, optionally bounded | +| `int` | Signed integer | +| `scalar` *scale* [*offset* [*units*]] | Fixed-point value with scale and optional offset/units | +| `bit` | Single bit | +| `action` | Write-only trigger with no data payload | +| `lut` | 5-input lookup table function | +| `enum` | Enumeration; followed by indented `number string` pairs | +| `position` | Position value | +| `time` | Time value | + +## Register file `registers` + +This file assigns hardware registers to each block and field. Its structure +mirrors `config` but replaces field type specifications with register +assignments. + +**Block definition syntax:** + +``` +block-name { [ "S" ] block-register | "X" } [ extension-module ] + [ field-definition ]* +``` + +- Prefix `S` on *block-register* allows the same register to be shared across + multiple blocks. +- `X` instead of a register number marks a block with no register-mapped fields + (extension-only fields only). +- An optional *extension-module* name enables the extension register syntax for + this block. + +**Field register syntax by type:** + +| Class | Register syntax | +|------------------------|-------------------------------------------------------| +| `param` | *register* \| *write-extension* | +| `read` | *register* \| *read-extension* | +| `write` | *register* \| *write-extension* | +| `time` | *low-register* *high-register* | +| `bit_out` | `(`*bit-index*`)`N | +| `pos_out` | `(`*pos-index*`)`N | +| `ext_out timestamp` | *ext-index* *ext-index* | +| `ext_out` other | *ext-index* | +| `bit_mux` | *register* | +| `pos_mux` | *register* | +| `table` (short) | `short` *size* *init-reg* *fill-reg* *length-reg* | +| `table` (long/DMA) | `long` `2^`*size* *nbuf* *base-reg* *length-reg* | + +The notation `(...)`N means the register number is repeated N times, once per +hardware instance of the block. + +### Extension register syntax + +When an extension server is enabled (see {doc}`/how-to/startup` `-X` option) and +an extension module is associated with a block, `param`, `read`, and `write` +fields may be redirected to it: + +``` +read-extension = [ read-reg ]* "X" field-spec +write-extension = [ read-reg ]* [ "W" [ write-reg ]* ] "X" field-spec +``` + +*field-spec* is passed to the extension module to bind the field. See +{doc}`/reference/extension` for details on extension fields. + +## Description file `description` + +The entire file is optional. Its syntax is: + +``` +block-name block-description + field-name field-description +``` + +Descriptions are newline-terminated UTF-8 strings. diff --git a/docs/reference/streaming-tables.md b/docs/reference/streaming-tables.md index 58c9228a..1da8aadb 100644 --- a/docs/reference/streaming-tables.md +++ b/docs/reference/streaming-tables.md @@ -1,11 +1,81 @@ # Streaming tables -:::{admonition} 🚧 TODO — documentation stub -:class: note +Tables in PandA operate in two modes: **fixed** (a single write replaces the +whole table) and **streaming** (a sequence of writes fills a DMA-backed ring +buffer that the hardware consumes continuously). Streaming is used for long, +DMA-driven tables — for example the SEQ sequencer table — where the host needs +to supply data faster than a single write can deliver. -This page is a Stage A scaffold stub and has not yet been written. +## Write operators -**Status:** writable-now +The operator appended to the field name on the configuration port determines +which mode is used: -**Source:** NEW (Interview5 §8) — << / <<|, buffer sizes -::: +| Operator | Encoding | Meaning | +|----------|----------|--------------------------------------------------| +| `<` | text | Fixed table write | +| `<<` | text | Streaming table write (more data follows) | +| `<<\|` | text | Streaming table write, last chunk | +| ` Date: Thu, 11 Jun 2026 16:07:24 +0000 Subject: [PATCH 05/28] docs: convert commands, fields, capture; extract capture-options Prompt E (server hard files): - reference/commands: full RST->MyST; Configuration + System command tables preserved, *IDN? kept, *CAPTURE.OPTIONS? cross-linked to capture-options. - reference/fields: full conversion; MODE transition matrix and the Summary-of-Sub-Types / Summary-of-Attributes tables kept verbatim; streaming table-write detail points to reference/streaming-tables. - reference/capture: keeps the ~60 MB/s figure + 'webcontrol not installed' caveat; Capture Options subsection extracted to reference/capture-options; links meta-panda how-to/integrate-with-a-panda for getting data out; retains the Stage A PandABlocks-client xref probe. - reference/capture-options: field + connection options with the 'run *CAPTURE.OPTIONS? live' note; superset remains blocked: verify. Co-Authored-By: Claude Opus 4.8 --- docs/PROGRESS.md | 8 +- docs/reference/capture-options.md | 71 ++++- docs/reference/capture.md | 244 ++++++++++++++- docs/reference/commands.md | 318 ++++++++++++++++++- docs/reference/fields.md | 498 +++++++++++++++++++++++++++++- 5 files changed, 1110 insertions(+), 29 deletions(-) diff --git a/docs/PROGRESS.md b/docs/PROGRESS.md index cefd8af4..2c714415 100644 --- a/docs/PROGRESS.md +++ b/docs/PROGRESS.md @@ -25,10 +25,10 @@ Page list expanded from `06-source-provenance-map.md`. - ✅ explanations/architecture — skeleton authored from server code (Prompt D); depth = blocked: author ## Reference -- ◐ reference/commands — source: commands.rst — writable-now [Prompt E] -- ◐ reference/fields — source: fields.rst — writable-now [Prompt E] -- ◐ reference/capture — source: capture.rst — writable-now (60MB/s = verify) [Prompt E]. Holds the Stage A xref probe. -- ◐ reference/capture-options — source: capture.rst ⊃ + Interview5 §4 — partial (superset = verify) [Prompt E split] +- ✅ reference/commands — source: commands.rst — converted (Prompt E); Configuration + System command tables preserved, *IDN? kept; *CAPTURE.OPTIONS? points to capture-options +- ✅ reference/fields — source: fields.rst — converted (Prompt E); MODE matrix + Summary-of-Sub-Types + Summary-of-Attributes tables kept; table-streaming detail points to reference/streaming-tables +- ✅ reference/capture — source: capture.rst — converted (Prompt E); ~60 MB/s + "not installed" webcontrol caveat kept; Capture Options extracted to capture-options; data-out links meta-panda integrate-with-a-panda; holds the Stage A client xref probe +- ✅ reference/capture-options — source: capture.rst ⊃ + Interview5 §4 — written (Prompt E); field + connection options; "run *CAPTURE.OPTIONS? live" note added; superset still blocked: verify - ✅ reference/config — converted from config.rst (Prompt D) - ⛔ reference/extension — source: extension.rst — blocked: verify - ✅ reference/streaming-tables — authored from Interview5 §8 + fields.rst streaming section (Prompt D) diff --git a/docs/reference/capture-options.md b/docs/reference/capture-options.md index 1bc7a82c..d659283c 100644 --- a/docs/reference/capture-options.md +++ b/docs/reference/capture-options.md @@ -1,11 +1,72 @@ # Capture options -:::{admonition} 🚧 TODO — documentation stub -:class: note +This page is the canonical reference for the two distinct sets of "capture +options" exposed by the server: -This page is a Stage A scaffold stub and has not yet been written. +- **Field capture options** — the per-field `CAPTURE` enumeration that selects + *what* is captured for a `pos_out` field. +- **Connection capture options** — the option line sent when connecting to the + data port that selects *how* the captured stream is transmitted. -**Status:** partial (superset = blocked: verify) +:::{admonition} Verify against your firmware (blocked: verify) +:class: caution -**Source:** capture.rst (Capture Options) + Interview5 §4 — add "run *CAPTURE.OPTIONS? live" note +The list below is the documented superset. The authoritative, live list for +your firmware is always whatever `*CAPTURE.OPTIONS?` returns — **run it against +the running server** rather than relying on this page, as availability of some +options (e.g. `StdDev`) depends on the FPGA configuration. The full Interview5 §4 +superset still needs confirming against the implementation. ::: + +## Field capture options + +`*CAPTURE.OPTIONS?` lists the available capture options for `pos_out` fields. +The documented options are: + +| Option | Description | +|---|---| +| Value | The value at the time of trigger is captured. | +| Diff | The difference of values is captured. | +| Sum | The sum of all valid values is captured (64-bit; may be scaled if `PCAP.SHIFT_SUM` is set). | +| Mean | The average of all valid values is captured. | +| Min | The minimum of all valid values is captured. | +| Max | The maximum of all valid values is captured. | +| StdDev | The standard deviation of valid values is captured. Availability depends on the FPGA configuration. | + +These options are set per field via the field's `CAPTURE` attribute — see +{doc}`/reference/fields` for the `pos_out` and `ext_out` capture settings, and +{doc}`/reference/capture` for how capture is configured and armed. + +`*CAPTURE.ENUMS?` returns a curated enumeration of these selections, the same as +calling `*ENUMS.`*name*`.`*field*`.CAPTURE?` on any `pos_out` field. + +## Connection capture options + +A line of capture options *must* be sent after the initial connection to the +data port before any data is sent. It is a list of any of the following options +separated by whitespace, ending with a newline character. + +| Option | Description | | | +|---|---|---|---| +| ASCII | Data is sent as ASCII numbers. | 1 | D | +| BASE64 | Binary data is sent as a stream of base-64 strings. | 1 | | +| FRAMED | Binary data is sent as a sequence of sized frames. | 1 | | +| UNFRAMED | Binary data is sent as a raw stream of bytes. | 1 | R | +| SCALED | All scalable data is scaled and sent as doubles. | 2 | D | +| RAW | The captured binary data is sent without processing. | 2 | | +| NO_HEADER | The data header is omitted. | | R | +| NO_STATUS | The connection and end-of-experiment status strings are omitted. | | R | +| ONE_SHOT | Only one experiment will be transmitted. | | R | +| XML | The header will be sent in XML format. | | | +| BARE | Selects `UNFRAMED RAW NO_HEADER NO_STATUS ONE_SHOT`. | | | +| DEFAULT | Default options. | | D | + +Key: + +- **D** — Default option if no other option is specified. +- **R** — Option selected in response to the `BARE` option. +- **1** — Data transmission formats; one of these will be selected. +- **2** — Data processing formats; one of these will be selected. + +For how these formats appear on the wire (framing, base-64 layout, header +contents), see {doc}`/reference/capture`. diff --git a/docs/reference/capture.md b/docs/reference/capture.md index 501b666e..cdbd4d62 100644 --- a/docs/reference/capture.md +++ b/docs/reference/capture.md @@ -1,16 +1,246 @@ # Data capture -:::{admonition} 🚧 TODO — documentation stub -:class: note +## Capture configuration -This page is a Stage A scaffold stub and has not yet been written. +Both `pos_out` and `ext_out` fields can be configured for data capture through +the data capture port by setting the appropriate value in the `CAPTURE` +attribute. The possible capture settings depend on the field type as follows: -**Status:** writable-now (60 MB/s = blocked: verify) +`pos_out` +: | Value | Description | + |---|---| + | No | Capture is disabled for this field. | + | Value | The value at the time of trigger will be captured. | + | Diff | The difference of values is captured. | + | Sum | The sum of all valid values is captured. This is a 64-bit value, and may be further scaled if `PCAP.SHIFT_SUM` is set. | + | Mean | The average of all valid values is captured. | + | Min | The minimum of all valid values is captured. | + | Max | The maximum of all valid values is captured. | + | Min Max | Both minimum and maximum values are captured. | + | Min Max Mean | All three values — minimum, maximum, average — are captured. | -**Source:** capture.rst (1460w) — keep ~60 MB/s + webcontrol caveat; link meta-panda how-to/integrate-with-a-panda for data-out -::: +`ext_out` +: | Value | Description | + |---|---| + | No | This field will not be captured. | + | Value | This field will be captured. | + +See {doc}`/reference/capture-options` for the full list of capture options and +how they are selected. + +## Data capture port + +The default server port for the data interface is port 8889. The initial +exchange is in ASCII with newline-separated lines; subsequent data communication +is as selected in the initial connection. + +Data capture proceeds as follows: + +1. Connect to the data server port, default 8889. +2. Send the capture options string followed by a newline. The newline character + is mandatory. +3. The server responds with `OK` unless there was an error parsing the capture + options, or the `NO_STATUS` option was specified. If there was an error, the + server responds with `ERR` followed by an error message and the connection is + closed. +4. The server now ignores all further input from the client, and the connection + pauses until data capture is started via the `*PCAP.ARM=` command. +5. At the beginning of a round of data capture or "experiment", a header + detailing the data to be sent and the data format is sent in ASCII followed by + an empty line. If `NO_HEADER` was selected then the header and blank line are + omitted. +6. Captured data is sent in the requested format until the experiment is complete + (either internally disarmed or disarmed via the `*PCAP.DISARM=` command), or + there is a communication problem. +7. At the end of the experiment a completion code is sent as a single line in + ASCII starting with `END`, unless `NO_STATUS` was specified. +8. Unless `ONE_SHOT` was specified, the server pauses until the next experiment + (step 4). + +The line of capture options sent in step 2 selects the transmission format, +processing and framing of the data stream — see +{doc}`/reference/capture-options`. + +### Data transport formatting + +Note that all binary data is sent with the lowest-order byte first. + +`ASCII` +: Each value is formatted as an ASCII number, and transmitted with one line per + captured sample. + +`BASE64` +: The stream of binary data is converted to base-64 strings and transmitted as a + series of lines until the experiment is complete. Each base-64 string is + preceded by a single space, so the end of the stream is easy to identify. + +`FRAMED` +: The captured binary data is sent in blocks of unpredictable size. Each block is + preceded by 8 bytes: the first four are `BIN` followed by a space, the + remaining four are the length of the data block in bytes *including* the 8-byte + header. + +`UNFRAMED` +: The captured binary data is sent as is. In this mode it is difficult or + impossible to reliably detect the end of the data stream, so this is normally + best combined with `NO_STATUS` and `ONE_SHOT`. + +### Data header + +At the beginning of each experiment the following information is sent: + +| Field | Description | +|---|---| +| arm_time | System timestamp when the ARM command was sent. | +| start_time | Timestamp of when PCAP became both armed and enabled. Uses a hardware-provided timestamp (e.g. from an event receiver) if available, falling back to the system timestamp. | +| hw_time_offset_ns | Offset in ns from the hardware timestamp to system time at the start of the experiment. Only present when a hardware time source is used. Used to check that hardware and system times have not drifted too far apart. | +| missed | Number of samples missed by a late data-port connection. | +| process | Data processing option: Scaled, Unscaled, or Raw. | +| format | Data delivery formatting: ASCII, Base64, Framed, or Unframed. | +| sample_bytes | Number of bytes in one sample unless `format` is `ASCII`. | +| fields | Information about each captured field. | + +All timestamps are in +[ISO 8601 UTC format](https://en.wikipedia.org/wiki/ISO_8601) with nanosecond +resolution, i.e. `YYYY-MM-DDTHH:mm:ss.sssssssssZ`. + +`start_time` will be a hardware timestamp if a hardware time source which produces +non-zero timestamps is selected; otherwise it will be a system timestamp saved by +the driver. + +For each field the following information is sent: + +| Field | Description | | +|---|---|---| +| name | Name of captured field. | | +| type | Data type of transmitted field after data processing. | | +| capture | Value of `CAPTURE` field used to enable this field. | | +| scale | Scaling factor if scaled field. | S | +| offset | Offset if scaled field. | S | +| units | Units string if scaled field. | S | + +Key: **S** — only present if a scaled field. + +If the `XML` option is selected the header is structured as a single `header` +element containing `data` and `fields` elements. + +The `type` field can be one of the following strings: + +| String | Bytes | Description | +|---|---|---| +| int32 | 4 | Used for scalable values sent in unscaled modes. | +| uint32 | 4 | Used for bit masks. | +| int64 | 8 | Used for raw ADC mean and unscaled 48-bit encoder data. | +| double | 8 | Used for all scaled values when `SCALED` is selected. | + +### Experiment completion + +At the end of each capture experiment a single line is sent, e.g.: + +``` +END 10 Ok +``` + +This specifies the number of samples sent and gives a completion code, which can +be one of the following values: + +| Code | Meaning | +|---|---| +| Ok | Experiment completed without intervention. | +| Disarmed | Experiment manually completed by `*PCAP.DISARM=` command. | +| Early disconnect | Client disconnect detected. | +| Data overrun | Client not taking data quickly, or network congestion; internal buffer overflow. | +| Framing error | Triggers too fast for configured data capture. | +| Driver data overrun | Probable CPU overload on PandA; should not occur. | +| DMA data error | Data capture too fast for memory bandwidth. | + +## High performance mode + +To get the highest performance, use `FRAMED RAW` mode. This activates a special +passthrough mode which avoids copying memory as much as possible. In tests it has +been capable of sustaining 60 MBytes/s when panda-webcontrol is **not** installed. +The downside to this mode is that if capture fails for any reason, then the last +framed block of data that the server sent should be discarded, as it will have +been corrupted while being sent. + +## Examples + +Some examples of data capture for different options follow. + +Default: + +``` +arm_time: 2021-05-26T10:34:06.133Z +missed: 0 +process: Scaled +format: ASCII +fields: + PCAP.CAPTURE_TS double Trigger + COUNTER1.OUT double Triggered scale: 1 offset: 0 units: + COUNTER2.OUT double Triggered scale: 1 offset: 0 units: + PGEN1.OUT double Triggered scale: 1 offset: 0 units: + + 1e-06 0 0 262143 + 3e-06 0 0 262142 + 5e-06 0 0 262141 + 7e-06 0 0 262140 + 9e-06 0 0 262139 +END 5 Ok +``` + +`BASE64`: + +``` +arm_time: 2021-05-26T10:34:06.133Z +missed: 0 +process: Scaled +format: Base64 +sample_bytes: 32 +fields: + PCAP.CAPTURE_TS double Trigger + COUNTER1.OUT double Triggered scale: 1 offset: 0 units: + COUNTER2.OUT double Triggered scale: 1 offset: 0 units: + PGEN1.OUT double Triggered scale: 1 offset: 0 units: + + ju21oPfGsD4AAAAAAAAAAAAAAAAAAAAAAAAAAPj/D0FU5BBxcyrJPgAAAAAAAAAAAAAAAAAAAAAA + AAAA8P8PQfFo44i1+NQ+AAAAAAAAAAAAAAAAAAAAAAAAAADo/w9BuF8+WTFc3T4AAAAAAAAAAAAA + AAAAAAAAAAAAAOD/D0E/q8yU1t/iPgAAAAAAAAAAAAAAAAAAAAAAAAAA2P8PQQ== +END 5 Ok +``` + +`XML`: + +``` +
+ + + + + + + +
+ + 1e-06 0 0 262143 + 3e-06 0 0 262142 + 5e-06 0 0 262141 + 7e-06 0 0 262140 + 9e-06 0 0 262139 +END 5 Ok +``` + +## Getting captured data out + +This page describes the wire format on the data port. For end-to-end ways of +retrieving captured data — reading the binary stream directly, using the Python +client, or via EPICS/Tango — see the meta-panda how-to +[Integrate with a PandA](https://pandablocks.github.io/meta-panda/main/how-to/integrate-with-a-panda.html). -Captured data can be read out programmatically with the Python client's +As a starting point, captured data can be read out programmatically with the +Python client's [`BlockingClient`](xref:PandABlocks-client#pandablocks.blocking.BlockingClient). diff --git a/docs/reference/commands.md b/docs/reference/commands.md index eb8bb3f9..39d409f5 100644 --- a/docs/reference/commands.md +++ b/docs/reference/commands.md @@ -1,11 +1,317 @@ # Command interface -:::{admonition} 🚧 TODO — documentation stub -:class: note +The default server port for the command interface is port 8888. All commands and +responses are in ASCII with lines separated by newline characters (ASCII +character `0x0A`). -This page is a Stage A scaffold stub and has not yet been written. +All commands can be grouped into three forms (query, assignment, table +assignment) and two targets (system and fields). There are exactly four possible +response formats (ok, ok with value, error, multiple value). This section +describes the command interface. -**Status:** writable-now +The three basic command forms are: -**Source:** commands.rst (1984w: Configuration Commands, System Commands) — keep *IDN? -::: +| Name | Format | Description | +|---|---|---| +| Query | *target*`?` | Interrogates *target* for the current value; can return an error, a single value, or a list of multiple values. | +| Assignment | *target*`=`*value* | Updates *target* with the given value; can return an error or success. | +| Table | *target*`<`*format* | Command may be followed by lines of text, and *must* be terminated by a blank line. | + +The four basic command responses are: + +| Name | Format | Description | +|---|---|---| +| Success | `OK` | Returned by assignment and table commands to report a successful update. | +| Value | `OK =`*value* | Successful return of a single value from a query command. | +| Error | `ERR` *error* | Error string returned on any command failure. | +| Multi value | `!`*value* … `.` | Any number of values can be returned, each preceded by `!`, finally `.` by itself indicates end of input. | + +Command forms and their possible responses: + +| Form | Responses | +|---|---| +| Query | Error, Value, Multi value | +| Assignment | Error, Success | +| Table | Error, Success | + +Each individual query target will either return a single value or multi-value, +as documented below. + +Finally, there are two basic types of target: configuration commands and system +commands. + +## Configuration commands + +The entire hardware interface to PandA is structured into "blocks" and "fields", +and each field may have a number of "attributes" depending on the field type. +This structure is reflected in the form of the configuration commands tabulated +below: + +| Command syntax | Description | +|---|---| +| *block*[*number*]`.`*field*`?` | Return current value of field. | +| *block*[*number*]`.`*field*`=`*value* | Assign value to field. | +| *block*[*number*]`.`*field*`<`[[`<`][`\|`]][`B`] | Write table data to field. `<` writes a fixed table, `<<` writes a streaming table and `<<\|` writes the last streaming table. | +| *block*[*number*]`.`*field*`.`*attr*`?` | Return current value of field attribute. | +| *block*[*number*]`.`*field*`.`*attr*`=`*value* | Assign value to field attribute. | +| *block*[*number*]`.*?` | Returns list of fields. | +| *block*[*number*]`.`*field*`.*?` | Returns list of field attributes. | + +In all of these commands the number after the block is optional if there is only +one instance of that block, and is ignored for the two `.*?` commands. See the +description of the `.TABLE` fields in {doc}`/reference/fields` for an explanation +of the optional format characters in the table write command. + +## System commands + +All system commands are prefixed with a leading `*` character. The simplest +command is `*IDN?`, which returns a system identification string: + +``` +< *IDN? +> OK =PandA SW: 330bd94-dirty FPGA: 0.1.9 d1275f61 00000000 +``` + +The available system commands are tabulated here and listed in more detail +below: + +| Command | Description | +|---|---| +| `*IDN?` | Device identification. | +| `*ECHO `*string*`?` | Echo. | +| `*WHO?` | List connected clients. | +| `*BLOCKS?` | List device blocks. | +| `*DESC.`*block*`.`*field*[`.`*attr*]`?`
`*DESC.`*block*`.`*field*`[].`*subfield*`?` | Show description for field, attribute, or table subfield. | +| `*ENUMS.`*block*`.`*field*[`.`*attr*]`?`
`*ENUMS.`*block*`.`*field*`[].`*subfield*`?` | List enumerations for field, attribute, or table subfield. | +| `*CHANGES`[`.`*group*]`?` | Report changes to values. *group* can be any of `CONFIG`, `BITS`, `POSN`, `READ`, `ATTR`, or `TABLE`. | +| `*CHANGES`[`.`*group*]`=`[`E`\|`S`] | Reset reported changes; *group* as above. | +| `*CAPTURE?` | Report fields configured for capture. | +| `*CAPTURE.*?` | List all fields that can be captured. | +| `*CAPTURE.`*name*`?` | Interrogate capture options; *name* can be `OPTIONS` or `ENUMS`. | +| `*CAPTURE=` | Reset data capture. | +| `*POSITIONS?` | Enumerate possible capture positions. | +| `*BITS?` | Enumerate possible bit bus positions. | +| `*VERBOSE=`*value* | Control command logging. | +| `*PCAP.`*field*`?` | Special position capture status fields. *field* can be any of `STATUS`, `CAPTURED`, or `COMPLETION`. | +| `*PCAP.`*field*`=` | Position capture actions. *field* can be either `ARM` or `DISARM`. | +| `*SAVESTATE=` | Triggers immediate save to file of the persistence file state. | +| `*CLOCK_FREQ?` | Returns currently configured system clock frequency. | + +`*IDN?` +: Returns the system identification string, for example: + + ``` + OK =PandA SW: 1.1 FPGA: 0.1.9 d1275f61 00000000 rootfs: PandA 1.1 + ``` + + The first field after "PandA" is the software version, the second is the FPGA + version, the third the firmware build number, and the fourth identifies the + supporting firmware. The final fields (prefixed `rootfs:`) identify the + underlying system on which the server is running. + + Note that the `rootfs:` identification is new to version 1.1 of PandA. + +`*ECHO `*string*`?` +: Returns the string back to the caller. Not terribly useful. Note that the + echoed string cannot contain any of `?`, `=` or `<`, as this would cause the + command to be mistaken for another command format. Example usage: + + ``` + < *ECHO This is a test? + > OK =This is a test + ``` + +`*WHO?` +: Returns a list of client connections, for example: + + ``` + < *WHO? + > !2015-12-04T14:30:40.403Z config 127.0.0.1:34185 + > . + ``` + + The first field is the time the connection was made, the second is either + `config` or `data` depending on whether the configuration or data port is + connected, and the third is the remote IP address and socket. + +`*BLOCKS?` +: Returns a list of all the top-level blocks in the system. The order in which + the blocks are returned is somewhat arbitrary. For example (here the list has + been shortened in the middle): + + ``` + < *BLOCKS? + > !TTLIN 6 + > !OUTENC 4 + ... + > !CLOCKS 1 + > !BITS 1 + > !QDEC 4 + > . + ``` + + Block and field commands can be used to interrogate each block. The number + after each block records the number of instances of each block. + +`*DESC.`*block*`?`
`*DESC.`*block*`.`*field*`?`
`*DESC.`*block*`.`*field*`.`*attr*`?`
`*DESC.`*block*`.`*field*`[].`*subfield*`?` +: Returns the description string for the specified block, field, attribute, or + table subfield, e.g.: + + ``` + < *DESC.TTLIN? + > OK =TTL input + < *DESC.TTLIN.TERM? + > OK =Select TTL input termination + < *DESC.TTLIN.TERM.INFO? + > OK =Class information for field + ``` + +`*ENUMS.`*block*`.`*field*`?`
`*ENUMS.`*block*`.`*field*`.`*attr*`?`
`*ENUMS.`*block*`.`*field*`[].`*subfield*`?` +: Returns the list of enumerations for the given field, attribute, or table + subfield, if appropriate. + +`*CHANGES?`
`*CHANGES.CONFIG?`
`*CHANGES.BITS?`
`*CHANGES.POSN?`
`*CHANGES.READ?`
`*CHANGES.ATTR?`
`*CHANGES.TABLE?` +: Reports changes to the appropriate group of values. Changes are reported since + the last request on the connection, and on the first request the current value + for every field will be reported. `*CHANGES?` reports changes for all groups; + otherwise one of the following groups can be selected: + + | Group | Description | + |---|---| + | CONFIG | Configuration settings | + | BITS | Bits on the system bus | + | POSN | Positions | + | READ | Polled read values | + | ATTR | Attributes (including capture enable flags) | + | TABLE | Table changes | + + For example: + + ``` + < *CHANGES.CONFIG? + > !TTLIN1.TERM=High-Z + > !TTLIN2.TERM=50-Ohm + > !TTLIN3.TERM=High-Z + ... + > !QDEC2.B=TTLIN1.VAL + > !QDEC3.B=TTLIN1.VAL + > !QDEC4.B=TTLIN1.VAL + > . + ``` + + Here 804 (at the time of writing) lines have been deleted from the transcript. + If we repeat the call we see that no further changes have happened until + something is actually changed: + + ``` + < *CHANGES.CONFIG? + > . + < TTLOUT4.VAL=TTLIN3.VAL + > OK + < *CHANGES.CONFIG? + > !TTLOUT4.VAL=TTLIN3.VAL + > . + ``` + + Note that for tables only the fact that the table has changed is shown; no + attempt is made to show the current table value: + + ``` + < *CHANGES.TABLE? + > !PCOMP1.TABLE< + > !PCOMP2.TABLE< + > !PCOMP3.TABLE< + > !PCOMP4.TABLE< + > !PGEN1.TABLE< + > !PGEN2.TABLE< + > !SEQ1.TABLE< + > !SEQ2.TABLE< + > !SEQ3.TABLE< + > !SEQ4.TABLE< + > . + ``` + +`*CHANGES=`[`E`\|`S`]
`*CHANGES.CONFIG=`[`E`\|`S`]
`*CHANGES.BITS=`[`E`\|`S`]
`*CHANGES.POSN=`[`E`\|`S`]
`*CHANGES.READ=`[`E`\|`S`]
`*CHANGES.ATTR=`[`E`\|`S`]
`*CHANGES.TABLE=`[`E`\|`S`] +: These commands reset the change information for the corresponding group so that + only changes occurring after the reset are reported, or so that all changes are + reported. If `=` or `=E` (for End) is specified then only new changes are + reported; if `=S` (for Start) then change reporting is reset to the start as + for a new connection. For example: + + ``` + < TTLIN1.TERM=50-Ohm + > OK + < *CHANGES= + > OK + < *CHANGES.CONFIG? + > . + ``` + +`*CAPTURE?` +: Returns a list of all positions and bit masks that will be written to the data + capture port. This list is controlled by setting the `.CAPTURE` attribute on + the corresponding position fields. + +`*CAPTURE.*?` +: Returns a list of all fields that can be configured for capture. This includes + all `pos_out` and `ext_out` fields. + +`*CAPTURE.OPTIONS?` +: Lists the available capture options for `pos_out` fields. See + {doc}`/reference/capture-options` for the full set of options. + +`*CAPTURE.ENUMS?` +: Generates a curated list of capture option selections, designed for presenting + lists of available capture options as an enumeration. Returns the same as + calling `*ENUMS.`*name*`.`*field*`.CAPTURE?` on any `pos_out` field. + +`*CAPTURE=` +: Resets all `.CAPTURE` flags to zero so that no data will be captured. + +`*POSITIONS?` +: Lists all available position capture fields in order. + +`*BITS?` +: Lists all available bit bus positions, but not including the special values + `ZERO` and `ONE`. + +`*VERBOSE=`*value* +: If `*VERBOSE=1` is set then every command will be echoed to the server's log. + Set `*VERBOSE=0` to restore normal quiet behaviour. + +`*PCAP.STATUS?`
`*PCAP.CAPTURED?`
`*PCAP.COMPLETION?` +: Interrogates the status of position capture: + + | Field | Description | + |---|---| + | STATUS | Returns a string with three fields: "Busy" or "Idle", followed by the number of connected readers, and the number taking data. | + | CAPTURED | Returns the number of samples captured in the current or most recent data capture. | + | COMPLETION | Returns the completion status from the most recent data capture, as listed in the table below. | + + The completion codes have the following meaning: + + | Code | Meaning | + |---|---| + | Busy | Capture in progress. | + | Ok | Capture completed without error or intervention. | + | Disarmed | Capture was manually disarmed by `*PCAP.DISARM=` command. | + | Framing error | Data capture framing error, probably due to incorrectly configured capture. | + | DMA data error | Internal data error, should not occur. | + | Driver data overrun | Data capture too fast, internal buffers overrun. Can also occur if the PandA processor is overloaded. | + +`*PCAP.ARM=`
`*PCAP.DISARM=` +: Top-level capture control: + + | Field | Description | + |---|---| + | ARM | Initiates data capture. Will fail if capture is already in progress, or no fields are configured for capture. | + | DISARM | Halts ongoing data capture. | + +`*SAVESTATE=` +: Updates the persistence state file (as configured on the command line when + launched) with the current state. Returns after a filesystem `sync` call, so it + is safe to power off the system after this command has completed. + +`*CLOCK_FREQ?` +: Returns the currently configured FPGA clock frequency, as used to convert + between times in natural units and times in clock ticks. diff --git a/docs/reference/fields.md b/docs/reference/fields.md index 0bc42d18..4e6647f5 100644 --- a/docs/reference/fields.md +++ b/docs/reference/fields.md @@ -1,11 +1,495 @@ -# Field types and attributes +(fields)= +# Blocks, fields and attributes -:::{admonition} 🚧 TODO — documentation stub -:class: note +The set of hardware blocks can be interrogated with the `*BLOCKS?` command: -This page is a Stage A scaffold stub and has not yet been written. +``` +< *BLOCKS? +> !TTLIN 6 +> !OUTENC 4 +> !PCAP 1 +> !PCOMP 4 +> !TTLOUT 10 +> !ADC 8 +> !DIV 4 +> !INENC 4 +> !SLOW 1 +> !PGEN 2 +> !LVDSIN 2 +> !POSITIONS 1 +> !POSENC 4 +> !SEQ 4 +> !PULSE 4 +> !SRGATE 4 +> !LUT 8 +> !LVDSOUT 2 +> !COUNTER 8 +> !ADDER 1 +> !CLOCKS 1 +> !BITS 1 +> !QDEC 4 +> . +``` -**Status:** writable-now +For each block the number after the block tells us how many instances there are +of the block. Each block is controlled and interrogated through a number of +fields, and the `block.*?` command can be used to interrogate the list of +fields: -**Source:** fields.rst (2671w: Field Types, Field Sub-Types, Summary of Sub-Types, Summary of Attributes) — keep MODE matrix; point to streaming-tables -::: +``` +< TTLIN.*? +> !VAL 1 bit_out +> !TERM 0 param enum +> . +``` + +This tells us that block `TTLIN` has two fields, `TTLIN.VAL` and `TTLIN.TERM`. +The first field after the field name is a sequence number for user interface +display, and the rest of each response describes the "type" of the field. In +this case we see that `TTLIN.VAL` is a `bit_out` field, which means it can be +used for bit data capture and can be connected to any `param bit_mux` field as a +data source. + +Each field has one or more attributes depending on the field type. The list of +attributes can be interrogated with the `block.field.*?` command: + +``` +< TTLIN.VAL.*? +> !CAPTURE_WORD +> !OFFSET +> !INFO +> . +< TTLIN.TERM.*? +> !INFO +> . +``` + +All fields have the `.INFO` attribute, which just repeats the type information +already reported, e.g. `TTLIN1.VAL.INFO?` returns `bit_out` (note that a block +number must be specified when interrogating fields and attributes). + +## Field types + +Each field type determines the set of attributes available for the field. The +types and their attributes are documented below. + +| Field type | Description | +|---|---| +| `param` *subtype* | Configurable parameter. The *subtype* determines the precise behaviour and the available attributes. | +| `read` *subtype* | A read-only hardware field, used for monitoring status. Again, *subtype* determines available attributes. | +| `write` *subtype* | A write-only field; *subtype* determines possible values and attributes. | +| `time` | Configurable timer parameter. | +| `bit_out` | Bit output, can be configured as bit input for `bit_mux` fields. | +| `pos_out` | Position output, can be configured for data capture and as position input for `pos_mux` fields. | +| `ext_out` *extra* | Extended output values, can be configured for data capture, but not available on position bus. | +| `bit_mux` | Bit input with configurable delay. | +| `pos_mux` | Position input multiplexer selection. | +| `table` | Table data with special access methods. | + +`param` *subtype* +: All fields of this type contribute to the `*CHANGES.PARAM` change group and + are used to configure the behaviour of the corresponding block. Fields of this + type are used for input configuration and other behavioural settings. + +`read` *subtype* +: All fields of this type contribute to the `*CHANGES.READ` change group, but + are only checked when either the field is read or the change group is polled. + Fields of this type are used for monitoring the internal status of a block, + and they cannot be written to. + +`write` *subtype* +: Fields of this type can only be written and are used for immediate actions on + a block. The `action` subtype is used to support actions without any + parameters; for example the following command forces a soft reset on the given + pulse block: + + ``` + < PULSE1.FORCE_RESET= + > OK + ``` + +`time` +: Fields of this type are used for configuring delays. They also contribute to + `*CHANGES.PARAM`. The following attributes are supported: + + `UNITS` + : Can be set to any of the strings `min`, `s`, `ms`, or `us`, and is used to + interpret how values read and written to the field are interpreted. + + `RAW` + : Can be read or written to report or set the delay in FPGA ticks. + + The `UNITS` attribute determines how numbers read or written to the field are + interpreted. For example: + + ``` + < PULSE1.DELAY.UNITS=s + > OK + < PULSE1.DELAY=2.5 + > OK + < PULSE1.DELAY.RAW? + > OK =312500000 + < PULSE1.DELAY.UNITS=ms + > OK + < PULSE1.DELAY? + > OK =2500 + ``` + + Note that changing `UNITS` doesn't change the delay, only how it is reported + and interpreted. + +`bit_out` +: Fields of this type are used for block outputs which contribute to the + internal bit system bus, and they contribute to the `*CHANGES.BITS` change + group. They can be captured via the appropriate `PCAP.BITS`*n* block as + reported by the `CAPTURE_WORD` attribute. + + `CAPTURE_WORD` + : Identifies which `pos_out` value can be used to capture this bit. + + `OFFSET` + : The bit offset into the captured word of this particular bit. + + For example: + + ``` + < TTLIN1.VAL.CAPTURE_WORD? + > OK =PCAP.BITS0 + < TTLIN1.VAL.OFFSET? + > OK =2 + ``` + + This tells us that if `PCAP.BITS0` is captured then `TTLIN1.VAL` can be read as + bit 2 of this word, counting from the least significant bit. The field itself + can be read to return the current value of the bit. + +`pos_out` +: Fields of this type are used for block outputs which contribute to the + internal position bus, and they contribute to the `*CHANGES.POSN` change group. + The following attribute supports capture control: + + `CAPTURE` + : Manages capture of this field. One of the following enumeration values can be + written: + + | Value | Description | + |---|---| + | No | Capture is disabled for this field. | + | Value | The value at the time of trigger will be captured. | + | Diff | The difference of values is captured. | + | Sum | The sum of all valid values is captured. This is a 64-bit value, and may be further scaled if `PCAP.SHIFT_SUM` is set. | + | Mean | The average of all valid values is captured. | + | Min | The minimum of all valid values is captured. | + | Max | The maximum of all valid values is captured. | + | Min Max | Both minimum and maximum values are captured. | + | Min Max Mean | All three values — minimum, maximum, average — are captured. | + + The following attributes support formatting of the field when reading it; the + current value is returned subject to the formatting rules described below. + + `OFFSET`, `SCALE` + : Configure the conversion from the underlying position to the value captured + when scaling is enabled and read from the `SCALED` attribute. + + `UNITS` + : Can be set to any UTF-8 string, provided for the convenience of the user + interface and returned as part of the data capture heading. + + `SCALED` + : Returns the scaled value computed as `value * scale + offset`. + +`ext_out` *extra* +: Fields of this type represent values that can be captured but which are not + present on the position bus. These fields also support one capture control + field: + + `CAPTURE` + : As for `pos_out`, can be set to control capture of this field: + + | Value | Description | + |---|---| + | No | This field will not be captured. | + | Value | This field will be captured. | + + The *extra* field determines the detailed behaviour of this field, and will be + one of the following values: + + | extra value | Description | + |---|---| + | `timestamp` | Timestamps in clock ticks with optional scaling to seconds on data capture. | + | `samples` | Special internal field for counting captured samples. | + | `bits` | Used to implement bit-bus readout fields. Fields of this sub-type implement an extra `BITS` field. | + + Fields of type `ext_out bits` implement an extra attribute: + + `BITS` + : Returns a list of all bit fields associated with this field. Fields of this + type can be used to capture a snapshot of the bit bus at the trigger time. + +`bit_mux` +: Bit input selectors for blocks. Each of these fields can be set to the name of + a corresponding `bit_out` field, for example: + + ``` + < TTLOUT1.VAL=TTLIN1.VAL + > OK + ``` + + There are two attributes: + + `DELAY` + : Can be set to any value between 0 and `MAX_DELAY` to delay the bit input to + the block by the specified number of clock ticks. + + `MAX_DELAY` + : Returns the maximum delay that can be set for this input. + +`pos_mux` +: Position input selectors for blocks. Each of these fields can be set to the + name of a corresponding `pos_out` field, for example: + + ``` + < ADDER1.INPA=ADC2.OUT + > OK + ``` + +`table` +: Values of this type are used for long tables of numbers. This server imposes + no structure on these values apart from treating them as an array of 32-bit + integers. + + Table values are written with the special `<` syntax: + + | Operator | Description | + |---|---| + | `block`*number*`.field<` | Normal table write, fixed table | + | `block`*number*`.field<<` | Normal table write, streaming table | + | `block`*number*`.field<<\|` | Normal table write, last streaming table | + | `block`*number*`.field OK + < SEQ3.TABLE.LENGTH? + > OK =12 + ``` + + Note that when data is sent in base-64 format, each individual line must + encode a multiple of four bytes, otherwise the write will be rejected. For full + details of fixed vs. streaming (`<<` / `<<|`) table writes and DMA buffer + sizing, see {doc}`/reference/streaming-tables`. + + The following attributes are provided by this field type: + + `MAX_LENGTH` + : The maximum number of 32-bit words which can be stored in the table. + + `LENGTH` + : The current number of words in the table. + + `B` + : This read-only attribute returns the content of the table in base-64. + + `FIELDS` + : Returns a list of strings which can be used to interpret the content of the + table. Each line returned is of the following format: + + ``` + left:right field-name subtype + ``` + + Here *left* and *right* are bit field indices into a single table row, + consisting of a number of 32-bit words concatenated (in little-endian order) + with bits numbered from 0 in the least significant position up to + 32×`ROW_WORDS`−1, and *left* ≥ *right*. The name of the field is given by + *field-name*, and *subtype* can be one of `int`, `uint`, or `enum`. If + *subtype* is `enum` then the list of enums can be interrogated through the + command: + + ``` + *ENUMS.block.table[].field? + ``` + + where *block*, *table*, *field* are appropriate names. + + `ROW_WORDS` + : Returns the number of 32-bit words in a single row of the table. This can be + used to help interpret the `FIELDS` result. + + `QUEUED_LINES` + : When a fixed table is written, returns the number of lines in that table. + When streaming tables are written, returns the number of lines that have been + scheduled, including the ones currently being used by the FPGA. + + `MODE` + : Indicates the mode that the table is in as a consequence of the last table + write. The possible values are `INIT` (no table), `FIXED` (fixed table), + `STREAMING` (streaming table) and `STREAMING_LAST` (last streaming table). + Writing an empty table always moves the table to `INIT` state. In addition to + this mode, there is also an implicit completed state in the FPGA that happens + either when there is a sudden error or when the streaming is finished; it will + cause any future writes to be rejected and the `MODE` attribute will keep the + last value until a reset is done, to ensure the client is aware of any error. + + The following table summarises the mode transitions for each table command, + where `<0` represents writing an empty table: + + | MODE \ command | `<` | `<<` | `<<\|` | `<0` | + |---|---|---|---|---| + | `INIT` | `FIXED` | `STREAMING` | `STREAMING_LAST` | `INIT` | + | `FIXED` | `FIXED` | `STREAMING` | `STREAMING_LAST` | `INIT` | + | `STREAMING` | Reject | `STREAMING` | `STREAMING_LAST` | `INIT` | + | `STREAMING_LAST` | Reject | Reject | Reject | `INIT` | + | `COMPLETED[HEALTH]` | Reject | Reject | Reject | `INIT` | + + Writing empty tables with `<<` or `<<|` is rejected to avoid accidental + mistakes in streaming mode. + +## Field sub-types + +The following field sub-types can be used for `param`, `read` and `write` +fields. + +`uint` [*max-value*] +: The most basic type: the value read or written is an unsigned 32-bit number. + There is one fixed attribute: + + `MAX` + : Returns the maximum value that can be written to this field. + +`int` +: Similar to `uint`, but signed, and there is no upper limit on the value. + +`scalar` *scale* [*offset* [*units*]] +: Floating point values can be read or written, and are converted from and to the + underlying signed integer type via the equations below: + + ``` + value = scale * raw + offset + raw = (value - offset) / scale + ``` + + The following attributes are supported: + + `UNITS` + : Returns the configured units string. + + `RAW` + : Returns the underlying unconverted integer value. + + `SCALE` + : Returns the configured scaling factor. + + `OFFSET` + : Returns the configured scaling offset. + +`bit` +: A value which is 0 or 1; there are no extra attributes. + +`action` +: A value which cannot be read and always writes as 0. Only useful for `write` + fields. + +`lut` +: This field sub-type is used for the 5-input lookup table function calculation + field. This field can be set to any valid logical expression generated from + inputs `A` to `E` using the standard operators `&`, `|`, `^`, `~`, `?:` from C + together with `=` for equality and `=>` for implication (`A=>B` abbreviates + `~A|B`). All operations have C precedence, `=` has the same precedence as `==` + in C, and `=>` has precedence between `|` and `?:`. + + The following attribute is supported: + + `RAW` + : Returns the corresponding lookup table assignment as a 32-bit number. + + For example: + + ``` + < LUT2.FUNC=A=>B?C:D + > OK + < LUT2.FUNC? + > OK =A=>B?C:D + < LUT2.FUNC.RAW? + > OK =0xF0CCF0F0 + ``` + +`enum` +: Enumeration fields define a list of valid strings which can be written to the + field. To interrogate the list of valid enumeration values use the `*ENUMS` + command, for example: + + ``` + < *ENUMS.TTLIN1.TERM? + > !High-Z + > !50-Ohm + > . + ``` + +`time` +: Converts between time in specified units and time in FPGA clock ticks. The + following attributes are supported: + + `UNITS` + : Can be set to any of the strings `min`, `s`, `ms`, or `us`, and is used to + interpret how values read and written to the field are interpreted. + + `RAW` + : Can be read or written to report or set the delay in FPGA ticks. + +## Summary of sub-types + +| Sub-type | Attributes | Description | +|---|---|---| +| uint | MAX | Possibly bounded 32-bit unsigned integer value | +| int | | Unbounded 32-bit signed integer value | +| scalar | RAW, UNITS, SCALE, OFFSET | Scaled signed floating point value | +| bit | | Bit: 0 or 1 | +| action | | Write only, no value | +| lut | RAW | 5-input lookup table logical formula | +| enum | LABELS | Enumeration selection | +| time | RAW, UNITS | Time intervals converted to FPGA ticks | + +## Summary of attributes + +| Field (sub)type | Attribute | Description | R | W | C | M | +|---|---|---|:-:|:-:|:-:|:-:| +| (all) | INFO | Returns type of field | R | | | | +| uint | MAX | Maximum allowed integer value | R | | | | +| scalar | RAW | Underlying integer value | R | W | | | +| | UNITS | Configured units for scalar | R | | | | +| | SCALE | Configured scaling factor for scalar | R | | | | +| | OFFSET | Configured scaling offset for scalar | R | | | | +| lut | RAW | Computed lookup table 32-bit value | R | | | | +| time | UNITS | Units and scaling selection for time | R | W | C | | +| | RAW | Raw time in FPGA clock cycles | R | W | | | +| bit_out | CAPTURE_WORD | Capturable word containing this bit | R | | | | +| | OFFSET | Offset of this bit in captured word | R | | | | +| bit_mux | DELAY | Bit input delay in FPGA ticks | R | W | C | | +| | MAX_DELAY | Maximum valid delay | R | | | | +| pos_out | CAPTURE | Position capture control | R | W | C | | +| | OFFSET | Position offset | R | W | C | | +| | SCALE | Position scaling | R | W | C | | +| | UNITS | Position units | R | W | C | | +| | SCALED | Position after applying scaling | R | | | | +| ext_out bits | BITS | List of bit_out fields | R | | | M | +| table | MAX_LENGTH | Maximum table row count | R | | | | +| | LENGTH | Current table row count | R | | | | +| | B | Table data in base-64 | R | | | M | +| | FIELDS | Table field descriptions | R | | | M | +| | ROW_WORDS | Number of words in a table row | R | | | | + +Key: + +- **R** — Attribute can be read. +- **W** — Attribute can be written. +- **C** — Attribute contributes to the `*CHANGES.ATTR` change set. +- **M** — Attribute returns a multiple-value result. From 8816ea25af6c3af3d679dc71af0d600a92c8628c Mon Sep 17 00:00:00 2001 From: Tom Cobb Date: Fri, 12 Jun 2026 09:09:17 +0000 Subject: [PATCH 06/28] docs: apply fresh-eyes review fixes (source-verified) - streaming-tables: correct MODE transition table (Reject rows, completed [HEALTH] row) and implicit-completed prose; B = plain base-64 with the left:right format moved to a new FIELDS entry; LENGTH/MAX_LENGTH in 32-bit words - fields: *CHANGES.PARAM -> *CHANGES.CONFIG (groups verified in config_server.h/system_command.c); drop nonexistent enum LABELS attribute; table length units; StdDev / Mean StdDev capture rows - capture-options: make the canonical superset per pos_out.c (7 options, combinations, full curated enum list); caution downgraded to a note - capture, commands: StdDev rows; add missing METADATA change group - startup: duplicate -p -> -P pid-file; add -r rootfs-version (*IDN?) - building: tip points at the in-repo devcontainer - myst.yml: drop PandABlocks-devcontainer reference key (repo superseded by per-repo devcontainers) Co-Authored-By: Claude Opus 4.8 --- docs/how-to/building.md | 7 +++--- docs/how-to/startup.md | 5 ++++- docs/myst.yml | 1 - docs/reference/capture-options.md | 31 ++++++++++++++------------ docs/reference/capture.md | 2 ++ docs/reference/commands.md | 7 +++--- docs/reference/fields.md | 15 ++++++++----- docs/reference/streaming-tables.md | 35 ++++++++++++++++++------------ 8 files changed, 62 insertions(+), 41 deletions(-) diff --git a/docs/how-to/building.md b/docs/how-to/building.md index cb0f817f..9be667eb 100644 --- a/docs/how-to/building.md +++ b/docs/how-to/building.md @@ -1,9 +1,10 @@ # Build and test the server :::{tip} -For local development the [PandABlocks-devcontainer](https://github.com/PandABlocks/PandABlocks-devcontainer) -provides a pre-configured environment and is the recommended starting point. -The instructions below cover a native build for those who need it. +For local development the devcontainer defined in this repository +(`.devcontainer/devcontainer.json` and the top-level `Dockerfile`) provides a +pre-configured environment and is the recommended starting point. The +instructions below cover a native build for those who need it. ::: ## Dependencies diff --git a/docs/how-to/startup.md b/docs/how-to/startup.md index 8a9873d5..4f8054ed 100644 --- a/docs/how-to/startup.md +++ b/docs/how-to/startup.md @@ -43,7 +43,7 @@ The server can also be started from the command line with the following options. : Run the server as a daemon. This is the normal production mode; omit for debugging. -`-p` *pid-file* +`-P` *pid-file* : Write the server process ID to the given file; the file is deleted on exit. `-T` @@ -58,3 +58,6 @@ The server can also be started from the command line with the following options. `-X` *port* : Connect to an extension server running locally on the specified port. See {doc}`/reference/extension` for details. + +`-r` *rootfs-version* +: Specify the rootfs version string reported by the `*IDN?` command. diff --git a/docs/myst.yml b/docs/myst.yml index 485022d3..48f1c8e7 100644 --- a/docs/myst.yml +++ b/docs/myst.yml @@ -12,7 +12,6 @@ project: # uncomment in Stage F and upstream into python-copier-template. # meta-panda: https://pandablocks.github.io/meta-panda/main # PandABlocks-FPGA: https://pandablocks.github.io/PandABlocks-FPGA/main - # PandABlocks-devcontainer: https://pandablocks.github.io/PandABlocks-devcontainer # fastcs-PandABlocks: https://diamondlightsource.github.io/fastcs-PandABlocks/main toc: - file: index.md diff --git a/docs/reference/capture-options.md b/docs/reference/capture-options.md index d659283c..215cfa36 100644 --- a/docs/reference/capture-options.md +++ b/docs/reference/capture-options.md @@ -8,20 +8,17 @@ options" exposed by the server: - **Connection capture options** — the option line sent when connecting to the data port that selects *how* the captured stream is transmitted. -:::{admonition} Verify against your firmware (blocked: verify) -:class: caution - -The list below is the documented superset. The authoritative, live list for -your firmware is always whatever `*CAPTURE.OPTIONS?` returns — **run it against -the running server** rather than relying on this page, as availability of some -options (e.g. `StdDev`) depends on the FPGA configuration. The full Interview5 §4 -superset still needs confirming against the implementation. +:::{note} +The authoritative, live list for your firmware is always whatever +`*CAPTURE.OPTIONS?` returns from the running server — availability of some +options (`StdDev`, and therefore `Mean StdDev`) depends on the FPGA +configuration. ::: ## Field capture options -`*CAPTURE.OPTIONS?` lists the available capture options for `pos_out` fields. -The documented options are: +`*CAPTURE.OPTIONS?` lists the individual capture options available for +`pos_out` fields: | Option | Description | |---|---| @@ -31,15 +28,21 @@ The documented options are: | Mean | The average of all valid values is captured. | | Min | The minimum of all valid values is captured. | | Max | The maximum of all valid values is captured. | -| StdDev | The standard deviation of valid values is captured. Availability depends on the FPGA configuration. | +| StdDev | The standard deviation of valid values is captured. Only listed if supported by the FPGA configuration. | + +A field's `CAPTURE` attribute can be set to `No` (capture disabled), to a +single option, or to a space-separated combination of options (e.g. +`Min Max Mean`). + +`*CAPTURE.ENUMS?` returns a curated enumeration of these selections — the same +list as calling `*ENUMS.`*block*`.`*field*`.CAPTURE?` on any `pos_out` field: +`No`, `Value`, `Diff`, `Sum`, `Mean`, `Min`, `Max`, `Min Max`, `Min Max Mean`, +`StdDev`, `Mean StdDev`. These options are set per field via the field's `CAPTURE` attribute — see {doc}`/reference/fields` for the `pos_out` and `ext_out` capture settings, and {doc}`/reference/capture` for how capture is configured and armed. -`*CAPTURE.ENUMS?` returns a curated enumeration of these selections, the same as -calling `*ENUMS.`*name*`.`*field*`.CAPTURE?` on any `pos_out` field. - ## Connection capture options A line of capture options *must* be sent after the initial connection to the diff --git a/docs/reference/capture.md b/docs/reference/capture.md index cdbd4d62..12a7730a 100644 --- a/docs/reference/capture.md +++ b/docs/reference/capture.md @@ -18,6 +18,8 @@ attribute. The possible capture settings depend on the field type as follows: | Max | The maximum of all valid values is captured. | | Min Max | Both minimum and maximum values are captured. | | Min Max Mean | All three values — minimum, maximum, average — are captured. | + | StdDev | The standard deviation of valid values is captured. Only available if supported by the FPGA configuration. | + | Mean StdDev | Both average and standard deviation are captured. Only available if supported by the FPGA configuration. | `ext_out` : | Value | Description | diff --git a/docs/reference/commands.md b/docs/reference/commands.md index 39d409f5..b2ad1f9f 100644 --- a/docs/reference/commands.md +++ b/docs/reference/commands.md @@ -83,7 +83,7 @@ below: | `*BLOCKS?` | List device blocks. | | `*DESC.`*block*`.`*field*[`.`*attr*]`?`
`*DESC.`*block*`.`*field*`[].`*subfield*`?` | Show description for field, attribute, or table subfield. | | `*ENUMS.`*block*`.`*field*[`.`*attr*]`?`
`*ENUMS.`*block*`.`*field*`[].`*subfield*`?` | List enumerations for field, attribute, or table subfield. | -| `*CHANGES`[`.`*group*]`?` | Report changes to values. *group* can be any of `CONFIG`, `BITS`, `POSN`, `READ`, `ATTR`, or `TABLE`. | +| `*CHANGES`[`.`*group*]`?` | Report changes to values. *group* can be any of `CONFIG`, `BITS`, `POSN`, `READ`, `ATTR`, `TABLE`, or `METADATA`. | | `*CHANGES`[`.`*group*]`=`[`E`\|`S`] | Reset reported changes; *group* as above. | | `*CAPTURE?` | Report fields configured for capture. | | `*CAPTURE.*?` | List all fields that can be captured. | @@ -170,7 +170,7 @@ below: : Returns the list of enumerations for the given field, attribute, or table subfield, if appropriate. -`*CHANGES?`
`*CHANGES.CONFIG?`
`*CHANGES.BITS?`
`*CHANGES.POSN?`
`*CHANGES.READ?`
`*CHANGES.ATTR?`
`*CHANGES.TABLE?` +`*CHANGES?`
`*CHANGES.CONFIG?`
`*CHANGES.BITS?`
`*CHANGES.POSN?`
`*CHANGES.READ?`
`*CHANGES.ATTR?`
`*CHANGES.TABLE?`
`*CHANGES.METADATA?` : Reports changes to the appropriate group of values. Changes are reported since the last request on the connection, and on the first request the current value for every field will be reported. `*CHANGES?` reports changes for all groups; @@ -184,6 +184,7 @@ below: | READ | Polled read values | | ATTR | Attributes (including capture enable flags) | | TABLE | Table changes | + | METADATA | Metadata field changes | For example: @@ -231,7 +232,7 @@ below: > . ``` -`*CHANGES=`[`E`\|`S`]
`*CHANGES.CONFIG=`[`E`\|`S`]
`*CHANGES.BITS=`[`E`\|`S`]
`*CHANGES.POSN=`[`E`\|`S`]
`*CHANGES.READ=`[`E`\|`S`]
`*CHANGES.ATTR=`[`E`\|`S`]
`*CHANGES.TABLE=`[`E`\|`S`] +`*CHANGES=`[`E`\|`S`]
`*CHANGES.CONFIG=`[`E`\|`S`]
`*CHANGES.BITS=`[`E`\|`S`]
`*CHANGES.POSN=`[`E`\|`S`]
`*CHANGES.READ=`[`E`\|`S`]
`*CHANGES.ATTR=`[`E`\|`S`]
`*CHANGES.TABLE=`[`E`\|`S`]
`*CHANGES.METADATA=`[`E`\|`S`] : These commands reset the change information for the corresponding group so that only changes occurring after the reset are reported, or so that all changes are reported. If `=` or `=E` (for End) is specified then only new changes are diff --git a/docs/reference/fields.md b/docs/reference/fields.md index 4e6647f5..82ebe99d 100644 --- a/docs/reference/fields.md +++ b/docs/reference/fields.md @@ -87,7 +87,7 @@ types and their attributes are documented below. | `table` | Table data with special access methods. | `param` *subtype* -: All fields of this type contribute to the `*CHANGES.PARAM` change group and +: All fields of this type contribute to the `*CHANGES.CONFIG` change group and are used to configure the behaviour of the corresponding block. Fields of this type are used for input configuration and other behavioural settings. @@ -110,7 +110,7 @@ types and their attributes are documented below. `time` : Fields of this type are used for configuring delays. They also contribute to - `*CHANGES.PARAM`. The following attributes are supported: + `*CHANGES.CONFIG`. The following attributes are supported: `UNITS` : Can be set to any of the strings `min`, `s`, `ms`, or `us`, and is used to @@ -183,6 +183,11 @@ types and their attributes are documented below. | Max | The maximum of all valid values is captured. | | Min Max | Both minimum and maximum values are captured. | | Min Max Mean | All three values — minimum, maximum, average — are captured. | + | StdDev | The standard deviation of valid values is captured. Only available if supported by the FPGA configuration. | + | Mean StdDev | Both average and standard deviation are captured. Only available if supported by the FPGA configuration. | + + Combinations of the individual options can also be written as a + space-separated list — see {doc}`/reference/capture-options`. The following attributes support formatting of the field when reading it; the current value is returned subject to the formatting rules described below. @@ -455,7 +460,7 @@ fields. | bit | | Bit: 0 or 1 | | action | | Write only, no value | | lut | RAW | 5-input lookup table logical formula | -| enum | LABELS | Enumeration selection | +| enum | | Enumeration selection (labels listed via the `*ENUMS` command) | | time | RAW, UNITS | Time intervals converted to FPGA ticks | ## Summary of attributes @@ -481,8 +486,8 @@ fields. | | UNITS | Position units | R | W | C | | | | SCALED | Position after applying scaling | R | | | | | ext_out bits | BITS | List of bit_out fields | R | | | M | -| table | MAX_LENGTH | Maximum table row count | R | | | | -| | LENGTH | Current table row count | R | | | | +| table | MAX_LENGTH | Maximum table length in 32-bit words | R | | | | +| | LENGTH | Current table length in 32-bit words | R | | | | | | B | Table data in base-64 | R | | | M | | | FIELDS | Table field descriptions | R | | | M | | | ROW_WORDS | Number of words in a table row | R | | | | diff --git a/docs/reference/streaming-tables.md b/docs/reference/streaming-tables.md index 1da8aadb..870314f0 100644 --- a/docs/reference/streaming-tables.md +++ b/docs/reference/streaming-tables.md @@ -40,34 +40,41 @@ current state: | `STREAMING` | A streaming write is in progress (`<<` was last) | | `STREAMING_LAST` | The final streaming chunk was received (`<<|` was last) | -Writing an empty table (`<0`) always moves the table to `INIT`. If a streaming -error occurs, `MODE` transitions automatically to report the error condition so -the client can become aware of failures. +Writing an empty table (`<0`) always moves the table to `INIT`. In addition to +these modes there is an implicit *completed* state in the FPGA, entered either +on a sudden error or when streaming finishes: any further writes are rejected +and the `MODE` attribute keeps its last value until the table is reset with +`<0`, ensuring the client becomes aware of the error. ### MODE transition table -| Current MODE / command | `<` | `<<` | `<<\|` | `<0` | -|------------------------|---------|-------------|------------------|--------| -| `INIT` | `FIXED` | `STREAMING` | `STREAMING_LAST` | `INIT` | -| `FIXED` | `FIXED` | `STREAMING` | `STREAMING_LAST` | `INIT` | -| `STREAMING` | `FIXED` | `STREAMING` | `STREAMING_LAST` | `INIT` | -| `STREAMING_LAST` | `FIXED` | `STREAMING` | `STREAMING_LAST` | `INIT` | +| Current MODE / command | `<` | `<<` | `<<\|` | `<0` | +|------------------------|----------|-------------|------------------|--------| +| `INIT` | `FIXED` | `STREAMING` | `STREAMING_LAST` | `INIT` | +| `FIXED` | `FIXED` | `STREAMING` | `STREAMING_LAST` | `INIT` | +| `STREAMING` | *Reject* | `STREAMING` | `STREAMING_LAST` | `INIT` | +| `STREAMING_LAST` | *Reject* | *Reject* | *Reject* | `INIT` | +| completed (`[HEALTH]`) | *Reject* | *Reject* | *Reject* | `INIT` | ## Other table attributes `LENGTH` -: Current number of words in the table (read-only). +: Current number of 32-bit words in the table (read-only). `MAX_LENGTH` -: Maximum number of table rows (read-only). +: Maximum number of 32-bit words which can be stored in the table (read-only). `ROW_WORDS` : Number of 32-bit words per table row (read-only). `B` (base-64 read) -: Returns the current table content encoded in base-64. Each line has the - format `left:right:data` where *left* and *right* are bit-field indices into - a row and *data* is the base-64 encoded row content. +: Returns the current table content encoded in base-64 (read-only). + +`FIELDS` +: Returns a list of strings describing how to interpret a table row. Each line + has the format `left:right field-name subtype`, where *left* and *right* are + bit-field indices into a single row. See {doc}`/reference/fields` for + details. A `<<` write returns the number of lines accepted. A `<` (fixed) write returns the total number of lines in the new table. From 9c55c717329bdb9fa4bd4a32e6e2c14d9a0f2870 Mon Sep 17 00:00:00 2001 From: Tom Cobb Date: Fri, 12 Jun 2026 09:09:18 +0000 Subject: [PATCH 07/28] devcontainer: add ubuntu-devcontainer-based devcontainer Per-repo devcontainers replace the planned PandABlocks-devcontainer repo. Modelled on python-copier-template (ubuntu-devcontainer:noble, developer target) with C/Python build tools; the ARM cross-compiler is a candidate addition for on-target builds. Co-Authored-By: Claude Opus 4.8 --- .devcontainer/devcontainer.json | 31 +++++++++++++++++++++++++++++++ Dockerfile | 11 +++++++++++ 2 files changed, 42 insertions(+) create mode 100644 .devcontainer/devcontainer.json create mode 100644 Dockerfile diff --git a/.devcontainer/devcontainer.json b/.devcontainer/devcontainer.json new file mode 100644 index 00000000..10a2b008 --- /dev/null +++ b/.devcontainer/devcontainer.json @@ -0,0 +1,31 @@ +// For format details, see https://containers.dev/implementors/json_reference/ +{ + "name": "PandABlocks-server Developer Container", + "build": { + "dockerfile": "../Dockerfile", + "target": "developer" + }, + "remoteEnv": { + // Allow X11 apps to run inside the container + "DISPLAY": "${localEnv:DISPLAY}" + }, + "customizations": { + "vscode": { + // Add the IDs of extensions you want installed when the container is created. + "extensions": [ + "ms-vscode.cpptools", + "ms-python.python" + ] + } + }, + "runArgs": [ + // Allow the container to access the host X11 display and the + // simulation server ports + "--net=host", + // Make sure SELinux does not disable access to host filesystems like tmp + "--security-opt=label=disable" + ], + // Mount the parent as /workspaces so peer repos (PandABlocks-FPGA, + // PandABlocks-rootfs) are visible alongside this one + "workspaceMount": "source=${localWorkspaceFolder}/..,target=/workspaces,type=bind" +} diff --git a/Dockerfile b/Dockerfile new file mode 100644 index 00000000..035af0f6 --- /dev/null +++ b/Dockerfile @@ -0,0 +1,11 @@ +# The devcontainer should use the developer target and run as root with podman +# or docker with user namespaces. +FROM ghcr.io/diamondlightsource/ubuntu-devcontainer:noble AS developer + +# Add any system dependencies for the developer/build environment here. +# Candidates: an ARMv7-A cross-compiler toolchain for on-PandA builds; the +# native toolchain below is enough for the simulation server and docs. +RUN apt-get update -y && apt-get install -y --no-install-recommends \ + build-essential \ + python3 \ + && apt-get dist-clean From 0e0edb789b9131c31f9f0ebda9d10da21f80c14e Mon Sep 17 00:00:00 2001 From: Tom Cobb Date: Fri, 12 Jun 2026 09:09:18 +0000 Subject: [PATCH 08/28] ci: publish docs to gh-pages from the docs branch too Trigger Docs CI on pushes to docs as well as main/tags, and allow the gh-pages publish step on the docs branch so the rewritten docs deploy before merging to main. Co-Authored-By: Claude Opus 4.8 --- .github/workflows/docs.yml | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/.github/workflows/docs.yml b/.github/workflows/docs.yml index 9fd92ce4..14045a25 100644 --- a/.github/workflows/docs.yml +++ b/.github/workflows/docs.yml @@ -4,6 +4,7 @@ on: push: branches: - main + - docs tags: - '*' pull_request: @@ -68,7 +69,7 @@ jobs: run: python3 .github/pages/make_switcher.py --add $DOCS_VERSION ${{ github.repository }} .github/pages/switcher.json - name: Publish Docs to gh-pages - if: github.ref_type == 'tag' || github.ref_name == 'main' + if: github.ref_type == 'tag' || github.ref_name == 'main' || github.ref_name == 'docs' # We pin to the SHA, not the tag, for security reasons. # https://docs.github.com/en/actions/learn-github-actions/security-hardening-for-github-actions#using-third-party-actions uses: peaceiris/actions-gh-pages@4f9cc6602d3f66b9c108549d475ec49e8ef4d45e # v4.0.0 From 37c0ca0b97e3610bfa9e755ad42dcc658c4ff311 Mon Sep 17 00:00:00 2001 From: Tom Cobb Date: Fri, 12 Jun 2026 12:11:43 +0000 Subject: [PATCH 09/28] =?UTF-8?q?docs:=20Stage=20B=20=E2=80=94=20link=20Gi?= =?UTF-8?q?tHub=20issues=20into=20stubs=20and=20verify=20notes?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- docs/PROGRESS.md | 14 ++++++++++++-- docs/reference/c-standard.md | 2 +- docs/reference/capture-options.md | 2 +- docs/reference/capture.md | 2 +- docs/reference/commands.md | 2 +- docs/reference/extension.md | 2 +- 6 files changed, 17 insertions(+), 7 deletions(-) diff --git a/docs/PROGRESS.md b/docs/PROGRESS.md index 2c714415..b74ae48b 100644 --- a/docs/PROGRESS.md +++ b/docs/PROGRESS.md @@ -37,8 +37,18 @@ Page list expanded from `06-source-provenance-map.md`. ## Blocked (issues raised) Issues not yet created — Stage B (Prompt B) will create and link them. -- ⛔ reference/extension — issue #TBD — verify -- ⛔ reference/c-standard — issue #TBD — author +- ⛔ reference/extension — issue #80 — verify +- ⛔ reference/c-standard — issue #83 — author + +- ⛔ reference/capture-options (live confirmation) — issue #78 — verify + +- ⛔ reference/capture (60 MB/s figure) — issue #79 — verify + +- ⛔ reference/commands (*IDN? v4.x) — issue #81 — verify + +- ⛔ how-to/building (CONFIG syntax) — issue #82 — verify + +- ⛔ explanations/architecture (depth) — issue #84 — author ## Notes - **Legacy sources preserved.** Original Sphinx/RST tree moved to `docs/_legacy_rst/` (not in TOC, diff --git a/docs/reference/c-standard.md b/docs/reference/c-standard.md index bc38efb0..01b9c9ea 100644 --- a/docs/reference/c-standard.md +++ b/docs/reference/c-standard.md @@ -9,5 +9,5 @@ This page is a Stage A scaffold stub and has not yet been written. **Source:** NEW (Interview1 §4) -**Blocked:** a GitHub issue will be created in Stage B (Prompt B) and linked here. +**Blocked:** [PandABlocks/PandABlocks-server#83](https://github.com/PandABlocks/PandABlocks-server/issues/83) ::: diff --git a/docs/reference/capture-options.md b/docs/reference/capture-options.md index 215cfa36..701e38eb 100644 --- a/docs/reference/capture-options.md +++ b/docs/reference/capture-options.md @@ -12,7 +12,7 @@ options" exposed by the server: The authoritative, live list for your firmware is always whatever `*CAPTURE.OPTIONS?` returns from the running server — availability of some options (`StdDev`, and therefore `Mean StdDev`) depends on the FPGA -configuration. +configuration. Live confirmation tracked in [PandABlocks/PandABlocks-server#78](https://github.com/PandABlocks/PandABlocks-server/issues/78). ::: ## Field capture options diff --git a/docs/reference/capture.md b/docs/reference/capture.md index 12a7730a..60bbcabf 100644 --- a/docs/reference/capture.md +++ b/docs/reference/capture.md @@ -160,7 +160,7 @@ be one of the following values: To get the highest performance, use `FRAMED RAW` mode. This activates a special passthrough mode which avoids copying memory as much as possible. In tests it has -been capable of sustaining 60 MBytes/s when panda-webcontrol is **not** installed. +been capable of sustaining 60 MBytes/s when panda-webcontrol is **not** installed. The downside to this mode is that if capture fails for any reason, then the last framed block of data that the server sent should be discarded, as it will have been corrupted while being sent. diff --git a/docs/reference/commands.md b/docs/reference/commands.md index b2ad1f9f..2a88c4ea 100644 --- a/docs/reference/commands.md +++ b/docs/reference/commands.md @@ -69,7 +69,7 @@ command is `*IDN?`, which returns a system identification string: ``` < *IDN? -> OK =PandA SW: 330bd94-dirty FPGA: 0.1.9 d1275f61 00000000 +> OK =PandA SW: 330bd94-dirty FPGA: 0.1.9 d1275f61 00000000 ``` The available system commands are tabulated here and listed in more detail diff --git a/docs/reference/extension.md b/docs/reference/extension.md index 0ecf431c..62ee4d47 100644 --- a/docs/reference/extension.md +++ b/docs/reference/extension.md @@ -9,5 +9,5 @@ This page is a Stage A scaffold stub and has not yet been written. **Source:** extension.rst (739w: Extension Modules, Injected Values) — verify vs implementation -**Blocked:** a GitHub issue will be created in Stage B (Prompt B) and linked here. +**Blocked:** [PandABlocks/PandABlocks-server#80](https://github.com/PandABlocks/PandABlocks-server/issues/80) ::: From f607d0753546dc925ac8d7066f074ac907a4f2a8 Mon Sep 17 00:00:00 2001 From: Tom Cobb Date: Wed, 17 Jun 2026 10:53:18 +0000 Subject: [PATCH 10/28] docs: replace {doc} roles with markdown links The {doc} role is not supported by MyST and emits a build warning for every cross-reference (20 warnings across the reference and how-to pages). Convert them to plain markdown links, which MyST resolves the same way. Co-Authored-By: Claude Opus 4.8 --- docs/explanations/architecture.md | 10 +++++----- docs/how-to/startup.md | 2 +- docs/index.md | 1 - docs/reference/capture-options.md | 6 +++--- docs/reference/capture.md | 4 ++-- docs/reference/commands.md | 4 ++-- docs/reference/config.md | 6 +++--- docs/reference/fields.md | 4 ++-- docs/reference/streaming-tables.md | 4 ++-- 9 files changed, 20 insertions(+), 21 deletions(-) diff --git a/docs/explanations/architecture.md b/docs/explanations/architecture.md index 0b0a2d99..ea86d78f 100644 --- a/docs/explanations/architecture.md +++ b/docs/explanations/architecture.md @@ -15,11 +15,11 @@ them to the PandA FPGA hardware (or a software simulation of it). **Configuration port (default 8888)** : An ASCII command/response interface used to read and write block fields and - execute system commands. Described in {doc}`/reference/commands`. + execute system commands. Described in [](/reference/commands.md). **Data port (default 8889)** : A binary streaming interface for captured experiment data. Described in - {doc}`/reference/capture`. + [](/reference/capture.md). ## Internal structure @@ -43,13 +43,13 @@ The server is organised around three cooperating layers. ## Block model The server loads its block and field topology from the `config_d` configuration -files at startup (see {doc}`/reference/config`). Each block is a named, +files at startup (see [](/reference/config.md)). Each block is a named, potentially multi-instance hardware component (e.g. `PULSE[4]`). Fields within a block map to registers and expose typed operations (read, write, capture, etc.) -as documented in {doc}`/reference/fields`. +as documented in [](/reference/fields.md). ## Persistence Field values that survive a server restart are written to a persistence file -(see the `-f` and `-t` options in {doc}`/how-to/startup`). The file is updated +(see the `-f` and `-t` options in [](/how-to/startup.md)). The file is updated on a poll/holdoff/backoff schedule to reduce write pressure. diff --git a/docs/how-to/startup.md b/docs/how-to/startup.md index 4f8054ed..f72ba4d7 100644 --- a/docs/how-to/startup.md +++ b/docs/how-to/startup.md @@ -57,7 +57,7 @@ The server can also be started from the command line with the following options. `-X` *port* : Connect to an extension server running locally on the specified port. See - {doc}`/reference/extension` for details. + [](/reference/extension.md) for details. `-r` *rootfs-version* : Specify the rootfs version string reported by the `*IDN?` command. diff --git a/docs/index.md b/docs/index.md index 6c00c2f8..2cd059c2 100644 --- a/docs/index.md +++ b/docs/index.md @@ -7,7 +7,6 @@ site: :end-before: diff --git a/docs/tutorials.md b/docs/tutorials.md index 66c75886..4b59ac34 100644 --- a/docs/tutorials.md +++ b/docs/tutorials.md @@ -2,5 +2,5 @@ ```{note} The PandABlocks-server has no tutorials yet. New users should start with the -[meta-panda tutorials](https://PandABlocks.github.io/meta-panda). +[meta-panda tutorials](xref:meta-panda/tutorials). ``` From 536f1edd310c3a5751f52d1507138d2f92f208d5 Mon Sep 17 00:00:00 2001 From: Tom Cobb Date: Thu, 18 Jun 2026 08:37:12 +0000 Subject: [PATCH 16/28] docs: enable PandABlocks-FPGA xref and fix meta-panda xref path - myst.yml: uncomment the PandABlocks-FPGA xref reference (interim /docs target). folders is already enabled here. - reference/capture.md: update the meta-panda xref to the folder-qualified path (how-to/integrate-with-a-panda) now that meta-panda emits folder URLs. Co-Authored-By: Claude Opus 4.8 --- docs/myst.yml | 5 ++++- docs/reference/capture.md | 2 +- 2 files changed, 5 insertions(+), 2 deletions(-) diff --git a/docs/myst.yml b/docs/myst.yml index a794081a..c64d13c1 100644 --- a/docs/myst.yml +++ b/docs/myst.yml @@ -24,9 +24,12 @@ project: # note above (-> /stable once releases + the switcher action land). Backs the # xrefs in tutorials and reference/capture. meta-panda: https://pandablocks.github.io/meta-panda/docs + # PandABlocks-FPGA (MyST) docs-branch build; interim /docs target per the note + # above (-> /stable once releases + the switcher action land). Backs the xrefs + # in reference/fields from the FPGA block docs. + PandABlocks-FPGA: https://pandablocks.github.io/PandABlocks-FPGA/docs # Not deployed yet — kept commented (interim /docs target; -> /stable later). # Uncomment in Stage F and upstream into python-copier-template. - # PandABlocks-FPGA: https://pandablocks.github.io/PandABlocks-FPGA/docs # fastcs-PandABlocks: https://diamondlightsource.github.io/fastcs-PandABlocks/docs toc: - file: index.md diff --git a/docs/reference/capture.md b/docs/reference/capture.md index 73082bf0..b243b0ba 100644 --- a/docs/reference/capture.md +++ b/docs/reference/capture.md @@ -239,7 +239,7 @@ END 5 Ok This page describes the wire format on the data port. For end-to-end ways of retrieving captured data — reading the binary stream directly, using the Python client, or via EPICS/Tango — see the meta-panda how-to -[Integrate with a PandA](xref:meta-panda/integrate-with-a-panda). +[Integrate with a PandA](xref:meta-panda/how-to/integrate-with-a-panda). From d89ea5bcd761255b4188282be0bb59c7c75cd8a6 Mon Sep 17 00:00:00 2001 From: Tom Cobb Date: Thu, 18 Jun 2026 15:05:42 +0000 Subject: [PATCH 17/28] docs: build with myst --strict `make docs` now runs `myst build --html --strict`, so CI exits non-zero on any error-severity message (e.g. an unresolved cross-repo xref) rather than publishing broken links. Verified the docs build clean at exit 0 under --strict. Co-Authored-By: Claude Opus 4.8 --- Makefile | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/Makefile b/Makefile index f13c5d34..04f3b8a8 100644 --- a/Makefile +++ b/Makefile @@ -143,13 +143,15 @@ slow_load: $(SLOW_LOAD) # # Docs are built with MyST (mystmd), run through npx so no global install is # needed; pin the version with MYSTMD_VERSION (see CONFIG.example). `make docs` -# mirrors `myst build --html` and `make docs-dev` mirrors `myst start`, matching -# the docs/ task runner. MyST writes its output into docs/_build/html. +# mirrors `myst build --html --strict` and `make docs-dev` mirrors `myst start`, +# matching the docs/ task runner. MyST writes its output into docs/_build/html. +# --strict exits non-zero on any error-severity message (e.g. an unresolved +# cross-repo xref) so CI fails rather than publishing broken links. MYST = npx --yes --package mystmd@$(MYSTMD_VERSION) myst docs: - cd docs && $(MYST) build --html + cd docs && $(MYST) build --html --strict docs-dev: cd docs && $(MYST) start From 0a1666c0bb6ba293d55fa06d0716c7c0438686bb Mon Sep 17 00:00:00 2001 From: Tom Cobb Date: Thu, 18 Jun 2026 15:36:25 +0000 Subject: [PATCH 18/28] docs: enforce link quality via error_rules Escalate link-resolves, reference-target-resolves and link-text-exists to error so (with `make docs` / `myst build --strict`) broken links, unresolved cross references and empty auto-filled labels fail CI. See reference/documentation-standards in meta-panda for the conventions enforced. Co-Authored-By: Claude Opus 4.8 --- docs/myst.yml | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/docs/myst.yml b/docs/myst.yml index c64d13c1..72befaed 100644 --- a/docs/myst.yml +++ b/docs/myst.yml @@ -31,6 +31,18 @@ project: # Not deployed yet — kept commented (interim /docs target; -> /stable later). # Uncomment in Stage F and upstream into python-copier-template. # fastcs-PandABlocks: https://diamondlightsource.github.io/fastcs-PandABlocks/docs + # Escalate link/cross-reference quality rules from warning to error so that, + # combined with `myst build --strict`, a broken link, an unresolved cross + # reference, or an empty auto-filled label fails CI instead of publishing a + # dead link. See reference/documentation-standards in meta-panda for the + # linking conventions these rules enforce. + error_rules: + - rule: link-resolves + severity: error + - rule: reference-target-resolves + severity: error + - rule: link-text-exists + severity: error toc: - file: index.md - file: tutorials.md From 50eb3ec71a115299c59fdfbc8fe4af1fc15fc112 Mon Sep 17 00:00:00 2001 From: "Sode, Adedamola (DLSLtd,RAL,LSCI)" Date: Fri, 19 Jun 2026 09:29:20 +0100 Subject: [PATCH 19/28] removed AI comment as command still works --- docs/reference/commands.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/reference/commands.md b/docs/reference/commands.md index b3b452db..ef23867e 100644 --- a/docs/reference/commands.md +++ b/docs/reference/commands.md @@ -69,7 +69,7 @@ command is `*IDN?`, which returns a system identification string: ``` < *IDN? -> OK =PandA SW: 330bd94-dirty FPGA: 0.1.9 d1275f61 00000000 +> OK =PandA SW: 4.1-1-g2a34851 FPGA: 4.2.0 04e81f37 8a2b0249 rootfs: 2023.2+snapshot-bb27e6aef4dafd501bb5d72c95facd178c92dc48 (langdale) ``` The available system commands are tabulated here and listed in more detail From 71cebb17866f5f312a547b678edca1e8c49a893f Mon Sep 17 00:00:00 2001 From: Tom Cobb Date: Mon, 22 Jun 2026 09:10:32 +0000 Subject: [PATCH 20/28] Docs: migrate to myst-version-switcher assemble model (v0.5.0) Replace the legacy peaceiris/actions-gh-pages + keep_files publish (and the old switcher@v0.2.0 action) with the version-switcher assemble model: - ci.yml + _docs.yml + _publish.yml (the documented tutorial split). _docs.yml keeps `make docs` (npx mystmd) and now packs docs.zip (bare html/ root); _publish.yml runs assemble@v0.5.0 -> upload-pages-artifact -> deploy-pages. - Bump the version-switcher plugin v0.2.0 -> v0.5.0 in docs/myst.yml. - Publish the `docs` branch as the live version during migration (guard-default-branch: false; main has no MyST docs yet). - Remove the old docs.yml. The Pages-source flip + gh-pages backfill/delete are a separate cutover. Co-Authored-By: Claude Opus 4.8 --- .github/workflows/_docs.yml | 82 +++++++++++++++++++++++++++++++ .github/workflows/_publish.yml | 86 +++++++++++++++++++++++++++++++++ .github/workflows/ci.yml | 55 +++++++++++++++++++++ .github/workflows/docs.yml | 88 ---------------------------------- docs/myst.yml | 2 +- notes.txt | 1 + 6 files changed, 225 insertions(+), 89 deletions(-) create mode 100644 .github/workflows/_docs.yml create mode 100644 .github/workflows/_publish.yml create mode 100644 .github/workflows/ci.yml delete mode 100644 .github/workflows/docs.yml create mode 100644 notes.txt diff --git a/.github/workflows/_docs.yml b/.github/workflows/_docs.yml new file mode 100644 index 00000000..abd96832 --- /dev/null +++ b/.github/workflows/_docs.yml @@ -0,0 +1,82 @@ +on: + workflow_call: + outputs: + version-name: + description: The version name this build was served at (pr- | main | docs | ). + value: ${{ jobs.build.outputs.version-name }} + +# Build the docs at the versioned BASE_URL and upload this build's `docs` artifact +# (docs.zip, bare html/ root). This is the UNPRIVILEGED half: it runs for every +# event — PRs (including forks), pushes to main/docs, and tags — but never +# publishes. _publish.yml (nested by ci.yml on internal events) reconstructs the +# whole site from these artifacts + release assets and deploys it to Pages. +# +# This repo has no package.json: mystmd is run on demand via npx (pinned by +# MYSTMD_VERSION in CONFIG) and driven through the `make docs` task runner, so we +# only need Node on the PATH. +jobs: + build: + runs-on: ubuntu-latest + outputs: + version-name: ${{ steps.ver.outputs.version-name }} + steps: + - uses: actions/checkout@v5 + + - name: Setup Node + uses: actions/setup-node@v4 + with: + node-version: 20 + + # `make docs` reads build settings (incl. MYSTMD_VERSION) from CONFIG. + - name: Create CONFIG + run: cp CONFIG.example CONFIG + + # Version name = the site sub-dir this build is served at, and the BASE_URL it + # must be built with. pr- for PRs; otherwise the ref name (main, docs, or a + # tag without `/`). No sanitisation: every name is filesystem/URL-safe already. + - name: Compute version name + id: ver + run: | + set -euo pipefail + if [ "${{ github.event_name }}" = pull_request ]; then + name="pr-${{ github.event.pull_request.number }}" + else + name="${{ github.ref_name }}" + fi + echo "version-name=$name" >> "$GITHUB_OUTPUT" + + # BASE_URL must match the versioned sub-path the build is served at, or its + # root-absolute assets 404. assemble files this build's artifact at the same + # version name, so the two cannot drift. + - name: Build docs + env: + BASE_URL: /PandABlocks-server/${{ steps.ver.outputs.version-name }} + run: make docs + + # Pack the build as docs.zip with a bare html/ root — the durable contract + # both _publish.yml's gather and the docs-release asset rely on. + - name: Pack docs.zip (bare html/ root) + run: | + set -euo pipefail + ( cd docs/_build && zip -rq "$RUNNER_TEMP/docs.zip" html ) + + # compression-level 0: docs.zip is already compressed. + - name: Upload docs artifact + uses: actions/upload-artifact@v4 + with: + name: docs + path: ${{ runner.temp }}/docs.zip + compression-level: 0 + + # Fork PRs build + verify here but do not auto-publish (ci.yml's publish job + # excludes them — the security boundary). Surface that on the PR. + - name: Explain the fork-preview opt-in + if: >- + github.event_name == 'pull_request' && + github.event.pull_request.head.repo.full_name != github.repository + run: | + echo "::warning title=Docs preview not published::This is a fork PR, so the \ + versioned docs site is NOT auto-published (fork builds run with a read-only \ + token). A maintainer can publish a preview by running the Publish workflow \ + for PR #${{ github.event.pull_request.number }}: \ + https://github.com/${{ github.repository }}/actions/workflows/_publish.yml" diff --git a/.github/workflows/_publish.yml b/.github/workflows/_publish.yml new file mode 100644 index 00000000..7c69f499 --- /dev/null +++ b/.github/workflows/_publish.yml @@ -0,0 +1,86 @@ +name: Publish + +# Reconstruct the WHOLE versioned docs site from durable sources (the live branch's +# build, release docs.zip assets, open-PR build artifacts) and deploy it directly to +# GitHub Pages via the published myst-version-switcher `assemble` action — no +# gh-pages branch. This is the PRIVILEGED half, kept in its own file so it can only +# run two trusted ways: +# +# workflow_call — nested by ci.yml AFTER a successful build, for INTERNAL events +# only (internal PRs, pushes to main/docs/tags). ci.yml passes +# this build's `version-name`; assemble downloads this run's +# `docs` artifact and stages it directly (the run isn't a +# completed success yet, so the gather can't discover it). +# workflow_dispatch — a maintainer's opt-in to preview an EXTERNAL fork PR. +# +# guard-default-branch is false during the docs migration: the new MyST docs live on +# the `docs` branch, so the default branch (main) has no build to guard against yet. +on: + workflow_call: + inputs: + version-name: + description: Version name (pr- | main | docs | ) of the in-run build to inject. + required: true + type: string + workflow_dispatch: + inputs: + pr: + description: External fork PR number to approve (pins its head SHA) and preview. + required: false + +permissions: + contents: read # checkout + read release assets + actions: read # gh run download (this run's + cross-run docs artifacts) + pages: write # deploy to Pages + id-token: write # deploy-pages OIDC + statuses: write # set the preview-approved status on a fork PR head SHA + +concurrency: + group: pages + cancel-in-progress: false + +jobs: + publish: + # The canonical-repo guard lives in the caller (ci.yml's publish job), not here. + runs-on: ubuntu-latest + environment: + name: github-pages + url: ${{ steps.deployment.outputs.page_url }} + steps: + - uses: actions/checkout@v5 + with: + fetch-depth: 0 # tags, for version ordering + prerelease detection + + # workflow_dispatch (fork opt-in): pin THIS commit as approved. + - name: Approve fork PR head SHA + if: github.event_name == 'workflow_dispatch' && inputs.pr != '' + env: + GH_TOKEN: ${{ github.token }} + REPO: ${{ github.repository }} + PR: ${{ inputs.pr }} + run: | + set -euo pipefail + sha=$(gh pr view "$PR" --repo "$REPO" --json headRefOid -q .headRefOid) + gh api --method POST "repos/$REPO/statuses/$sha" \ + -f state=success -f context=preview-approved \ + -f description="Fork docs preview approved" + + # On the nested call, `artifact-version-name` tells assemble to download this + # run's `docs` artifact and stage it as that version (it isn't a completed + # success yet). Empty on workflow_dispatch -> a pure durable gather. + - name: Assemble versioned site + id: site + uses: DiamondLightSource/myst-version-switcher-plugin/assemble@v0.5.0 + with: + repo: ${{ github.repository }} + guard-default-branch: false + artifact-version-name: ${{ inputs.version-name }} + + - name: Upload Pages artifact + uses: actions/upload-pages-artifact@v3 + with: + path: ${{ steps.site.outputs.dir }} + + - name: Deploy to GitHub Pages + id: deployment + uses: actions/deploy-pages@v4 diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml new file mode 100644 index 00000000..0a48b02a --- /dev/null +++ b/.github/workflows/ci.yml @@ -0,0 +1,55 @@ +name: Docs CI + +# Build + verify the docs on every event, then publish on INTERNAL events. The +# build (`_docs.yml`) runs for PRs (including forks), pushes to main/docs, and tags, +# and uploads each build's `docs` artifact. Publishing is nested here (the `publish` +# job -> `_publish.yml`) so its status is visible on the PR/commit, but ONLY for +# internal events on this repo: a fork PR's build runs with a read-only token and +# must never deploy. +# +# `docs` is a publish trigger during the docs migration: the new MyST docs live on +# the `docs` branch (main has none yet), so `docs` is the live version. +on: + pull_request: + push: + branches: [main, docs] + tags: ['*'] # '*' never matches '/' + +jobs: + docs: + uses: ./.github/workflows/_docs.yml + + # Tag-only: attach this build's docs.zip (bare html/ root) to the GitHub Release + # so `assemble` can reconstruct that released version on future deploys. + docs-release: + needs: [docs] + if: github.ref_type == 'tag' + runs-on: ubuntu-latest + permissions: + contents: write + steps: + - uses: actions/download-artifact@v4 + with: + name: docs + - env: + GH_TOKEN: ${{ github.token }} + run: gh release upload "${{ github.ref_name }}" docs.zip --clobber --repo "${{ github.repository }}" + + # Internal events only: an internal PR, or a push to main/docs/tag, has a + # same-repo build we can trust + deploy. Fork PRs (head repo != this repo) are + # excluded — _docs.yml's build job warns them instead. + publish: + needs: [docs] + if: >- + github.repository == 'PandABlocks/PandABlocks-server' && + ( github.event_name != 'pull_request' || + github.event.pull_request.head.repo.full_name == github.repository ) + uses: ./.github/workflows/_publish.yml + with: + version-name: ${{ needs.docs.outputs.version-name }} + permissions: + contents: read + actions: read + pages: write + id-token: write + statuses: write diff --git a/.github/workflows/docs.yml b/.github/workflows/docs.yml deleted file mode 100644 index c518b2c5..00000000 --- a/.github/workflows/docs.yml +++ /dev/null @@ -1,88 +0,0 @@ -name: Docs CI - -# Mirrors DiamondLightSource/myst-version-switcher-plugin -# .github/workflows/_docs.yml, but as a normal (non-reusable) workflow. This repo -# has no package.json: mystmd is run on demand via npx (pinned by MYSTMD_VERSION -# in CONFIG) and driven through the `make docs` task runner. switcher.json and the -# root redirect are written by the published version-switcher `switcher` action. - -on: - push: - branches: - - main - - docs - tags: - - '*' - pull_request: - -permissions: - contents: write - -jobs: - build: - runs-on: ubuntu-latest - - steps: - - name: Avoid git conflicts when tag and branch pushed at same time - if: github.ref_type == 'tag' - run: sleep 60 - - - name: Checkout - uses: actions/checkout@v5 - with: - # Need tags (for the switcher version list) and origin/gh-pages. - fetch-depth: 0 - - # No package.json here; mystmd is fetched on demand by npx (see the docs - # target in the Makefile), so we only need Node on the PATH. - - name: Setup Node - uses: actions/setup-node@v4 - with: - node-version: 20 - - # `make docs` reads build settings (incl. MYSTMD_VERSION) from CONFIG. - - name: Create CONFIG - run: cp CONFIG.example CONFIG - - - name: Sanitize ref name for docs version - run: echo "DOCS_VERSION=${GITHUB_REF_NAME//[^A-Za-z0-9._-]/_}" >> $GITHUB_ENV - - # BASE_URL is required so assets/links resolve under the versioned Pages - # sub-path (https://pandablocks.github.io/PandABlocks-server/$DOCS_VERSION/). - - name: Build docs - env: - BASE_URL: /PandABlocks-server/${{ env.DOCS_VERSION }} - run: make docs - - # Two layouts from the one build: artifact/html for the uploaded artifact - # (downstream relies on the `html` dir name), and pages/ for the - # gh-pages publish tree, into which the action also writes switcher.json - # and the root redirect. - - name: Stage built docs - run: | - mkdir -p $RUNNER_TEMP/artifact $RUNNER_TEMP/pages - cp -r docs/_build/html $RUNNER_TEMP/artifact/html - cp -r docs/_build/html $RUNNER_TEMP/pages/$DOCS_VERSION - - - name: Upload built docs artifact - uses: actions/upload-artifact@v4 - with: - name: docs - path: ${{ runner.temp }}/artifact - - - name: Write switcher.json + redirect - uses: DiamondLightSource/myst-version-switcher-plugin/switcher@v0.2.0 - with: - version: ${{ env.DOCS_VERSION }} - repo: ${{ github.repository }} - output-dir: ${{ runner.temp }}/pages - - - name: Publish Docs to gh-pages - if: github.ref_type == 'tag' || github.ref_name == 'main' || github.ref_name == 'docs' - # We pin to the SHA, not the tag, for security reasons. - # https://docs.github.com/en/actions/learn-github-actions/security-hardening-for-github-actions#using-third-party-actions - uses: peaceiris/actions-gh-pages@4f9cc6602d3f66b9c108549d475ec49e8ef4d45e # v4.0.0 - with: - github_token: ${{ secrets.GITHUB_TOKEN }} - publish_dir: ${{ runner.temp }}/pages - keep_files: true diff --git a/docs/myst.yml b/docs/myst.yml index 72befaed..64d2c2b5 100644 --- a/docs/myst.yml +++ b/docs/myst.yml @@ -3,7 +3,7 @@ project: title: PandABlocks-server github: https://github.com/PandABlocks/PandABlocks-server plugins: - - https://github.com/DiamondLightSource/myst-version-switcher-plugin/releases/download/v0.2.0/version-switcher.mjs + - https://github.com/DiamondLightSource/myst-version-switcher-plugin/releases/download/v0.5.0/version-switcher.mjs # Cross-repository references (mystmd xref + Sphinx intersphinx). # PROTOTYPE for upstreaming into the DLS python-copier-template (Stage A spec §6). # diff --git a/notes.txt b/notes.txt new file mode 100644 index 00000000..60ef38bf --- /dev/null +++ b/notes.txt @@ -0,0 +1 @@ +FPGA docs link go to FPGA docs not repo From 475655557ba4ff86d01effc483f65d3ce61957bc Mon Sep 17 00:00:00 2001 From: "Sode, Adedamola (DLSLtd,RAL,LSCI)" Date: Fri, 19 Jun 2026 08:46:42 +0100 Subject: [PATCH 21/28] changed documentation from sphinx to mystmd --- docs/how-to/building.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/docs/how-to/building.md b/docs/how-to/building.md index 9be667eb..a5be04ab 100644 --- a/docs/how-to/building.md +++ b/docs/how-to/building.md @@ -49,13 +49,13 @@ The following symbols must point to the appropriate dependencies: The following symbols can be left at their defaults: `BUILD_DIR` -: Destination for built files. +: Path for built files. `PYTHON` -: Python interpreter used during the build. +: Path for python interpreter used during the build -`SPHINX_BUILD` -: `sphinx-build` script for documentation builds. +`MYSTMD_VERSION` +: `mystmd version No.` Version of MyST required to build docs. defaults to 1.10.1. `DEFAULT_TARGETS` : Makefile targets built by `make` or `make default`. Default list: `driver`, From 6caa79d00add7d8317343bbd0eb429cde04618e7 Mon Sep 17 00:00:00 2001 From: "Sode, Adedamola (DLSLtd,RAL,LSCI)" Date: Fri, 19 Jun 2026 09:22:52 +0100 Subject: [PATCH 22/28] adding node to CI container for building docs during zpkg build --- .github/workflows/code.yml | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/.github/workflows/code.yml b/.github/workflows/code.yml index 103ec395..c6e3f38a 100644 --- a/.github/workflows/code.yml +++ b/.github/workflows/code.yml @@ -42,7 +42,13 @@ jobs: cd PandABlocks-rootfs ln -s CONFIG.example CONFIG make kernel PLATFORM=${{matrix.platform}} - + + # zpkg requires the docs built by myst, so we need to add node to + # the container. + - name: Setup Node + uses: actions/setup-node@v4 + with: + node-version: 20 - name: Make zpkg run: | cd PandABlocks-server From bd5fb6f71b3b60d229839652f8801ecadf91dd8e Mon Sep 17 00:00:00 2001 From: "Sode, Adedamola (DLSLtd,RAL,LSCI)" Date: Fri, 19 Jun 2026 11:19:39 +0100 Subject: [PATCH 23/28] Removed rootfs kernel and zpkg --- .github/workflows/code.yml | 6 ------ docs/how-to/building.md | 15 +-------------- 2 files changed, 1 insertion(+), 20 deletions(-) diff --git a/.github/workflows/code.yml b/.github/workflows/code.yml index c6e3f38a..872a6335 100644 --- a/.github/workflows/code.yml +++ b/.github/workflows/code.yml @@ -43,12 +43,6 @@ jobs: ln -s CONFIG.example CONFIG make kernel PLATFORM=${{matrix.platform}} - # zpkg requires the docs built by myst, so we need to add node to - # the container. - - name: Setup Node - uses: actions/setup-node@v4 - with: - node-version: 20 - name: Make zpkg run: | cd PandABlocks-server diff --git a/docs/how-to/building.md b/docs/how-to/building.md index a5be04ab..02e049bb 100644 --- a/docs/how-to/building.md +++ b/docs/how-to/building.md @@ -19,9 +19,6 @@ The following must be available before building the server. : Must be available before building the server; it contains the configuration file defining the low-level register interface to the PandA firmware. -**PandABlocks-rootfs** -: Provides the kernel build tree and the `zpkg` build tool. - ## Setting up the `CONFIG` file Copy `CONFIG.example` to `CONFIG` in the base directory and edit as needed, @@ -39,13 +36,6 @@ The following symbols must point to the appropriate dependencies: : Path to the Zynq cross-compiler toolchain if it is not on `PATH`. Not required for the simulation server or documentation builds. -`KERNEL_DIR` -: Path to the kernel build tree from the PandABlocks-rootfs build. Required to - build the kernel module. - -`PANDA_ROOTFS` -: Path to the PandABlocks-rootfs checkout where the `zpkg` build tool is found. - The following symbols can be left at their defaults: `BUILD_DIR` @@ -59,7 +49,7 @@ The following symbols can be left at their defaults: `DEFAULT_TARGETS` : Makefile targets built by `make` or `make default`. Default list: `driver`, - `server`, `sim_server`, `docs`, `zpkg`. + `server`, `sim_server`, `docs`. ## Build targets @@ -70,7 +60,6 @@ The following symbols can be left at their defaults: | `server` | Server binary to run on PandA | | `sim_server` | Simulation server to run on the local PC | | `docs` | HTML documentation | -| `zpkg` | `panda-server` zpkg package | | `clean` | Removes the entire `$(BUILD_DIR)` directory | ## Generated files @@ -86,5 +75,3 @@ After a successful build, `$(BUILD_DIR)` contains: `html/` : HTML documentation. -`panda-server@`*version*`.zpg` and `zpkg-panda-server/` -: The zpkg package for installation onto PandA. From cde459c7cbe4bcbc42f668b38ce8c0c081133ae6 Mon Sep 17 00:00:00 2001 From: Tom Cobb Date: Mon, 22 Jun 2026 11:07:23 +0000 Subject: [PATCH 24/28] Docs: re-deploy docs version after closing the docs->main preview race Co-Authored-By: Claude Opus 4.8 From 7b19bb3c973b299499d372f9204bde1997c2b51e Mon Sep 17 00:00:00 2001 From: Tom Cobb Date: Mon, 22 Jun 2026 11:18:41 +0000 Subject: [PATCH 25/28] Docs: overlay legacy main/ from gh-pages during migration main has no MyST docs build yet, so the assemble model drops /main/. Until docs merges to main, stage the existing gh-pages main/ build into the assemble site dir so generate lists main in switcher.json and /main/ keeps serving. Co-Authored-By: Claude Opus 4.8 --- .github/workflows/_publish.yml | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) diff --git a/.github/workflows/_publish.yml b/.github/workflows/_publish.yml index 7c69f499..9594d139 100644 --- a/.github/workflows/_publish.yml +++ b/.github/workflows/_publish.yml @@ -51,6 +51,26 @@ jobs: with: fetch-depth: 0 # tags, for version ordering + prerelease detection + # MIGRATION SHIM: the new docs live on the `docs` branch; `main` has no MyST + # build yet, so the assemble model would drop /main/. Until `docs` merges to + # `main`, overlay the existing gh-pages main/ build into the assemble site dir + # ($RUNNER_TEMP/site, which assemble.sh mkdir -p's but never wipes) so generate + # lists `main` in switcher.json and /main/ keeps serving. Remove this step (and + # delete gh-pages) once main itself is migrated. + - name: Stage legacy main/ from gh-pages (migration shim) + env: + GH_TOKEN: ${{ github.token }} + run: | + set -euo pipefail + if git fetch --quiet origin gh-pages 2>/dev/null \ + && git cat-file -e origin/gh-pages:main/index.html 2>/dev/null; then + mkdir -p "$RUNNER_TEMP/site/main" + git archive origin/gh-pages main | tar -x --strip-components=1 -C "$RUNNER_TEMP/site/main" + echo "Staged legacy main/ from gh-pages" + else + echo "::warning::no gh-pages main/ to stage — skipping" + fi + # workflow_dispatch (fork opt-in): pin THIS commit as approved. - name: Approve fork PR head SHA if: github.event_name == 'workflow_dispatch' && inputs.pr != '' From 6b55daafb5e4ba1e5b83dd82593e761d8c8d0ef6 Mon Sep 17 00:00:00 2001 From: Tom Cobb Date: Mon, 22 Jun 2026 12:05:30 +0000 Subject: [PATCH 26/28] Docs: verify the deployed Pages origin matches the assembled switcher.json Mirror the upstream myst-version-switcher _publish.yml change: after deploy-pages, poll the live switcher.json and fail if it does not byte-match the assembled one, so a stale/wedged Pages origin surfaces as a red check instead of silently serving old docs. Co-Authored-By: Claude Opus 4.8 --- .github/workflows/_publish.yml | 38 ++++++++++++++++++++++++++++++++++ 1 file changed, 38 insertions(+) diff --git a/.github/workflows/_publish.yml b/.github/workflows/_publish.yml index 9594d139..5bbfa403 100644 --- a/.github/workflows/_publish.yml +++ b/.github/workflows/_publish.yml @@ -104,3 +104,41 @@ jobs: - name: Deploy to GitHub Pages id: deployment uses: actions/deploy-pages@v4 + + # deploy-pages reports success once the deployment RECORD flips active — even + # if the Pages backend silently fails to update the SERVED origin (a wedged + # site can serve a stale artifact indefinitely; see the 2026-06 incident where + # a gh-pages→Actions cutover froze the origin while every deploy reported OK). + # Confirm the live switcher.json byte-matches the one we just assembled, polling + # for up to 60s to let the origin/CDN propagate. A mismatch fails the run, so a + # stale origin surfaces as a red check instead of silently serving old docs. + - name: Verify the deployed origin matches what we assembled + env: + PAGE_URL: ${{ steps.deployment.outputs.page_url }} + SITE_DIR: ${{ steps.site.outputs.dir }} + run: | + set -euo pipefail + [ -n "$PAGE_URL" ] || { echo "::error::deploy-pages did not report a page_url"; exit 1; } + local_json="$SITE_DIR/switcher.json" + [ -f "$local_json" ] || { echo "::error::no switcher.json at $local_json"; exit 1; } + # Command substitution strips trailing newlines from both sides, so the + # comparison is robust to a curl/file trailing-newline difference. + want=$(cat "$local_json") + url="${PAGE_URL%/}/switcher.json" + deadline=$(( $(date +%s) + 60 )) + attempt=0 + while :; do + attempt=$((attempt + 1)) + got=$(curl -fsS "$url?cb=$(date +%s%N)" || true) # cache-bust → hit the origin + if [ -n "$got" ] && [ "$got" = "$want" ]; then + echo "origin matches the assembled switcher.json (attempt $attempt)" + exit 0 + fi + if [ "$(date +%s)" -ge "$deadline" ]; then + echo "::error::live switcher.json still does not match the assembled one after 60s — Pages origin likely stale/wedged" + echo "--- assembled ($local_json) ---"; printf '%s\n' "$want" + echo "--- live ($url) ---"; printf '%s\n' "$got" + exit 1 + fi + sleep 1 + done From 69e14997970a10f3291957bbe77e8d69595d9486 Mon Sep 17 00:00:00 2001 From: Tom Cobb Date: Wed, 24 Jun 2026 11:21:03 +0000 Subject: [PATCH 27/28] Docs CI: consume shared reusable workflows @v0.7.0 Replace this repo's docs-build/publish workflows with calls to the shared myst-version-switcher-plugin reusable workflows pinned at v0.7.0: - docs.yml@v0.7.0 builds with `cp CONFIG.example CONFIG && make docs` - publish.yml@v0.7.0 reconstructs the whole versioned site from durable sources (main's build persisted at _sources/main.zip, release docs.zip assets, open-PR artifacts) and deploys to Pages directly (no gh-pages) main is the docs branch and the default branch, so it is self-durable and guard-default-branch stays "true": every deploy asserts main is present, so a publish can never silently drop /main/. Delete the old per-branch _docs.yml/_publish.yml (incl. the gh-pages main shim and assemble@v0.5.0). Add preview-fork.yml, a workflow_dispatch wrapper calling publish.yml with `pr` (a reusable workflow can't be dispatched cross-repo) for the maintainer fork-PR preview opt-in; docs.yml's fork warning links to it. code.yml and tests.yml are unchanged. Co-Authored-By: Claude Opus 4.8 Claude-Session: https://claude.ai/code/session_01RBtsHDzshkwFgTJtrDVveU --- .github/workflows/_docs.yml | 82 ---------------- .github/workflows/_publish.yml | 144 ----------------------------- .github/workflows/ci.yml | 38 +++++--- .github/workflows/preview-fork.yml | 31 +++++++ 4 files changed, 56 insertions(+), 239 deletions(-) delete mode 100644 .github/workflows/_docs.yml delete mode 100644 .github/workflows/_publish.yml create mode 100644 .github/workflows/preview-fork.yml diff --git a/.github/workflows/_docs.yml b/.github/workflows/_docs.yml deleted file mode 100644 index abd96832..00000000 --- a/.github/workflows/_docs.yml +++ /dev/null @@ -1,82 +0,0 @@ -on: - workflow_call: - outputs: - version-name: - description: The version name this build was served at (pr- | main | docs | ). - value: ${{ jobs.build.outputs.version-name }} - -# Build the docs at the versioned BASE_URL and upload this build's `docs` artifact -# (docs.zip, bare html/ root). This is the UNPRIVILEGED half: it runs for every -# event — PRs (including forks), pushes to main/docs, and tags — but never -# publishes. _publish.yml (nested by ci.yml on internal events) reconstructs the -# whole site from these artifacts + release assets and deploys it to Pages. -# -# This repo has no package.json: mystmd is run on demand via npx (pinned by -# MYSTMD_VERSION in CONFIG) and driven through the `make docs` task runner, so we -# only need Node on the PATH. -jobs: - build: - runs-on: ubuntu-latest - outputs: - version-name: ${{ steps.ver.outputs.version-name }} - steps: - - uses: actions/checkout@v5 - - - name: Setup Node - uses: actions/setup-node@v4 - with: - node-version: 20 - - # `make docs` reads build settings (incl. MYSTMD_VERSION) from CONFIG. - - name: Create CONFIG - run: cp CONFIG.example CONFIG - - # Version name = the site sub-dir this build is served at, and the BASE_URL it - # must be built with. pr- for PRs; otherwise the ref name (main, docs, or a - # tag without `/`). No sanitisation: every name is filesystem/URL-safe already. - - name: Compute version name - id: ver - run: | - set -euo pipefail - if [ "${{ github.event_name }}" = pull_request ]; then - name="pr-${{ github.event.pull_request.number }}" - else - name="${{ github.ref_name }}" - fi - echo "version-name=$name" >> "$GITHUB_OUTPUT" - - # BASE_URL must match the versioned sub-path the build is served at, or its - # root-absolute assets 404. assemble files this build's artifact at the same - # version name, so the two cannot drift. - - name: Build docs - env: - BASE_URL: /PandABlocks-server/${{ steps.ver.outputs.version-name }} - run: make docs - - # Pack the build as docs.zip with a bare html/ root — the durable contract - # both _publish.yml's gather and the docs-release asset rely on. - - name: Pack docs.zip (bare html/ root) - run: | - set -euo pipefail - ( cd docs/_build && zip -rq "$RUNNER_TEMP/docs.zip" html ) - - # compression-level 0: docs.zip is already compressed. - - name: Upload docs artifact - uses: actions/upload-artifact@v4 - with: - name: docs - path: ${{ runner.temp }}/docs.zip - compression-level: 0 - - # Fork PRs build + verify here but do not auto-publish (ci.yml's publish job - # excludes them — the security boundary). Surface that on the PR. - - name: Explain the fork-preview opt-in - if: >- - github.event_name == 'pull_request' && - github.event.pull_request.head.repo.full_name != github.repository - run: | - echo "::warning title=Docs preview not published::This is a fork PR, so the \ - versioned docs site is NOT auto-published (fork builds run with a read-only \ - token). A maintainer can publish a preview by running the Publish workflow \ - for PR #${{ github.event.pull_request.number }}: \ - https://github.com/${{ github.repository }}/actions/workflows/_publish.yml" diff --git a/.github/workflows/_publish.yml b/.github/workflows/_publish.yml deleted file mode 100644 index 5bbfa403..00000000 --- a/.github/workflows/_publish.yml +++ /dev/null @@ -1,144 +0,0 @@ -name: Publish - -# Reconstruct the WHOLE versioned docs site from durable sources (the live branch's -# build, release docs.zip assets, open-PR build artifacts) and deploy it directly to -# GitHub Pages via the published myst-version-switcher `assemble` action — no -# gh-pages branch. This is the PRIVILEGED half, kept in its own file so it can only -# run two trusted ways: -# -# workflow_call — nested by ci.yml AFTER a successful build, for INTERNAL events -# only (internal PRs, pushes to main/docs/tags). ci.yml passes -# this build's `version-name`; assemble downloads this run's -# `docs` artifact and stages it directly (the run isn't a -# completed success yet, so the gather can't discover it). -# workflow_dispatch — a maintainer's opt-in to preview an EXTERNAL fork PR. -# -# guard-default-branch is false during the docs migration: the new MyST docs live on -# the `docs` branch, so the default branch (main) has no build to guard against yet. -on: - workflow_call: - inputs: - version-name: - description: Version name (pr- | main | docs | ) of the in-run build to inject. - required: true - type: string - workflow_dispatch: - inputs: - pr: - description: External fork PR number to approve (pins its head SHA) and preview. - required: false - -permissions: - contents: read # checkout + read release assets - actions: read # gh run download (this run's + cross-run docs artifacts) - pages: write # deploy to Pages - id-token: write # deploy-pages OIDC - statuses: write # set the preview-approved status on a fork PR head SHA - -concurrency: - group: pages - cancel-in-progress: false - -jobs: - publish: - # The canonical-repo guard lives in the caller (ci.yml's publish job), not here. - runs-on: ubuntu-latest - environment: - name: github-pages - url: ${{ steps.deployment.outputs.page_url }} - steps: - - uses: actions/checkout@v5 - with: - fetch-depth: 0 # tags, for version ordering + prerelease detection - - # MIGRATION SHIM: the new docs live on the `docs` branch; `main` has no MyST - # build yet, so the assemble model would drop /main/. Until `docs` merges to - # `main`, overlay the existing gh-pages main/ build into the assemble site dir - # ($RUNNER_TEMP/site, which assemble.sh mkdir -p's but never wipes) so generate - # lists `main` in switcher.json and /main/ keeps serving. Remove this step (and - # delete gh-pages) once main itself is migrated. - - name: Stage legacy main/ from gh-pages (migration shim) - env: - GH_TOKEN: ${{ github.token }} - run: | - set -euo pipefail - if git fetch --quiet origin gh-pages 2>/dev/null \ - && git cat-file -e origin/gh-pages:main/index.html 2>/dev/null; then - mkdir -p "$RUNNER_TEMP/site/main" - git archive origin/gh-pages main | tar -x --strip-components=1 -C "$RUNNER_TEMP/site/main" - echo "Staged legacy main/ from gh-pages" - else - echo "::warning::no gh-pages main/ to stage — skipping" - fi - - # workflow_dispatch (fork opt-in): pin THIS commit as approved. - - name: Approve fork PR head SHA - if: github.event_name == 'workflow_dispatch' && inputs.pr != '' - env: - GH_TOKEN: ${{ github.token }} - REPO: ${{ github.repository }} - PR: ${{ inputs.pr }} - run: | - set -euo pipefail - sha=$(gh pr view "$PR" --repo "$REPO" --json headRefOid -q .headRefOid) - gh api --method POST "repos/$REPO/statuses/$sha" \ - -f state=success -f context=preview-approved \ - -f description="Fork docs preview approved" - - # On the nested call, `artifact-version-name` tells assemble to download this - # run's `docs` artifact and stage it as that version (it isn't a completed - # success yet). Empty on workflow_dispatch -> a pure durable gather. - - name: Assemble versioned site - id: site - uses: DiamondLightSource/myst-version-switcher-plugin/assemble@v0.5.0 - with: - repo: ${{ github.repository }} - guard-default-branch: false - artifact-version-name: ${{ inputs.version-name }} - - - name: Upload Pages artifact - uses: actions/upload-pages-artifact@v3 - with: - path: ${{ steps.site.outputs.dir }} - - - name: Deploy to GitHub Pages - id: deployment - uses: actions/deploy-pages@v4 - - # deploy-pages reports success once the deployment RECORD flips active — even - # if the Pages backend silently fails to update the SERVED origin (a wedged - # site can serve a stale artifact indefinitely; see the 2026-06 incident where - # a gh-pages→Actions cutover froze the origin while every deploy reported OK). - # Confirm the live switcher.json byte-matches the one we just assembled, polling - # for up to 60s to let the origin/CDN propagate. A mismatch fails the run, so a - # stale origin surfaces as a red check instead of silently serving old docs. - - name: Verify the deployed origin matches what we assembled - env: - PAGE_URL: ${{ steps.deployment.outputs.page_url }} - SITE_DIR: ${{ steps.site.outputs.dir }} - run: | - set -euo pipefail - [ -n "$PAGE_URL" ] || { echo "::error::deploy-pages did not report a page_url"; exit 1; } - local_json="$SITE_DIR/switcher.json" - [ -f "$local_json" ] || { echo "::error::no switcher.json at $local_json"; exit 1; } - # Command substitution strips trailing newlines from both sides, so the - # comparison is robust to a curl/file trailing-newline difference. - want=$(cat "$local_json") - url="${PAGE_URL%/}/switcher.json" - deadline=$(( $(date +%s) + 60 )) - attempt=0 - while :; do - attempt=$((attempt + 1)) - got=$(curl -fsS "$url?cb=$(date +%s%N)" || true) # cache-bust → hit the origin - if [ -n "$got" ] && [ "$got" = "$want" ]; then - echo "origin matches the assembled switcher.json (attempt $attempt)" - exit 0 - fi - if [ "$(date +%s)" -ge "$deadline" ]; then - echo "::error::live switcher.json still does not match the assembled one after 60s — Pages origin likely stale/wedged" - echo "--- assembled ($local_json) ---"; printf '%s\n' "$want" - echo "--- live ($url) ---"; printf '%s\n' "$got" - exit 1 - fi - sleep 1 - done diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 0a48b02a..4517592d 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -1,23 +1,33 @@ name: Docs CI # Build + verify the docs on every event, then publish on INTERNAL events. The -# build (`_docs.yml`) runs for PRs (including forks), pushes to main/docs, and tags, -# and uploads each build's `docs` artifact. Publishing is nested here (the `publish` -# job -> `_publish.yml`) so its status is visible on the PR/commit, but ONLY for -# internal events on this repo: a fork PR's build runs with a read-only token and -# must never deploy. +# build (the shared `docs.yml`) runs for PRs (including forks), pushes to main, and +# tags, and uploads each build's `docs` artifact. Publishing is nested here (the +# `publish` job -> the shared `publish.yml`) so its status is visible on the +# PR/commit, but ONLY for internal events on this repo: a fork PR's build runs with a +# read-only token and must never deploy. # -# `docs` is a publish trigger during the docs migration: the new MyST docs live on -# the `docs` branch (main has none yet), so `docs` is the live version. +# Both halves are consumed from DiamondLightSource/myst-version-switcher-plugin at a +# pinned tag — this repo carries no docs-build/publish workflow of its own. +# publish.yml reconstructs the whole versioned site from durable sources (main's +# latest build persisted at _sources/main.zip, each release's docs.zip asset, every +# open PR's build artifact) and deploys it to Pages directly; there is no gh-pages +# branch. on: pull_request: push: - branches: [main, docs] + branches: [main] tags: ['*'] # '*' never matches '/' jobs: docs: - uses: ./.github/workflows/_docs.yml + uses: DiamondLightSource/myst-version-switcher-plugin/.github/workflows/docs.yml@v0.7.0 + with: + # `make docs` drives npx mystmd (pinned by MYSTMD_VERSION) and reads build + # settings from CONFIG, so create it first. eval runs the compound command. + build-command: cp CONFIG.example CONFIG && make docs + # The fork-PR warning links here so a maintainer can publish a fork preview. + preview-workflow: preview-fork.yml # Tag-only: attach this build's docs.zip (bare html/ root) to the GitHub Release # so `assemble` can reconstruct that released version on future deploys. @@ -35,16 +45,18 @@ jobs: GH_TOKEN: ${{ github.token }} run: gh release upload "${{ github.ref_name }}" docs.zip --clobber --repo "${{ github.repository }}" - # Internal events only: an internal PR, or a push to main/docs/tag, has a - # same-repo build we can trust + deploy. Fork PRs (head repo != this repo) are - # excluded — _docs.yml's build job warns them instead. + # Internal events only: an internal PR, or a push to main/tag, has a same-repo + # build we can trust + deploy. Fork PRs (head repo != this repo) are excluded — + # docs.yml's build job warns them instead. publish.yml defaults guard-default-branch + # to true, so every deploy asserts main is present and can never silently drop + # /main/ (main is self-durable via _sources/main.zip). publish: needs: [docs] if: >- github.repository == 'PandABlocks/PandABlocks-server' && ( github.event_name != 'pull_request' || github.event.pull_request.head.repo.full_name == github.repository ) - uses: ./.github/workflows/_publish.yml + uses: DiamondLightSource/myst-version-switcher-plugin/.github/workflows/publish.yml@v0.7.0 with: version-name: ${{ needs.docs.outputs.version-name }} permissions: diff --git a/.github/workflows/preview-fork.yml b/.github/workflows/preview-fork.yml new file mode 100644 index 00000000..fbdad8ae --- /dev/null +++ b/.github/workflows/preview-fork.yml @@ -0,0 +1,31 @@ +name: Preview fork PR + +# Maintainer opt-in to publish a versioned docs preview for an EXTERNAL fork PR. +# Fork PRs build + verify in Docs CI but never auto-publish (their build runs with a +# read-only token), so this is the only path that deploys one. A reusable workflow +# can't be workflow_dispatch'd cross-repo, so this thin wrapper lives here and passes +# the PR number through to the shared publish.yml via workflow_call. +# +# It approves the fork PR's CURRENT head SHA (a `preview-approved` commit status) and +# assembles the site with that PR's artifact included. A later push to the PR (new +# SHA) drops the preview until you re-run this. version-name is empty: there is no +# in-run build to inject, so publish.yml does a pure durable gather. +on: + workflow_dispatch: + inputs: + pr: + description: Fork PR number to approve + preview. + required: true + +jobs: + preview: + uses: DiamondLightSource/myst-version-switcher-plugin/.github/workflows/publish.yml@v0.7.0 + with: + version-name: "" + pr: ${{ inputs.pr }} + permissions: + contents: read + actions: read + pages: write + id-token: write + statuses: write From be4ad7a3c746c7bc5aa2e0e4c34046082b06326e Mon Sep 17 00:00:00 2001 From: Tom Cobb Date: Wed, 24 Jun 2026 16:01:35 +0000 Subject: [PATCH 28/28] docs: pin favicon to the PandA logo + set logo_text The docs set no favicon, so `myst build` fetched MyST's default favicon from mystmd.org on every build; when that remote fetch flakes ("Premature close") the build fails (exit 2). Pin the favicon to the bundled PandA logo so the build is self-contained, and set logo_text to the repo name. Co-Authored-By: Claude Opus 4.8 Claude-Session: https://claude.ai/code/session_01RBtsHDzshkwFgTJtrDVveU --- docs/myst.yml | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/docs/myst.yml b/docs/myst.yml index 64d2c2b5..e24d3a39 100644 --- a/docs/myst.yml +++ b/docs/myst.yml @@ -85,4 +85,9 @@ site: # default flattened /commands. folders: true logo: images/PandA-logo-for-black-background.svg + logo_text: PandABlocks-server + # Pin the favicon to the bundled PandA logo so `myst build` never fetches the + # default favicon from mystmd.org (a flaky remote fetch that intermittently + # fails the build). + favicon: images/PandA-logo-for-black-background.svg github_url: https://github.com/PandABlocks/PandABlocks-server