docs: fix dead heading anchors and fail the docs build on warnings

.github/workflows/docs.yaml

@@ -21,10 +21,13 @@ on:
   workflow_dispatch:
   push:
     branches: [ "main" ]
-    paths:
+    paths: &docs-paths
       - .asf.yaml
       - .github/workflows/docs.yaml
       - docs/**
+  # Build pull requests too, so a docs warning fails the PR instead of reaching main.
+  pull_request:
+    paths: *docs-paths
 
 name: Deploy DataFusion Ballista site
 
@@ -37,6 +40,7 @@ jobs:
         uses: actions/checkout@v7.0.1
 
       - name: Checkout asf-site branch
+        if: github.event_name != 'pull_request'
         uses: actions/checkout@v7.0.1
         with:
           ref: asf-site
@@ -55,13 +59,19 @@ jobs:
           pip install -r docs/requirements.txt
 
       - name: Build docs
+        env:
+          # Pull requests fail on warnings; main publishes regardless, so one that
+          # slips through (e.g. two PRs that are only broken together) can't block
+          # the site from updating.
+          SPHINXOPTS: ${{ github.event_name == 'pull_request' && '-W --keep-going' || '' }}
         run: |
           set -x
           source venv/bin/activate
           cd docs
           ./build.sh
 
       - name: Copy & push the generated HTML
+        if: github.event_name != 'pull_request'
         run: |
           set -x
           cd asf-site/

docs/README.md

@@ -48,6 +48,9 @@ inside a Python virtualenv.
 ./build.sh
 ```
 
+The build fails on any Sphinx warning, such as a broken cross-reference, and CI runs the same
+check on pull requests that touch the docs.
+
 ## Release
 
 The documentation is published from the `asf-site` branch of this repository.

docs/build.sh

@@ -18,4 +18,5 @@
 # under the License.
 
 rm -rf build
-make html
+# Fail on warnings unless the caller sets SPHINXOPTS (an empty value disables it).
+make html SPHINXOPTS="${SPHINXOPTS--W --keep-going}"

docs/source/conf.py

@@ -59,6 +59,9 @@
     '.md': 'markdown',
 }
 
+# Emit heading ids so file.md#heading links resolve.
+myst_heading_anchors = 4
+
 # Add any paths that contain templates here, relative to this directory.
 templates_path = ['_templates']
 

docs/source/contributors-guide/code-organization.md

@@ -17,36 +17,36 @@
   under the License.
 -->
 
-## Ballista Code Organization
+# Ballista Code Organization
 
 This section provides links to the source code for major areas of functionality.
 
-### ballista-core crate
+## ballista-core crate
 
 - [Crate Source](https://github.com/apache/datafusion-ballista/tree/main/ballista/core)
 - [Protocol Buffer Definition](https://github.com/apache/datafusion-ballista/blob/main/ballista/core/proto/ballista.proto)
 - [Execution Plans](https://github.com/apache/datafusion-ballista/tree/main/ballista/core/src/execution_plans)
 - [Ballista Client](https://github.com/apache/datafusion-ballista/blob/main/ballista/core/src/client.rs)
 
-### ballista-scheduler crate
+## ballista-scheduler crate
 
 - [Crate Source](https://github.com/apache/datafusion-ballista/tree/main/ballista/scheduler)
 - [Distributed Query Planner](https://github.com/apache/datafusion-ballista/blob/main/ballista/scheduler/src/planner.rs)
 - [gRPC Service](https://github.com/apache/datafusion-ballista/blob/main/ballista/scheduler/src/scheduler_server/grpc.rs)
 - [REST API](https://github.com/apache/datafusion-ballista/tree/main/ballista/scheduler/src/api)
 - [Prometheus Integration](https://github.com/apache/datafusion-ballista/blob/main/ballista/scheduler/src/metrics/prometheus.rs)
 
-### ballista-executor crate
+## ballista-executor crate
 
 - [Crate Source](https://github.com/apache/datafusion-ballista/tree/main/ballista/executor)
 - [Flight Service](https://github.com/apache/datafusion-ballista/blob/main/ballista/executor/src/flight_service.rs)
 - [Executor Server](https://github.com/apache/datafusion-ballista/blob/main/ballista/executor/src/executor_server.rs)
 
-### ballista crate
+## ballista crate
 
 - [Crate Source](https://github.com/apache/datafusion-ballista/tree/main/ballista/client)
 - [Context Extensions](https://github.com/apache/datafusion-ballista/blob/main/ballista/client/src/extension.rs)
 
-### PyBallista
+## PyBallista
 
 - [Source](https://github.com/apache/datafusion-ballista/tree/main/python)

docs/source/index.rst

@@ -15,9 +15,9 @@
 .. specific language governing permissions and limitations
 .. under the License.
 
-=====================
+==========================
 Apache DataFusion Ballista
-=====================
+==========================
 
 Table of content
 ================

docs/source/user-guide/introduction.md

@@ -24,7 +24,7 @@ Ballista is a distributed compute platform primarily implemented in Rust, and po
 Ballista has both scheduler and an executor component processes that are standard Rust executables.
 
 Dockerfiles are also provided to build images for use in containerized environments, such as Docker, Docker Compose,
-and Kubernetes. See the [deployment guide](deployment/) for more information.
+and Kubernetes. See the [deployment guide](deployment/index.rst) for more information.
 
 SQL and DataFrame queries can be submitted from Python and Rust, and SQL queries can also be submitted using the
 Ballista CLI.