esops-dev · dj-wasabi · May 24, 2026 · May 24, 2026
@@ -0,0 +1,48 @@
+name: Nightly benchmarks
+
+# Micro-benchmarks gated out of the per-PR ``pytest`` run because shared
+# CI throughput fluctuates. The runner's hot-path suffix lookup is the
+# only benchmark today (see ``forecaster/tests/perf/`` and the
+# ``benchmark`` pytest marker); this workflow opts in so a regression
+# like "lru_cache silently dropped" is caught within 24 hours instead
+# of waiting for someone to run the marker locally.
+#
+# Failure here does NOT block any merge — it surfaces as a job-level
+# failure in the Actions tab. Treat it as a heads-up signal.
+
+on:
+  schedule:
+    # 03:17 UTC — picked off the hour so the run doesn't pile onto the
+    # GitHub-wide top-of-hour scheduling spike. Daily cadence keeps the
+    # signal fresh without burning CI minutes.
+    - cron: "17 3 * * *"
+  workflow_dispatch:
+
+permissions:
+  contents: read
+
+jobs:
+  bench:
+    name: Microbenchmarks (opt-in)
+    runs-on: ubuntu-latest
+    defaults:
+      run:
+        working-directory: forecaster
+    steps:
+      - uses: actions/checkout@v6
+
+      - uses: astral-sh/setup-uv@v7
+        with:
+          python-version: "3.12"
+          enable-cache: true
+          cache-dependency-glob: "forecaster/pyproject.toml"
+
+      - run: uv sync --all-extras
+
+      # ``-m benchmark`` flips the marker filter set in
+      # ``forecaster/pyproject.toml`` (which defaults to
+      # ``-m 'not benchmark'``) so only the perf suite runs. Keeping the
+      # marker name in lockstep with the test module is intentional —
+      # adding a benchmark elsewhere requires the same marker.
+      - name: pytest -m benchmark
+        run: uv run pytest -m benchmark tests/perf -v
@@ -28,7 +28,7 @@ make chart-lint       # helm lint + kubeconform
   changes and require a deprecation period.
 - `mypy --strict` and `ruff` must be clean.
 - Adding a model? Update three places: `forecaster/src/promforecast/models/`,
-  `pyproject.toml` entry points, and `docs/models.md`.
+  `pyproject.toml` entry points, and `docs/reference/models.md`.
 - Adding a Helm value? Regenerate `values.schema.json` (`make chart-schema`).
 
 ## Commit style

@@ -39,7 +39,7 @@ VM with native future timestamps, and the bundled ServiceMonitor drops
 the redundant `*_forecast` / `*_forecast_lower` / `*_forecast_upper`
 snapshot families from the scrape so dashboards plot a smooth forward
 curve rather than averaging the sink push against a 5-minute snapshot
-staircase. See [`docs/architecture/emission-paths.md`](docs/architecture/emission-paths.md)
+staircase. See [`docs/reference/emission-paths.md`](docs/reference/emission-paths.md)
 for what each path emits and the dual-source trap that motivates the
 defaults. Existing snapshot-only installs upgrading to the new defaults
 should follow [`docs/how-to/migrate-to-sink-first.md`](docs/how-to/migrate-to-sink-first.md)
@@ -49,7 +49,7 @@ Editing the YAML config does **not** require a pod restart: the
 forecaster watches the mounted ConfigMap, validates the new config
 in-place, and rolls back on validation failure (`kubectl apply` to the
 ConfigMap is enough). `POST /-/reload` and `SIGHUP` trigger the same
-path. See [`docs/cli.md`](docs/cli.md) for `promforecast validate`,
+path. See [`docs/reference/cli.md`](docs/reference/cli.md) for `promforecast validate`,
 which CI users can run against a config file before merging.
 
 See [`docs/`](docs/) for configuration, model selection, and operational

@@ -42,7 +42,7 @@ on demand:
 helm test <release>
 ```
 
-See [`docs/architecture/emission-paths.md`](../../docs/architecture/emission-paths.md)
+See [`docs/reference/emission-paths.md`](../../docs/reference/emission-paths.md)
 for what the two paths emit, why running both unguarded is a silent
 trap, and the `metric_relabel_configs` snippet to paste into an
 externally-managed Prometheus.

@@ -5,7 +5,7 @@
     up automatically. VictoriaMetrics is registered as the default so the
     bundled dashboards land on the sink-fed copy of the forecast metrics
     (smooth curves) rather than the scraped snapshot (5-min staircase).
-    See ``docs/architecture/emission-paths.md`` for why the default matters.
+    See ``docs/reference/emission-paths.md`` for why the default matters.
 */}}
 {{- $vm := .Values.datasources.victoriaMetrics }}
 {{- $prom := .Values.datasources.prometheus }}

@@ -110,7 +110,7 @@ spec:
               echo "Remediation: promforecast.serviceMonitor.enabled=true (the umbrella default)" >&2
               echo "             so the standalone chart drops the snapshot families. For an" >&2
               echo "             externally-managed Prometheus, add the metric_relabel_configs" >&2
-              echo "             snippet from docs/architecture/emission-paths.md." >&2
+              echo "             snippet from docs/reference/emission-paths.md." >&2
               echo "$dup_out" >&2
               exit 1
             fi

@@ -64,7 +64,7 @@ promforecast:
   # fail with ``no matches for kind ServiceMonitor`` on a cluster without
   # the CRD. For installs that scrape with a plain Prometheus (no
   # operator), flip this off and paste the ``metric_relabel_configs``
-  # snippet from ``docs/architecture/emission-paths.md`` into your scrape
+  # snippet from ``docs/reference/emission-paths.md`` into your scrape
   # config — the snapshot families must be dropped somewhere or the
   # dual-source trap reappears.
   serviceMonitor:

@@ -74,7 +74,7 @@ release-name-agnostically:
 > forecaster pod's `instance` need to be updated to the new stable
 > values before upgrading.
 
-See [`docs/architecture/emission-paths.md`](../../docs/architecture/emission-paths.md)
+See [`docs/reference/emission-paths.md`](../../docs/reference/emission-paths.md)
 for what each emission path emits and the dual-source trap the drop
 prevents.
 

@@ -15,7 +15,7 @@
 */}}
 {{- $explicit := .Values.serviceMonitor.dropForecastSnapshots }}
 {{- if and .Values.existingConfigMap (not (kindIs "bool" $explicit)) }}
-{{- fail "serviceMonitor.dropForecastSnapshots must be set explicitly (true/false) when existingConfigMap is set — the chart cannot read the sink toggle from an externally-managed ConfigMap. See docs/architecture/emission-paths.md for the dual-source labelset trap." }}
+{{- fail "serviceMonitor.dropForecastSnapshots must be set explicitly (true/false) when existingConfigMap is set — the chart cannot read the sink toggle from an externally-managed ConfigMap. See docs/reference/emission-paths.md for the dual-source labelset trap." }}
 {{- end }}
 {{- $sinkOn := dig "sink" "remote_write" "enabled" false .Values.config }}
 {{- $drop := ternary $explicit $sinkOn (kindIs "bool" $explicit) }}
@@ -56,7 +56,7 @@ spec:
       # making the example alerts in ``examples/alerts/promforecast-rules.yaml``
       # (which interpolate ``{{`{{ $labels.instance }}`}}`` to identify the
       # node-exporter target) report the wrong host. Matches the
-      # ``honor_labels: true`` snippet in ``docs/architecture/emission-paths.md``.
+      # ``honor_labels: true`` snippet in ``docs/reference/emission-paths.md``.
       honorLabels: true
       interval: {{ .Values.serviceMonitor.interval }}
       scrapeTimeout: {{ .Values.serviceMonitor.scrapeTimeout }}

@@ -27,7 +27,7 @@ config:
   # Schema version. The forecaster routes on this string and warns when
   # it is missing. ``promforecast.io/v1`` is the stable schema;
   # ``promforecast.io/v1alpha1`` still loads (with a DeprecationWarning)
-  # for configs predating the stable cut-over. See docs/config-schema.md
+  # for configs predating the stable cut-over. See docs/reference/config-schema.md
   # for the deprecation policy.
   apiVersion: promforecast.io/v1
 

@@ -62,7 +62,7 @@ scrape_configs:
     # Operational metrics (``forecast_failures_total``, ``forecast_quality_score``,
     # ``forecast_run_duration_seconds``, etc.) stay scraped — they only
     # exist on /metrics regardless of sink configuration. See
-    # docs/architecture/emission-paths.md for the dual-source trap and the
+    # docs/reference/emission-paths.md for the dual-source trap and the
     # cross-metric-join labelset asymmetry this drop preserves.
     metric_relabel_configs:
       - source_labels: [__name__]

@@ -1,38 +1,68 @@
 # Documentation
 
-The docs follow the [Diataxis](https://diataxis.fr/) framework:
+Organised by pipeline position — input → model → output — then operations,
+guides, and reference around the edges. If you're not sure where a topic
+lives, the section descriptions below should point you in the right direction.
 
-- **Tutorials** — learning-oriented, end-to-end walkthroughs.
-- **How-tos** — task-oriented, "how do I X?" recipes.
-- **Reference** — information-oriented, exhaustive details.
-- **Explanation** — understanding-oriented, design and trade-offs.
+## Reference
 
-## Index
+Information-oriented, exhaustive details. Look here when you know the name
+of the thing and want its full specification.
+
+- [Config schema](reference/config-schema.md) — `apiVersion`, stability guarantees, and the `v1alpha1` → `v1` migration
+- [CLI](reference/cli.md) — `promforecast run`, `promforecast validate`, `promforecast diagnose`
+- [Models](reference/models.md) — built-in forecasters, auto-select, accuracy metrics
+- [Emission paths (snapshot vs. sink + VM)](reference/emission-paths.md) — what each path emits, the dual-source trap, and why the official charts default to sink + VM
+
+## Inputs
+
+Everything that shapes the data flowing into the forecasting model.
+
+- [Preprocessing](inputs/preprocessing.md) — gap imputation and outlier scrubbing
+- [Change-points](inputs/change-points.md) — input-side break detection and lookback adaptation
+- [Seasonality](inputs/seasonality.md) — periodicity detection
+- [Cold-start](inputs/cold-start.md) — sparse-history behaviour and peer-sibling borrowing
+- [Regressors](inputs/regressors.md) — calendar, custom PromQL, and holiday regressors, plus conditional forecasts (baseline given the current regressor state)
+- [Series discovery](inputs/discovery.md) — template-driven auto-fan-out over label values
+
+## Outputs
+
+Everything the forecaster emits beyond raw forecast curves — derived
+metrics, calibration, explainability, and ad-hoc what-if endpoints.
+
+- [Capacity planning](outputs/capacity.md) — damped trend, time-to-threshold ETAs, growth rate, will-breach predictive boolean, queue burn-down (drain) forecasting
+- [SLO error-budget forecasting](outputs/slo.md) — burn rate, exhaustion projection, multi-window alerting
+- [Maintenance window recommendations](outputs/maintenance-windows.md) — top-N lowest-load windows for scheduling reboots, rollouts, and cert rotations
+- [Conformal calibration](outputs/conformal.md) — empirical-coverage band correction
+- [Explainability (decomposition)](outputs/explainability.md) — trend / seasonal / level / residual split of the forecast curve
+- [Forecast drift](outputs/drift.md) — prediction-vs-prediction shift telemetry
+- [Forecast preview](outputs/preview.md) — opt-in ad-hoc what-if endpoint
+- [Canary forecasts](outputs/canary.md)
+
+## Operations
+
+Running the forecaster in production — diagnosis, capacity safeguards,
+scheduling, and serving telemetry.
 
-- [Emission paths (snapshot vs. sink + VM)](architecture/emission-paths.md) — what each path emits, the dual-source trap, and why the official charts default to sink + VM
-- [Config schema](config-schema.md) — `apiVersion`, stability guarantees, and the `v1alpha1` → `v1` migration
-- [Models](models.md) — built-in forecasters, auto-select, accuracy metrics
-- [CLI](cli.md) — `promforecast run`, `promforecast validate`, `promforecast diagnose`
 - [Reload verification](operations/reload.md) — confirming a config reload actually picked up the expected queries
 - [Duplicate-labelset diagnosis](operations/diagnose.md) — detecting and remediating the scrape+sink dual-source state
 - [Cardinality budgeting](operations/cardinality.md) — per-family multipliers and the `--strict-cardinality` CI gate
-- [High availability](ha.md) — Lease-based leader election, Redis snapshot cache
-- [Query cache](cache.md) — in-memory LRU + optional Redis dedup for overlapping PromQL
-- [Series discovery](discovery.md) — template-driven auto-fan-out over label values
-- [Preprocessing](preprocessing.md) — gap imputation and outlier scrubbing
-- [Change-points](change-points.md) — input-side break detection and lookback adaptation
-- [Conformal calibration](conformal.md) — empirical-coverage band correction
-- [Capacity planning](capacity.md) — damped trend, time-to-threshold ETAs, growth rate, will-breach predictive boolean, queue burn-down (drain) forecasting
-- [SLO error-budget forecasting](slo.md) — burn rate, exhaustion projection, multi-window alerting
-- [Maintenance window recommendations](operations/maintenance-windows.md) — top-N lowest-load windows for scheduling reboots, rollouts, and cert rotations
-- [Conditional forecasts](conditional-forecasts.md) — baseline given the current regressor state, for "unusual *for current conditions*" alerts
-- [Explainability (decomposition)](explainability.md) — trend / seasonal / level / residual split of the forecast curve
-- [Forecast drift](drift.md) — prediction-vs-prediction shift telemetry
-- [Reading the numbers](reading-the-numbers.md) — how to interpret quality and accuracy in practice (mean vs p95, horizon paradox, "wait three cycles", when to actually worry)
-- [Scheduling](scheduling.md) — per-query refresh intervals, max-concurrent-fits cap
-- [Forecast preview](preview.md) — opt-in ad-hoc what-if endpoint
-- [Regressors](regressors.md) — calendar, custom PromQL, and holiday regressors
-- [Development](development.md) — local toolchain, CI overview, adding models
+- [Scheduling](operations/scheduling.md) — per-query refresh intervals, max-concurrent-fits cap
+- [High availability](operations/ha.md) — Lease-based leader election, Redis snapshot cache
+- [Query cache](operations/cache.md) — in-memory LRU + optional Redis dedup for overlapping PromQL
+- [Telemetry](operations/telemetry.md) — internal metrics catalogue
+
+## Guides
+
+Cross-cutting explanations that don't fit cleanly under one feature.
+
+- [Reading the numbers](guides/reading-the-numbers.md) — how to interpret quality and accuracy in practice (mean vs p95, horizon paradox, "wait three cycles", when to actually worry)
+
+## Contributing
+
+- [Development](contributing/development.md) — local toolchain, CI overview, adding models
+
+---
 
 For a copy-and-edit reference deployment (multi-group config, HA values,
 ArgoCD/Flux manifests, architecture diagram), see

@@ -56,7 +56,7 @@ Three files must stay in sync:
 
 1. `forecaster/src/promforecast/models/<name>.py` (model wrapper).
 2. `forecaster/pyproject.toml` (entry-point registration).
-3. `docs/models.md` (user docs).
+3. `docs/reference/models.md` (user docs).
 
 CI automatically tests the full registered model set.
 

@@ -25,7 +25,7 @@ There is **no universal “good MAPE”** — it depends heavily on the signal.
 | Bursty                | `50–100 %`         | Disk I/O with occasional spikes |
 | Sparse / near-zero    | `100 % +` (or NaN) | Rare error rates |
 
-For sparse/near-zero series, MAPE becomes unreliable. Set `data_profile: intermittent` on the query to switch to MASE scoring (see [models.md](models.md)).
+For sparse/near-zero series, MAPE becomes unreliable. Set `data_profile: intermittent` on the query to switch to MASE scoring (see [`../reference/models.md`](../reference/models.md)).
 
 ## Mean quality vs. p95 MAPE — why they can disagree
 
@@ -85,7 +85,7 @@ For per-series drill-down (quality, drift, deviation, durations, errors) switch
 
 ## See also
 
-- [telemetry.md](telemetry.md) — full list of emitted metrics and labels  
-- [conformal.md](conformal.md) — how bands are empirically calibrated  
-- [drift.md](drift.md) — `forecast_drift_score` semantics  
-- [scheduling.md](scheduling.md) — how `refresh_interval` and per-query overrides work
+- [`../operations/telemetry.md`](../operations/telemetry.md) — full list of emitted metrics and labels  
+- [`../outputs/conformal.md`](../outputs/conformal.md) — how bands are empirically calibrated  
+- [`../outputs/drift.md`](../outputs/drift.md) — `forecast_drift_score` semantics  
+- [`../operations/scheduling.md`](../operations/scheduling.md) — how `refresh_interval` and per-query overrides work
@@ -51,5 +51,5 @@ Dashboard tip: stack the static `defaults.lookback` against `forecast_input_effe
 
 ## See also
 
-- [Models](models.md) — how the adjusted lookback feeds the fit.
+- [Models](../reference/models.md) — how the adjusted lookback feeds the fit.
 - [Input preprocessing](preprocessing.md) — outlier scrubbing runs *before* change-point detection so a single spike cannot masquerade as a regime shift.
@@ -82,5 +82,5 @@ With `outliers.enabled: true, method: hampel, sensitivity: medium` the spike is
 
 ## Interaction with conformal calibration
 
-Outlier scrubbing runs **before** conformal calibration (see [`conformal.md`](conformal.md)).  
+Outlier scrubbing runs **before** conformal calibration (see [`../outputs/conformal.md`](../outputs/conformal.md)).  
 A clean hold-out window produces tighter, more honest quantile bands.