perf(astrodome): reuse exact hot-path state

.env.example

@@ -27,14 +27,16 @@ ASTRO_DIRECTIONAL_QUEUE_SIZE=8
 # is benchmarked. Bot, worker, and operator CLI must use the same value (1..32).
 ASTRO_DIRECTIONAL_CONCURRENCY=1
 # Docker hard limit includes Go, CDO/ecCodes children, and charged file cache.
-# 24g is the safe initial candidate for the full footprint, CDO/ecCodes and
-# charged file cache, not a scientific constant. Benchmark before reducing it.
-ASTRO_DIRECTIONAL_WORKER_MEMORY_LIMIT=24g
+# The 32g production headroom covers an observed 18.53 GB peak without cgroup
+# pressure; a 24 GiB trial reached its hard boundary. It is not a scientific
+# constant or a measured minimum. Repeat the resource gate before reducing it.
+ASTRO_DIRECTIONAL_WORKER_MEMORY_LIMIT=32g
 ASTRO_DIRECTIONAL_WORKER_GOMEMLIMIT=12GiB
 # The production cold-run benchmark showed that four CPUs make the complete
-# dense 72-frame calculation approach three hours. Eight leaves four logical
-# CPUs on the 12-thread host for model sync and ordinary bot forecasts.
-ASTRO_DIRECTIONAL_WORKER_CPU_LIMIT=8.00
+# dense 72-frame calculation approach three hours. Ten leaves two logical
+# CPUs on the 12-thread host for model sync and ordinary bot forecasts; repeat
+# the non-regression measurements after a host or workload change.
+ASTRO_DIRECTIONAL_WORKER_CPU_LIMIT=10.00
 # Optional independent NASA GEOS-CF AOD550/ozone enrichment for the
 # Reference V-band diagnostic. Forecast delivery remains fail-open when false
 # or when NASA is temporarily unavailable.

README.md

@@ -146,13 +146,15 @@ and HTTPS access to DWD, NASA GEOS-CF, Telegram, VK, and the configured atlas so
 required; neither platform's long polling needs an inbound application port.
 
 The optional Astrodome deployment adds an isolated directional worker shared
-with Horizon. The initial, deliberately conservative candidate gives that
-worker four CPUs, a `24 GB` cgroup hard limit, and `GOMEMLIMIT=12GiB`. It also
+with Horizon. The current host-specific candidate gives that
+worker ten CPUs, a `32 GB` cgroup hard limit, and `GOMEMLIMIT=12GiB`. It also
 enforces a `400 GiB` project-footprint ceiling and a `150 GiB` free-space
 reserve before dome admission. These are rollout guardrails, not measured
-minimum requirements: do not reduce them or publish a smaller production
-profile until cold/warm full-dome, concurrent model-sync, memory-pressure, and
-ordinary-forecast latency benchmarks pass. The shared active-slot limit is
+minimum requirements. A measured full run peaked at 18.53 GB without cgroup
+pressure under this limit, whereas a 24 GiB trial reached its hard boundary;
+do not reduce the headroom or publish a smaller production profile until
+same-run cold/warm, concurrent model-sync, memory-pressure, and ordinary-forecast
+latency benchmarks pass. The shared active-slot limit is
 `ASTRO_DIRECTIONAL_CONCURRENCY=1` by default; increasing it can multiply the
 full-dome resident footprint and requires repeating those resource tests. The
 optional website is deployed from the independent `site-astrosferum`

README.ru.md

@@ -164,13 +164,15 @@ HTTPS-доступ к DWD, NASA GEOS-CF, Telegram, VK и настроенным
 порт для long polling обеих платформ не нужен.
 
 Опциональный астрокупол добавляет изолированный directional worker, общий с
-«Горизонтом». Начальный намеренно консервативный профиль выделяет ему четыре
-CPU, жёсткий cgroup-лимит `24 GB` и `GOMEMLIMIT=12GiB`; admission дополнительно
+«Горизонтом». Текущий профиль для production-хоста выделяет ему десять
+CPU, жёсткий cgroup-лимит `32 GB` и `GOMEMLIMIT=12GiB`; admission дополнительно
 соблюдает потолок footprint проекта `400 GiB` и резерв не менее `150 GiB`
 свободного места. Это защитные параметры rollout, а не измеренные минимальные
-требования. Уменьшать их или объявлять меньший production-профиль нельзя до
-успешных холодных/тёплых расчётов полного купола, одновременной синхронизации
-модели, проверки давления на память и задержки обычного прогноза. Общий лимит
+требования. Полный измерительный прогон достиг peak 18,53 GB без событий
+давления cgroup при этом лимите, тогда как проба с 24 GiB упёрлась в жёсткую
+границу. Уменьшать запас или объявлять меньший production-профиль нельзя до
+same-run cold/warm расчётов, одновременной синхронизации модели, проверки
+давления на память и задержки обычного прогноза. Общий лимит
 активных слотов по умолчанию равен `ASTRO_DIRECTIONAL_CONCURRENCY=1`; его
 увеличение может кратно увеличить resident footprint полного купола и требует
 повторить эти ресурсные тесты. Необязательный сайт

config/config.example.yaml

@@ -19,7 +19,7 @@ horizon_analysis:
   queue_size: 4
   concurrency: 1
   # Shared by Horizon extraction and Astrodome native-column preload inside
-  # the isolated directional worker. Match the default eight-CPU worker quota.
+  # the isolated directional worker. The current production quota is ten CPUs.
   cdo_workers: 8
   job_timeout: 10m
   cache_ttl: 48h

docker-compose.yml

@@ -165,8 +165,8 @@ services:
     security_opt:
       - no-new-privileges:true
     pids_limit: 128
-    mem_limit: ${ASTRO_DIRECTIONAL_WORKER_MEMORY_LIMIT:-24g}
-    cpus: "${ASTRO_DIRECTIONAL_WORKER_CPU_LIMIT:-8.00}"
+    mem_limit: ${ASTRO_DIRECTIONAL_WORKER_MEMORY_LIMIT:-32g}
+    cpus: "${ASTRO_DIRECTIONAL_WORKER_CPU_LIMIT:-10.00}"
     stop_grace_period: 2m
     healthcheck:
       test: ["CMD", "/usr/local/bin/bot_astrosferum_directional_worker", "--healthcheck", "http://127.0.0.1:18084/healthz"]

docs/architecture.en.md

@@ -1460,7 +1460,8 @@ workers. The current composition implements this boundary:
 Each extracted CDO table is normalized as its bounded subprocess finishes and the
 raw table is released; only the normalized 72-frame result remains for the
 calculation. The directional worker's Compose hard limit is configured with
-`ASTRO_DIRECTIONAL_WORKER_MEMORY_LIMIT` (`24g` by default), and its lower Go
+`ASTRO_DIRECTIONAL_WORKER_MEMORY_LIMIT` (`32g` on the measured production
+host), and its lower Go
 heap target with `ASTRO_DIRECTIONAL_WORKER_GOMEMLIMIT` (`12GiB` by default).
 Neither is a scientific constant. Any change must follow measured peak RSS
 during a full-run smoke test and leave headroom for CDO, PostgreSQL, and the
@@ -1598,7 +1599,10 @@ terrain-blocked refracted ray keeps `direction_at_model_top_ecef=null`: the
 worker does not fabricate a tangent at a model boundary that the ray never
 reached. The tangent remains mandatory for every available refracted node.
 
-Refraction v3 performs two forward production passes and accepts a ray only
+Refraction v4 (`dormand-prince-5-4-fsal-event-v4`) retains the seventh DOPRI
+stage as the canonical accepted endpoint and reuses its normalized-start
+derivative only when the next state has the identical binary64 representation.
+It performs two forward production passes and accepts a ray only
 when their endpoint, direction, and optical-path diagnostics converge within
 the versioned limits. The reverse pass is not repeated for ordinary work; it
 is enabled by the reference/strict mode used for regression and release

docs/architecture.ru.md

@@ -1477,7 +1477,8 @@ Horizon и астрокупол не должны занимать workers об
 Каждая таблица CDO нормализуется сразу после завершения bounded subprocess, после
 чего raw table освобождается; до расчёта сохраняется только нормализованный
 результат из 72 frames. Жёсткий Compose-лимит directional worker задаётся
-`ASTRO_DIRECTIONAL_WORKER_MEMORY_LIMIT` (`24g` по умолчанию), а более низкая
+`ASTRO_DIRECTIONAL_WORKER_MEMORY_LIMIT` (`32g` на измеренном production-хосте),
+а более низкая
 цель Go heap — `ASTRO_DIRECTIONAL_WORKER_GOMEMLIMIT` (`12GiB` по умолчанию).
 Ни одно значение не является научной константой. Изменение допускается только
 после измерения peak RSS в полном smoke-расчёте с запасом для CDO, PostgreSQL и
@@ -1615,7 +1616,10 @@ ECEF-систему луча в восстановленном трёхмерн
 границе модели, которой луч не достиг. Для каждого доступного преломлённого
 узла касательная остаётся обязательной.
 
-Интегратор рефракции `dormand-prince-5-4-event-v3` в production принимает луч
+Интегратор рефракции `dormand-prince-5-4-fsal-event-v4` сохраняет седьмую
+стадию DOPRI как каноническое принятое конечное состояние и повторно использует
+производную нормализованного начала только тогда, когда следующее состояние
+имеет побитово идентичное представление binary64. В production луч принимается
 только после согласования двух последовательно уточнённых прямых проходов.
 Дополнительный обратный проход и проверка обратимости выполняются только в
 reference-режиме для калибровки, регрессии и release verification. Принятый

docs/configuration.en.md

@@ -117,9 +117,9 @@ memory, CPU and ordinary-forecast latency benchmarks justify more.
 | `ASTRO_ASTRODOME_RESIDENT_LIMIT` | No | `10GiB` | Maximum projected resident footprint of preloaded native ICON-EU columns for one Astrodome job. Validation accepts `1GiB..20GiB`; keep it below the worker cgroup hard limit with headroom for Go, CDO/ecCodes, and charged file cache. |
 | `ASTRO_ASTRODOME_JOB_TIMEOUT` | No | `0s` | Astrodome wall-clock deadline. `0s` disables it so a live calculation is not cancelled by an estimate; positive values must be `1m..1h`. Explicit user cancellation and process shutdown remain effective. |
 | `ASTRO_DIRECTIONAL_INTERNAL_REQUEST_TIMEOUT` | No | `0s` | Bot-to-worker HTTP deadline. `0s` disables the transport cutoff; positive values must be `1s..8h`. It must be `0s` when the Astrodome job timeout is disabled, or exceed a finite Astrodome job timeout. |
-| `ASTRO_DIRECTIONAL_WORKER_MEMORY_LIMIT` | No | `24g` until benchmarked | Compose hard cgroup limit for Go, CDO/ecCodes children, and charged file cache. It is an initial conservative candidate, not a measured minimum. |
-| `ASTRO_DIRECTIONAL_WORKER_GOMEMLIMIT` | No | `12GiB` until benchmarked | Go heap target inside the worker hard limit. It is not a total-process or cgroup limit. |
-| `ASTRO_DIRECTIONAL_WORKER_CPU_LIMIT` | No | `8.00` on the 12-thread production host | Compose CPU quota for the isolated worker. The retained single directional slot leaves four logical CPUs for model sync and ordinary forecasts. Rebenchmark before applying this host-specific value elsewhere. |
+| `ASTRO_DIRECTIONAL_WORKER_MEMORY_LIMIT` | No | `32g` on the measured production host | Compose hard cgroup limit for Go, CDO/ecCodes children, and charged file cache. It preserves headroom above the measured 18.53 GB peak; it is not a measured minimum. |
+| `ASTRO_DIRECTIONAL_WORKER_GOMEMLIMIT` | No | `12GiB` on the measured production host | Go heap target inside the worker hard limit. It is not a total-process or cgroup limit. |
+| `ASTRO_DIRECTIONAL_WORKER_CPU_LIMIT` | No | `10.00` on the 12-thread production host | Compose CPU quota for the isolated worker. The retained single directional slot leaves two logical CPUs for model sync and ordinary forecasts. Rebenchmark before applying this host-specific value elsewhere. |
 
 Relevant YAML defaults are:
 
@@ -143,13 +143,21 @@ astrodome:
   min_free_inodes: 10000
 ```
 
-The current bounded profile measured a 300-column preload at `244.151 s` with
-four CDO workers and five production-v2 node calculations at
-`0.969..1.423 s` each. For the production-v2 geometry (129 nodes over 72
-frames) on the configured eight-CPU worker, including the existing per-frame
-scheduling bound, these samples project `22.8..33.1 min`. Production preload
-is configured for eight CDO workers, but the documentation does not infer an
-unmeasured speed-up from the worker-count change. The `30m` value is therefore
+Two complete current-writer measurements at 129 nodes over 72 frames used ten
+node workers, eight CDO workers, `GOMEMLIMIT=12GiB`, and the bounded two-frame
+prepared-state window. Run `2026081606` completed all 9,288 node-hours in
+`26m07.679s`, including a `4m41.816s` preload. The later immutable run
+`2026081612`, with a warmer model/file cache, completed in `23m33.700s`,
+including a `3m52.033s` preload; its science phase was `19m41.667s`. This is
+not a same-run cold/warm pair. One node on the later run retained a pre-existing
+fail-closed 1.842-cm quadrature-floor result; a focused replay on the
+unmodified `main` failed in the same cell, interval, and component with ratio
+1.58747 versus 1.58745, so it is not a
+prepared-state, persistent-pool, or FSAL regression. Under a 32-GiB cgroup the
+later run peaked at 18,532,704,256 bytes with no `memory.events`; the earlier
+24-GiB trial reached its hard boundary and accumulated pressure events. The
+32-GiB value is operational headroom, not a minimum requirement. The `30m`
+value is therefore
 queue/UI scheduling guidance only, not a completion guarantee or an execution
 deadline. Production uses `0s` for both the Astrodome job timeout and the
 bot-to-worker transport timeout, so a healthy calculation is not truncated by
@@ -159,8 +167,9 @@ finite guards within the validation ranges above; the transport guard must then
 exceed the job guard, and it cannot remain finite while the job guard is
 disabled. The historical v28/path-v22 baseline covered all 9,288
 node-hours in 33 min 14 s: 9,284 were available and four sub-GL2 physical
-spans failed closed. This single run does not replace repeated cold/warm,
-simultaneous-sync, payload, or observational gates.
+spans failed closed. The current measurements do not replace a same-run
+cold/warm pair, simultaneous-sync, payload, ordinary-Forecast/Horizon latency,
+or observational gates.
 Completed results are
 bounded by TTL and entry count. Admission also accounts for staging, leases,
 temporary data, the complete project disk ceiling, free bytes, and free