WIP(logging): fail-closed application-log write guard, UNVERIFIED (BACKLOG #122)

*** THIS IS A RESCUE COMMIT, NOT A FINISHED LANE. DO NOT MERGE AS-IS. ***

The building agent was cut off by usage-pool exhaustion mid-lane. This commit
exists only to give the work a SHA: 13 files were uncommitted, including
messagefoundry/logging_guard.py, tests/test_log_write_guard.py and a NEW ADR
(0162), none of which existed anywhere else. The harness at-risk capture does NOT
cover this worktree -- it scans .claude/worktrees/ only, and new.ps1 lanes are
siblings at MessageFoundry-<name>.

*** ADR 0162 CARRIES AN ALLOCATED NUMBER I DID NOT VERIFY. *** Whoever resumes
MUST confirm it came from scripts/coord/alloc.ps1 and was not chosen by grepping.
A grepped ADR number merges clean and silently corrupts the ledger; it has fired
three times in this repo.

NOT VERIFIED: no full suite, no baseline diff, no red-first evidence. --no-verify
used deliberately and recorded rather than hidden; the alternative was losing
untracked files outright.

Scope was #122 on the owner's principle "we never want to process stuff if the
processing cannot be logged" -- count-and-log applied to the application log.
TWO STAGES, and they must not be collapsed: roll to a fresh file and RECOVER
first; stop the connection ONLY if the replacement also cannot be written. A
single-stage stop turns a transient lock into an outage.

Author: wshallwshall

docs/PHI.md

@@ -99,7 +99,7 @@ destruction) are documented in [§3](#3-encryption-at-rest) under the matching h
 | `[backup].destination/mefor-backup-*.mfbak` (ADR 0049 DR backup) | **SQLite only** carries bodies | **SQLite: Yes** — a consistent store snapshot (full inbound + outbound bodies) + the config bundle. **SQL Server / Postgres: No** — config bundle only | **Yes** — `.mfbak` chunked-AEAD codec under the **store DEK** (`resolve_active_key`); an identity-cipher (no-key) box is **refused** unless `[backup].allow_unencrypted` writes a `.mfbak.plain` | **PL-1** (SQLite) / **PL-4** (server backends) | On a **server-DB store `snapshot_to` raises `DbaDelegatedError`**, so the BackupRunner writes a **config-only** archive — or skips entirely when `[backup].config_only_on_server_db = false`. There is therefore **no `.mfbak` containing message bodies on SQL Server or Postgres**; the DB-tier backup there is `BACKUP DATABASE` / Always On / `pg_dump` / PITR, infra-owned. Where bodies *are* present it is a second at-rest PHI copy, bounded by keep-N retention; like `uploads_dir` it is **not** re-encrypted by `rotate-key`. The share's own ACLs are infra-owned | ``keep-N `[backup].retention_keep` `` |
 | `mefor-backup-*` / `mefor-tar-*` / `mefor-verify-*` staging dirs (OS temp dir, ADR 0049) | SQLite carries bodies; server backends config-only | **Yes** — a full store snapshot, and on verify a **decrypted** archive | **No** — the snapshot keeps the store's own column cipher, but the staging tar and the verify extraction are **plaintext on disk**; no engine ACL (`_secure_file` is never called on these paths) | **PL-1** | `run_backup` snapshots the store to `<tmp>/store.db` and tars it **plaintext** before sealing it into the `.mfbak` (`pipeline/dr_backup.py`), and `[backup].verify_after_backup` (**default `true`**) decrypts the archive straight back out to a second temp dir on **every** run — independent of `full_restore_verify`. Transient (the `TemporaryDirectory` unlinks on exit) but **not** on a crash or `SIGKILL`. Lives under `%TEMP%` / `TMPDIR`, **not** the ACL'd data dir: cover the temp volume with FDE and point `TMP`/`TMPDIR` at an owner-only path ([§10](#10-secure-deployment--operations-checklist)) | `UNBOUNDED — honest gap` |
 | File-connector output / spill dirs (`.hl7`, `.processed`, `.error`) | all | **Yes** — plaintext on disk | **No** — no cipher at all on this path | **PL-1** | Written by the File transport; treat the directory as PHI and cover it with volume/share encryption + an ACL | `UNBOUNDED — honest gap` |
-| Application log files (`[logging].log_dir`; under NSSM, `<DataDir>\logs\service.out.log` and `service.err.log`) | all (filesystem, not the DB) | **Possibly** — redaction is best-effort; a single-token identifier can survive it | **No** — plaintext on disk, no app-level cipher | **PL-1** | The engine installs no file handler; NSSM captures stdout/stderr. The defence is the three handler filters + `safe_exc()`/`safe_text()` + the never-log-bodies rule ([§7](#7-logging--phi-redaction) row 1), and the residual is stated there. The directory ACL is the NSSM installer's **best-effort** `icacls /inheritance:r`; age deletion is `[retention].app_log_days` (files by **mtime** — content is never read, so nothing selective happens here) and optional in-place gzip is `[retention].app_log_compress_days` (the compressor **does** read a file's bytes to archive + integrity-verify them, but only in-process — nothing is logged, and the archive stays inside the same ACL'd directory at the source's mtime). A support bundle copies a 500-line tail of this file out of the ACL'd directory entirely ([§7](#7-logging--phi-redaction)). Cover the volume with FDE ([§10](#10-secure-deployment--operations-checklist)) | `` `[retention].app_log_days` `` |
+| Application log files (`[logging].log_dir`; under NSSM, `<DataDir>\logs\service.out.log` and `service.err.log`) | all (filesystem, not the DB) | **Possibly** — redaction is best-effort; a single-token identifier can survive it | **No** — plaintext on disk, no app-level cipher | **PL-1** | NSSM captures stdout/stderr; the engine writes a log file of its own only when the opt-in `[logging].file` is set (#122, [ADR 0162](adr/0162-fail-closed-application-log-write-guard-detect-roll-and-stop.md) — same three handler filters, engine-owned rotation, refused inside `log_dir`), together with the `*.broken-*` files a write failure rolls aside, which sit outside `log_dir` and are therefore NOT swept by `[retention].app_log_days`. The defence is the three handler filters + `safe_exc()`/`safe_text()` + the never-log-bodies rule ([§7](#7-logging--phi-redaction) row 1), and the residual is stated there. The directory ACL is the NSSM installer's **best-effort** `icacls /inheritance:r`; age deletion is `[retention].app_log_days` (files by **mtime** — content is never read, so nothing selective happens here) and optional in-place gzip is `[retention].app_log_compress_days` (the compressor **does** read a file's bytes to archive + integrity-verify them, but only in-process — nothing is logged, and the archive stays inside the same ACL'd directory at the source's mtime). A support bundle copies a 500-line tail of this file out of the ACL'd directory entirely ([§7](#7-logging--phi-redaction)). Cover the volume with FDE ([§10](#10-secure-deployment--operations-checklist)) | `` `[retention].app_log_days` `` |
 | `messages.summary` | all three | **Yes** — MRN / patient name / order | **Yes, when a key is set** — store cipher; AAD `("messages","summary",id)`; store DEK (EF-3) | **PL-2** | Ingest-derived; no SQL search or index exists on it, so encrypting it costs nothing. NULL/blank stay as-is | ``rides `[security].delete_message_bodies_after_days` `` |
 | `messages.metadata` | all three | **Yes** — operator/handler-attached values | **Yes, when a key is set** — store cipher; AAD `("messages","metadata",id)`; store DEK (EF-3) | **PL-2** | **Nulled by `purge_message_bodies` on the `[retention].messages_days` window, in the same statement as the body** (ASVS 14.2.7) — see [§8](#8-retention--purge) | ``rides `[security].delete_message_bodies_after_days` `` |
 | `messages.error` | all three | **Possibly** — may embed raw fragments from exceptions | **Yes, when a key is set** — store cipher; AAD `("messages","error",id)`; store DEK (WP-5) | **PL-2** | Also `safe_exc()`-redacted **before** write. NULL/blank values stay as-is | ``rides `[security].delete_message_bodies_after_days` `` |
@@ -962,7 +962,7 @@ with materially different PHI profiles, so they get their own rows; stream 4 is
 
 | Stream | Events logged | Format | Where stored | How used | Access control | Retention | PHI / sensitive free text + redaction |
 |---|---|---|---|---|---|---|---|
-| **1. General application log** | operational events, worker/connection lifecycle, exception **types**, warnings, every alert that the `LoggingAlertSink` fallback implements when no `[alerts]` transport is configured (see row 13 for the two it does not) | single-line text (`[logging].format = "text"`, the default) or one JSON object per line (`"json"`); UTC `Z` timestamps in both | stdout only — the engine installs **no file handler**; under NSSM the supervisor captures stdout/stderr to `<DataDir>\logs\service.out.log` / `service.err.log` | day-to-day operations, incident triage, and the source of the support-bundle tail in row 14 | at rest: the NSSM installer creates `<DataDir>\logs` and locks the whole DataDir with `icacls /inheritance:r` to SYSTEM + Administrators + the service account (best-effort — a failure warns, never aborts). Over the API: `GET /logs/tail` requires the dedicated **`logs:view`** permission **and** `require_phi_read`, and every served page writes a `logs_view` audit row (line **count** only, never content) | NSSM rotates by **size** (`AppRotateBytes` 10 MB) and never deletes by age; age deletion is `[retention].app_log_days` over `[logging].log_dir` (`.log`/`.txt`, by mtime, **content never read**), optionally preceded by in-place gzip on `[retention].app_log_compress_days` (integrity-validated before the original is removed; the archive keeps the source's mtime, so the same delete window ages it out). Both default 0 = keep forever, uncompressed | **Can contain PHI.** The engine's own permission catalog classifies this as a PHI read surface (`logs:view`: "best-effort redaction, residual single-token PHI possible"). Defence: never-log-bodies rule, `safe_exc()` at the source, the three handler filters, python-hl7 loggers silenced. **Residual:** a single-token identifier is not scrubbed |
+| **1. General application log** | operational events, worker/connection lifecycle, exception **types**, warnings, every alert that the `LoggingAlertSink` fallback implements when no `[alerts]` transport is configured (see row 13 for the two it does not) | single-line text (`[logging].format = "text"`, the default) or one JSON object per line (`"json"`); UTC `Z` timestamps in both | stdout by default — under NSSM the supervisor captures stdout/stderr to `<DataDir>\logs\service.out.log` / `service.err.log`; **plus** the opt-in engine-owned `[logging].file` when configured (#122, ADR 0162), which carries the identical filter chain and whose write failures roll it aside and, on a second failure, stop this process's connections | day-to-day operations, incident triage, and the source of the support-bundle tail in row 14 | at rest: the NSSM installer creates `<DataDir>\logs` and locks the whole DataDir with `icacls /inheritance:r` to SYSTEM + Administrators + the service account (best-effort — a failure warns, never aborts). Over the API: `GET /logs/tail` requires the dedicated **`logs:view`** permission **and** `require_phi_read`, and every served page writes a `logs_view` audit row (line **count** only, never content) | NSSM rotates by **size** (`AppRotateBytes` 10 MB) and never deletes by age; age deletion is `[retention].app_log_days` over `[logging].log_dir` (`.log`/`.txt`, by mtime, **content never read**), optionally preceded by in-place gzip on `[retention].app_log_compress_days` (integrity-validated before the original is removed; the archive keeps the source's mtime, so the same delete window ages it out). Both default 0 = keep forever, uncompressed | **Can contain PHI.** The engine's own permission catalog classifies this as a PHI read surface (`logs:view`: "best-effort redaction, residual single-token PHI possible"). Defence: never-log-bodies rule, `safe_exc()` at the source, the three handler filters, python-hl7 loggers silenced. **Residual:** a single-token identifier is not scrubbed |
 | **2. `uvicorn` request/access log** (sub-stream of 1) | one line per HTTP request — method, **full request line including the query string**, status, timing | inherits stream 1's format | inherits stream 1's sink | request tracing, latency and error triage | inherits stream 1's | inherits stream 1's | **Can contain PHI.** `configure_logging` clears uvicorn's own handlers and propagates to the root, so the three filters apply; `serve` passes `log_config=None` and never disables `access_log`, so at the default `INFO` level every request is logged. OIDC `code`/`state` **are** scrubbed. **Not** scrubbed: PHI-shaped search needles on GET routes (`?content=…`, `?field_value=…`) — the single-token residual above |
 | **3. `messagefoundry.audit` off-box tee** (sub-stream of 1) | one JSON object per **committed** `audit_log` row: `event`/`ts`/`action`/`actor`/`channel_id`/`client`/`detail` | JSON | emitted after the row is durably committed and **outside** the store write lock; rides stream 1's handlers | shipping audit evidence to a SIEM so it survives a host compromise | inherits stream 1's | inherits stream 1's | `detail` is passed through the `safe_text` PHI chokepoint **before** it leaves the process; `client` is forwarded verbatim as a discrete field so a SIEM can index it. Best-effort: a logging failure is caught, never raised into the audit write. **Pinned to `INFO`** — it is emitted even at `[logging].level = WARNING` |
 | **4. Off-box syslog/SIEM forwarder** — the shared **transport** for 1–3 | a copy of every record from 1–3 | `forward_format`, default **JSON** (independent of the stdout format) | the operator's collector (`forward_host`/`_port`) | off-box evidence retention / SIEM correlation | **default-on when a collector is named.** Transport: `udp` (default) / `tcp` / **`tls`** (RFC 5425, CA-anchored, verified by default). `serve` gates the hop on the shared posture gradient before the handler is installed: verified TLS ungated; otherwise loopback / attested / synthetic ALLOW, non-enforcing PHI WARN, **enforcing PHI REFUSE (exit 2)** | the collector's, not the engine's | the identical three filters are installed on this handler, so the forwarded copy is PHI-redacted — but it still carries usernames, connection names, message ids, client addresses and the audit chain. That is the engine's own stated reason for gating the hop |