Skip to content

Latest commit

 

History

History
1387 lines (1288 loc) · 113 KB

File metadata and controls

1387 lines (1288 loc) · 113 KB

Changelog

All notable changes to MessageFoundry are documented here. The format follows Keep a Changelog; versions follow Semantic Versioning.

Added

  • A startup preflight that reads the store principal's effective privileges, so the least-privilege grant the runbooks prescribe stops being a claim the engine cannot check. DEPLOY-SERVER-DB.md told operators exactly which grant the engine's database login needs, and the engine had no way to see what it had actually been given: no fixed-server-role probe and no database-role probe existed anywhere, and [store].require_managed_identity constrains the credential's kind rather than its privilege — a sysadmin gMSA satisfies it clean. On a first deployment an over-granted store principal would therefore have gone unobserved. serve now reads fixed server-role and database-role membership plus CONTROL SERVER / database CONTROL on SQL Server, and role attributes (SUPERUSER, CREATEROLE, CREATEDB, REPLICATION, BYPASSRLS), assumable predefined roles and database ownership on PostgreSQL — before any listener binds. The PostgreSQL attributes are read across every role the principal may assume, not only its own row: attributes are never inherited, but a member may SET ROLE to the holder and exercise them, so a wrapper role carrying CREATEROLE is named (CREATEROLE via role site_ops) instead of reading clean. It observes and warns; it does not refuse by default — refusing on an over-grant could block a legitimate deployment mid-setup, and the engine does not own the grant. Every start logs what it saw, writes a store_privilege_preflight audit row, and names each excess grant in security_loosenings() and GET /security/posture. Set [store].require_least_privilege = true to turn the warning into a refusal (refuse/warn splits on [security].enforcement, exactly like require_managed_identity). It does not fail open, and that is the part to know before reading its output. A probe that cannot run — permission denied, a driver error, a store handle with no probe — reports unobservable, which is a different result from "observed, and it is fine" in the log line, in the audit row and in the posture response, and which a declared require_least_privilege also refuses. SQLite reports not_applicable and says why: a local file has no server principal, and the control there is the filesystem ACL. The PostgreSQL least-privilege grant is now documented (DEPLOY-SERVER-DB.md §1.2), which it previously was not. (BACKLOG #1008)

Removed

  • BREAKING: [security].handles_real_patient_data is gone, and with it the whole data-class axis. Every instance carries patient data; the PHI gates apply unconditionally. Setting the key — or its pre-ADR-0118 spelling [ai].data_class — now refuses at load with a message naming the switch to reach for instead. Removed with it: the DataClass enum, HopPosture.is_phi, the data_class and synthetic_relaxation fields on SecurityPosture, and data_class on AiPolicy. derived_posture() / require_posture() return the production tier alone. Why, in one line: it turned off nineteen start-up gates on one line, and it was not the audited opt-out the documentation claimed. security_loosenings() never named it, so the serve-time loosening warning — the thing that fires for every other deviation — did not fire for the widest relaxation the product shipped. The completeness test that should have caught that exempted the field with a reason that was false, in an exemption branch that could never execute. What to use instead: the gate you actually mean. Each is separately named, separately audited and separately reported — allow_unencrypted_phi (plus allow_unencrypted_phi_under_strict_enforcement under the shipped enforcement = enforce), block_unlisted_outbound, allow_keeping_phi_indefinitely, allow_single_factor_admin_when_exposed, allow_unverified_alert_smtp_tls, [alerts].security_notifications_required, a per-connection cleartext_accepted / tls_revocation_attested, or the [security].enforcement dial. What this costs: a box that ran key-free on the declaration now needs a key or the audited per-gate ack. Nothing is deployed (there is no migration), and both in-repo users of the declaration — CI's SQL Server load leg and the failover load harness — moved to per-gate relaxations that are narrower than what they replace. See ADR 0186 and BACKLOG #1279.

Changed

  • An Active Directory login is now identified by the directory's immutable id, not by sAMAccountName. A directory frees a deleted account's name and may reissue it to a different person. The engine resolved an AD principal by that name, so a recycle without a matching MessageFoundry delete_user adopted the departed operator's row and re-bound its user_id -- the value uploaded-file ownership, the per-uploader quota and saved search presets all key on. Nothing reported it. AdPrincipal now carries the normalised objectGUID, users.directory_object_id stores it on all three store backends (nullable, in-place upgraded, no index), and _upsert_ad_user resolves by that id. A login whose id disagrees with the row holding its username is refused and audited (directory_identity_conflict), never adopted or backfilled -- backfilling on first sight would leave the recycle window open for every account that had not signed in yet. A directory that returns no immutable identifier still resolves by username, and the engine warns once per distinct cause -- the attribute absent, or present in a shape it cannot read -- so a site on that path is told rather than left to assume the control is running. A directory-side rename now keeps the account instead of minting a second one, which is the other half of the same defect: before this, a rename resolved to nothing and silently orphaned the uploads and presets keyed to the first row. (BACKLOG #1471)
  • The directory session reconciler is keyed on that same immutable id, and the stored username is now a cache the directory refreshes. Identifying a login by objectGUID while reconcile_directory_sessions went on probing resolve_principal(<the stored name>) left a renamed account reading as absent on every pass -- the same answer a deleted or disabled account gives -- so at the shipped ad_session_recheck_seconds = 300 and ad_session_recheck_strikes = 2 a deploying site would have seen a renamed person's sessions revoked, a security notice emailed, and the cycle restart at the next sign-in, roughly every ten minutes and with no administrative escape, because nothing in the engine could write users.username. _probe_principal now asks the directory by directory_object_id where the row carries one, and the directory's current sAMAccountName is copied down onto the row -- from the login path and from the reconciler pass alike -- so the user_id that uploaded-file ownership, the per-uploader quota and saved search presets key on never moves. A rename onto a name another row already holds is refused, not forced (username is NOT NULL UNIQUE): the login path refuses at the directory_identity_conflict guard, and the reconciler leaves both rows alone and audits auth.ad_username_refresh_conflict, costing the renamed person neither their session nor their roles. A directory returning no readable objectGUID still probes by name, unchanged. set_user_username is engine-internal and reachable from no API route, deliberately -- an operator able to set it could point a row at a directory account it is not bound to, which is the privilege transfer #1471 closes; there is still no setter for directory_object_id. (BACKLOG #1532)
  • Web console engine UI seam 93ba1f10b9dccfc8 -> b93f38d097f97a45. SecurityPosture gained the additive store_privilege object above, and StorePrivilegeView joins the discovered surface. Additive with a default, so an older console ignores it; the seam still moves because the golden seam contract introspects that model's field set.
  • DEPLOY-SERVER-DB.md §1.2 posture B now states its prerequisite. "A DBA pre-creates the objects" is not sufficient on its own: the engine skips its DDL batch only when the schema_meta marker records the current batch, and on PostgreSQL CREATE TABLE IF NOT EXISTS against an existing table is still refused for a role holding only USAGE (the schema ACL is checked before the existence skip, measured on 16.14). Bootstrap once with a DDL-capable principal, then hand over.
  • messagefoundry audit-anchor, and audit-verify --expected-anchor / --expected-anchor-file to check one back. The audit hash chain links each row to its predecessor, so deleting the newest rows leaves a shorter chain that still walks cleanly — audit-verify on its own reports OK after a tail-truncation, which is the shape an attacker hiding what they just did leaves behind. The store could always compare against an external anchor; nothing exposed it, so the capability was unreachable. audit-anchor prints COUNT:HEAD (a row count plus a digest — no PHI, no secret, safe to hold in a ticket or an object store); passing it back reports truncated or rewritten when the live chain differs. Know what it is before you build a job on it: an EXACT point-in-time seal, comparing the count and the head hash. The head half is not redundant — an attacker who cuts the newest rows and forges the same number of replacements restores the count and leaves a chain that walks cleanly, so the head is the only thing that differs. The cost of that detection is that a chain which merely grew also reports truncated or rewritten. So it seals a chain at rest across a gap in custody: quiesce the engine, anchor, hold the value off-box, re-verify while the chain is still quiesced — around a maintenance window, a database move, a backup/restore, a hand-off. Anchoring and immediately re-verifying compares a value to itself; re-checking a held anchor against a running engine alarms on every ordinary boot. For continuous coverage of a live engine the off-box log forward / tee remains the control, and [integrity].audit_verify_on_start is unchanged — it is a bare walk and stays blind to a truncated tail. (BACKLOG #328)

Changed

  • An API request body with an unknown or misspelled key is now refused with HTTP 422 instead of being accepted and silently dropped. Pydantic's default is extra="ignore", and none of the 125 models in messagefoundry/api/models.py and messagefoundry/api/auth_models.py overrode it — so a key the engine did not recognise vanished and the route answered success. The sharpest case was PUT /users/{id}/channel-scope: channels is optional and None means all channels, so {"chanels": ["IB_ACME_ADT"]} asked for one connection and granted every one of them. The posture is request-scoped, and that is the whole design. The 32 models FastAPI parses out of a request body now subclass messagefoundry.api.request_model.RequestModel, which forbids unknown keys; the 93 response-only models stay tolerant, because messagefoundry.apiclient reads engine responses into those same classes and the web console ships as a separately-versioned wheel — a strict response model would make an older client raise on a newer engine that merely grew a field. Five shapes (AdGroupMap, AdGroupMapEntry, AdGroupScopeEntry, AdGroupScopeMap, ChannelScope) travel in both directions; they carry the request rule because a dropped key on the RBAC writes is a mis-grant, so adding a field to one of them needs the client bump in the same release. (BACKLOG #1109)

  • A fhir_lookup search value now states its KIND, and a plain string carrying one of FHIR's value-layer separators is refused rather than sent. Percent-encoding is a URL-layer control: it stops one value becoming two search parameters, and it cannot help at the FHIR value layer, where , | and $ are FHIR's own separators. The FHIR specification is explicit that a server percent-decodes a parameter value first and reads FHIR's syntax second (R4 section 3.1.1.4.19, R5 section 3.2.1.5.7), so %7C arrives as a live token separator. A message-derived value carrying one could therefore change what the search means. Three kinds, because one string cannot carry two provenances. A plain str is data and raises a PHI-safe error if it carries , | or $ — the error names the parameter key and the character, never the value. FhirToken(system, code) splits "MRN|" + mrn into its two halves: the system is your literal and passes through, the code is data and screens. FhirRaw("...") is FHIR search syntax you wrote — a composite, a quantity, a comma-separated OR or _sort list — percent-encoded only. Refusal rather than FHIR's backslash escape, deliberately: the escape is correct only if the far end implements the unescape, and server behaviour there varies, whereas a value that never leaves the process cannot be misread by any server. Escaping stays available as an additive fourth kind for a site that has a real FHIR server and can verify it. Migration: {"identifier": "MRN|" + mrn} becomes {"identifier": FhirToken("MRN", mrn)}, which puts identical bytes on the wire. Import FhirToken / FhirRaw from messagefoundry. A non-string scalar also raises now — it was never in the declared type, but urlencode used to coerce it, so {"_count": 50} has to become {"_count": "50"}. What is NOT screened: the backslash. FHIR names it alongside these three because it introduces the escape, so a server that implements the unescape reads a bare \ as an introducer. Widening a refusal is a behaviour change that should be ruled, so it is recorded on messagefoundry/fhirsearch.py rather than folded in here. (BACKLOG #1243, ADR 0043)

  • The authorization-grant audit trail now defaults ON, so a deployment records every authorization grant rather than only the state-changing ones. [security].audit_all_authorization_decisions and the internal [diagnostics].audit_all_authz it desugars to both default true. Until now only a fixed set of state-change / configuration / user-management permissions wrote an auth.permission_granted row, so every authenticated read was authorized and never recorded — and a site could not reconstruct a read history afterwards, because the rows did not exist. What the old default guarded against was measured, and it named the wrong surface. The reason on record was that full tracing would flood the hash-chained audit log through console polling and the /ws/stats feed. The web console never traverses require() — it is server-rendered in-process and gates on its own cookie-world check, which records denials only — and WebSocket authorization fires once per connection, not per message. The volume moves rather than vanishing, so size it. The JSON API is the surface that changes: 33 require()-gated GET routes in api/app.py and 9 more in api/auth_routes.py go from no grant row to one row per authenticated request (a per-request ceiling of one, whatever a route's permission count), bounded by your API clients' polling cadence. Nothing prunes audit_log[retention].audit_days is reserved and unenforced by design — so [retention].max_db_mb is the signal to watch. The per-request cost is a commit, not just a row: the grant write is awaited before the route body runs, takes the store write lock, and commits standalone (audit is excluded from the group committer), so a busy JSON-API deployment pays one extra commit per authenticated request on the same lock the pipeline handoffs use. Set [security].audit_all_authorization_decisions = false to restore the previous narrow trail; that is now reported as a loosening at serve and on GET /security/posture. PHI-view grants stay excluded at either value, because the PHI-access audit path already records them. (BACKLOG #1277, ADR 0118 §5 amended)

  • A PHI instance reached through a declared reverse proxy with [security].require_mfa explicitly off would refuse to start on first deployment, where it previously would not have. The MFA-at-exposure gate derived "is this instance exposed?" from [api].serve_ui, a field the ADR 0143 console degrade arms rewrite in place earlier in the same startup. On the topology the runbooks recommend — a loopback bind behind a declared TLS terminator, with the web console left at its default — the auto-degrade cleared that flag first, so the gate evaluated "not exposed" and the refusal was unreachable, while the ASVS 11.7.1 arm in the same startup classified the identical boot as exposed. The gate now reads a single console-independent predicate (an off-loopback bind or [api].tls_terminated_upstream), so it also fires when the console is auto-degraded, when serve_web_console = false disables it outright, and when the console package is simply not installed: the surface authenticating with one factor is the JSON operator API, which the proxy serves either way. The #189 dual-control advisory reads the same predicate and gains the same reach (still warn-only). Who this would bite: a deploying site that has explicitly set require_mfa = false on a PHI-carrying environment behind a declared TLS terminator, under enforcement = enforce. Two remedies, both existing: set [security].require_mfa = true, or set the already-shipped acknowledgment [security].allow_single_factor_admin_when_exposed = true, which downgrades the refusal to a loud audited warning. A plain loopback bind with nothing declared is not exposed and is byte-identical. An undeclared proxy (web_console_public_address set, no tls_terminated_upstream) deliberately still does not refuse — exposure there would be an inference — but it no longer passes in silence: a new warning names single-factor admin directly on a PHI instance with require_mfa off. (BACKLOG #326, ADR 0140 amendment)

  • BREAKING — an [[alerts.rules]] block that routes to an unconfigured transport now refuses at startup instead of being silently ignored. notifier_from_settings returned early when no transport was configured, before the loop that cross-checks each rule's transports against the ones that exist. So the fail-loud guarantee held everywhere except the state an operator is most likely to be in while first setting alerts up: with one transport configured, a rule naming a different one was a hard ValueError at startup; with zero configured, the identical rule was accepted and then never applied, and nothing said so. Validation now runs first. Who this bites: an instance where [[alerts.rules]] exist AND at least one rule or escalation tier sets a non-empty transports AND no transport is actually configured (webhook_url unset, and not all three of email_smtp_host + email_from + email_to set — the email transport needs all three, which is what makes a half-filled [alerts] block look configured). Such an instance starts today and will refuse after upgrading. Why this is safe to take: in that state the rule has never routed a single alert. The refusal removes no working behaviour — it converts a permanent silent no-op into a startup error that names the exact keys to add. A rule that names no transport is unaffected and still starts (now with a warning when rules exist that cannot notify anyone), so the ordinary "write the rules first, wire the transport later" flow keeps working. The same cross-check now runs at authoring time: messagefoundry alert add (which the VS Code "New Alert" command shells) refuses a rule routing to an unconfigured transport rather than persisting a file that only fails at the next boot. It is scoped to the rule being added, so a file that already contains a bad rule can still be repaired with alert remove.

  • A mail-only or Direct-only PHI instance can now satisfy the open-egress startup gate by declaring its destinations. [egress] has eight allowed_* lists and all eight are enforced downstream, but the startup gate hand-enumerated six: allowed_smtp and allowed_direct were absent, so an instance whose only egress is Email() or Direct() exited 2 with "outbound egress is UNRESTRICTED" while holding a fully-enumerated allow-list, and nothing in the message named the two lists that did not count. The two are now counted — deliberately only when [security].block_unlisted_outbound is left unset, which is exactly the state the deny-by-default flip turns ON, so such an instance starts fail-closed. An instance that explicitly set block_unlisted_outbound = false is unchanged and still refused, because there the other six transports stay allow-any; the refusal now names that override as the reason. No shipped refusal stops firing.

  • BREAKING — a non-loopback DICOM C-STORE SCP now requires a verifiable peer control; calling_ae_allowlist no longer satisfies the gate on its own. The fail-closed peer-control check refuses a remotely-reachable SCP that has no peer control, and it accepted any one of three: calling_ae_allowlist, source_ip_allowlist, or mTLS. It counted them rather than weighing them. But a Calling AE Title is a string the caller asserts about itself in the association request — no key, no signature, nothing to verify — and AE Titles are published in conformance statements and visible in any capture. An SCP whose only control was an AE-title list was therefore reachable by anyone who could route to it and knew one string, while passing a check named "fail-closed peer controls". Server TLS does not close this: without tls_ca_file there is no client certificate, so the cleartext bind guard (confidentiality) and this gate (authentication) are orthogonal. What changed: off-loopback, the gate now requires source_ip_allowlist or mTLS (tls + tls_ca_file). calling_ae_allowlist is kept and still enforced at association time — it is a genuinely useful filter that catches a misrouted sender and pins intent — it simply has to be paired with one of the two. Measured: AE-title-only off-loopback goes from starting to refused; AE-title paired with an IP allowlist starts; IP-only and mTLS-only are unchanged; and every loopback bind (the common dev/single-box case) is unchanged. Who this bites: a site running a non-loopback SCP whose only peer control is calling_ae_allowlist. It starts today and will refuse after upgrading. The fix is one line — add source_ip_allowlist=[...] to the inbound(...) call, which for a DICOM SCP is the only authoring surface — and the refusal names it. Keep the AE list; it is still doing work. Tracked as BACKLOG #316. Options considered and declined: an audited opt-out switch, and documenting the weakness without changing the gate.

  • BREAKING — an unrecognized key in a known config section now fails the start instead of loading silently. Every section model inherits extra="ignore", so a mistyped or stale key in messagefoundry.toml loaded clean, did nothing, and said nothing: an operator could misspell block_unlisted_outbound and believe a posture control was on while the engine applied its permissive default. Exactly one section, [security], warned about it; the other 27 were silent. The loader now refuses, naming the section and the offending key and suggesting the closest valid field name. Scope, and it is deliberate — the refusal covers the config FILE only. MEFOR_* environment variables and CLI flags are still accepted silently, because about a dozen documented MEFOR_* variables (the Vault store and secrets providers, the TLS revocation attestation, the lane timing probes) are read straight from os.environ by consumers that are not settings fields — so refusing an unrecognized environment key would refuse a deployment configured exactly as the shipped documentation instructs. [security] is the exception and is refused from the environment too. The check lives in the loader rather than a pydantic extra="forbid" for a second reason: pydantic echoes the offending value in its error, and the CLI prints validation errors verbatim to stderr, which the Windows service captures to a log file — so a mistyped secret key would have written the secret to disk. Both existing config refusals name keys only, never values. Who this would bite on first deployment: a config file carrying a key that is not a field of its section — a typo, a key copied from newer documentation, or a setting since removed from the engine. Remedy: correct the spelling; the error names the section and the key. Nothing needs migrating, because a key that is refused now was doing nothing before.

Security

  • The web console's step-up actions would have refused an MFA-pending session without the audit row the console's other MFA refusals write. require_ui_step_up and require_ui_step_up_action switched off require_ui's second-factor gate to keep their /ui/reauth?next= continuation, then refused a pending session with a bare redirect of their own. On a first deployment with [security].require_mfa on, which is the default, a stolen password-only session cookie would have probed all 42 step-up route gates and left no auth.mfa_denied row. The same switch put the permission check first, so the refusal would also have shown which of those permissions the account holds, and it spent the account's admin-write budget before refusing. The gate now refuses, audits and orders its checks as it does on every other /ui route. Only where it sends the browser differs. The JSON API was never affected. (BACKLOG #1542)
  • The web console's message editor would have opened the raw body to a custom role holding messages:edit without messages:view_raw. GET /ui/messages/{id}/edit and POST /ui/messages/{id}/edit-resend gated on messages:edit alone, while the JSON handler they call in-process (GET /messages/{id}, require_phi_read(messages:view_raw)) has its own gate skipped by that direct call — so the console re-asserted a different permission than the one it stood in for. Both verbs now require both permissions and fail closed on either, and both charge the per-actor PHI-read budget (require_ui_step_up gained phi=). Custom-role minting is deliberately unchanged: messages:edit is still not in CUSTOM_ROLE_FORBIDDEN_PERMISSIONS, so a role meaning "may resubmit, must not read" remains mintable — it simply cannot open an editor that displays the body it edits. Who this would bite: a deploying org whose admin had minted such a custom role; that role would have exceeded its stated scope (HIPAA minimum-necessary) on first deployment. No built-in role reaches it — ADMINISTRATOR and OPERATOR grant both permissions — and every such read was already audited. (BACKLOG #324)

Fixed

  • The shipped VS Code snippet generated a FHIR lookup the engine now refuses. The meforfhirlookup snippet built its search by concatenating a message field into a flat ?-query — the form removed along with [egress].fhir_require_structured_params — so the snippet emitted a Handler that raises on first use. The FhirLookup docstring, the fhir_lookup docstring and the Steps palette taught the same removed form. All now use the per-value-encoded params= form; the read-by-id form is unchanged. Why it shipped broken: nothing read that file. A new test parses every shipped snippet body and asserts none teaches the removed form — a test that merely checked the JSON parses would not have caught it. (ADR 0043)
  • A CR/LF inside an exception message could forge a whole log line on the text sink. ControlCharScrubFilter escaped only the rendered message, and logging.Formatter appends a record's traceback (exc_text) and stack dump (stack_info) verbatim — so a newline-bearing exception string landed at column 0 on its own physical line, where a payload padded to the record layout was byte-indistinguishable from a real entry to an operator or a line-oriented SIEM parser. Both fields are now scrubbed too (ASVS 16.4.1; the residual ADR 0034 §1 disclosed, BACKLOG #335). Visible change: a traceback is not collapsed onto one line — its line breaks are kept and every line is indented with |, so it stays readable while no line of it can start at column 0. A log parser keyed on Traceback (most recent call last): at the start of a line needs that prefix added. The JSON sink is unchanged in substance (json.dumps already escaped these fields); its exception and stack values now carry the same indent.
  • The DICOM C-STORE SCP's fail-closed refusal named a settings key that does not exist. It told the operator to set [inbound].source_ip_allowlist; InboundSettings has no such field and section models ignore unknown keys, so an operator following the engine's own error message wrote a key into messagefoundry.toml that was accepted and silently discarded — leaving a non-loopback SCP with no peer-IP gate while believing it had one. Aggravated by the construction gate counting controls: a calling_ae_allowlist (a caller-asserted AE Title with no cryptographic binding) plus the discarded key passed the check. The message now names the working surface — the inbound(...) keyword, which for a DICOM SCP is the only one, since DICOM() is not authorable in connections.toml — and distinguishes it from the connections.toml [[inbound]] key that is real, so a site running MLLP alongside DICOM cannot read it as licence to delete a working allowlist. The same wrong spelling is corrected in the module docstring, the gate comment, config/wiring.py, config/settings.py, docs/SECURITY.md and docs/ASVS-L2-PHASE0-CHANGES.md. Whether an AE-title list alone should keep satisfying that gate is tracked as BACKLOG #252 — it is an ADR 0025 §9 contract change and is deliberately not decided here.
  • Two startup gates described themselves against the deployment tier rather than the enforcement dial. Comments on the managed-identity and security-notification gates read "refuse (production) / warn (non-production)" over branches that read enforcing — and enforce is the shipped default on dev and staging as much as on prod, so all three refuse. Comment-only, no behaviour change, but these are the comments two published documentation defects were copied from.
  • The load harness's no-loss reconcile did not enforce the read >= sent // 2 intake guarantee 0.3.2 documented. The unconfirmed-send excusal is capped at max(connections, half the run), but that max() takes the connection count as a floor, and every call site passes a connection count — so on a short, low-rate step (connscale-smoke's N=100 cell: ~105 sends, 100 connections) the count won the max() and the intake bound degraded to read >= 5, the very vacuity the cap exists to prevent. Nothing clamped the excusal to sent either, so timeouts > sent degraded it to read >= 0. The half-the-run cap still decides the systemic no-ACK verdict (0.3.2's de-flake is unchanged), and an unconditional intake floor the excusal cannot lower now enforces read >= sent // 2 in all three reconcile copies, at every call site. The estate copy also gained the honest-reporting branch its two siblings had: it previously printed read>=sent, … on a bounded-excused run whose read was demonstrably below sent, and its over-budget detail string now matches theirs — a test pins the three in step, since nothing enforced the claim that they were. Known gap, unfixed: the systemic no-ACK verdict is still gated on the same capped budget, so a dead ACK path that nonetheless delivered everything still passes when connections >= sent; an intake floor cannot catch a fault whose signature is a high read with no ACKs. Bounding that arm needs its own change.

0.3.2 — 2026-07-28 — Early Access

A patch release for one adopter-facing defect shipped in 0.3.1, plus two gates that were passing without being able to fail.

Fixed

  • The scaffolded supply-chain gate named a repository adopters cannot read. messagefoundry init writes a fail-closed CI gate that runs gh attestation verify --repo … on the pinned engine wheel before installing it. It named the retired private development vault rather than the public repository that actually mints the attestations, so every project scaffolded by 0.3.1 shipped a provenance gate that fails on its owner's first CI run. If you scaffolded against 0.3.1, either re-run messagefoundry init, or edit that one --repo argument in .github/workflows/check.yml to MEFORORG/MessageFoundry; setting the repository variable MEFOR_VERIFY_ENGINE=off skips the job entirely as a stopgap. The scaffold test had pinned the wrong value, so the defect was being actively enforced; it now also pins the negative.
  • The weekly vulnerability-metrics job measured an empty window instead of failing. vuln-metrics.yml invokes scripts/security/vuln_metrics.py with no --repo, so the argparse default silently decided what was measured — and it named the same retired vault, whose Dependabot PRs a public token cannot read. All seven KPIs were computed over zero pull requests rather than erroring. The default is now $GITHUB_REPOSITORY, so the job measures the repository it runs in.
  • A partially-failed release could not be retried. The GitHub release step failed when the tag or release already existed, so a publish that died midway (as 0.3.0's did) left no clean path forward. It is now idempotent.
  • The load harness's no-loss reconcile false-failed a demonstrably zero-loss run. It excuses a send left unconfirmed at connection teardown, capped so a dead ACK path cannot pass as zero-loss — but the cap modelled legitimate stranding as "~one in-flight frame per connection". The sender keeps an unbounded in-flight window and paces open-loop sends by offered rate, so real stranding scales with rate × ACK-latency instead. A run that stranded 14 of 90 sends while the engine read and delivered every one of the rest was reported as message loss. The cap is now max(connections, half the run), which keeps read >= sent // 2 always required.

0.3.1 — 2026-07-27 — Early Access

Security

  • BREAKING — multi-factor authentication is now an ACCESS gate, not only a step-up gate (ASVS 6.3.3). An MFA-pending session is refused on every authorized route with 403 + X-MFA-Required: 1; a browser session is confined to the new /ui/mfa page until it verifies. Previously the second factor was demanded only at the step-up (sensitive-action) boundary, so a password-only session could read the whole estate. [security].require_mfa_scope (new, default every_local_account) widens who must enrol from the Administrator role to every local account; set it to administrators to keep the previous posture. It is reported as a loosening on GET /security/posture but never refuses to boot. Who is affected on upgrade:
    • Existing sessions are unaffected until they expire — they were minted MFA-satisfied and the stamp is honoured. The change bites at the next sign-in.
    • Every un-enrolled local account must now enrol a factor before it can do anything else. The escape path is reachable from the pending session itself (/me/reauth/me/mfa/enroll/me/mfa/confirm, or /ui/account in the browser), and confirming satisfies that same session. Note the consequence: for an un-enrolled account, whoever holds the password can self-enrol the second factor. The out-of-band MFA_ENABLED notification is the compensating control.
    • Non-interactive local service accounts using bearer tokens will start failing with X-MFA-Required and cannot enrol unattended. Move them to mTLS (require_service_cert is exempt by design) or to AD, or set require_mfa_scope = "administrators".
    • The PySide6 test harness needs an enrolled account or require_mfa_scope = "administrators": its API client only retries an X-MFA-Required refusal when an MFA handler is registered, and none is wired today.
    • Caveat on factor strength: a WebAuthn passkey is asserted at user_verification=preferred, so for a passkey-only account the second factor may be device possession alone. This is the owner-signed L3 relaxation, recorded in docs/SECURITY.md, not an oversight.
  • BREAKING (federated deployments only) — a directory session is no longer minted MFA-satisfied unconditionally (ASVS 6.3.4). _complete_ad_login now takes the grant per mechanism: AD simple-bind and Kerberos still pass it under the owner-signed delegated-directory-MFA relaxation (the engine learns nothing about directory-side strength from a bind or a ticket), while the federated (OIDC) leg passes [auth].oidc_require_mfa_claim — the one directory signal the engine actually verifies. With the claim gate on (the default) nothing changes: a token carrying no configured amr/acr was already refused at claims validation. With oidc_require_mfa_claim = false, federated sessions are now minted un-verified and refused — turn the claim gate on, move those users to AD, or set [security].require_mfa = false, which remains the global off-switch. Also fail-closed: the directory exemption is now an allow-list (provider == "ad") rather than a denylist (!= "local"), so an unrecognized provider value requires a second factor instead of silently skipping one.
  • New auth.mfa_denied audit event. The MFA gate sits above the permission loop, so auth.permission_denied never fires for a pending session; without its own row a stolen password-only token could enumerate the entire authenticated surface leaving no trace.
  • ENGINE_UI_SEAM 13 → 14 — api.security.require() gained an mfa_gate keyword, and the console imports it directly. A console wheel older than this engine refuses to mount, as designed.
  • In-use data protection for PHI is now declared and reported (ADR 0152, ASVS 11.7.1). An exposed PHI instance (a non-loopback bind or a declared [api].tls_terminated_upstream, which includes the recommended loopback-behind-proxy topology) that has not set [security].memory_encryption_operator_declared = true — the operator's declaration that the host provides hardware memory encryption (AMD SEV-SNP / Intel TDX) — now warns at every start. Not a breaking change: nothing that boots today stops booting. The refusal is opt-in via the new [security].require_memory_encryption_declaration (default false), because this is a host property that no operator can satisfy on Windows (where the platform read-out is always null) — the same scoping rule as [security].allowed_client_networks' companion refusal (ADR 0151). Loopback and synthetic instances are byte-identical. See OFF-LOOPBACK-DEPLOYMENT.md ladder row 12 and SYSTEM-REQUIREMENTS.md.
  • Report-only platform memory-encryption read-out on GET /security/posture (ADR 0152 rung 1; ENGINE_UI_SEAM 12 → 13) — memory_encryption_self_reported_capability / ..._self_reported_active / ..._self_reported_mechanism / memory_encryption_readout_source, plus memory_encryption_operator_declared, the tri-state memory_encryption_readout_contradicts_declaration (null = nothing was measured that could contradict anything) and memory_encryption_note, the disclaimer carried in the response body so it travels with any copy of the artifact. Linux reads /proc/cpuinfo flags (capability) and /dev/sev-guest / /dev/tdx_guest character devices (activation) as separate facts; everywhere else every field is null. No value of any of these satisfies ASVS 11.7.1 — they are what the host says about itself, and cryptographic attestation (rung 3) is not built. The read-out is never accepted as a substitute for the declaration, in either direction.
  • Windows crash dumps of the engine process are suppressed — a WER dump is a full PHI disclosure written to disk. serve applies the process-local half itself (SetErrorMode OR-ed into the inherited mode, WerSetFlags(NOHEAP | NO_UI | DISABLE_SNAPSHOT_*)); the machine-policy half no process can reach is opt-in via install-service.ps1 -SuppressCrashDumps (WER ExcludedApplications, registered for both messagefoundry.exe and the venv python.exe, plus a narrowing-only LocalDumps override that is written only where LocalDumps is already configured — creating that key would switch dump collection ON). Residuals in SERVICE.md.

Fixed

  • The harness message list could livelock and stop updating entirely. MessagesPanel._apply cleared its in-flight guard, re-fired any refresh latched during the read, then returned before rendering — discarding the snapshot as superseded. Whenever refreshes arrive faster than a read completes there is always a latch waiting when the read lands, so every snapshot was discarded and the table never updated: permanently stuck, not merely slow (measured: 391 reads served, 0 rendered). It now renders first and drains afterwards, costing at most one read of staleness while still converging on the latest filter. This surfaced as an intermittent CI failure that no timeout or retry could fix, because neither addresses a livelock.
  • Two config blocks in the off-loopback runbook aborted at load[diagnostics].audit_all_authz and [ai].data_class had been relocated by ADR 0118, so an operator copy-pasting either block got an immediate start failure. Corrected to [security].audit_all_authorization_decisions and [security].handles_real_patient_data, and every fenced toml block in that runbook is now pinned by a test that loads it and fails on a silently-ignored key.

0.3.0 — 2026-07-13 — Early Access

Highlights since 0.2.15 — streaming attachments end-to-end, a copy-on-Send message model, richer connectivity, and a run of security hardening. (Concise highlights; the git history is the full change set.)

Added

  • Streaming large attachments end-to-end (#149, ADR 0105) — very-large OBX-5 documents are detached from the message skeleton at ingress and streamed through routing → transform → delivery on all three stores (SQLite / SQL Server / Postgres), with an operator read/download surface.
  • Copy-on-Send message model (ADR 0104) — Send snapshots the message at construction (opt-in [pipeline].snapshot_on_send), plus Message.copy() and a recognition-first message_type_of(accepts=) predicate; and in the IDE, a point-and-click HL7 field picker and the Steps-view authoring palette (ADR 0103 / 0106).
  • Connectivity — generic HTTP auth (OAuth 2.0 client-credentials + Digest) and HTTP response-header capture (#154, #65); a connector SecretProvider seam with a HashiCorp Vault backend (#196).
  • Monitoring — an engine-wide KPI roll-up with a saturation-derivative alert and DB-pool metrics (#93).

Changed

  • pipeline: batch_handoff_statements now defaults ON on SQL Server (ADR 0075) — per-hop SQL statement batching (a distance-insurance lever: folds each message's per-hop store round-trips into the fewest pyodbc.execute() T-SQL batches, cutting network round-trips, NOT transactions — the single per-hop COMMIT is untouched, commits/msg stays 2.000) now activates by default on the SQL Server store. Promoted 2026-07-08 after Bench B showed harmless-near (batch ON vs OFF within ±0.4% at ~0.28 ms RTT, zero-loss) and helps-far (constant ~−18% ACK-p99, absolute saving widening with RTT: −84 ms @ +20 ms, −212 ms @ +50 ms) over a green SS correctness precondition (tests/test_adr0075_batch_sqlserver.py, 9 passed). Fail-closed + SQL-Server-only — Postgres (asyncpg) and SQLite are byte-identical no-ops; the flag is retained only as an emergency off-switch: set [pipeline].batch_handoff_statements = false to disable.

Security

  • The published PyPI source distribution no longer ships the whole repo (#1020) — the sdist is pinned to the messagefoundry package + its metadata, with a fail-closed "sdist is package-only" gate in the release workflow. No release ever exposed PHI or customer data.
  • Transport hardening — certificate-revocation refusal extended to outbound-connector TLS (#201); in-use memory protection for the store (#198); the publish deny-list is read from the ref being published (#983).

Fixed

  • CI reliability — bound PHI-retention on the Windows service smoke (#1011); wrap pyodbc-heavy SQL Server steps with a native-crash retry (#1010); multi-message finalizers take their per-message locks in canonical order (#980).

0.2.15 — 2026-07-06 — Early Access

Thread-hop fusion (ADR 0071 B5, flagged default-OFF) + the browser ops dashboard, pooled-claimer primitives, and opt-in persistent outbound MLLP. The headline engine change is B5 thread-hop fusion — a SQL-Server-only path that fuses each off-loop CPU stage with its store handoff onto one synchronous-pyodbc worker hop to cut the per-completion executor→loop marshaling wall; it ships behind [pipeline].fuse_thread_hops (default off) pending the SQL-Server throughput bench. Everything else below is additive / opt-in.

Added

  • pipeline: thread-hop fusion — B5, opt-in (ADR 0071; [pipeline].fuse_thread_hops default off, SQL Server + pooled claim mode only) — fuses each off-loop CPU stage (route_only / transform_one) with its adjacent store handoff onto one synchronous-pyodbc worker hop, so a multi-statement aioodbc handoff marshals back to the loop once instead of per statement (attacks the per-completion marshaling ceiling the 2026-07-04 profile named). Fuses thread hops, not transactions — commits/msg are identical (the poison-guard is intact); Postgres (asyncpg loop-native) and SQLite (loop-affine handoff lock) keep the async path by construction. Activation is fail-closed: fusion only turns on when the dedicated sync pyodbc pool + per-stage fusing executors open cleanly, otherwise the async path runs unchanged. Ships with the self-contained SQLite/Proactor crossing-count micro-bench + a Windows CI mechanism gate, and a connscale B0/B1 fusion A/B harness axis (fuse_ab profile, trials-banked); the throughput GO/NO-GO (ship-by-default vs escalate to free-threading, ADR 0053) is a separate SQL-Server bench, not a merge gate.
    • Promote-gate resolved 2026-07-06 — NO-GO (ADR 0071 §8/§10, #787): the SQL-Server fuse_ab bench measured a real but sub-threshold +6.5 / +9.3 / +10.0 % lift (below the ≥10% bar; zero-loss held), so fuse_thread_hops stays default-OFF and the lever escalates to free-threading (ADR 0053). Bench-SHA provenance: the run was at commit 8bab40e2, which is not an ancestor of main (PR5 was squash-merged as 90f80a3, #780) — but git diff 8bab40e2 90f80a3 is empty, so the NO-GO bench ran a code tree byte-identical to merged PR5.
  • store: pooled-claimer primitives claim_fifo_heads / list_fifo_lanes / release_claimed on all three backends (ADR 0066 PR 2 — now wired by the StageDispatcher and the default claim path since #755/#744, see Changed below): a FIFO-safe multi-lane head-claim (probe-then-claim, EMPTY-on-locked-head — never a #285 skip to seq N+1), a read-only head-due-aware lane discovery for the pooled sweep, and an attempts-neutral claim release. claim_next_fifo / claim_next_fifo_batch / claim_ready are untouched.
  • pipeline: bounded pooled-mode infra-fault handling (T17) (ADR 0070; #766) — in pooled mode a lane whose head keeps failing on an infrastructure fault now re-pends its head at an exponential-capped backoff (cap [pipeline].infra_fault_backoff_cap, default 60 s) instead of spinning the ~4×/s discovery sweep, and after [pipeline].infra_fault_stop_after (default 10) consecutive zero-progress faults (~4 min of wall clock) applies [pipeline].infra_fault_policy (default stop): STOP-the-lane with a throttled lane_stuck alert — never auto-dead-letter. retry_forever instead keeps re-pending at the cap and alerts once the horizon is crossed. per_lane mode is unaffected.
  • Browser ops dashboard (read-only, M1) (#75, ADR 0065) — an opt-in ([api].serve_ui, default off), same-origin, zero-install browser ops view served under /ui by the engine's FastAPI app. Read-only: a live-polling connections dashboard (In/Out/Queued/ Errors/Last-Activity), a message log with filters, the audited raw-message view (reuses the exact GET /messages/{id} PHI path — RBAC + per-access audit + redaction), and a dead-letter list. Auth is a new HttpOnly + SameSite=Strict session cookie confined to /ui — the JSON API stays Authorization-header-only, so a request bearing only the cookie is still rejected. Ships a strict CSP (script-src 'self', no unsafe-*), Cache-Control: no-store on /ui + PHI reads, and autoescape-by- default rendering (a stdlib HTML builder — no new runtime dependency, no npm). A JSON-only deployment is byte-identical (serve_ui off); off-loopback requires TLS (refused even under --allow-insecure-bind).
  • Browser ops dashboard — connection controls (M2a) (#75, ADR 0065) — the dashboard adds safe operator actions: inbound connection start / stop / restart, reusing the JSON control handlers (connections:control + per-channel guard). CSRF is defended in depth by an Origin / Sec-Fetch-Site same-origin check on top of the SameSite=Strict session cookie — token-free (no crypto import).
  • Browser ops dashboard — message replay + step-up re-auth (M2b) (#75, ADR 0065) — single-message replay from the browser (a Replay button on the message detail). Replay is require_step_up in the JSON API, so the /ui route uses a cookie-world step-up gate (require_ui_step_up): if the session hasn't recently stepped up it redirects to a /ui re-auth page (password, + TOTP when MFA is required) instead of a 403 header a browser can't act on; after re-auth the browser auto-retries the pending replay. The re-auth next target is validated to be a /ui replay action only (anti open-redirect).
  • Browser ops dashboard — dead-letter bulk replay (M3) (#75, ADR 0065) — a per-channel Replay all dead action on the dead-letters page, re-queuing every dead delivery for a channel via the JSON replay_dead_letters handler. Same step-up gate as message replay (require_ui_step_up → /ui re-auth + auto-retry; the channel is in the action path so the body-less re-POST carries it), and it honors the dual-control approval gate — when a replay is held for a second approver it surfaces a "held for approval" page instead of redirecting.
  • Browser ops dashboard — live /ws/stats channel (M-ws) (#75, ADR 0065) — the dashboard now shows a live queue-status strip pushed over the engine's /ws/stats WebSocket (previously only the desktop path existed and it was unused). A browser can't set the WS Authorization header, so a same-origin browser handshake authenticates via the mf_session cookie it carries; CSWSH is defended by a same-origin Origin-vs-Host check plus the SameSite=Strict cookie (a cross-site handshake carries no cookie). The native (header) path is unchanged — a client without an Origin falls through to it. The the WS strip degrades to empty if the socket can't connect. The connections table itself now updates live over the socket too: the server pushes the rendered (already-escaped) connections fragment, the client swaps it in and stops polling, and polling resumes as a fallback if the socket drops.
  • Browser ops dashboard — HL7 parse-tree view (#75, ADR 0065) — the message-detail page links to a GET /ui/messages/{id}/parse-tree view that renders the HL7 segment/field tree server-side via the pure parsing lib. It reuses the single audited GET /messages/{id} PHI path (no new PHI egress), every field value is escaped (attacker-influenced HL7 can't inject markup), and a non-HL7 body (X12/DICOM/ binary) surfaces a "no parse tree" notice rather than an error.
  • Browser ops dashboard — per-destination dead-letter replay (#75, ADR 0065) — the dead-letters page now offers Replay per (channel, destination) buttons alongside the per-channel "Replay all dead" (POST /ui/dead-letters/{channel_id}/{destination_name}/replay), same step-up + approval-gate + path- based auto-retry as the channel-wide action. is_safe_ui_action was widened to the two-segment path and hardened to reject any .. traversal marker.
  • Browser ops dashboard — off-loopback exposure via [api].public_origin (#75, ADR 0065) — a new opt-in [api].public_origin (e.g. https://ops.example.com) makes the dashboard's same-origin CSRF and CSWSH checks work when /ui is reached off-loopback through a reverse proxy that doesn't preserve the Host header: the browser Origin is matched against the configured public origin instead of the request Host. Default (unset) is unchanged — loopback / Host-preserving-proxy behavior. The safe defaults stand: [api].host is 127.0.0.1, and serve_ui off-loopback still requires TLS (exposure_protected), refused even under --allow-insecure-bind. (Phishing-resistant MFA / managed- admin-host controls for off-loopback admin remain a separate posture decision — WebAuthn #11 + the ASVS 8.4.2 residual.)
  • mllp: persistent outbound connections — OPT-IN (ADR 0067; persistent=false default this release, per-outbound opt-in via persistent=true) — the MLLP destination can reuse one lazily-established connection across deliveries (with idle_timeout_seconds / max_connection_age_seconds freshness knobs and reconnect-before-first-byte), eliminating the per-message TCP/TLS handshake and the TIME_WAIT ephemeral-port exhaustion measured on the 2026-07-02 load campaign. It ships opt-in: the default stays connect-per-message (today's proven posture — no behavior change for existing deployments), and persistent=true is a documented opt-in for sustained high-rate lanes.
    • Shipped default: persistent=false (connect-per-message). Existing outbounds are byte-for-byte unchanged. Set persistent = true on an MLLP() destination to opt into connection reuse (recommended on sustained high-rate lanes; see docs/SERVICE.md "High-delivery-rate TCP tuning"). The default flips to persistent=true in a subsequent release once the ADR 0067 §8 trigger is met: a real deployment runs persistent=true clean on a live feed and the mid-transaction stray-frame correctness edge is closed (a test / the MSA-2↔MSH-10 correlation, BACKLOG #82) or field-confirmed benign.

Removed

  • Frozen zero-Python Windows console installer retired (#39, ADR 0032 Phase B Amendment (2026-07-01)). The PyInstaller --onedir + Inno Setup channel added in 0.2.11 is removed: packaging/console-installer/, the release-console-installer job in the release workflow, and its AC-linked tests are deleted, along with the Qt-LGPL frozen-binary written-offer/bundled-license apparatus and the pending Authenticode signing-cert requirement. Rationale: zero uptake (the CI leg failed on every tag release since it merged; one out-of-band .exe with no downloads), the no-Python/no-IT demand gate never fired (adopters are pip + IT-covered), and it only ever shipped unsigned. The desktop console is unaffected — it stays installable via pip install messagefoundry[console] + the ADR 0032 Phase A gui-script and shortcut scripts; only the frozen, zero-Python conveyance is gone. The zero-install audience is now served by the browser ops dashboard (BACKLOG #75).

Changed

  • Server-DB store opens now skip the schema DDL batch when it already ran (ADR 0064). A single-row schema_meta marker records the content hash of the shipped DDL batch; a re-open of a current database skips the whole guarded batch and the exclusive schema lock (previously every open re-ran dozens of check-then-create statements under sp_getapplock/the schema advisory lock — the measured N≥4 co-start convoy of the WS-B bench, and wasted round-trips on every single-engine restart). Any edit to the DDL batch changes the hash and forces exactly one full idempotent run, so upgraded databases still adopt on-open migrations (ADR 0060) unchanged. Operational note: out-of-band schema surgery is no longer self-healed at the next restart — run DELETE FROM schema_meta afterward to force one full run. SQLite is unaffected.
  • Startup crash recovery (reset_stale_inflight) is now index-seekable (ADR 0064): the all-stages pass runs one UPDATE per pipeline stage against the existing ix_queue_ready(stage, status, …) index instead of one unindexed status-only full scan of the queue (Postgres additionally drops an unsargable OR $n IS NULL form). Same rows recovered, same single transaction — all three backends.
  • Default staged-pipeline claim path flipped to pooled per-stage claimers (ADR 0066; [pipeline].claim_mode default per_lanepooled — issue #744, shipped via PR #765). The StageDispatcher was first wired in behind [pipeline].claim_mode default-OFF (#755, ADR 0066 PR4) and is now the default claim topology: one dispatcher per stage running a handful of pooled claimer tasks over the claim_fifo_heads / list_fifo_lanes / release_claimed primitives (batch-claiming head-prefixes across lanes), collapsing the ~4,500 per-(lane×stage) claim loops that saturated a shared server DB at high fan-out and holding zero-loss where per_lane dropped messages. per_lane stays fully selectable as the byte-identical opt-out ([pipeline].claim_mode = "per_lane"), enforced by the zero-pooled-construction test sentinel. Reliability-core — read once at engine start (a /config/reload does not toggle it; restart to change). Single-node scope; the at-least-once / per-lane-FIFO / poison-guard invariants are unchanged in both modes.

0.2.14 — 2026-07-01 — Early Access

Delta security-audit remediation. A focused security audit of the surface added since the 2026-06-10 full review (v0.2.0 → v0.2.13) surfaced seven verified findings; this release fixes all of them. No new critical, no SQL injection, no auth bypass, no RCE — the most serious was an unauthenticated memory-exhaustion DoS in the new default HL7 parser. Each fix ships with a regression test. See docs/reviews/DELTA-REVIEW-2026-07-01.md, a maintainer-internal document — docs/SECURITY-DOCS-POLICY.md states what is withheld and what you can request.

Security

  • Bounded the built-in HL7 rich-text repetition escape (DELTA-01/02; _builtin_hl7.py). The tolerant built-in parser (now the default hot-path backend, ADR 0054) expanded \.inN\-style repetition escapes with no cap, so a ~15-byte inbound field (\.in2000000000\) allocated gigabytes synchronously on the event loop before the ACK — an unauthenticated OOM/denial-of-service. The count is now clamped (MAX_ESCAPE_REPEAT = 512), and a malformed count no longer raises out of a field read — that had severed the connection and dropped a parseable message with no disposition, breaking the count-and-log invariant.
  • XML-DSig verify() now requires an explicit trust anchor (DELTA-03; parsing/xml/signature.py). Called with neither x509_cert nor ca_pem_file, it previously fell back to signxml's default of trusting any certificate that chains to the host's system CA store (origin-blind verification); it now raises ValueError. Behavior change for the opt-in [xml] codec — a caller must pin the expected signer or a partner CA. No in-repo caller relied on the old default.
  • FhirLookup SMART token endpoint is now egress-gated (DELTA-04; [egress].allowed_http). A fhir_lookup connection composed with with_smart_backend() POSTs a signed client_assertion to its smart_token_url; that host was not checked against the egress allowlist (only the FHIR base host was), so a crafted smart_token_url could exfiltrate the assertion to an un-allowlisted host. The lookup and outbound arms now share one gate (ADR 0043 §D3).
  • Support bundle no longer discloses the store host/database; its log redaction was widened (DELTA-05/07; support/). The offline support bundle's status.json carried the SQL Server host/database verbatim — it is now reduced to the backend kind (file basename only for SQLite). The bundled-log redactor previously used a fixed HL7-segment allowlist with no free-text name/DOB heuristics; it now delegates to the engine redactor (messagefoundry.redaction) for parity with stored-error redaction.
  • Inbound HTTP listener rejects ambiguous framing (DELTA-06; transports/http_listener.py). A duplicate Content-Length, a duplicate Transfer-Encoding, or the two present together are now refused with 400 per RFC 7230 §3.3.3 — closing an HTTP request-smuggling / desync surface behind a fronting proxy.

0.2.13 — 2026-07-01 — Early Access

The store connection-scale sizing wave — right-size the server-DB connection pool to the measured inverted-U optimum, guard against over-provisioning, and guarantee the message store stays unified. All changes are server-DB-only; the single-node SQLite default is unaffected.

Added

  • Soft store-pool over-provisioning warning (ADR 0062) — a server-DB engine now logs an advisory WARNING at graph start if [store].pool_size is sized past the connection-pool inverted-U optimum: at/beyond the ~80 catastrophic cliff, or oversized for the engine's inbound-interface count (~2.5 × interfaces). Advisory only — it never blocks startup; SQLite has no pool so it is skipped, and the default (40) never trips it. Guards the "set a huge pool for 1500 connections" footgun (which is a sharding problem, not a pool one).

Changed

  • Default server-DB store connection pool size raised 5 → 40 ([store].pool_size, env MEFOR_STORE_POOL_SIZE; ADR 0062). A three-sweep connection-scale study found the pool is an inverted-U: it helps up to ~40 per engine, and over-provisioning is catastrophic — past ~40 the extra connections thrash one shared SQL instance (WRITELOG serialization + per-message finalizer applocks), and ACK latency explodes 30–90×. 40 is the measured optimum — do not set it higher to chase connection count. Server-DB backends only (Postgres / SQL Server) — the default single-node SQLite backend is unaffected (fixed read pool + single writer; never reads pool_size). Existing explicit [store].pool_size / MEFOR_STORE_POOL_SIZE values are unchanged — only the unset default moves. Behavioral deltas on server-DB engines: ~ the steady-state DB sessions per engine, and the startup pool pre-warm rises from ~2 to ~20 connections per engine (bounded by warm_pool_timeout, off the intake path, self-releasing, never raises). Connection-budget caution: pool_size is per engine, so on a shared server DB engines × pool_size all count against one max_connections (Postgres default ~100 → ~2 engines at 40) — raise max_connections, front the DB with a pooler (PgBouncer), or use SQL Server; or size pool_size down. Never split the store to fit the budget (ADR 0063). See docs/DEPLOY-SERVER-DB.md §3.
  • No split data store: multi-shard engine sharding now requires a server DB (ADR 0063, amends ADR 0037). messagefoundry supervise with more than one shard on a SQLite store is now refused at startup — the old SQLite-file-per-shard behavior split the message store into one database per shard, fragmenting search/reporting/audit/replay. A sharded deployment must share one unified store, so >1 shard requires [store].backend = 'postgres' or 'sqlserver' (every shard connects to the same database). A single un-sharded engine on SQLite is unaffected (byte-identical to serve). Migrating an existing SQLite-sharded deployment: drain each shard store to empty, then re-point supervise at one server DB (not an offline store merge).

0.2.12 — 2026-07-01 — Early Access

The throughput & connection-scale wave. The staged-queue per-message commit chain is shortened (opt-in inline fast-path + batch-claim, plus a result-preserving seq-only FIFO ordering that drops a per-handoff round-trip); a connection-scale measurement harness + read-only engine instrumentation lands; per-lane wake events (opt-in) eliminate the thundering-herd empty-claim storm that dominates at high connection counts; and ADR 0059's seq-only FIFO index re-key now reaches upgraded databases via a one-time on-open migration. All new runtime behavior is opt-in / off-by-default unless noted — the seq-only ordering (B3) and the index migration (B10) are result-preserving.

Added

  • Inline Step-A fast-path (ADR 0057) — opt-in per inbound via inline: for the pure all-deliver message (no filter/state/pass-through), fuse route+transform+handoff into one committed transaction, cutting the per-message commit depth from 7 to 5 durable round-trips. Off by default → byte-identical to the split path; ineligible messages fall back automatically.
  • Batch-claim (#671, ADR 0058) — opt-in via [store].fifo_claim_batch (>1): the INGRESS/ROUTED FIFO claim takes the contiguous due head-prefix in one commit instead of one row per commit, processed in strict FIFO order. Default 1 = off (byte-identical); preserves per-lane FIFO (#285) and at-least-once.
  • Per-lane wake events (#678, ADR 0061) — opt-in via [pipeline].per_lane_wake: a committed message wakes only its own (stage, lane) worker instead of every worker of that stage, eliminating the thundering-herd empty-claim storm at high connection counts (~1,500 inbounds). Default off + byte-identical; the FIFO claim and the lost-wakeup poll backstop are unchanged (a missed wake self-heals). Env override MEFOR_PIPELINE_PER_LANE_WAKE for the harness A/B.
  • Connection-scale measurement harness + read-only engine instrumentation (#675) — a headless harness that spins N inbound connections at a low per-connection rate and reads the connection-scale walls (executor saturation, server-store pool wait, idle-poll storm, FD/socket count, config-reload + ACK latency) vs connection count. The supporting engine instrumentation is additive + read-only, surfaced via /stats + /status: empty-claim counters split into idle-poll vs per-commit wake-fanout, and (on a server store) connection-pool acquire-wait percentiles + size/idle occupancy. Counters default to 0 / None — byte-identical when unused.

Changed

  • Seq-only per-lane FIFO ordering (#673, ADR 0059) — the per-lane FIFO claim now orders by the DB-assigned seq (rowid on SQLite) alone instead of (created_at, seq), and the per-insert SELECT MAX(created_at) clamp is removed from every stage handoff (one fewer round-trip per produced row). Result-preserving (proven order-isomorphic to the prior clamped ordering) and strictly more robust under clock skew / failover (seq has no wall-clock dependence). created_at stays a real ingest-time/metrics timestamp — it is simply no longer an ordering key. The FIFO covering indexes re-key to trail in seq (see the migration below).
  • Rename-based FIFO covering-index migration (#676, ADR 0060) — ADR 0059 re-keyed the per-lane FIFO indexes to trail in seq for the seq-only claim, but kept their names under IF NOT EXISTS guards, so only fresh databases adopted the new index — an upgraded DB silently kept its old created_at-trailing index and never got ADR 0059's throughput win. The seq-trailing indexes are now named ix_queue_fifo_in_seq / ix_queue_fifo_out_seq, and a one-time, idempotent on-open migration drops the old-named index and builds the new one on all three backends, so upgraded databases adopt it. Correctness is unchanged (the claim orders by seq/rowid and names no index, so the migration only restores speed). Operational notes: the first open after upgrade pays a one-time index rebuild on the queue table (SQLite/Postgres blocking, SQL Server offline — bounded by live queue depth, at cold start before serving); on a very large SQLite queue a concurrent second opener may hit a transient, non-corrupting open failure during the rebuild; the shared-DB backends (SQL Server / Postgres) should upgrade stop-the-world / under a drain window (a mixed-version fleet or a live rejoin can re-create or contend on the index); a downgrade re-creates the old-named index (drop ix_queue_fifo_in/out manually if downgrading permanently).
  • /status DB observability — the SQLite journal mode and synchronous durability setting are now surfaced in the DB status (synchronous=NORMAL remains the crash-safe-under-WAL default).

0.2.11 — 2026-06-29 — Early Access

The Plan-6 disaster-recovery + cloud/HA wave — turnkey DR backup/restore-verify and a third-tier DR standby, Kubernetes/cloud HA deployment packaging, and a frozen zero-Python Windows console installer — alongside the free-threading-keystone built-ins HL7 parser and the first SQLite durable-write group-commit lever. All on-prem and code-first; new behavior is opt-in / off-by-default unless noted.

Added

  • Turnkey DR backup + restore-verify (#60, ADR 0049) — an engine-managed scheduled/on-demand backup that bundles the loaded --config dir + a consistent SQLite store snapshot into one AES-256-GCM-encrypted .mfbak archive (chunked-AEAD, fail-closed on tamper/truncate/reorder, keyed by the existing store DEK — no new key), to an operator-set local/UNC path (no cloud target) under keep-N retention. The snapshot runs read-only off the event loop and never touches a staged-queue row; each run restore-verifies the archive (decrypt → integrity_check → row-count) and audits a PHI-free dr_backup row. New messagefoundry backup / restore-verify CLI. Off by default ([backup].enabled = false); SQLite-only (server-DB stores are DBA-delegated, backed up config-only); leader-gated under HA; a keyless PHI instance refuses to write a cleartext archive unless the audited [backup].allow_unencrypted escape is set.
  • Third-tier DR standby (#61, ADR 0048) — a right-sized disaster-recovery box that activates only when the whole active-passive HA pair/site (or its shared store) is gone, running a reduced high-priority feed set in an accepted degraded mode. Adds: a per-connection priority tier (critical/normal/low, [delivery].priority default normal + per-connection override); a startup DR run-profile ([dr]) that starts only connections at/above priority_threshold (default critical), the rest reporting status:"filtered", behind an acquire-VIP-or-abort takeover; and a cold seed from #60's encrypted .mfbak (restore-verify, local/UNC only). Activation is manual only — audited POST /dr/activate / /dr/release gated by a new dr:operate permission; activation_mode='auto' is rejected at config load. No [dr] section = a no-op, unaffected.
  • Cloud / Kubernetes HA deployment packaging (#41, ADR 0047) — packages the already-shipped active-passive HA into a copyable cloud target. Packaging + docs only — no engine code changed. Adds a Postgres-backed multi-replica k8s reference manifest (docker/k8s/ha-postgres.yaml: replicas: 3, [cluster].enabled, a PodDisruptionBudget, terminationGracePeriodSeconds > leader_lease_ttl_seconds so a drained leader releases its lease before SIGKILL, hardened securityContext, secrets via secretKeyRef) — no PVC, since durability lives in external Postgres. The default compose.yaml stays single-node SQLite; a new ha profile runs Postgres + warm standby locally. New docs/CLOUD-DEPLOYMENT.md (primary-only L4 NLB MLLP recipe; no L7/HPA for MLLP; SQL Server AG variant) and docs/CLOUD-PHI-HIPAA.md (BAA, KMS CMEK layered with the engine's own AES-256-GCM, PrivateLink). Active-passive only; demand-gated.
  • Frozen zero-Python Windows console installer (#39, ADR 0032 Phase B) — the PySide6 admin console now ships as a self-contained Windows installer (a PyInstaller --onedir freeze wrapped in an Inno Setup .exe) with Desktop/Start-Menu shortcuts and an Add/Remove-Programs uninstall entry — no Python, venv, or pip install on the box. Per-user / no-elevation by default (opt-in all-users via /ALLUSERS); this packages the console client only — the engine NSSM service and the 127.0.0.1:8765 API boundary are unchanged. Frozen from the same wheel the release publishes, by an isolated job that never reds an engine release. Authenticode signing is gated on an owner-provisioned cert — until that secret lands the installer ships unsigned (SmartScreen "Unknown publisher"). Windows-only; no MSIX/Store, no auto-update.
  • SQLite app-side group-commit committer (#64, ADR 0055) — an opt-in durable-write lever for the single-writer SQLite backend: a committer coroutine coalesces the grouped staged-queue handoffs into one commit under the writer lock, amortizing fsyncs/msg, while the claim / reference-snapshot / audit writes stay standalone and every staged-queue invariant (count-and-log, at-least-once, FIFO) is preserved. Off by default[store].group_commit_window_ms = 0.0 builds no committer and is byte-identical to today; set it (with group_commit_max_batch, default 64) to enable. The win is largest under synchronous=FULL and muted under the default NORMAL. SQLite only — the server-DB backends ignore these knobs (native concurrent-pool group-commit is a later increment); the absolute enterprise throughput figure stays pending hardware-matched measurement.
  • Background store connection-pool pre-warm (#661) — on graph start/promotion the engine fires a best-effort background task that pre-opens pooled connections on the server-DB backends (Postgres / SQL Server), so a connection burst — the post-promotion delivery workers in active-passive HA, or a cold start — finds them warm instead of paying cold connects (TCP+TLS+login). On by default via [store].warm_pool (+ warm_pool_timeout / warm_pool_target), capped to ≤ half the pool; a no-op on SQLite. Cancellation- and shutdown-safe — it never strands or hangs the engine on a failover to a dead node.
  • Single project-root config anchoring (#33-A, ADR 0050) — one opt-in --project-root (= [environments].base_dir) anchors the whole config bundle (the --config graph, environments/<env>.toml, messagefoundry.toml, and [store].path) under one root with a single precedence (explicit-absolute > project-root > CWD), so a serve launched from a non-repo CWD (the NSSM case) no longer silently reads empty env() values or creates the DB in the wrong place. Three PHI-safe startup diagnostics: a hard-fail when an explicit root + an env()-referencing graph is missing its <env>.toml, a WARNING when CWD differs from the root, and a WARNING for the NSSM silent-miss. The --project-root / --env / --service-config flags are extended to the offline validate / graph / dryrun / check subcommands (value resolution only — not serve's required-env / posture refusal), and check suppresses its messagefoundry.toml upward-walk when those flags are passed.

Changed

  • Tolerant HL7 parser re-backed by a low-allocation built-ins model (#88, ADR 0054) — the hot-path Peek/Message tolerant tier now parses over native dict/list/str instead of python-hl7, a behaviour-identical drop-in (public API, field-path semantics, escape rules, MSH-1/2 raw handling, and encode() round-trips all byte-parity-verified against python-hl7 over the golden corpus). MSH parses eagerly, other segments lazily on first field-path touch. On by default, with a per-parse python-hl7 fallback kept for this release — a contract HL7PeekError still raises and dead-letters, while an unexpected internal error falls back to python-hl7 and is logged, never crashing a connection. The free-threading keystone for ADR 0053 and a large single-thread parse win; the strict hl7apy validate() tier and parse_tree / RawMessage are untouched. python-hl7 stays a dependency for the fallback window (removal is a follow-up release).
  • A set project root (--project-root or [environments].base_dir) now anchors the store DB too, not just environments/. A deployment that runs serve/supervise with a project root and a relative --db / [store].path (or relies on the default relative messagefoundry.db) will now find/create the DB under the root instead of the process CWD — including each shard's <stem>_<shard>.db. --project-root additionally anchors a relative --config / --service-config (a file-only [environments].base_dir anchors the DB + env values but not those two, since they are resolved before the settings load). This is the intended fix for the split-store footgun, but it relocates an existing relative DB: pass an absolute [store].path / --db to keep the DB where it is (absolute paths bypass the root), or accept the new location. Deployments with no project root, or with an absolute DB path, are unaffected. The new CWD-mismatch WARNING surfaces any move at startup.

0.2.10 — 2026-06-27 — Early Access

The Plan-5 "v0.3 candidate" wave — completing the deferred connector/codec set and the Corepoint parity gaps, built across two multisession waves (L1–L9) and adversarially reviewed. All on-prem, code-first, no behavior change to existing graphs.

Added

  • Inbound HTTP / REST listener (#7, ADR 0023) — a connector-owned bound asyncio HTTP/1.1 socket source in transports/ (not api/), feeding the payload-agnostic ingress (ADR 0004) as a RawMessage. ACK-on-receipt (respond-with-receipt after the raw is durably committed), with oversize/malformed/slow-loris hardening surfaced as connection_events; new ConnectorType.HTTP. The substrate for the future inbound FHIR facade (#20) / DICOMweb receiver (#24). (SOAP-envelope sync-reply, intake-socket auth, and method/path routing metadata are deferred follow-ons.)
  • fhir_lookup(connection, query) (#58, ADR 0043) — a Handler-callable, read-only FHIR read/search that extends the ADR 0010 db_lookup carve-out to FHIR: off the event loop, raises on a Router / in dry-run, reuses the SMART Backend bearer (ADR 0024) + [egress].allowed_http, GET-only.
  • Email / SMTP outbound destination (#23, ADR 0029) — a stdlib Email()/SMTP() connector; STARTTLS-by-default, AUTH-over-TLS-only, a new deny-by-default [egress].allowed_smtp arm. (IMAP/POP read + XOAUTH2 is a deferred Phase 2.)
  • X12 strict implementation-guide validation via pyx12 (#32) behind the tolerant X12Peek/X12Message hot path (messagefoundry[x12]; yields 997/999 acks), and a structured [xml] codec layer (#31) — XmlMessage (XPath read/set + ns-aware re-encode) over hardened lxml + optional xmlschema/signxml (messagefoundry[xml]; XXE / entity-expansion / external-DTD refused).
  • Operator alert-state (#56, ADR 0044) — a new alert_instance store table (open / acknowledged / resolved + first/last-seen + count) across all three backends, de-duped on the ADR-0014 throttle key; GET /alerts/active + ack/resolve (RBAC MONITORING_DIAGNOSE); the per-connection alerts_active count is now real; a console Alerts tab. Metadata-only.
  • User-definable custom RBAC roles (#57, ADR 0045) — an admin-defined named role = a chosen subset of the existing Permission catalog (no new permission kinds), persisted via an additive roles migration (3 backends), gated by USERS_MANAGE; the six built-ins stay; narrowing revokes on live sessions.
  • Message-content search (#51, ADR 0046) — HL7 field-path / raw-content matching by scan-and-decrypt-per-row (the store is AES-GCM-encrypted at rest, so a plain LIKE is impossible): metadata-pre-filtered, hard row/result caps (truncate-and-tell), decrypt off the event loop, behind messages:view_* + step-up + a message_search audit row that never logs the search needle.
  • HL7 timestamp helpers on Message (#59) — age-from-DOB, length-of-stay, and the tolerant HL7-TS parse surfaced on the Message API (reusing timezone.py; no duplicate parser).
  • messagefoundry support-bundle CLI (#49) — a PHI-safe diagnostic zip (no message bodies, no secrets; redacted log tail) — and a zero-egress version update-check (#30, ADR 0026): a no-network pinned-vs-current diff surfaced as a /status field + an update_available alert + a console banner (on by default; mode=live rejected at load).

Changed

  • [egress] gains allowed_smtp (email); the read-only-lookup carve-out (CLAUDE.md §2/§8) now names fhir_lookup alongside db_lookup.
  • New connectors/codecs are documented in docs/CONNECTIONS.md and the update-check in docs/CONFIGURATION.md ([update_check]).

Security

  • All new live-lookup / search paths stay on-prem and gated: fhir_lookup and the update-check are zero-/ allow-listed-egress; content search is step-up-gated + audited and never weakens at-rest encryption (the cleartext key-field index was declined; a keyed-token index is a deferred 2nd slice). New crypto sites (transports/http_listener.py TLS) are registered in the ASVS-11.1.3 crypto inventory.

Dependencies

  • New optional extras only: messagefoundry[x12] (pyx12) and messagefoundry[xml] (lxml/xmlschema/signxml); the base install is unchanged. Lockfile re-exported.

0.2.9 — 2026-06-27 — Early Access

A retention + security-hardening + observability release: per-connection retention and embedded-document pruning windows, dual-control config reloads with startup code-attestation, operational-health metrics, and a fix for the intermittent Windows listener-teardown / CI hang.

Added

  • Per-connection retention windows (ADR 0027). Optional messages_days (inbound) and dead_letter_days (outbound) on a connection, layered over the global [retention] window and authored on the connection spec or connections.toml (the same override idiom as the delivery knobs): None inherits the global window, 0 keeps forever. The RetentionRunner threads a per-connection cutoff through the body and dead-letter purge on all three store backends; the never-purge-an-in-flight-body guard and the single per-pass audit row (now recording the overrides) are unchanged. (#34)
  • Embedded-document pruning (ADR 0042). Optional prune_documents_after (+ a size threshold) per inbound connection: after the window, bulky base64 embedded documents — HL7 OBX-5 ED and the generic mfb64:v1: carriage — are stripped in place to a small size/content-type tombstone (via the parsed model / codec, never string-slicing HL7), keeping the rest of the message parseable; the row is never deleted and a documents_pruned flag is set. All three backends. (The ingest-time offload variant remains deferred.) (#47)
  • Dual-control config:reload (ADR 0041 D2). config_reload is now a gateable [approvals].operations op — a distinct second approver must release a live config reload (the requester can never self-approve; both identities land in the hash-chained audit). Opt-in / deny-by-default, so single-operator deployments are unchanged. (#53)
  • Startup code self-attestation (ADR 0041 D3). At startup the engine hashes its loaded modules against the wheel's dist-info/RECORD; on drift it records a hash-chained, off-box-teed startup_integrity audit row and raises an alert (alert-only by default; opt-in [integrity].fail_closed_on_drift refuses to start). A no-op on an editable (pip install -e .) install, so development is never bricked. (#54)
  • Operational-health metrics. GET /status now meters the app-log directory's disk usage alongside the database, and a per-connection message-stall alert rule fires when a connection's oldest-undelivered age crosses a configurable threshold. (#50)

Changed

  • Non-editable, hash-locked wheel is the enforced production default (ADR 0017 amendment). The prior recommendation is now the default for production deployments; editable installs remain a no-op for development. (#54)

Fixed

  • Intermittent Windows listener-teardown hang. MLLPSource / TcpSource / X12Source no longer await server.wait_closed() / writer.wait_closed() unbounded on the Windows Proactor loop during teardown — a wait that never completes can no longer stall a shared event loop (the same class as the resolved py3.11 hang). Added CI guards (a per-test faulthandler stack dump and a step-level watchdog) so a future hang fails fast and names itself instead of silently timing out. (#55)

Docs

  • Refreshed benchmarks/TUNING-BASELINE.md with measured multi-process sharding throughput from the Windows Server 2025 box (η ≈ 0.85 speedup shape; per-shard E_core ≈ 42 msg/s — a test-box SQLite floor), plus the still-unmeasured hardware-gated follow-ups (enterprise E_core, the shared-DB commit-wall sweep). (#28, #29)
  • Authored ADR 0027 (per-connection retention) and ADR 0042 (embedded-document pruning); added EARS acceptance criteria to ADR 0041 D2/D3; amended ADR 0017 for the enforced wheel.

CI

  • Locked the smoke job's config directory (#603) and skipped a mirror-only Dependabot guardrail test on the OSS mirror (#606), greening main CI post-0.2.8.

0.2.8 — 2026-06-27 — Early Access

A tooling/ops release: the load harness gains a multi-shard driver so one harness can drive a supervise cluster (unblocking the multi-core throughput measurement), supervise resolves --env files for its shards, and a prominent upgrade note for the config-directory permission guard introduced in 0.2.6.

⚠ Upgrading from ≤ 0.2.5 — tighten config-dir ACLs first

The config-directory permission guard (SEC-003 / ADR 0036), added in 0.2.6, refuses to load a --config directory that is writable by a broad principal (e.g. Authenticated Users / S-1-5-11). A deployment whose config dir inherits that write — common under C:\srv\… — will fail to start on first upgrade to ≥ 0.2.6 with "refusing to load config from writable-by-others path …". Before upgrading, tighten the directory (elevated):

icacls "<config-dir>" /inheritance:d /T
icacls "<config-dir>" /remove:g *S-1-5-11 /T          # drop Authenticated Users
icacls "<config-dir>" /grant *S-1-5-18:(OI)(CI)F /grant *S-1-5-32-544:(OI)(CI)F /T  # SYSTEM + Admins

See docs/SERVICE.mdUpdate to a new build and Lock down the config directory (CONFIG-2).

Added

  • Multi-shard load driving (messagefoundry-harness). python -m harness gains --skip-preflight (drive shard MLLP ports that no single --engine owns) and a repeatable --shard-engine <url>: the engine poller now takes a list of shard APIs and sums each shard's /stats (read/written/backlog/in_pipeline/queue_depth/dead) into one cluster sample, so the no-loss reconcile and drain are cluster-aggregate — a healthy K-shard run reports pass, not a false "lost on intake". With no --shard-engine the behavior is byte-identical to before. Two sample graphs ship for the throughput suite: harness/config/store_once (the dedup-triggering one-handler-list[Send]-of-identical-body shape for store-once) and harness/config/passthrough (an internal PassThrough() re-ingress hop); the load graph (harness/config/load) is now shard-taggable via MEFOR_LOAD_SHARD_ADT/_RESULTS/_OTHER. (#604)

Fixed

  • supervise --project-root. supervise now accepts --project-root and forwards it to each spawned serve --shard, so supervise --config <dir> --env <env> resolves each shard's environments/<env>.toml (previously the shards resolved nothing from their spawned cwd and required an explicit --service-config posture). Backward compatible — no --project-root is unchanged. (#602)

0.2.7 — 2026-06-27 — Early Access

A docs/packaging release that fixes the broken badge images on the PyPI project page and adds a config-check pre-commit hook.

Fixed

  • Broken badge images in the PyPI project description. The CI and Security status badges in the README pointed at the private source repo, so they rendered as broken images on the public PyPI page — an anonymous viewer can't fetch a private repo's GitHub Actions badge SVG (it 404s). The README now points at the public mirror (MEFORORG/MessageFoundry), and the release build additionally rewrites any remaining wshallwshallMEFORORG repo slug in the README before it is embedded as the PyPI long_description, so the rendered badges resolve anonymously. (#568)

Added

  • messagefoundry check pre-commit hook. A VS Code-extension-generated .mefor-hooks/pre-commit runs messagefoundry check so a commit can't introduce a broken config (skips cleanly if python or the package isn't importable; bypass with --no-verify). (#568)

Docs

  • Backlog #47 — base64 embedded-document (attachment) pruning (Mirth attachment-handler / data-pruner parity); and a Changelog link in the README. (#568)

0.2.6 — 2026-06-27 — Early Access

A large release: the throughput-maximization build (high-fan-out store-once, multi-process sharding, and internal pass-through connectors with full Postgres/SQL Server parity), a console + IDE "fleet" tier for managing multiple engine shards, and a broad security-hardening wave from the 2026-06 audit.

Added

  • Multi-process sharding (L3). An inbound connection can carry an optional shard tag; serve --shard <id> runs an engine process that owns only that shard's inbound connections (outbound + routing/handlers are shared), and a new supervise command spawns, monitors, and restarts one serve subprocess per shard (each with its own SQLite db file and API port). Per-connection sharding parallelizes intake across CPU cores; per-channel FIFO is preserved within a shard. (#584)
  • Internal pass-through (PT) connectors (L4). A Handler may Send into an internal PassThrough() inbound that carries its own router; the message re-ingresses as a new content-addressed child message inside the same transaction (at-least-once, count-and-log, and single-finalizer authority all preserved), bounded by a correlation-depth loop guard. This generalizes the ADR 0013 re-ingress primitive. Implemented on all three store backends — SQLite, plus full Postgres and SQL Server parity for the atomic re-ingress. (#585, #590)
  • Store-once-deliver-many (L2b). A high-fan-out outbound now stores the message body once (content-addressed, reference-counted shared_body) instead of once per destination; single-destination delivery is unchanged (inline, byte-identical). (#580)
  • Fleet tier — manage multiple engine shards. The console can register and switch between multiple engine endpoints (#582); the IDE promote flow can target a specific engine instance/shard (#583).
  • IDE editor productivity. A MessageFoundry build toolbar + CodeLens on config files (#593), an "Insert Element" quick-pick with expanded transform-idiom snippets (#595), a Wizards group with collapsible Home groups (#578), and a vsce VSIX packaging script (#577).
  • Config-fingerprint attestation. Config reloads record a config fingerprint in the reload audit (ADR 0041 load-path attestation). (#597)

Changed

  • Faster fan-out. On a fan-out the engine parses the per-message payload once where it is value-identical, avoiding redundant re-parsing. (#581)

Fixed

  • Fail-fast pass-through guard. A graph with a PT inbound on a store backend that does not implement PT re-ingress is now rejected at startup and on reload/dry-run (a clear configuration error, HTTP 422) — before any listener binds — instead of failing at the first Send. (#587)
  • Auth hardening. Tighter field-level authorization, a last-admin guard, a corrected TOTP window, and rate-limit documentation fixes. (#563)
  • API / store. Channel-scoped event and topology reads, faster WebSocket session revocation, and atomic bootstrap-secret creation. (#565)
  • IDE. Workspace-trust gating, machine-scoped promote targets, and a fail-closed AI-assist policy (SEC-004/005/022). (#561)

Security

The 2026-06 security-audit remediation wave (in-repo remediation ledger, #566):

  • Transport TLS / SSRF / injection: FTPS TLS verification, an FHIR-path SSRF guard, and read-only enforcement on db_lookup (SEC-001/010/009). (#560)
  • Listener hardening: a cleartext-bind guard plus source-IP allowlist for the raw-TCP/X12 listeners. (#558)
  • DICOM: fail-closed C-STORE SCP peer controls (calling-AE + peer-IP) and a passphrase-key callback (SEC-012/016). (#559)
  • Pipeline: off-event-loop router/transform execution and a non-HL7 ingress size cap (SEC-013/017). (#562)
  • Config trust: enforce Windows config-source trust and scope the sibling-helper finder (SEC-003/019). (#564)
  • PHI redaction: narrowed a free-text PHI residual and added an advisory raise-fstring lint (SEC-023). (#557)
  • Supply chain: Dependabot security-track guardrails and adopter-scaffold hash-pinning. (#556)
  • Static analysis: resolved two real CodeQL findings (webview HTML attribute escaping; owner-only file-delivery fallback) (#554) and adopted a CodeQL triage policy + accepted-risk register (ADR 0034). (#567)

Docs

  • ADRs 0037–0040 record the throughput-build decisions (multi-process sharding, pass-through connectors, the shelved L5 DB-sharding design, and the not-adopted free-threading assessment) (#591); design notes for L5 DB-sharding (#588) and cp314t readiness (#589); and the Secure AI-Assisted Development Standards updated with the audit lessons (#576).

0.2.5 — 2026-06-26 — Early Access

A bug-fix release hardening SQL Server cluster cold-start.

Fixed

  • SQL Server: concurrent schema-init race on a virgin DB (HA cold start). Two cluster nodes starting simultaneously against an empty database both ran the IF OBJECT_ID(...) IS NULL CREATE TABLE guards with no cross-node lock, so both issued CREATE and the loser died at startup on a 2714 ("There is already an object named ..."). _ensure_schema now takes an exclusive sp_getapplock (mefor:schema_init) around the DDL — the T-SQL analog of the PostgreSQL store's existing schema advisory lock — so the second node serializes and runs the now-no-op guarded CREATEs cleanly. Single-node and pre-created schema are unaffected; SQLite and PostgreSQL were already race-safe. (#553)

Changed

  • Docs: the [cluster] settings docstring and the pool-size validation error now name both postgres and sqlserver (the cross-section validator already admitted both). (#553)

0.2.4 — 2026-06-26 — Early Access

A bug-fix release that completes the EF-6 SQL Server fix shipped in 0.2.3.

Fixed

  • SQL Server: EF-6 "Connection is busy with results for another command" fully resolved (0.2.3's fix was incomplete). v0.2.3 (#543) switched the FIFO claim read to fetchall, but draining the UPDATE...OUTPUT rows does not free the statement handle — without MARS the pooled connection was still returned to the aioodbc pool busy, so the error reproduced at every cold start. All pooled cursor sites now close the cursor (SQLFreeStmt/SQLCloseCursor) via a new _cursor context manager before the connection is released, on both the success and exception paths; claim_ready (another UPDATE...OUTPUT) and the DELETE...OUTPUT handoffs had the same latent gap and are covered too. A driver-free unit test now asserts the close-before-release invariant so the regression can't recur. SQLite and PostgreSQL were unaffected. (#550)

0.2.3 — 2026-06-26 — Early Access

A bug-fix + feature release: the SQL Server store no longer raises "connection busy" errors under concurrent load, plus connection/transport event logging, GUI-managed translation tables, and inbound listener port-conflict detection.

Fixed

  • SQL Server: "Connection is busy with results for another command" under concurrent load (EF-6). claim_next_fifo — and three sibling sites (_maybe_finalize, consume_recovery_code_hash, consume_totp_step) — read a result-set-returning statement with a lone fetchone() and could return the pooled connection to the pool with the result set still pending, so the next borrower's first command raced an HY000 busy error (ODBC Driver 18, no MARS). All affected sites now fully drain the result set (fetchall) before commit/release. SQLite and PostgreSQL were unaffected (asyncpg materializes rows; SQLite has no shared pooled-connection single-result-set constraint). (#543)

Added

  • Connection/transport event log + "Response Sent" ACK capture (ADR 0020 / ADR 0021). A new id-keyed, metadata-only connection_event table records inbound connection lifecycle, pre-ingress failures, and outbound lane transitions, with a [diagnostics] config block (per-connection overrides + retention), a GET /events read API, and a console Event Log page. Event reasons are scrubbed and encrypted at rest. (#541)
  • GUI-managed translation tables (code sets) (ADR 0033). A code-set CLI + writer and a VS Code extension grid editor / Translation Tables view for maintaining code-set mappings. (#540)
  • Inbound listener port-conflict detection — static + runtime checks that flag two inbound connections bound to the same host:port before they collide at startup. (#538)

Changed

  • Docs: README install instructions are now version-agnostic and link the website docs; the roadmap section is replaced with a features summary. (#542, #544)

0.2.2 — 2026-06-24 — Early Access

A security-hardening release: PHI-at-rest encryption is closed across every backend, the active-passive cluster gains a store-checked split-brain fence, outbound delivery is effectively-once, and the at-rest cipher becomes crypto-agile — all additive, with the on-disk mfenc:v1 format byte-identical.

Changed

  • BREAKING — Python 3.14 is now the only supported runtime. requires-python is raised to >=3.14 (was >=3.11), and the ruff/mypy targets, CI matrix (Linux + Windows Server 2022/2025, all on 3.14), Docker base image, lockfiles, and adopter scaffold move with it. Adopters and engine hosts must be on Python 3.14 — a 3.11/3.12/3.13 host will refuse to install the wheel. The 3.11/3.12/3.13-specific test apparatus is retired with this change (the MEFOR_PY311_QUARANTINE conftest lever, the py3.11 store soak CI job, and scripts/soak/store_soak.py; the underlying BACKLOG #17 asyncio↔aiosqlite concern is still mitigated by the shared session loop in pyproject.toml).

Security

  • PHI-at-rest encryption closed across all three backends. The patient summary (MRN + name) and metadata columns are now encrypted at rest (previously cleartext even with encryption enabled), and the SQL Server error / last_error / message_events.detail columns are brought to parity with SQLite and Postgres — every cipher column is now AES-256-GCM at rest. Coverage is surfaced by a new authenticated, audited GET /security/posture route (reports the active-key fingerprint + per-backend column coverage; never key bytes).
  • Fail-closed for PHI without a key. An instance declared data_class = phi now refuses to start without an encryption key (previously it started in cleartext with a warning), unless explicitly overridden by the new, audited [store].allow_unencrypted_phi.
  • Crypto-agility marker (additive). The at-rest cipher marker is now version/algorithm-aware (mfenc:v2:<alg>:…) so a future algorithm can be introduced without a data migration. The mfenc:v1 format is byte-identical and AES-256-GCM remains the only algorithm; decryption fails closed on an unknown marker version or algorithm.
  • Database-TLS hardening. A new [store].ssl_root_cert pins a private database CA (Postgres), with machine-store CA-import and certificate-rotation operator runbooks. The DPAPI key file's ACL now grants the service account read access without broadening exposure.

Added

  • Active-passive split-brain fence. A monotonic leader-epoch fencing token on the leadership lease, validated inside the FIFO claim transaction, so a superseded or paused ex-leader that resumes is fenced out (it claims nothing) — backed by continuous "at most one leader" SLO checks and a real-handover failover test. SQLite (single-node) behavior is unchanged.
  • Effectively-once outbound delivery. A same-transaction idempotency ledger skips re-delivery of an already-delivered message after a failover or crash-recovery re-claim, without re-ordering a lane; an operator-initiated replay still re-sends.
  • Pre-side-effect leadership re-checks so a node that loses leadership between claiming and sending re-queues the work rather than emitting it as a stale leader.
  • messagefoundry verify --check-disposition for post-deploy disposition validation.

Fixed

  • CycloneDX SBOM generation on Python 3.14.
  • PyPI long-description rendering (version pins, links).
  • De-flaked several intermittent CI tests (failover-load timeouts, a harness server port-bind race, the startup fault-isolation recovery assertion, and the docker-smoke shutdown-marker check).

0.2.1 — 2026-06-23 — Early Access

Fixed

  • Windows: messagefoundry --help crashed on a legacy codepage — the top-level help rendered a non-cp1252 character (a -> arrow in the adr-analyze subcommand help, new in 0.2.0), so --help aborted with UnicodeEncodeError on a cp1252/charmap console (cmd, PowerShell, or any redirected stdout). main() now reconfigures stdout/stderr with errors="replace" and the help text is ASCII; the machine-read JSON introspection subcommands are unaffected (json.dumps(ensure_ascii=True)).
  • verify --section host crashed without the [console] extracheck_console_no_window() resolved a console submodule via find_spec, which imported the console package and its eager httpx dependency, so a [sqlserver]-only install aborted with ModuleNotFoundError: No module named 'httpx' instead of skipping the console check. The console package now imports its API client lazily (PEP 562 __getattr__), so resolving a submodule no longer requires httpx, and the check degrades to SKIP if a console dependency is absent.

0.2.0 — 2026-06-23 — Early Access

Added

  • One-click console launch — a windowed messagefoundry-console launcher ([project.gui-scripts], no flashing console window) carrying the MessageFoundry badge as the window/taskbar icon, plus scripts/console/install-console-shortcut.ps1 to drop Desktop / Start-Menu shortcuts (per-user, or -AllUsers for machine-wide). Operators open the admin console by double-clicking an icon instead of running a Python command. See ADR 0032.
  • SQL Server 2025 support — the SQL Server store + Database connector are now validated against SQL Server 2025 (17.x) in addition to 2022 (16.x): both majors are exercised by the gated CI legs (store, coordinator, failover, and load smoke). No schema or T-SQL change was needed — ODBC Driver 18 (18.5+) covers both. The supported-version matrix moves from 2019/2022 to 2022/2025. Note: SQL Server 2025 requires an AVX-capable CPU.

Security

  • Dependency fast-response program — a KEV→EPSS→CVSS triage policy with a ≤72h fast lane for actively-exploited dependency CVEs (.github/SECURITY.md, docs/security/DEP-CVE-RUNBOOK.md); a daily SCA cron; Dependabot moved to the native uv ecosystem with automatic hashed-lock re-export; scoped auto-merge of safe patches with a supply-chain cooldown; weekly RV.2 metrics (docs/security/DEPENDENCY-METRICS.md); and an adopter remediation SLA + advisory process (docs/SUPPORT-POLICY.md, docs/security/ADVISORY-PROCESS.md).
  • Adopter "vulnerable pin" tripwiremessagefoundry init's scaffolded CI gains an audit-pin job that reds an adopter's build when their pinned engine or its dependencies have a known published advisory (docs/ADOPTER-CI.md).
  • Release-sync drift guard — a tag/PyPI/public-mirror version-consistency tripwire + a publish-time version guard, so the git tag, the PyPI wheel, and the OSS mirror can't silently diverge.

0.1.0 — 2026-06-18 — Early Access

First public Early Access release: the feature set is complete and validated by the project's own tests, but the external code review + penetration test (the bar for a security-certified v1.0) happen after launch — so this is not yet "GA / independently security-reviewed". See docs/EARLY-ADOPTER-GUIDE.md.

Added

  • Engine + staged pipeline — code-first Connection / Router / Handler model on a durable staged queue (ingress → routed → outbound) with at-least-once handoff, retry/backoff, dead-letter, and replay. Count-and-log: every received message is persisted with its disposition before the ACK.
  • Transports — MLLP and File (source & destination); REST, SOAP, and Database destinations; a Database poll source. Payload-agnostic ingress (HL7 v2.x by default; JSON / XML-SOAP / X12 / DB records).
  • Server-DB store backends (production) — PostgreSQL and Microsoft SQL Server, alongside the zero-config single-node SQLite (WAL) default. Byte-identical single-node behaviour on every backend.
  • Active-passive high availability — self-fencing leadership lease, leader-gated message graph, claim-time per-lane FIFO across nodes, cross-node convergence, and read-only /cluster/* observability (surfaced as a leader/role/lease + node-roster view on the console's Engine Status page), on both PostgreSQL and SQL Server. A two-node failover-load test harness (SIGKILL-the-primary under load) proves recovery + no acknowledged loss + preserved per-lane ordering.
  • Security — authentication + RBAC (local and AD: LDAP/Kerberos), deny-by-default per-route permissions, opaque sessions, a user-attributed tamper-evident (hash-chained) audit log, AES-256-GCM body encryption at rest with key rotation, native transport TLS (API HTTPS/WSS + MLLP-over-TLS) with an off-loopback bind guard and a certificate-expiry monitor, deny-by-default egress controls, PHI log redaction, and a centrally-governed, PHI-safe AI-assist policy.
  • Operability & tooling — a localhost HTTP/WebSocket API; a PySide6 admin console; the messagefoundry CLI (serve / validate / graph / dryrun / check / connection / generate / …); a VS Code extension (setup, promote, test bench); a headless load + failover test harness; and a published throughput + active-passive failover baseline (docs/benchmarks/TUNING-BASELINE.md).
  • Alerting — a logging sink plus a webhook/email notifier; queue-buildup and certificate-expiry alerts.
  • Deployment — runs as a Windows service via NSSM; a channel × TLS-posture deployment matrix (docs/DEPLOYMENT.md); a staged Lab → Shadow → Limited → Full early-adopter guide.

Notes

  • Throughput is hardware-dependent (a durable-write-bound path); the published numbers are "as measured on a reference config", not a guarantee — re-run the method on your hardware. See docs/benchmarks/TUNING-BASELINE.md.
  • Releases are built, SBOM'd (CycloneDX), and signed with Sigstore — see the release workflow.