M5 W7: composed E2E, fault, concurrency, and SLO gates

data/m5_baseline_defects.json

@@ -2,7 +2,5 @@
   "schema_version": 1,
   "baseline_commit": "8001643fe503254ed9f12f0996894a49efe52342",
   "contract": "A probe passes only when its positive control works and the named baseline defect is behaviorally reproduced.",
-  "defects": [
-    {"id": "M5-DEF-11", "owner_wave": "W7", "test": "tests/test_m5_baseline_defects.py::test_defect_latency_harness_rejects_chunked_readback", "expected_reason": "latency_harness_expects_immediate_quote_number"}
-  ]
+  "defects": []
 }

data/m5_e2e_matrix.json

@@ -0,0 +1,104 @@
+{
+  "schema_version": 1,
+  "contract": "Every row has an executable production-ASGI boundary and focused positive/fault oracles. E2E-16 here is only the offline surrogate; real-phone proof is an external W8 gate.",
+  "cases": [
+    {
+      "id": "E2E-01",
+      "production_node": "tests/test_m5_e2e_composed.py::test_e2e01_spoken_order_to_guarded_provider_acceptance_and_download",
+      "positive_nodes": ["tests/test_quote_worker.py::test_untampered_positive_control_delivers_exactly_once"],
+      "fault_nodes": ["tests/test_quote_worker.py::test_fail_closed_guard_called_once_blocks_and_never_delivers"]
+    },
+    {
+      "id": "E2E-02",
+      "production_node": "tests/test_m5_e2e_composed.py::test_e2e02_03_natural_edits_pending_quantity_and_no_default",
+      "positive_nodes": ["tests/test_order_integration.py::test_multiline_order_ordinal_reference_targets_the_right_line"],
+      "fault_nodes": ["tests/test_order.py::test_attribute_reference_matching_multiple_lines_disambiguates"]
+    },
+    {
+      "id": "E2E-03",
+      "production_node": "tests/test_m5_e2e_composed.py::test_e2e02_03_natural_edits_pending_quantity_and_no_default",
+      "positive_nodes": ["tests/test_order_integration.py::test_discriminating_confirmation_reply_confirms_the_line"],
+      "fault_nodes": ["tests/test_order_integration.py::test_discriminating_confirmation_denial_withdraws_the_phantom_line"]
+    },
+    {
+      "id": "E2E-04",
+      "production_node": "tests/test_m5_e2e_composed.py::test_e2e04_05_all_sizes_require_readback_and_edit_restarts_chunks",
+      "positive_nodes": ["tests/test_m5_baseline_defects.py::test_short_order_requires_readback_then_distinct_assent"],
+      "fault_nodes": ["tests/test_order_integration.py::test_assent_refused_on_empty_order"]
+    },
+    {
+      "id": "E2E-05",
+      "production_node": "tests/test_m5_e2e_composed.py::test_e2e04_05_all_sizes_require_readback_and_edit_restarts_chunks",
+      "positive_nodes": ["tests/test_order.py::test_chunk_lines_respects_speech_duration_budget"],
+      "fault_nodes": ["tests/test_conversation_store.py::test_readback_and_assent_receipts_bind_exact_chunk_coverage"]
+    },
+    {
+      "id": "E2E-06",
+      "production_node": "tests/test_m5_e2e_composed.py::test_e2e06_authoritative_replay_and_fork_are_idempotent_and_fail_closed",
+      "positive_nodes": ["tests/test_runtime_conversation_durability.py::test_agent_restart_restores_pending_order_and_replay_is_exact"],
+      "fault_nodes": ["tests/test_runtime_conversation_durability.py::test_history_regression_fork_gap_and_agent_rebinding_fail_closed"]
+    },
+    {
+      "id": "E2E-07",
+      "production_node": "tests/test_m5_e2e_composed.py::test_e2e07_revision_is_same_number_fresh_n_plus_one",
+      "positive_nodes": ["tests/test_quote_store.py::test_atomic_revision_commits_new_row_and_supersedes_prior_together"],
+      "fault_nodes": ["tests/test_quote_store.py::test_atomic_revision_faults_leave_original_active_and_no_descendant"]
+    },
+    {
+      "id": "E2E-08",
+      "production_node": "tests/test_m5_e2e_composed.py::test_e2e08_cross_account_reference_is_neutral_and_read_only",
+      "positive_nodes": ["tests/test_spoken_quote_reference.py::test_latest_quote_is_store_resolved_then_discriminatingly_confirmed"],
+      "fault_nodes": ["tests/test_spoken_quote_reference.py::test_suffix_cannot_disclose_foreign_tenant_or_account"]
+    },
+    {
+      "id": "E2E-09",
+      "production_node": "tests/test_m5_e2e_composed.py::test_e2e09_10_atomic_lineage_and_two_api_writers_have_no_orphans",
+      "positive_nodes": ["tests/test_quote_store.py::test_atomic_issue_commits_quote_lineage_and_both_outboxes_once"],
+      "fault_nodes": ["tests/test_quote_store.py::test_atomic_issue_faults_roll_back_every_binding_row_and_never_reuse_number"]
+    },
+    {
+      "id": "E2E-10",
+      "production_node": "tests/test_m5_e2e_composed.py::test_e2e09_10_atomic_lineage_and_two_api_writers_have_no_orphans",
+      "positive_nodes": ["tests/test_runtime_conversation_durability.py::test_concurrent_hosted_callers_never_share_order_state"],
+      "fault_nodes": ["tests/test_quote_store.py::test_concurrent_revision_cas_allows_one_descendant"]
+    },
+    {
+      "id": "E2E-11",
+      "production_node": "tests/test_m5_e2e_composed.py::test_e2e11_12_guarded_customer_bytes_and_capability_refusals_share_route",
+      "positive_nodes": ["tests/test_quote_artifact_layout.py::test_maximum_layout_has_full_parity_and_printable_page_bounds"],
+      "fault_nodes": ["tests/test_quote_renderers_golden.py::test_every_customer_visible_fact_is_individually_guarded_in_both_formats"]
+    },
+    {
+      "id": "E2E-12",
+      "production_node": "tests/test_m5_e2e_composed.py::test_e2e11_12_guarded_customer_bytes_and_capability_refusals_share_route",
+      "positive_nodes": ["tests/test_quote_retrieval_route.py::test_http_retrieval_is_neutral_and_serves_only_persisted_guarded_bytes"],
+      "fault_nodes": ["tests/test_quote_artifacts.py::test_capability_refusal_matrix_is_neutral"]
+    },
+    {
+      "id": "E2E-13",
+      "production_node": "tests/test_m5_e2e_composed.py::test_e2e13_missing_contact_blocks_before_quote_with_success_control",
+      "positive_nodes": ["tests/test_m5_w5_delivery_worker.py::test_completion_gate_atomic_leases_and_two_channel_cardinality"],
+      "fault_nodes": ["tests/test_m5_w5_delivery_worker.py::test_crash_before_send_retries_but_after_send_boundary_requires_manual_review"]
+    },
+    {
+      "id": "E2E-14",
+      "production_node": "tests/test_m5_e2e_composed.py::test_e2e14_stale_source_refuses_through_asgi_boundary",
+      "positive_nodes": ["tests/test_source_snapshots.py::test_fresh_source_time_is_preserved_not_replaced_with_gateway_now"],
+      "fault_nodes": ["tests/test_source_snapshots.py::test_stale_price_snapshot_cannot_be_spoken"]
+    },
+    {
+      "id": "E2E-15",
+      "production_node": "tests/test_m5_e2e_composed.py::test_e2e15_restart_recovers_issued_work_without_rerendering_in_call",
+      "positive_nodes": ["tests/test_production_controls.py::test_sqlite_hardening_online_encrypted_backup_and_clean_restore"],
+      "fault_nodes": ["tests/test_production_controls.py::test_backup_tamper_and_nonempty_restore_fail_closed"]
+    },
+    {
+      "id": "E2E-16",
+      "production_node": "tests/test_m5_e2e_composed.py::test_e2e16_offline_hosted_surrogate_counts_as_preproduction_only",
+      "positive_nodes": ["tests/test_voice_stream.py::test_stream_bridge_runs_a_gateway_turn_with_readback_gate"],
+      "fault_nodes": ["tests/test_gateway_voice.py::test_degraded_transcript_routes_to_readback_not_silent_id"],
+      "claim": "offline_surrogate_only",
+      "external_evidence_required": true
+    }
+  ]
+}

data/m5_w7_fault_matrix.json

@@ -0,0 +1,25 @@
+{
+  "schema_version": 1,
+  "claim": "accelerated_offline_fault_and_recovery_matrix; wall_clock_24h_staging_soak_requires_external_P12_environment",
+  "faults": [
+    {"transition": "conversation snapshot/event/turn commit", "node": "tests/test_conversation_store.py::test_event_write_failure_rolls_back_snapshot_and_processed_turn"},
+    {"transition": "assent/quote/lineage/outbox commit", "node": "tests/test_quote_store.py::test_atomic_issue_faults_roll_back_every_binding_row_and_never_reuse_number"},
+    {"transition": "revision/supersede/outbox commit", "node": "tests/test_quote_store.py::test_atomic_revision_faults_leave_original_active_and_no_descendant"},
+    {"transition": "two API writers", "node": "tests/test_m5_e2e_composed.py::test_e2e09_10_atomic_lineage_and_two_api_writers_have_no_orphans"},
+    {"transition": "simultaneous authenticated hangups", "node": "tests/test_m5_e2e_composed.py::test_e2e10_simultaneous_authenticated_hangups_serialize_store_transactions"},
+    {"transition": "two worker leases", "node": "tests/test_m5_baseline_defects.py::test_artifact_queue_claim_is_exclusive"},
+    {"transition": "artifact format transaction", "node": "tests/test_quote_artifacts.py::test_artifact_transaction_fault_has_no_partial_format"},
+    {"transition": "document parity tamper", "node": "tests/test_quote_worker.py::test_fail_closed_guard_called_once_blocks_and_never_delivers"},
+    {"transition": "provider pre-send/post-send crash", "node": "tests/test_m5_w5_delivery_worker.py::test_crash_before_send_retries_but_after_send_boundary_requires_manual_review"},
+    {"transition": "provider 429/5xx/timeout/hard failure", "node": "tests/test_delivery_adapters.py::test_twilio_sms_is_link_only_and_classifies_http"},
+    {"transition": "delayed authenticated hangup", "node": "tests/test_post_call.py::test_delayed_event_is_accepted_using_signed_delivery_time"},
+    {"transition": "permanently lost hangup", "node": "tests/test_post_call.py::test_reconciliation_qualifies_once_and_permanent_loss_holds_with_alert"},
+    {"transition": "duplicate/out-of-order callback", "node": "tests/test_m5_w5_delivery_worker.py::test_duplicate_and_out_of_order_callbacks_do_not_regress_final_state"},
+    {"transition": "process restart with pending order", "node": "tests/test_runtime_conversation_durability.py::test_agent_restart_restores_pending_order_and_replay_is_exact"},
+    {"transition": "process restart with issued work", "node": "tests/test_m5_e2e_composed.py::test_e2e15_restart_recovers_issued_work_without_rerendering_in_call"},
+    {"transition": "disk headroom/readiness pressure", "node": "tests/test_production_controls.py::test_production_validator_separates_policy_from_operational_failures"},
+    {"transition": "backup corruption/clean restore", "node": "tests/test_production_controls.py::test_backup_tamper_and_nonempty_restore_fail_closed"},
+    {"transition": "poison artifact queue isolation", "node": "tests/test_m5_w5_delivery_worker.py::test_quarantined_poison_artifact_does_not_block_later_work"},
+    {"transition": "PostHog exporter outage", "node": "tests/test_production_controls.py::test_posthog_export_failure_cannot_change_agent_turn"}
+  ]
+}

data/m5_w7_reference_profile.json

@@ -0,0 +1,18 @@
+{
+  "schema_version": 1,
+  "claim": "offline_reference_profile_not_tenant_P12",
+  "steady_quotes": 100,
+  "burst_quotes": 100,
+  "burst_concurrency": 2,
+  "channels": ["email"],
+  "observation_cutoff_seconds": 120,
+  "thresholds": {
+    "hangup_to_all_channel_accept_p95_seconds": 60,
+    "minimum_acceptance_fraction": 0.95,
+    "minimum_one_sided_95_lower_bound": 0.95
+  },
+  "quantile": "nearest_rank_no_trimming",
+  "confidence": "one_sided_95_wilson",
+  "warm_cold_policy": "warm app; cold process is reported separately in W8",
+  "stop_rule": "run every preregistered quote; do not replace failures"
+}

docs/DECISION_LOG.md

@@ -1664,3 +1664,84 @@ Phoenix remains an optional W7 local comparison, not a production dependency.
 readiness, PII-free journey events, worker spans, alert thresholds, and the M5
 operations runbook are implemented. **Status:** code verified offline; P06/P07
 policy, clean-host container evidence, and live sink receipts remain external.
+
+## 2026-07-11 — D24: W7 reference evidence is executable but cannot impersonate tenant P12 or real-phone proof
+
+**Decision.** W7 freezes an offline reference profile with 100 steady quotes,
+100 quotes at two-writer burst, nearest-rank untrimmed percentiles, a one-sided
+95% Wilson lower bound, a 120-second observation cutoff, and a rule that every
+admitted quote stays in the denominator. The controlled turn cohort contains
+1,050 order operations across 300 independent one-to-six-line calls. Reports
+are generated under ignored `state/` paths and carry the exact commit,
+environment, workload, sample count, p50/p75/p95/p99/max where applicable, and
+a shared PostHog-compatible report ID.
+
+This profile is explicitly `offline_reference_profile_not_tenant_P12`. It does
+not select the tenant's real workload, provider/channel mix, cold policy,
+retention, or live-phone strata. Likewise, E2E-16's offline hosted surrogate is
+not real-phone evidence, the readback estimator is not actual TTS audio, and the
+accelerated fault matrix is not a wall-clock 24-hour staging soak. Those claims
+remain blocked under `owieschon-f6p` and W8 until policy, staging, credentials,
+real providers, handsets/carriers/speakers, and a human audio audit exist.
+
+Query-rewrite racing remains outside the binding path. The W7 feasibility study
+merges raw and rewritten BM25 candidate sets only, always retains raw fallback,
+exposes no binding field, and measures quality, fallback, outage, cost, and
+latency. The observed scripted study produced no top-five recall gain and higher
+latency/cost, so the locked recommendation is not to enable it. A later tenant
+experiment would require preregistration; fastest-response-wins can never bind
+identity, quantity, account, price, availability, quote number, or a document.
+
+**Why.** Small offline samples and simulated providers are useful engineering
+evidence but become dangerous when relabeled as production proof. Separating the
+executable reference from the external promotion claim lets CI catch regressions
+without manufacturing unavailable tenant decisions or LTE/audio evidence.
+
+**Outcome.** The composed E2E, fault, latency, load, PostHog correlation, and
+rewrite-eval manifests fail closed on missing nodes or inflated claims. The load
+run passes at 100/100 accepted in both cohorts; burst hangup-to-provider-accept
+p95 is 9.26 seconds and the one-sided lower bound is 0.974. A two-writer run also
+found and fixed transaction interleaving on the shared SQLite connection by
+serializing every public adapter operation. **Status:** W7 offline implementation
+verified; tenant P12, actual 24-hour staging soak, actual TTS/LTE timing, and
+real-phone evidence remain explicit external gates. No deployment was attempted.
+
+## 2026-07-11 — D25: hosted grades are naturalness evidence; deterministic raw/local oracles own safety
+
+**Decision.** The W8 voice-eval contract has four distinct layers. The checked-in
+v2 visible suite covers every E2E-01 through E2E-16 row and all named strata; its
+binding oracle is the production-ASGI test node, not scenario prose. Current
+ElevenLabs Agent Tests may grade naturalness only. Their invocation is safety
+evidence only when raw user/agent history, paired tool requests/results, and the
+gateway `say` survive export; exact-say and internal-state checks are rerun over
+those raw rows, and the pinned local deterministic suite always runs as well.
+Incomplete P15 export selects the local fallback explicitly—it never turns a
+native pass into a binding-fact verdict.
+
+The implementation repository contains no authored “blind” cases. An isolated
+author must provide at least 32 cases covering the visible strata plus a reserve,
+and a hash-only seal records exactly one run. `NOT_EXERCISED` and `INFRA_ERROR`
+are failures. Similarly, the checked-in phone plan is a deterministic 30-call
+schedule template, not evidence. It becomes frozen only after P12 and assignment
+receipts; results append in call-index order, cannot replace a failed/infra call,
+and cannot finalize before every scheduled call is counted.
+
+Canary entry and broader enablement are separate owner authorizations. Evidence
+readiness cannot be overridden by an authorization receipt. Rollback must restore
+the prior agent version and image, preserve issued artifact hashes, and replay no
+accepted or terminal job. The checked-in rollout plan has
+`deployment_authorized=false`.
+
+**Why.** A hosted model can sound convincing while changing a quantity or
+omitting a tool call, and an implementation-authored “blind” set is not blind.
+Making result states, raw-evidence completeness, one-shot lifecycle, and owner
+stops executable prevents those category errors.
+
+**Outcome.** `scripts/m5_w8_eval.py`, `agent_grid.py`,
+`m5_blind_holdout.py`, `m5_phone_matrix.py`, and
+`m5_release_candidate.py` implement the complete offline/future-external control
+plane. The executed visible suite passes all 16 scenario rows while the release
+bundle truthfully remains `PREPRODUCTION_BLOCKED`. **Status:** offline W8 build
+implemented; isolated blind/reserve, current hosted IDs/raw export, staging,
+providers, real-phone/audio/LTE, tenant policy, canary, and rollback receipts are
+external under `owieschon-ix3.13`. No deployment was attempted.