STAGE_E_STREAMING_ENABLED defaults False on a fresh instance: monolith exits 0 writing no evidence

[WilfordGrimley]

Sub-item of #626 ("default the default things, disable them with flags", owner ruling 2026-07-29). This one is a worse variant than the command-flag cases that issue anticipates, because the switch is an environment variable rather than a management-command flag, so its absence flips behaviour off silently rather than surfacing in --help.

Finding

docker/docker-compose.prod.yml sets STAGE_E_STREAMING_ENABLED=${STAGE_E_STREAMING_ENABLED:-False} on both services. The only reason it evaluates True on the current production box is line 8 of docker/.env, and that file is untracked — git ls-files docker/.env returns nothing. The tracked template docker/.env.dist has no STAGE_E_STREAMING_ENABLED line at all.

The consequence for a fresh instance: clone, no docker/.env, compose default False applies, and dispatch_micro_batch returns status="disabled" for every chunk at MPCAutofill/cardpicker/stage_e_dispatch.py:1484. Stage C and Stage D write nothing. Stage 0, Stage E and channel_report still run, and run_pipeline exits 0. A new maintainer observes a successful whole-catalogue run that produced no evidence.

Three things compound it:

• run_pipeline's module docstring describes itself as a complete from-scratch whole-catalogue run that writes.
• PR Streaming pipeline default: micro-batch Stage C/D dispatch and verdict transfer #666's description stated it removed the STAGE_E_STREAMING_ENABLED gate so streaming runs unconditionally. The gate is still present at the line above.
• MPCAutofill/cardpicker/tests/test_run_pipeline.py:231 has an autouse fixture that monkeypatches the flag True for the whole suite, so no test exercises the default and CI cannot catch the regression.

This does not affect the 2026-08-04 pass, which ran with the flag genuinely set.

Shape of a fix

Defaulting the behaviour on and providing a disable flag matches the #626 directive. Candidate approach: invert the default so streaming is on unless explicitly disabled, and keep an off switch for the case where an operator wants Stage 0/E only. Whatever the mechanism, the documented reproducibility property is that a fresh clone with no local env file runs the monolith and writes evidence.

Two related considerations rather than requirements:

• STAGE_E_MICRO_BATCH_SIZE is the neighbouring env var mentioned in the same context; Measure STAGE_E_MICRO_BATCH_SIZE at the next real bulk inflow (25/50/100 protocol) #463 covers measuring it at the next bulk inflow and is a separate question from whether it needs to be set at all.
• The autouse True fixture is the reason this was invisible. Test coverage for the default path is what would make a future inversion detectable.