fix(ops): scope terminal veto to recovery epoch

[Pigbibi]

Summary

• make terminal auth timeout/disconnect sticky only within the current immutable recovery epoch
• reset the epoch only before explicit initial/restart/recreate attempts; stable readiness still succeeds immediately
• preserve RFC3339 nanosecond filtering, ignore untimestamped events, and remove ambiguous dialog dismissal from progress
• add executable policy regressions to CI

Contract evidence

• pre-epoch terminal ignored
• in-epoch terminal remains a veto after later generic progress
• explicit new epoch resets the veto
• stable readiness precedes activity policy
• sub-second cutoff preserved; untimestamped events ignored
• forward/reverse timeout messages covered
• shell parameter-expansion closing brace is not part of the terminal regex

Validation

• all tests/test_*.sh
• python3 -m py_compile 2fa_bot.py
• bash -n for shell scripts/tests/overrides
• actionlint .github/workflows/*.yml
• focused shellcheck -S warning
• git diff --check

Safety

No deploy, restart, recreate, Scheduler trigger, /run, /dry-run, order, funds, position, leverage, live config, or secret operation was performed.

[github-actions]

🤖 Codex PR Review

🚫 Merge blocked: 1 serious issue(s) found in high-risk files

🚫 Blocking Issues

These issues must be fixed before this PR can be merged:

1. 🟠 [HIGH] Logic in scripts/recover_ib_gateway_ready.sh

attempt_start is set immediately before docker compose restart / docker compose up --force-recreate, while terminal_regex still treats generic shutdown lines like Connection reset by peer and Server disconnected as terminal. On restart, those messages can be emitted by the old container during the controlled stop and are then included in the new epoch, so gateway_epoch_activity() returns terminal and suppresses progress-based waiting for the replacement startup. This makes explicit recovery self-veto on normal restart noise. (line 19)

Suggestion: Do not open the new epoch until after the old instance has been stopped/replaced, or narrow terminal detection so restart/shutdown disconnects from the previous instance cannot count against the new attempt. A robust fix is to anchor the epoch to the new container/process instance rather than the restart command timestamp.

---

Review by Codex PR Review bot • PR

[Pigbibi]

PERMANENT_FREEZE / HUMAN_REQUIRED

Stopping under the cross-reslice repeated-finding rule at head dbff862236ae43763c481ba0008ab8be2be8ab3e (HIGH, fingerprint 6459a4ba12cafa080cad). No closure, human exception, rerun, merge, bypass, or live action will be attempted.

Boundary design required before any future implementation:

• begin the new epoch only after controlled termination of the old container completes and the replacement container identity plus started_at are confirmed;
• isolate the old container/log stream from the replacement epoch;
• apply sticky terminal veto only to the replacement identity and events at or after its epoch start.

The current reslice starts the epoch before restart/recreate, so disconnect emitted by the old container can self-veto replacement startup. No new Gateway fix PR will be created in this run.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: dbff862236

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

Keep progress checks from reusing stale epoch logs

With progress_extensions greater than 1, this re-scans from the original attempt start on every extension check, so a single early line such as IBC: Starting Gateway makes gateway_epoch_activity return progress for every loop iteration even after another full progress_wait_seconds passes with no new activity. In a hung startup that logs once and then stalls, the recovery now burns all configured extensions before restarting/recreating instead of requiring fresh progress for each extension; keep the terminal veto epoch-wide, but base progress on activity since the previous wait/check.

Useful? React with 👍 / 👎.

[chatgpt-codex-connector]

Do not backdate second-precision file log entries

For file logs that only include whole-second timestamps, this normalizes the event to the start of that second. Since attempt_start is recorded with nanoseconds, any real in-epoch file log emitted later in the same second (for example 2fa.log, whose Python formatter uses %Y-%m-%d %H:%M:%S) compares as older than the epoch and is ignored. When the only 2FA/login progress after a restart happens in that sub-second window, recovery will miss it and proceed to the next external action instead of extending the wait.

Useful? React with 👍 / 👎.

[chatgpt-codex-connector]

Preserve the sticky veto across verbose file logs

This fallback only classifies the last 400 lines even though the new policy makes a terminal auth/disconnect line sticky for the whole recovery epoch. If Docker logs are unavailable/rotated or time out and an in-epoch terminal line in launcher.log or 2fa.log scrolls past those last 400 lines before the readiness wait finishes, the fallback can return later progress (or nothing) and allow extensions/retries instead of honoring the terminal veto; scan from the epoch boundary or otherwise remember terminal state across the full epoch.

Useful? React with 👍 / 👎.