Skip to content

fix(proxy): hold fenced hard turns through cooldown - #1739

Open
kevinsslin wants to merge 3 commits into
Soju06:mainfrom
kevinsslin:codex/upstream-hard-turn-cooldown-recovery
Open

fix(proxy): hold fenced hard turns through cooldown#1739
kevinsslin wants to merge 3 commits into
Soju06:mainfrom
kevinsslin:codex/upstream-hard-turn-cooldown-recovery

Conversation

@kevinsslin

Copy link
Copy Markdown

Closes #1737.

Summary

  • keep an explicitly enabled hard turn-state request pending through an active HTTP bridge retry-circuit cooldown
  • require a live durable session and owner epoch, zero response events, and no response id before waiting
  • send nothing upstream during the wait; after cooldown, preserve the existing durable operation-ledger arbitration and one-shot recovery cap
  • fail closed when durable proof is unavailable or the request budget expires
  • preserve the default fail_closed behavior
  • add low-cardinality wait telemetry and strict OpenSpec coverage

Why

Production telemetry showed two eventless upstream failures opening a 60-second retry circuit. Codex Desktop retried in milliseconds despite the HTTP/SSE retry hints, exhausting its retry budget against startup 503 responses. Turn-state-only hard continuity was rejected before the durable operation ledger could arbitrate bounded recovery.

The recommended deployment mode for this incident class is server_anchored_replay_once; this PR does not make recovery default-on or add unbounded retries.

Validation

  • 5871 passed, 70 skipped — complete unit suite on the fork-main branch
  • 553 passed — HTTP bridge unit file
  • 129 passed — HTTP bridge integration suite
  • clean upstream branch focused regressions: 5 passed
  • Ruff check and format pass
  • ty check passes
  • proxy architecture check passes
  • strict OpenSpec validation passes for this change
  • git diff --check passes

The full repository OpenSpec sweep contains unrelated pre-existing incomplete changes; this change validates independently under --strict.

@kevinsslin
kevinsslin marked this pull request as ready for review August 14, 2026 06:14

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 5e284873d4

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread app/modules/proxy/_service/http_bridge/streaming.py
@Komzpa
Komzpa force-pushed the codex/upstream-hard-turn-cooldown-recovery branch from 5e28487 to 4a2d3b5 Compare August 14, 2026 06:27
@Komzpa

Komzpa commented Aug 14, 2026

Copy link
Copy Markdown
Collaborator

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: dbc805dec8

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread app/modules/proxy/_service/http_bridge/streaming.py Outdated
Comment thread app/modules/proxy/_service/http_bridge/streaming.py Outdated
Comment thread app/modules/proxy/_service/http_bridge/streaming.py
Comment thread app/modules/proxy/_service/http_bridge/streaming.py Outdated
Comment thread tests/unit/test_proxy_http_bridge.py
@Komzpa Komzpa added the 🤖 codex: needs work [@codex review] raised an issue label Aug 14, 2026
@Komzpa Komzpa removed the 🤖 codex: needs work [@codex review] raised an issue label Aug 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

HTTP bridge cooldown startup 503s can exhaust Codex retry budget for hard turn-state recovery

2 participants