Skip to content

fix(core): preserve explicit Codex final answers - #794

Merged
100yenadmin merged 4 commits into
mainfrom
fix/1.6-final-answer-extraction
Jul 28, 2026
Merged

fix(core): preserve explicit Codex final answers#794
100yenadmin merged 4 commits into
mainfrom
fix/1.6-final-answer-extraction

Conversation

@100yenadmin

@100yenadmin 100yenadmin commented Jul 28, 2026

Copy link
Copy Markdown
Owner

Outcome

Preserve Codex's explicit phase: "final_answer" as the authoritative final
message even when later assistant commentary is appended to the same task.
This unblocks the Hermes-first LCO return leg used by Eva to observe a Codex
acknowledgment.

What changed

  • Recognize explicit final-answer phase on normalized event_msg and
    response_item shapes.
  • Prevent later heuristic commentary from replacing an explicit protocol final.
  • Persist a public-safe final-answer reason code so incremental append indexing
    retains the same precedence.
  • Add full-parse and append-delta regressions using synthetic fixtures.

Validation

  • Regression-first proof: both new tests failed before implementation.
  • node --test --import tsx --test-name-pattern='explicit Codex final_answer|append indexing preserves' tests/codex-index.test.ts
    — 2 passed.
  • node --test --import tsx tests/codex-index.test.ts
    — 35 passed.
  • node --test --import tsx tests/index-fast-skip.test.ts tests/session-diff.test.ts
    — 70 passed.
  • npm run build — passed.
  • Private Hermes/Python MCP reproduction: 59 source files, zero parse errors,
    acknowledgment marker found through exact and recent final extraction.

npm run typecheck remains red on the same pre-existing test-only errors on
clean main@48b3d9928498c1bc5a04987e82657a2a0f824e70; repository-wide validation
is delegated to the canonical CI workflow.

Safety and proof boundary

  • No live action is performed by this PR.
  • No active Eva package, profile, MCP configuration, or database is changed.
  • No raw transcript, prompt, query, thread ID, source path, database,
    credential, environment value, or copied runtime configuration is included.
  • Merge proves source integration only. The direct Eva↔Codex LCO round trip is
    rerun after the exact merged candidate is packaged.
  • This PR does not publish npm, create a Git tag or GitHub Release, merge PR
    fix(qa): accept documented post-create diagnostics #785, or claim customer readiness.

Agent-authored PR.

Closes #793
Related: #791

Agent provenance

Summary by CodeRabbit

  • Bug Fixes
    • Explicit final answers are now preserved even when later assistant commentary appears.
    • Re-indexing appended session content no longer replaces an established authoritative final answer.
    • Legacy sessions are safely reprocessed before relying on final-answer authority metadata.

@100yenadmin

Copy link
Copy Markdown
Owner Author

@codex review

@coderabbitai

coderabbitai Bot commented Jul 28, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: 66cba51e-c8ef-4a9d-a443-6f8548a2c2c2

📥 Commits

Reviewing files that changed from the base of the PR and between f70fad8 and 94f4583.

📒 Files selected for processing (2)
  • packages/core/src/index.ts
  • tests/index-fast-skip.test.ts
📜 Recent review details
⏰ Context from checks skipped due to timeout. (2)
  • GitHub Check: CodeQL (javascript-typescript)
  • GitHub Check: test
🔇 Additional comments (4)
packages/core/src/index.ts (3)

5367-5378: 🩺 Stability & Availability

Already flagged: instr() substring match is brittle to future reason_codes_json serialization changes.

Same concern as the open bot review thread on this range (no reply yet) — using instr(reason_codes_json, ?) for exact-token matching works for the current array-of-strings JSON serialization, but silently stops matching if serialization ever changes (pretty-printing, object form, etc.), which would quietly reintroduce the append-side final-answer-clobbering regression this PR fixes.


20043-20052: 🎯 Functional Correctness

Already flagged: isExplicitCodexFinalAnswer trusts a structurally-injected phase field without cross-envelope validation.

Same as the open bot review thread — phase is read from item.response_item?.phase / event_msg?.phase / etc. independent of the envelope that supplied role/rangeKind, so a mismatched cross-envelope phase could in theory promote a non-final message. No reply/won't-fix recorded for this specific instance yet.


2685-2685: LGTM!

Also applies to: 3065-3065, 5339-5339, 5479-5498, 19465-19465, 19543-19572, 19597-19597, 19974-20009

tests/index-fast-skip.test.ts (1)

817-818: LGTM!

Also applies to: 829-899


📝 Walkthrough

Walkthrough

Adds authoritative handling for Codex final_answer phases, persists that state in prepared-source reason codes, preserves it during incremental merges, and adds full-parse and append-indexing regressions.

Changes

Codex final-answer authority

Layer / File(s) Summary
Final-answer detection and selection
packages/core/src/index.ts
Explicit final_answer phases are detected across supported item shapes and take precedence over heuristic assistant commentary.
Prepared-source persistence and session recovery
packages/core/src/index.ts
Authoritative state is carried into final-message event records, marked with codex_explicit_final_answer, and recovered for existing sessions.
Incremental merge and regression coverage
packages/core/src/index.ts, tests/codex-index.test.ts, tests/index-fast-skip.test.ts
Append merges preserve authoritative finals, while tests cover explicit phases, appended commentary, and legacy metadata reparsing.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Sequence Diagram(s)

sequenceDiagram
  participant CodexJSONL
  participant FinalAnswerDetector
  participant SessionImporter
  participant PreparedSourceRanges
  participant IncrementalIndexer
  CodexJSONL->>FinalAnswerDetector: provide assistant item
  FinalAnswerDetector->>SessionImporter: identify explicit final_answer
  SessionImporter->>PreparedSourceRanges: persist authoritative final-message reason code
  IncrementalIndexer->>SessionImporter: import appended content
  SessionImporter->>IncrementalIndexer: merge with authoritative final precedence
Loading

Possibly related PRs

Suggested labels: codex

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title is concise and accurately reflects the main change: preserving explicit Codex final answers.
Linked Issues check ✅ Passed The changes match #793 by giving explicit final_answer precedence, preserving heuristic fallback, and adding full-parse and append-indexing regressions.
Out of Scope Changes check ✅ Passed The patch stays focused on final-answer preservation, metadata versioning, and regression tests with no clear unrelated code changes.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/1.6-final-answer-extraction

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot added the safety Safety, approval, privacy, or control-boundary issue label Jul 28, 2026
@evaos-code-review-bot

evaos-code-review-bot Bot commented Jul 28, 2026

Copy link
Copy Markdown

evaOS review status: completed

PR: #794 - fix(core): preserve explicit Codex final answers
Head: 796b384c158239e4e41d8b935b295edf6b3a0e8d
Updated: 2026-07-28T21:20:25.044Z

evaOS review completed for this PR head.

Automation note: agents should wait for this comment to reach completed, stale_head, closed_or_merged_before_review, skipped, or failed before treating evaOS review as settled for this head. provider_deferred means evaOS still intends to retry.

PR URL: #794

Review URL: #794 (review)

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@packages/core/src/index.ts`:
- Around line 5367-5375: Update existingCodexSessionHasRangeReasonCode so the
reasonCode value is escaped for SQL LIKE metacharacters, including underscores,
percent signs, and the escape character, before constructing the JSON substring
pattern. Add the corresponding LIKE escape declaration to the query while
preserving the existing quoted reason-code matching behavior.

In `@tests/codex-index.test.ts`:
- Around line 240-258: Update the test fixture in the final-answer detection
test so the event_msg and response_item shapes are exercised independently, with
each case containing only its own final-bearing item. Parameterize or split the
cases while preserving the existing assertions for both shapes, ensuring
event_msg.phase regressions cannot be masked by a response_item final.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: a1673c8f-ab65-4f38-a12c-476be4ee62fc

📥 Commits

Reviewing files that changed from the base of the PR and between 48b3d99 and 796b384.

📒 Files selected for processing (2)
  • packages/core/src/index.ts
  • tests/codex-index.test.ts
📜 Review details
⏰ Context from checks skipped due to timeout. (2)
  • GitHub Check: CodeQL (javascript-typescript)
  • GitHub Check: test
🔇 Additional comments (3)
packages/core/src/index.ts (2)

3062-3065: LGTM!

Also applies to: 5336-5339, 5476-5479, 5496-5501, 19462-19465


19542-19572: LGTM!

Also applies to: 19587-19599, 19965-20012, 20043-20051

tests/codex-index.test.ts (1)

2-2: LGTM!

Also applies to: 282-324

Comment thread packages/core/src/index.ts Outdated
Comment thread tests/codex-index.test.ts Outdated

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 796b384c15

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread packages/core/src/index.ts

@evaos-code-review-bot evaos-code-review-bot Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Walkthrough

PR: #794 - fix(core): preserve explicit Codex final answers
Head: 796b384c158239e4e41d8b935b295edf6b3a0e8d into main. Review event: COMMENT.
Provider: GLM/Z.ai through ZCode (zcode-glm, zcode, model GLM-5.2).

Estimated review effort: 1/5 (~14 min)

Changed Files

File Status Churn Purpose Risk
packages/core/src/index.ts modified +42/-3 Changed file Moderate: validated P3 finding
tests/codex-index.test.ts modified +93/-1 Test coverage Low

Review Signal

Validated inline findings: 3 (P0: 0, P1: 0, P2: 0, P3: 3).
Dropped findings before posting: 0. High-severity findings: 0.

Risk Taxonomy

  • API compatibility: 1
  • Auth: 1
  • Runtime correctness: 1

Validation and Proof

1 required validation/proof recommendation(s) selected from changed files.

  • required: TypeScript/web build or CI proof - Runtime TypeScript/web files or package/config files changed. Proof: npm run build; typecheck; focused Vitest; green GitHub check.
    Proof status: sufficient - PR metadata mentions acceptable proof for each required validation recommendation.
    Profile validation hints: Call out evidence leakage, replay/collision risks, duplicate side effects, and brittle sanitizer logic.
    Profile proof expectations: Look for focused sanitizer, signature, orchestration, or fixture proof.

Related Context

Related issues/PRs: #785, #793, #791.
Suggested labels: tests.
Suggested reviewers: none from current metadata.

Review Settings Preview

  • Profile: assertive
  • Enabled sections: Review summary (inline_review); Walkthrough (inline_review); Changed-files table (walkthrough); Effort estimate (walkthrough); Related issues/PRs (walkthrough); Suggested labels (suggestion_only); Review status comment (sticky_status)
  • Path instructions: none
  • Label suggestions: orchestration, security, regression-hardening
  • Reviewer suggestions: none
  • Suggestion behavior: suggestions only; labels and reviewers are not auto-applied.
  • Roadmap-only settings: auto-apply labels; auto-request reviewers; required status checks

Pre-merge checklist

  • Inline comments target current RIGHT-side diff lines.
  • No secret-like content survived into posted inline comments.
  • REQUEST_CHANGES is only used when eligible P0/P1 findings survive validation.
  • Required behavior proof is present or not applicable.
  • Labels and reviewers are suggestions only; the bot did not auto-apply them.

Comment thread packages/core/src/index.ts
Comment thread packages/core/src/index.ts
Comment thread packages/core/src/index.ts
@coderabbitai coderabbitai Bot removed the safety Safety, approval, privacy, or control-boundary issue label Jul 28, 2026
@evaos-code-review-bot

evaos-code-review-bot Bot commented Jul 28, 2026

Copy link
Copy Markdown

evaOS review status: stale head

PR: #794 - fix(core): preserve explicit Codex final answers
Head: e20f1d7c2b83643fea97b369e631c5335ab3e3c0
Updated: 2026-07-28T21:25:05.405Z

evaOS review stopped because this queued head is no longer the live PR head.

Automation note: agents should wait for this comment to reach completed, stale_head, closed_or_merged_before_review, skipped, or failed before treating evaOS review as settled for this head. provider_deferred means evaOS still intends to retry.

PR URL: #794

coderabbitai[bot]
coderabbitai Bot previously approved these changes Jul 28, 2026
@coderabbitai coderabbitai Bot added the codex Codex session indexing or control label Jul 28, 2026
@evaos-code-review-bot

evaos-code-review-bot Bot commented Jul 28, 2026

Copy link
Copy Markdown

evaOS review status: completed

PR: #794 - fix(core): preserve explicit Codex final answers
Head: f70fad852267bc1755f46ce14e693170fcae64de
Updated: 2026-07-28T21:29:17.482Z

evaOS review completed for this PR head.

Automation note: agents should wait for this comment to reach completed, stale_head, closed_or_merged_before_review, skipped, or failed before treating evaOS review as settled for this head. provider_deferred means evaOS still intends to retry.

PR URL: #794

Review URL: #794 (review)

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: f70fad8522

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread packages/core/src/index.ts Outdated

@evaos-code-review-bot evaos-code-review-bot Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Walkthrough

PR: #794 - fix(core): preserve explicit Codex final answers
Head: f70fad852267bc1755f46ce14e693170fcae64de into main. Review event: COMMENT.
Provider: GLM/Z.ai through ZCode (zcode-glm, zcode, model GLM-5.2).

Estimated review effort: 2/5 (~32 min)

Changed Files

File Status Churn Purpose Risk
packages/core/src/index.ts modified +45/-6 Changed file Moderate: validated P3 finding
tests/codex-index.test.ts modified +104/-1 Test coverage Moderate: validated P3 finding
tests/index-fast-skip.test.ts modified +70/-1 Test coverage Low
tests/prepared-cards.test.ts modified +1/-1 Test coverage Low
tests/prepared-source-ranges.test.ts modified +1/-1 Test coverage Low
tests/session-diff.test.ts modified +3/-3 Test coverage Low

Review Signal

Validated inline findings: 3 (P0: 0, P1: 0, P2: 0, P3: 3).
Dropped findings before posting: 0. High-severity findings: 0.

Risk Taxonomy

  • Data loss: 1
  • Flaky test risk: 1
  • Runtime correctness: 1

Validation and Proof

1 required validation/proof recommendation(s) selected from changed files.

  • required: TypeScript/web build or CI proof - Runtime TypeScript/web files or package/config files changed. Proof: npm run build; typecheck; focused Vitest; green GitHub check.
    Proof status: sufficient - PR metadata mentions acceptable proof for each required validation recommendation.
    Profile validation hints: Call out evidence leakage, replay/collision risks, duplicate side effects, and brittle sanitizer logic.
    Profile proof expectations: Look for focused sanitizer, signature, orchestration, or fixture proof.

Related Context

Related issues/PRs: #785, #793, #791.
Suggested labels: tests.
Suggested reviewers: none from current metadata.

Review Settings Preview

  • Profile: assertive
  • Enabled sections: Review summary (inline_review); Walkthrough (inline_review); Changed-files table (walkthrough); Effort estimate (walkthrough); Related issues/PRs (walkthrough); Suggested labels (suggestion_only); Review status comment (sticky_status)
  • Path instructions: none
  • Label suggestions: orchestration, security, regression-hardening
  • Reviewer suggestions: none
  • Suggestion behavior: suggestions only; labels and reviewers are not auto-applied.
  • Roadmap-only settings: auto-apply labels; auto-request reviewers; required status checks

Pre-merge checklist

  • Inline comments target current RIGHT-side diff lines.
  • No secret-like content survived into posted inline comments.
  • REQUEST_CHANGES is only used when eligible P0/P1 findings survive validation.
  • Required behavior proof is present or not applicable.
  • Labels and reviewers are suggestions only; the bot did not auto-apply them.

Comment thread tests/codex-index.test.ts
Comment thread packages/core/src/index.ts
Comment thread packages/core/src/index.ts
@evaos-code-review-bot

evaos-code-review-bot Bot commented Jul 28, 2026

Copy link
Copy Markdown

evaOS review status: completed

PR: #794 - fix(core): preserve explicit Codex final answers
Head: 94f4583160a30bac4f252bdd9aacda230d55cc43
Updated: 2026-07-28T21:38:42.862Z

evaOS review completed for this PR head.

Automation note: agents should wait for this comment to reach completed, stale_head, closed_or_merged_before_review, skipped, or failed before treating evaOS review as settled for this head. provider_deferred means evaOS still intends to retry.

PR URL: #794

Review URL: #794 (review)

@evaos-code-review-bot evaos-code-review-bot Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Walkthrough

PR: #794 - fix(core): preserve explicit Codex final answers
Head: 94f4583160a30bac4f252bdd9aacda230d55cc43 into main. Review event: COMMENT.
Provider: GLM/Z.ai through ZCode (zcode-glm, zcode, model GLM-5.2).

Estimated review effort: 1/5 (~16 min)

Changed Files

File Status Churn Purpose Risk
packages/core/src/index.ts modified +43/-4 Changed file Low
tests/codex-index.test.ts modified +104/-1 Test coverage Low
tests/index-fast-skip.test.ts modified +73/-1 Test coverage Low

Review Signal

No validated inline findings.
Dropped findings before posting: 0. High-severity findings: 0.

Risk Taxonomy

No finding categories.

Validation and Proof

1 required validation/proof recommendation(s) selected from changed files.

  • required: TypeScript/web build or CI proof - Runtime TypeScript/web files or package/config files changed. Proof: npm run build; typecheck; focused Vitest; green GitHub check.
    Proof status: sufficient - PR metadata mentions acceptable proof for each required validation recommendation.
    Profile validation hints: Call out evidence leakage, replay/collision risks, duplicate side effects, and brittle sanitizer logic.
    Profile proof expectations: Look for focused sanitizer, signature, orchestration, or fixture proof.

Related Context

Related issues/PRs: #785, #793, #791.
Suggested labels: tests.
Suggested reviewers: none from current metadata.

Review Settings Preview

  • Profile: assertive
  • Enabled sections: Review summary (inline_review); Walkthrough (inline_review); Changed-files table (walkthrough); Effort estimate (walkthrough); Related issues/PRs (walkthrough); Suggested labels (suggestion_only); Review status comment (sticky_status)
  • Path instructions: none
  • Label suggestions: orchestration, security, regression-hardening
  • Reviewer suggestions: none
  • Suggestion behavior: suggestions only; labels and reviewers are not auto-applied.
  • Roadmap-only settings: auto-apply labels; auto-request reviewers; required status checks

Pre-merge checklist

  • Inline comments target current RIGHT-side diff lines.
  • No secret-like content survived into posted inline comments.
  • REQUEST_CHANGES is only used when eligible P0/P1 findings survive validation.
  • Required behavior proof is present or not applicable.
  • Labels and reviewers are suggestions only; the bot did not auto-apply them.

@100yenadmin
100yenadmin merged commit 94c5d90 into main Jul 28, 2026
4 checks passed
@100yenadmin
100yenadmin deleted the fix/1.6-final-answer-extraction branch July 28, 2026 21:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

codex Codex session indexing or control

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bug(codex): explicit final_answer is lost after later commentary

1 participant