Fix Rescan ready files after spec producers exit to avoid skipping verification #155 - #159
Fix Rescan ready files after spec producers exit to avoid skipping verification #155#159lztmdrclht wants to merge 7 commits into
Conversation
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: b9a19828e7
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 23d42fa350
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
Commented out the section for the oh-my-openagent plugin.
|
@haoran-ding This PR needs to be reviewed. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 670b82f84b
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
|
It looks confusing that there are many commits but the files changed is empty. Please see it on GitHub webpage. |
I accidentally pushed changes from another branch to this one, and the last commit overrode the previous one. |
2 participants
Summary
This fixes issue #155 : the race in streaming_reasoner where the watcher could exit without verifying files that became ready between the regular directory scan and the spec-producer-exit check.
The change adds a final post-producer rescan inside streaming_reasoner in src/verification.py:
A small helper _submit_file() is introduced so both the regular scan and the catch-up scan submit files the same way.
After all spec producers are observed to be done, the watcher re-scans every expected file that is neither processed nor submitted.
Any file that is now ready is submitted to the executor and processed in the same invocation.
Only if the post-producer rescan finds no newly ready files does the watcher report pending files as missing specs and exit.
Problem streaming_reasoner scans files for complete [SPEC]/[INFO] markers and only afterward checks whether all spec-generation producers have finished. If a producer wrote the final markers after that scan but before the producer-exit check, the watcher would see:
is_file_ready(file) -> False during the scan
all spec_procs done -> True immediately after
unready = expected - processed containing the now-complete file
no reasoning/validation futures
It then broke out with the misleading "Spec generation process(es) exited ... but no files received [SPEC]/[INFO] markers" warning, even though the file was complete on disk. The file was never submitted to _verify_single_file, so its verification result was missing and coverage was under-reported.
Result
After this change, the watcher performs one final readiness scan after producers exit and submits any files that have since become ready. Files that were complete but previously missed are now verified in the same invocation, and the missing-spec warning is only emitted for files that remain unready after the final scan.
Example behavior now:
Before: verify calls = [], processed = set(), warning incorrectly claims no specs were received.
After: the race file is submitted, verified, and returned in processed; the warning is reserved for genuinely incomplete files.