Recover orphaned resources_restart markers - #166
Draft
robjarawan wants to merge 2 commits into
Draft
Conversation
robjarawan
force-pushed
the
fix/resource-restart-marker-recovery
branch
from
September 6, 2026 06:30
ccdef9c to
81d73d6
Compare
Test Results378 tests 377 ✅ 1m 43s ⏱️ Results for commit 81d73d6. ♻️ This comment has been updated with latest results. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #165.
The
resources_restartmarker only contained a timestamp, so another instance could tell that the marker existed but could not tell whether its owner was still running. If the owner died after creating the marker and before the restarted instance removed it, every later resource restart waited forever andsr3kept the configuration in theresources_restartstate.So what I did was make marker acquisition exclusive and record the owner PID, process creation time, and timestamp. A contender still waits while that exact owner is alive. If the owner is gone, it verifies that the marker did not change, removes the orphan, and claims a new marker. Timestamp-only markers from the current release get a 60-second compatibility window before they are treated as legacy orphans.
The regression uses the real
Resources.restart()path in child processes. One child creates the marker and exits with status 17 at the exec boundary. On the current baseline, the recovery child is still waiting after two seconds. With this fix, it detects the dead owner and reaches its own exec boundary. A separate control keeps the first owner alive and proves that a contender does not take over until the owner finishes.Before:
After:
Tests:
The existing inherited-descriptor cleanup from MetPX#1652 remains in the same restart path and is unchanged.
This branch also includes the local-fixture CI repair from #140 so the fork maintenance jobs do not depend on public dynamic flows.
Related history: MetPX#1439 and merged MetPX#1553 introduced the coordination marker. I did not find an existing issue for recovery after its owner dies.