Skip to content

Managed Claudexor startup kills the elected daemon before journal preparation publishes its control endpoint #720

Description

@ouroboros-agent

Ouroboros can repeatedly fail to wake its managed Claudexor daemon even though the exact pinned runtime and existing data root can start successfully. The managed launcher allows 20 seconds for an authenticated control endpoint, then terminates its own child and returns daemon_spawn_failed. In the observed case, the child had already acquired root authority and was still preparing the existing journals before either listener was bound.

Observed on macOS on 2026-09-06, with Ouroboros source 5ea0a54 and managed Claudexor 3.9.8, build 529e512f5f112af9009dea55d0b2c2d08b57b6a5. The launcher module is byte-identical in the newer official target 5c3f47c. This finding is independent of the Fable request-shape change in #717.

At 19:54:22 UTC, replacement PID 96540 acquired root authority but never logged a control listener. A subsequent product POST /api/claudexor/wake started PID 99751 at 19:58:46 UTC. It also acquired root authority, then the product returned daemon_spawn_failed after its startup window and terminated the child. No listener was published by that attempt.

A controlled recovery then launched the same resolved managed command, version, build and owned root through the existing process-custody mechanism, retaining the live server's session identity and allowing a longer observation period outside the product's 20-second waiter. No source, settings, authentication or journal contents were manually changed, and no second daemon root was introduced. PID 5184 acquired root authority at 20:03:34.231 UTC and bound its control listener at 20:03:56.253 UTC, 22.022 seconds later. The operator observed an authenticated protocol-3 handshake after 24.32 seconds and servingMode=normal after 43.36 seconds. Startup logs independently record normal admission opening at 20:04:16.638 UTC. These observations prove this startup can succeed after the current cutoff; they do not establish subsequent long-term daemon stability.

The pinned Claudexor bundle synchronously prepares the global journal and registered project partitions before creating its listeners. The existing root contained 72 journal.bin files totaling 547,816,516 bytes. This explains a concrete pre-listener work path that can exceed the host's deadline; it is not evidence of journal corruption, and it does not justify deleting or rewriting journals. Earlier successful starts that day already took roughly 16.6–18.2 seconds from root-authority acquisition to control binding.

The relevant immutable Ouroboros code is the 20-second constant, the authenticated-endpoint wait and timeout termination, and the child cleanup helper. The cleanup helper drops the Popen reference and sends termination without waiting for exit. As a secondary cleanup observation, PID 99751 remained a zombie under the live server until a later ordinary subprocess operation reaped it. This does not explain the original daemon's death, and native lease handling of a still-present zombie was not established.

The original daemon, PID 21121, disappeared before these replacement attempts. Its death cause remains unknown. The old 3.8.2 text appended from daemon.log was stale diagnostic tail, not evidence that the replacement used the wrong runtime or why the original daemon died.

Related issues cover different stages. #446 was closed by #457 after fixing a losing concurrent launcher that stopped waiting before the winning daemon appeared within the existing window. Here the elected daemon itself needs longer than that window. #604 covers session-based reaping and the separate five-second wait for normal admission after control is reachable. Neither distinction should be lost by treating every startup failure as one timeout.

The required outcome is a managed wake that can complete this legitimate pre-listener startup while preserving single-owner custody and explicit cancellation or terminal-failure handling. Verification should distinguish root acquisition, authenticated control reachability, and normal product admission, and should ensure terminated children are reaped. This report deliberately does not prescribe a new numeric default, a journal redesign, or a cause for PID 21121's disappearance.

After this successful startup, a separate requested restart of the live application changed the server generation. Its ordinary custody reaper removed PID 5184 at 20:05:49 UTC as a previous-session process; that later event is distinct from the pre-listener timeout.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions