feat(ext-tasks): durable storage, retention, and multi-instance task resumption - #19
Merged
Merged
Conversation
…resumption The 2026-07-28 Tasks extension shipped with an in-memory store only, while the superseded 2025 crate carried SQLite/Postgres/DynamoDB. That is now inverted: turul-mcp-ext-tasks owns task persistence and carries all four backends, laid out like the sibling storage crates (traits.rs + one module per backend). Every status rule, owner check and tasks/update key decision lives once in traits.rs as pure transitions; a backend only does load -> apply -> store under its own atomicity boundary (a row lock for SQL, optimistic concurrency for DynamoDB, which has no read-modify-write transaction). Cross-backend parity is therefore structural, and enforced by a 14-invariant contract run against all four — the check the 2025 crate had in design and never wired up. Behavioural changes: - Multi-instance resumption. A parked worker waited on an in-process oneshot, so a tasks/update landing on any other instance was silently lost and the task sat in input_required forever — precisely the deployment a shared store is for. The worker now watches the store, the same store-and-fetch shape SessionStorage already uses for SSE events. Pinned by a two-server e2e. - Retention: orphan recovery, per-task ttlMs, terminal deletion, plus DynamoDB's native ttlEpoch. One with_ext_tasks_retention call configures the sweep loop AND the per-item TTL writer; without it nothing sweeps, which is the correct default and now a testable one. - MRTR-before-task composition (SEP-2663 §Composition), reachable via ExtTaskElection::with_mrtr_first. - Client: call_tool_or_task_with_input_responses. The existing MRTR retry routed through the strict resultType check, which rejects "task", so no client could consume the composition at all. Found by the e2e — server, wire and spec were each correct and the pair was still broken. The SEP prose is now vendored and checksummed alongside the schema: the rules that govern sequencing exist only in prose, so pinning the schema alone left them resting on an unpinned web page. Backends are gated by scripts/ext-tasks-backends.sh, which owns its services — probes Postgres, fetches and starts DynamoDB Local (jar, no docker) — and FAILS rather than skips when one is unreachable. turul-mcp-task-storage is deprecated with an owner and a removal trigger. Versions: turul-mcp-ext-tasks 0.1.3, turul-mcp-client 0.4.1, turul-mcp-server 0.4.4. Breaking for out-of-workspace TaskStore implementors: sweep is a new required method and provide_input takes an untyped map. Gates: 79 PASS / 0 FAIL, conformance 37/37 scored.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #86, #88, #89 (internal task list).
What
The 2026-07-28 Tasks extension shipped with an in-memory store only, while the
superseded 2025 crate carried SQLite/Postgres/DynamoDB. That is now inverted:
turul-mcp-ext-tasksowns task persistence and carries all four backends, laid outlike the sibling storage crates (
traits.rs+ one module per backend).Every status rule, owner check and
tasks/updatekey decision lives once intraits.rsas pure transitions. A backend only does load → apply → store under itsown atomicity boundary — a row lock for SQL, optimistic concurrency (
rev+ConditionExpression) for DynamoDB, which has no read-modify-write transaction. Thatmakes cross-backend parity structural rather than hoped for, and it is enforced by a
14-invariant contract run against all four.
Behavioural changes
oneshot, so atasks/updatelanding on any other instance was silently lost and the task sat ininput_requiredforever — precisely the deployment a shared store exists for. Theworker now watches the store, the same store-and-fetch shape
SessionStoragealreadyuses for SSE events. Pinned by a two-server e2e that fails without it.
ttlMs, terminal deletion, plus DynamoDB'snative
ttlEpoch. Onewith_ext_tasks_retentioncall configures both the sweep loopand the per-item TTL writer. Without it nothing sweeps, which is the correct default
and now a testable one.
ExtTaskElection::with_mrtr_first.call_tool_or_task_with_input_responses. The existing MRTR retryrouted through the strict
resultTypecheck, which rejects"task"— so no clientcould consume the composition at all. Found by the e2e: server, wire and spec were
each correct and the pair was still broken.
Provenance
The SEP prose is vendored and checksummed alongside the schema. The rules that govern
sequencing exist only in prose, so pinning the schema alone left them resting on an
unpinned web page. Verified against the published extension overview; it confirms the
tasks/updateinert-key decision verbatim ("Ignore responses for unknown oralready-satisfied keys") and surfaced one untested case, now covered.
Gating
scripts/ext-tasks-backends.showns its services — probes Postgres, fetches and startsDynamoDB Local (jar, no docker) — and fails rather than skips when one is
unreachable.
turul-mcp-task-storageis deprecated with an owner and a removal trigger.Verification
15 server wire, 21 crate
Every behavioural claim was proven by breaking the code and watching the test fail.
Versions
turul-mcp-ext-tasks0.1.3 ·turul-mcp-client0.4.1 ·turul-mcp-server0.4.4Breaking for out-of-workspace
TaskStoreimplementors:sweepis a new requiredmethod,
provide_inputtakes an untyped map, andTaskStoreError::UnknownInputKeyisreplaced by
InvalidInputResponse { key, detail }. Patch bumps perdocs/rules/crate-versioning.md; flagged because 0.1.x gives no other signal.