Skip to content

Verify live_write notification teardown by end state - #313

Merged
millsmillsymills merged 2 commits into
mainfrom
issue-296-teardown-gate
Aug 3, 2026
Merged

Verify live_write notification teardown by end state#313
millsmillsymills merged 2 commits into
mainfrom
issue-296-teardown-gate

Conversation

@millsmillsymills

Copy link
Copy Markdown
Collaborator

_delete_notification_any_state decided whether a seeded notification leaked by counting failed deleteNotification mutations. The bin holding the notification is unknown at teardown, so both bins are tried and at least one call is a no-op — and the API may report a no-op either as success or as an error. The count therefore tracks error conventions rather than whether anything leaked: it raises a CLEANUP FAILED banner for a notification the delete test already removed, swallows a real failure when the wrong-bin delete no-ops successfully, and surfaces only failures[0], which is the benign wrong-bin error.

Changes

  • delete_notification_any_state reads the end state back after both delete attempts and reports an orphan only when a bin still holds the id. Both bins are paged to the end, since QUERY_NOTIFICATIONS requests no sort order. Every delete error is attached to an ExceptionGroup, so both causes reach stderr through run_cleanup. A read-back that itself fails is reported rather than treated as success.
  • The session-end orphan scan read structured_content as a bare list. For a tool returning list[Notification] FastMCP always returns a {"result": [...]} envelope, so the isinstance(notifs, list) guard returned early on every run and the scan had never reported an orphan. It now unwraps the envelope through select_mcptest_orphans, lists ARCHIVE as well as UNREAD, passes an explicit page limit instead of the tool's default of 50, and still names any orphans it collected when a later bin fails to enumerate.
  • build_raw_client and the teardown helpers move from tests/live_write/conftest.py to the new fixture-free tests/live_write/_teardown.py, following _gates.py: importing the conftest registers its session-scoped autouse fixtures and would skip the run.
  • tests/unit/test_live_write_teardown.py drives the helper offline against a fake client that models both wrong-bin API behaviours. It covers an already-deleted notification not being an orphan, a single real failure alongside a no-op success not being swallowed, both delete errors reaching the report, an orphan detected in the archive bin and past the first page, single-page bins costing one request, an unverifiable end state raising, and the client being closed on the raising path. A separate test pins the structured_content envelope the orphan scan parses, so a FastMCP change fails in CI rather than silently emptying the backstop.

Eight of the nineteen tests fail against the previous implementation.

tests/live_write/ remains gated behind the live_write marker, UNRAID_ALLOW_LIVE_WRITES=1, and the mcptest* name guard, and is deselected in CI.

Closes #296

🤖 Generated with Claude Code

millsmillsymills and others added 2 commits August 3, 2026 12:53
Teardown decided whether a seeded notification leaked by counting failed
deleteNotification mutations. The bin holding the notification is unknown
at teardown, so both bins are tried and at least one call is a no-op; the
count tracks the API's error conventions rather than whether anything
leaked. It false-alarms when the delete test already removed the entry,
and hides a real failure when the wrong-bin delete no-ops successfully.

delete_notification_any_state now pages both bins after the delete
attempts and reports an orphan only when the id is still present, with
every delete error attached to an ExceptionGroup. A read-back that
itself fails is reported rather than treated as success.

The session-end orphan scan parsed structured_content as a bare list,
which the tool never returns, so it had never reported an orphan; it now
unwraps the result envelope and scans the archive bin as well as unread.

The helpers move to tests/live_write/_teardown.py, fixture-free like
_gates.py, so unit tests can import them without activating the live
suite's session-scoped autouse gate.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@millsmillsymills
millsmillsymills merged commit c52a436 into main Aug 3, 2026
11 checks passed
@millsmillsymills
millsmillsymills deleted the issue-296-teardown-gate branch August 3, 2026 20:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

live_write notification teardown: failure-count gate conflates real-orphan vs already-gone

1 participant