feat: human-in-the-loop approval checkpoints before expensive stages - #171
Closed
lucastononro wants to merge 4 commits into
Closed
feat: human-in-the-loop approval checkpoints before expensive stages#171lucastononro wants to merge 4 commits into
lucastononro wants to merge 4 commits into
Conversation
…108) Backend: - New `request-approval` capability skill: the agent posts a consequential decision (target column, prep/leakage plan, model shortlist, expensive run) as an actionable card and BLOCKS on the user's Approve/Edit. Reuses services/clarifications.py's future registry (timeouts, session-cleanup cancellation); 30-min window, timeout tells the agent to proceed but flag the decision unapproved. - services/approvals.py: per-session opt-in flag + apply_approval_gate, an identity function when the flag is off (same list, same prompt string object) — default behavior provably unchanged. The runner applies it per agent run, so delegated sub-agents inherit the gate through the shared session_id without parameter threading. - POST /sessions/{id}/approvals/{approval_id} resolves the pending gate (approve | edit; edit requires non-empty edits). - MessageCreate.approvals (optional): each run-triggering message re-asserts the session flag; omitted/false keeps gates off. Frontend: - ShieldCheck toggle in the chat input bar (OFF by default); the toggle state rides along on sendMessage only when ON. - ApprovalCard in components/chat/: kind badge, proposed decision (markdown), Approve / Edit-with-revision actions, resolved states (approved / edited / timeout / cancelled). - `approval_request` / `approval_resolved` SSE events: reducer cases in useSessionStream plus restore-on-reload handling so a reload mid-gate brings the card back actionable instead of stalling the session. Tests: 13 new (gate identity when disabled, skill+prompt injection, send_message flag wiring incl. default-off, handler block/approve/edit/ timeout/validation, resolve endpoint 200/404/400). Closes #108 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Wp66uSq9saSpRq9Bbmjxj4
- ApprovalCard: surface verdict-submission failures inline instead of only console.error — the buttons re-enable on failure but the user previously had no indication their Approve/Edit never reached the agent (expired approval, backend restart, ...). - ApprovalCard: named ErrorBoundary import per components/ style guide. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
# Conflicts: # backend/routers/sessions.py
lucastononro
added a commit
that referenced
this pull request
Jul 29, 2026
…ive stages (closes #108)
Owner
Author
|
Merged into integration branch |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The orchestrator plans then auto-executes end-to-end;
request-clarificationexists but is agent-initiated and ad hoc. This adds opt-in approval gates: when the user flips the toggle, agents post consequential decisions — target column, prep/leakage plan, model shortlist, expensive runs — as an actionable card and block until the user Approves or Edits. Default behavior (toggle off) is provably unchanged.Stacked on #170 (
fix/106-resume-sessions).Changes
Backend
request-approvalcapability skill (backend/skills/request-approval/): registers a blocking future in the existingservices/clarifications.pyregistry (same timeout machinery + session-cleanup cancellation the clarification flow uses), publishes anapproval_requestSSE/persisted event, blocks on the future, then publishesapproval_resolvedand returns the outcome to the model (APPROVED/EDITED — revision replaces your proposal/NO RESPONSE — proceed but flag unapproved/CANCELLED). 30-minute window.services/approvals.py: per-session opt-in flag +apply_approval_gate(session_id, skills, prompt). When the flag is off it returns its inputs unchanged (the same list and string objects), so the default path is byte-identical. When on, it appends the skill + a system-prompt block defining which decisions must be gated. Applied by the runner per agent run — delegated sub-agents inherit through the sharedsession_idwith no parameter threading.POST /sessions/{id}/approvals/{approval_id}with{decision: approve|edit, edits}resolves the gate (400 on empty edit, 404 unknown/expired).MessageCreate.approvals(optional bool): each run-triggering message re-asserts the session flag, so the toggle takes effect on the very next turn.Frontend
ApprovalCard(components/chat/): kind badge (Target column / Prep plan / Model shortlist / Decision), markdown proposal, context line, Approve and Edit (textarea revision) actions, and resolved renderings for approved/edited/timeout/cancelled.useSessionStream: reducer cases forapproval_request/approval_resolvedplus restore-on-reload handling — a reload mid-gate brings the card back actionable (the backend agent is still blocked on the future), instead of stalling the session until the window times out.api.replyApproval(),SSEEventunion +ChatItem'approval'type.Test plan
Existing (no regression): full backend suite green — 319 passed, 8 skipped;
tsc --noEmit,next lint,next build,ruff checkall green.New (
backend/tests/test_approvals.py, 13 tests):apply_approval_gatereturns the same skills list object and the same prompt string object; asend_message(run_agent=true)without the field leaves gates disabledapprovals: trueenables for the launch, next message without it disables againrequired: [title, decision])APPROVED, edit → revision text returned, timeout → "proceed but flag unapproved" +approval_resolved(decision=timeout), missing args →is_errorwith nothing publishedCloses #108
🤖 Generated with Claude Code
https://claude.ai/code/session_01Wp66uSq9saSpRq9Bbmjxj4
Greptile Summary
This PR adds opt-in human-in-the-loop approval gates: a
request-approvalskill that blocks the agent on an actionable card until the user approves or edits a consequential decision (target column, prep plan, model shortlist, expensive run). The feature reuses the existing clarification registry for blocking futures, timeout handling, and session-cleanup cancellation, so it's well-integrated with the existing flow.request-approvalskill registers a future inservices.clarifications, publishesapproval_request/approval_resolvedSSE events, and returns a structured outcome to the agent.services/approvals.pymanages the opt-in flag and injects the skill + system-prompt block perrun_agentcall, with the default path provably unchanged.ApprovalCardrenders the pending decision with Approve/Edit actions and proper resolved states;useSessionStreamadds reducer cases forapproval_request/approval_resolvedand a restore-on-reload path that brings back pending cards actionable after a page refresh mid-gate.Confidence Score: 5/5
Safe to merge. The default (toggle-off) path is provably unchanged — apply_approval_gate returns its input objects as-is — and the new approval path is well-isolated behind a per-session in-memory flag.
The implementation reuses the existing clarification registry, timeout machinery, and session-cleanup hooks rather than introducing new concurrency primitives. The default path was verified both by code inspection (same object identity) and by dedicated tests. The 13 new tests cover the critical paths (approve, edit, timeout, cancellation, missing args, endpoint 404/400), and the full test suite passes without regression.
Files Needing Attention: No files require special attention.
Important Files Changed
Sequence Diagram
sequenceDiagram participant U as User participant FE as Frontend participant BE as Backend Router participant R as Runner (run_agent) participant H as request-approval handler participant C as clarifications registry U->>FE: Toggle approvals ON + send message FE->>BE: "POST /sessions/{id}/messages {approvals: true}" BE->>BE: approvals_svc.set_enabled(session_id, true) BE->>R: run_agent(...) R->>R: apply_approval_gate → inject skill + prompt R->>H: Agent calls request-approval tool H->>C: "register(session_id, approval_id, timeout=1800s) → future" H->>BE: publish_fn(approval_request) BE-->>FE: SSE: approval_request FE->>FE: Render ApprovalCard (pending) H->>H: await future BLOCKED U->>FE: Click Approve FE->>BE: "POST /sessions/{id}/approvals/{approval_id}" BE->>C: resolve_clarification(session_id, approval_id) C-->>H: "future resolves with decision=approve" H->>BE: publish_fn(approval_resolved) BE-->>FE: SSE: approval_resolved FE->>FE: Update ApprovalCard to Approved H->>R: return APPROVED proceed R->>R: Agent continuesReviews (2): Last reviewed commit: "Merge branch 'staging-v0.0.5' into fix/1..." | Re-trigger Greptile