Skip to content

feat: human-in-the-loop approval checkpoints before expensive stages - #171

Closed
lucastononro wants to merge 4 commits into
fix/106-resume-sessionsfrom
fix/108-hitl-approvals
Closed

feat: human-in-the-loop approval checkpoints before expensive stages#171
lucastononro wants to merge 4 commits into
fix/106-resume-sessionsfrom
fix/108-hitl-approvals

Conversation

@lucastononro

@lucastononro lucastononro commented Jul 19, 2026

Copy link
Copy Markdown
Owner

Summary

The orchestrator plans then auto-executes end-to-end; request-clarification exists but is agent-initiated and ad hoc. This adds opt-in approval gates: when the user flips the toggle, agents post consequential decisions — target column, prep/leakage plan, model shortlist, expensive runs — as an actionable card and block until the user Approves or Edits. Default behavior (toggle off) is provably unchanged.

Stacked on #170 (fix/106-resume-sessions).

Changes

Backend

  • New request-approval capability skill (backend/skills/request-approval/): registers a blocking future in the existing services/clarifications.py registry (same timeout machinery + session-cleanup cancellation the clarification flow uses), publishes an approval_request SSE/persisted event, blocks on the future, then publishes approval_resolved and returns the outcome to the model (APPROVED / EDITED — revision replaces your proposal / NO RESPONSE — proceed but flag unapproved / CANCELLED). 30-minute window.
  • services/approvals.py: per-session opt-in flag + apply_approval_gate(session_id, skills, prompt). When the flag is off it returns its inputs unchanged (the same list and string objects), so the default path is byte-identical. When on, it appends the skill + a system-prompt block defining which decisions must be gated. Applied by the runner per agent run — delegated sub-agents inherit through the shared session_id with no parameter threading.
  • POST /sessions/{id}/approvals/{approval_id} with {decision: approve|edit, edits} resolves the gate (400 on empty edit, 404 unknown/expired).
  • MessageCreate.approvals (optional bool): each run-triggering message re-asserts the session flag, so the toggle takes effect on the very next turn.

Frontend

  • ShieldCheck toggle in the ChatPane input bar (OFF by default; only serialized on the wire when ON — the default request payload is unchanged).
  • ApprovalCard (components/chat/): kind badge (Target column / Prep plan / Model shortlist / Decision), markdown proposal, context line, Approve and Edit (textarea revision) actions, and resolved renderings for approved/edited/timeout/cancelled.
  • useSessionStream: reducer cases for approval_request/approval_resolved plus restore-on-reload handling — a reload mid-gate brings the card back actionable (the backend agent is still blocked on the future), instead of stalling the session until the window times out.
  • api.replyApproval(), SSEEvent union + ChatItem 'approval' type.

Test plan

Existing (no regression): full backend suite green — 319 passed, 8 skipped; tsc --noEmit, next lint, next build, ruff check all green.

New (backend/tests/test_approvals.py, 13 tests):

  • default-path byte-identity: with the flag off, apply_approval_gate returns the same skills list object and the same prompt string object; a send_message(run_agent=true) without the field leaves gates disabled
  • flag wiring: approvals: true enables for the launch, next message without it disables again
  • skill discovery + schema (required: [title, decision])
  • handler: blocks until resolved (task not done before resolve), approve → APPROVED, edit → revision text returned, timeout → "proceed but flag unapproved" + approval_resolved(decision=timeout), missing args → is_error with nothing published
  • endpoint: resolves the pending future with the user verdict (200), 404 unknown id, 400 edit-without-edits leaves the future pending

Closes #108

🤖 Generated with Claude Code

https://claude.ai/code/session_01Wp66uSq9saSpRq9Bbmjxj4

Greptile Summary

This PR adds opt-in human-in-the-loop approval gates: a request-approval skill that blocks the agent on an actionable card until the user approves or edits a consequential decision (target column, prep plan, model shortlist, expensive run). The feature reuses the existing clarification registry for blocking futures, timeout handling, and session-cleanup cancellation, so it's well-integrated with the existing flow.

  • Backend: New request-approval skill registers a future in services.clarifications, publishes approval_request/approval_resolved SSE events, and returns a structured outcome to the agent. services/approvals.py manages the opt-in flag and injects the skill + system-prompt block per run_agent call, with the default path provably unchanged.
  • Frontend: ApprovalCard renders the pending decision with Approve/Edit actions and proper resolved states; useSessionStream adds reducer cases for approval_request/approval_resolved and a restore-on-reload path that brings back pending cards actionable after a page refresh mid-gate.

Confidence Score: 5/5

Safe to merge. The default (toggle-off) path is provably unchanged — apply_approval_gate returns its input objects as-is — and the new approval path is well-isolated behind a per-session in-memory flag.

The implementation reuses the existing clarification registry, timeout machinery, and session-cleanup hooks rather than introducing new concurrency primitives. The default path was verified both by code inspection (same object identity) and by dedicated tests. The 13 new tests cover the critical paths (approve, edit, timeout, cancellation, missing args, endpoint 404/400), and the full test suite passes without regression.

Files Needing Attention: No files require special attention.

Important Files Changed

Filename Overview
backend/services/approvals.py New module: per-session opt-in flag and skill/prompt injection. Default path is identity (same objects returned unchanged). Clean implementation.
backend/skills/request-approval/handler.py New skill handler: registers a blocking future in the clarification registry, publishes approval_request/approval_resolved events, and maps outcomes (approve/edit/timeout/cancelled) to structured tool results.
backend/routers/sessions.py Adds approval flag assertion on send_message and a new POST /sessions/{id}/approvals/{approval_id} endpoint that resolves the pending future; mirrors the existing clarification endpoint pattern.
frontend/src/components/chat/ApprovalCard.tsx New component: renders a pending approval card with Approve/Edit actions, inline error display on failure, and resolved state renderings for all four outcomes (approve/edit/timeout/cancelled).
frontend/src/lib/useSessionStream.ts Adds approval_request/approval_resolved SSE reducer cases and a restore-on-reload path that correctly patches pending cards to resolved when approval_resolved rows are present in history.
backend/tests/test_approvals.py 13 tests covering default-path identity, flag toggling, skill discovery, handler blocking/approve/edit/timeout/missing-args, and endpoint resolution/404/400 paths.
backend/schemas.py Adds optional approvals field to MessageCreate and a new ApprovalReply schema (decision: approve
frontend/src/lib/types.ts Adds ApprovalRequestData/ApprovalResolvedData interfaces and extends the SSEEvent discriminated union with approval_request/approval_resolved cases.
frontend/src/app/page.tsx Adds approvalsEnabled state + ref, threads the flag through all five sendMessage call sites, and passes toggle props to ChatPane.
backend/services/agent/runner.py Adds apply_approval_gate call before each agent run; no-op when flag is off; delegated sub-agents inherit via shared session_id.

Sequence Diagram

sequenceDiagram
    participant U as User
    participant FE as Frontend
    participant BE as Backend Router
    participant R as Runner (run_agent)
    participant H as request-approval handler
    participant C as clarifications registry

    U->>FE: Toggle approvals ON + send message
    FE->>BE: "POST /sessions/{id}/messages {approvals: true}"
    BE->>BE: approvals_svc.set_enabled(session_id, true)
    BE->>R: run_agent(...)
    R->>R: apply_approval_gate → inject skill + prompt
    R->>H: Agent calls request-approval tool
    H->>C: "register(session_id, approval_id, timeout=1800s) → future"
    H->>BE: publish_fn(approval_request)
    BE-->>FE: SSE: approval_request
    FE->>FE: Render ApprovalCard (pending)
    H->>H: await future BLOCKED
    U->>FE: Click Approve
    FE->>BE: "POST /sessions/{id}/approvals/{approval_id}"
    BE->>C: resolve_clarification(session_id, approval_id)
    C-->>H: "future resolves with decision=approve"
    H->>BE: publish_fn(approval_resolved)
    BE-->>FE: SSE: approval_resolved
    FE->>FE: Update ApprovalCard to Approved
    H->>R: return APPROVED proceed
    R->>R: Agent continues
Loading

Reviews (2): Last reviewed commit: "Merge branch 'staging-v0.0.5' into fix/1..." | Re-trigger Greptile

…108)

Backend:
- New `request-approval` capability skill: the agent posts a
  consequential decision (target column, prep/leakage plan, model
  shortlist, expensive run) as an actionable card and BLOCKS on the
  user's Approve/Edit. Reuses services/clarifications.py's future
  registry (timeouts, session-cleanup cancellation); 30-min window,
  timeout tells the agent to proceed but flag the decision unapproved.
- services/approvals.py: per-session opt-in flag + apply_approval_gate,
  an identity function when the flag is off (same list, same prompt
  string object) — default behavior provably unchanged. The runner
  applies it per agent run, so delegated sub-agents inherit the gate
  through the shared session_id without parameter threading.
- POST /sessions/{id}/approvals/{approval_id} resolves the pending gate
  (approve | edit; edit requires non-empty edits).
- MessageCreate.approvals (optional): each run-triggering message
  re-asserts the session flag; omitted/false keeps gates off.

Frontend:
- ShieldCheck toggle in the chat input bar (OFF by default); the toggle
  state rides along on sendMessage only when ON.
- ApprovalCard in components/chat/: kind badge, proposed decision
  (markdown), Approve / Edit-with-revision actions, resolved states
  (approved / edited / timeout / cancelled).
- `approval_request` / `approval_resolved` SSE events: reducer cases in
  useSessionStream plus restore-on-reload handling so a reload mid-gate
  brings the card back actionable instead of stalling the session.

Tests: 13 new (gate identity when disabled, skill+prompt injection,
send_message flag wiring incl. default-off, handler block/approve/edit/
timeout/validation, resolve endpoint 200/404/400).

Closes #108

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Wp66uSq9saSpRq9Bbmjxj4
Comment thread frontend/src/components/chat/ApprovalCard.tsx
lucastononro and others added 3 commits July 29, 2026 16:16
- ApprovalCard: surface verdict-submission failures inline instead of
  only console.error — the buttons re-enable on failure but the user
  previously had no indication their Approve/Edit never reached the
  agent (expired approval, backend restart, ...).
- ApprovalCard: named ErrorBoundary import per components/ style guide.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
# Conflicts:
#	backend/routers/sessions.py
lucastononro added a commit that referenced this pull request Jul 29, 2026
@lucastononro

Copy link
Copy Markdown
Owner Author

Merged into integration branch staging-v0.0.5 (see the STAGING-v0.0.5.md ledger on that branch for merge order, Greptile follow-ups and test results). These changes will land on main via the staging merge — closing to clear the queue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant