Skip to content

Commit e8468ee

Browse files
authored
docs: define governed refund paid pilot (#21)
Co-authored-by: fly1d <fly1d@users.noreply.github.com>
1 parent 1ccd652 commit e8468ee

8 files changed

Lines changed: 452 additions & 1 deletion

File tree

CHANGELOG.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,3 +15,4 @@ for public contracts once they are declared stable.
1515
- Human approval queue with single-decision enforcement and append-only audit events.
1616
- PostgreSQL system of record with transactional audit writes, Alembic migrations, readiness
1717
checks, and database-level audit mutation protection.
18+
- Reproducible governed-refund demo, paid-pilot boundary, and customer-discovery criteria.

Makefile

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,7 @@ lint:
1212
python -m ruff check .
1313

1414
type:
15-
python -m mypy src
15+
python -m mypy src examples
1616

1717
unit:
1818
python -m pytest tests/unit --cov=agent_control_plane --cov-report=term-missing

README.md

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -25,6 +25,19 @@ Not in the initial scope:
2525
- a replacement for Temporal, Mem0, LangSmith, or DSPy;
2626
- Kubernetes before workload and isolation requirements justify it.
2727

28+
## First validation scenario
29+
30+
The initial commercial hypothesis is governed refund approval for an existing customer-support
31+
agent. The repository includes a reproducible end-to-end demo, a bounded paid-pilot offer, and a
32+
customer-discovery guide:
33+
34+
- [`examples/refund_approval.py`](examples/refund_approval.py)
35+
- [`docs/PAID_PILOT.md`](docs/PAID_PILOT.md)
36+
- [`docs/CUSTOMER_DISCOVERY.md`](docs/CUSTOMER_DISCOVERY.md)
37+
38+
These are validation materials, not a production-readiness claim. Major provider integrations
39+
remain deferred until customer evidence identifies a concrete blocker.
40+
2841
## Quick start
2942

3043
Requirements: Python 3.11 or newer.

docs/CUSTOMER_DISCOVERY.md

Lines changed: 66 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,66 @@
1+
# Customer Discovery
2+
3+
The purpose of discovery is to falsify or support the paid-pilot hypothesis in
4+
`docs/PAID_PILOT.md`. It is not a product pitch and should not be used to manufacture positive
5+
feedback.
6+
7+
## Recruitment
8+
9+
Interview people responsible for an agent already used in customer support, finance operations,
10+
security operations, or another workflow with consequential tool calls. Prioritize the
11+
engineering owner, the operator who handles exceptions, and the person accountable for risk or
12+
budget. Avoid counting people who are only exploring agents and have no deployed workflow.
13+
14+
Do not put customer names, personal data, credentials, incident details, or confidential logs in
15+
this public repository. Store interview notes in an approved private system and use anonymous
16+
identifiers in aggregate results.
17+
18+
## Thirty-minute interview
19+
20+
Spend the first 20 minutes on past behavior before showing the demo.
21+
22+
1. What agent workflow is running today, and who owns it when something goes wrong?
23+
2. Tell me about the last incorrect or risky action. What happened next?
24+
3. Which actions require a person to review them today? Walk through the actual process.
25+
4. How often are actions delayed, repeated, silently dropped, or approved without enough context?
26+
5. How do you determine which agent, prompt, tool call, and person caused an outcome?
27+
6. When did you last pause or roll back an agent? How long did diagnosis and recovery take?
28+
7. What does the current process cost in engineering time, operator time, refunds, or risk?
29+
8. Which controls are mandatory before this workflow can handle more volume?
30+
9. Who owns the budget for fixing this, and what has already been purchased or built?
31+
10. Is there a time-bound project where a three-week staging pilot could be evaluated?
32+
33+
Then show the refund-approval scenario. Ask what would have to change for it to fit their actual
34+
workflow, which evidence is missing, who else must approve a pilot, and what decision date is
35+
realistic. Do not ask whether they "like" the product or whether they "would use" it someday.
36+
37+
## Evidence record
38+
39+
Record one row per interview outside the public repository:
40+
41+
| Field | Evidence to capture |
42+
| --- | --- |
43+
| Segment | Industry, team function, and deployed-agent maturity |
44+
| Last incident | Date range, failure type, impact, and current recovery path |
45+
| Frequency | Consequential actions and manual reviews per week |
46+
| Current spend | Tools plus engineering and operator effort |
47+
| Required control | Approval, audit, pause, replay, evaluation, or another named need |
48+
| Buying process | Budget owner, security owner, procurement steps, and deadline |
49+
| Commitment | Introduction, data sample, technical session, trial, or paid pilot |
50+
| Disconfirming evidence | Reason the problem is unimportant or already solved |
51+
52+
Separate direct quotes and observed behavior from interpretation. A verbal compliment is not a
53+
commitment; access to a technical owner, sanitized cases, scheduled integration time, or payment
54+
is progressively stronger evidence.
55+
56+
## Decision thresholds
57+
58+
Run 20 qualified interviews before broadening the product. Continue toward the refund pilot if
59+
at least eight teams report the problem from recent experience, five provide a technical follow-
60+
up or representative cases, two commit engineering time to an evaluation, and one signs a paid
61+
pilot within eight weeks.
62+
63+
Stop or change the segment if fewer than five teams report a recent consequential failure, the
64+
problem has no named budget owner, or existing observability and workflow tools already solve it
65+
without meaningful friction. Build Mem0, DSPy, Temporal, or another major adapter only when
66+
repeated interview and pilot evidence identifies that capability as the blocker.

docs/PAID_PILOT.md

Lines changed: 88 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,88 @@
1+
# Paid Pilot: Governed Refund Approvals
2+
3+
This document defines a commercial hypothesis to test, not a claim that the current project is
4+
production-ready or that demand has already been proven.
5+
6+
## Buyer and problem
7+
8+
The initial buyer is the engineering or risk owner of a customer-support agent that can propose
9+
refunds or credits. Their problem is not generating another answer. It is proving that a risky
10+
action waited for the right human decision, preserving who decided it and why, and recovering
11+
when an agent version behaves badly.
12+
13+
The qualifying scenario has all of these properties:
14+
15+
- an agent already runs in a test or production workflow;
16+
- at least one action can create financial, compliance, or customer harm;
17+
- humans review some of those actions today, even if the process is manual;
18+
- the team cannot reconstruct decisions quickly from its current logs;
19+
- a named owner has budget or authority to sponsor an operational pilot.
20+
21+
## Offer
22+
23+
The starting commercial hypothesis is a three-week, CNY 30,000 paid pilot in an isolated or
24+
staging environment. Taxes, travel, custom infrastructure, and production support are outside
25+
that price. Any quote to a real buyer must state the exact deployment boundary and data policy.
26+
27+
The pilot covers:
28+
29+
- one existing customer-support agent;
30+
- one high-risk refund or credit action;
31+
- registration, pause/activate, approval, decision, and audit integration;
32+
- one policy workshop and one operator handoff session;
33+
- a final evidence report against the agreed metrics.
34+
35+
The customer continues to own agent reasoning and execution. The control plane records and
36+
governs the action request; it does not move money. No custom dashboard, compliance
37+
certification, 24/7 SLA, model hosting, Mem0, DSPy, or autonomous prompt changes are included.
38+
39+
## Acceptance metrics
40+
41+
Before integration, both parties choose a representative set of at least 20 synthetic or
42+
sanitized refund cases. The pilot succeeds technically when:
43+
44+
- every above-threshold test action is held until an explicit decision;
45+
- every sampled decision can be reconstructed from actor, reason, status, and audit events;
46+
- stale lifecycle writes and repeated decisions fail closed;
47+
- the customer can pause the integrated agent and observe the state change;
48+
- the integration requires no more than five customer engineering days.
49+
50+
Commercial validation is separate from technical success. Continue investing after the pilot
51+
only if a named buyer agrees the problem is material, uses the workflow with real operators, and
52+
offers a paid production next step. Do not treat repository stars, compliments, or an unpaid
53+
demo as purchase evidence.
54+
55+
## Readiness boundary
56+
57+
The current `main` branch is suitable for local and isolated evaluation. A pilot using
58+
customer-accessible infrastructure must wait for authentication in issue #15 and pull request
59+
#16 to receive the required independent security reviews. Production use additionally requires
60+
idempotency, backup and restore rehearsal, tenant/resource authorization, token rotation or
61+
OIDC, rate limiting, and an agreed operational owner.
62+
63+
## Demo
64+
65+
Start the development API and run the reproducible scenario from another shell:
66+
67+
```bash
68+
make run
69+
python examples/refund_approval.py
70+
```
71+
72+
The script registers a uniquely named support agent, activates it, requests approval for a CNY
73+
1,280 refund, verifies that the request is pending, records a human decision, and checks the four
74+
audit events. It does not call a payment or refund system.
75+
76+
For an authenticated environment, configure a bearer token for each principal:
77+
78+
```bash
79+
export ACP_DEMO_OPERATOR_TOKEN='<operator-token>'
80+
export ACP_DEMO_AGENT_TOKEN='<agent-token>'
81+
export ACP_DEMO_REVIEWER_TOKEN='<reviewer-token>'
82+
python examples/refund_approval.py
83+
```
84+
85+
The subjects configured for those tokens must match `ACP_DEMO_OPERATOR_ACTOR`,
86+
`ACP_DEMO_AGENT_ACTOR`, and `ACP_DEMO_REVIEWER_ACTOR`. `ACP_DEMO_TOKEN` is a convenience fallback
87+
only when all three actor variables are also set to that token's subject. Separate
88+
least-privilege identities are required for a real pilot.

examples/__init__.py

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
"""Executable product scenarios for evaluating Agent Control Plane."""

0 commit comments

Comments
 (0)