RFC 004: Budget allocate and enforce - #6
Conversation
Proposes split allocate request/response schemas and enforce semantics. Co-authored-by: Cursor <cursoragent@cursor.com>
brettin
left a comment
There was a problem hiding this comment.
Test - checking if review posting is permitted
brettin
left a comment
There was a problem hiding this comment.
Review: RFC 004 — Budget allocate and enforce
Overall this RFC correctly names a real and consequential divergence between the spec single-shape BudgetPolicy and the platform allocate/enforce split, but as written it is a stub — three problem bullets and one one-line proposal — that leaves most of the actual contract work undone. A few observations and suggestions below.
brettin
left a comment
There was a problem hiding this comment.
Review: RFC 004 — Budget allocate and enforce
Overall this RFC correctly names a real and consequential divergence between the spec's single-shape BudgetPolicy and the platform's allocate/enforce split, but as written it is a stub that leaves most of the actual contract work undone. A few observations and suggestions below.
Note: Apologies for the noise — earlier in this PR's history my tooling fragmented this review across many comments and a couple of probe messages. This is the consolidated version; the fragments above can be ignored.
✅ What works well
- Gap is real and well-scoped. The spec at
contracts/openapi/gmp-core-v3.yaml:586-620usesbudget-policy.schema.jsonas both theallocateBudgetrequest body and theBudgetPolicyCreatedresponse, withbudgetIdmarked required — meaning a client must invent the id before the server has done anything. The platform implementation inaria-server/aria_server/api/routes/budgets.py(AllocateBudgetRequest,BudgetResponse,allocate_budget) correctly treats the id as server-generated and adds lifecycle fields (status,createdAt,expiresAt,usedTokens/usedToolCalls/usedCostUsd) that have no home in the spec schema. - Enforce divergence accurately identified. Spec
enforceBudgetaccepts the tier-basedModelRoutingPolicy(defaultTierin{cheap_fast, balanced, expensive_smart},fallbackTiers,budgetAware), whilearia-client/aria_client/models/requests.py:EnforceBudgetRequestand the server route ship{budgetId, fallbackModel, routingRules}— a fundamentally different shape. The client documents the split viaSpecTierRoutingPolicyvs.ModelRoutingPolicyaliases inmodels/common.py. - TTL/expiry surfaced. Calling out
ttlSecondsmatters: it is load-bearing in the implementation (expires_at = now + timedelta(seconds=ttl_seconds), default 86400) but absent frombudget-policy.schema.json.
🔍 Suggestions
- Write the actual schemas, not just point at the gap. The RFC 002/003 pattern is to propose concrete request/response shapes; this RFC should propose
AllocateBudgetRequest(nobudgetId, withttlSeconds, optional caps) andAllocateBudgetResponse/Budget(withstatus,createdAt,expiresAt,usedTokens,usedToolCalls,usedCostUsd) as JSON Schema sketches. - Reconcile the two enforce models — don't pick one. Spec tier routing and platform
budgetId+fallbackModelanswer different questions (policy template vs. runtime binding). Propose either (a) keep both as distinct operations (enforceBudget= tier policy, newbindBudgetRouting= runtime), or (b) a union shape withoneOf. Today the silent override breaks conformance. - Cover enforcement-point taxonomy. The real enforcement lives in
aria-server/aria_server/services/mag.py:check_budget(pre-call, on tokens/cost) and_raise_if_tool_calls_exceeded(pre/post on tool calls), raisingBudgetExceededErrorconsumed inservices/run_executor.py. The RFC should document where enforcement happens (per LLM call, per tool call, per run) and thehard_stop/degrade_model/alert_onlyenum frombudget-policy.schema.json— the implementation only doeshard_stoptoday. - Specify accounting units precisely. Spec field is
maxSpendUsd; platform field ismaxCostUsd. Specscopeenum is{task, tenant, user}; platform leavesscopeas a free string. These are silent ABI breaks; flag them explicitly. - Tie to
UsageMeter/usage-meter.schema.json. Allocate/enforce are meaningless without an accounting event model. The specUsageMeter(modelTokens,toolCalls,externalSpendUsd,recordedAt) and the platform's richerUsageMeterResponse(inputTokens/outputTokens/totalTokens,provider,model, per-budget aggregation inGET /v3/usage/tasks/{taskId}) diverge along the same axes asBudgetPolicy; this RFC should treat them together.
❓ Open questions
- What happens on overrun mid-step — does an in-flight tool call complete (post-check) or get cancelled?
_raise_if_tool_calls_exceededwithwhen=postsuggests post-only enforcement for tool counts; is that intentional? - Do allocations survive server restart?
Budgetis persisted in SQLAlchemy butexpires_atis wall-clock; is there a sweeper, or do expired budgets linger until nextcheck_budget? - How do nested or child runs inherit a parent budget — single
budgetIdshared, or sub-allocation?run.budget_idis a nullable scalar today. - Should
enforceBudgetbe idempotent / replace prior policy, or append? The current route returns 202 withstatus=acceptedwithout persisting the routing rules (the code comment says "in production this would update the MAG config"). - What is the failure mode for spec clients that POST a tier-shaped
ModelRoutingPolicyto the platform/v3/budgets/enforce— 422, or silent partial accept?
Summary
Adds RFC 004 for
allocateBudget/enforceBudgetrequest-response shapes and budget-bound routing.Gap IDs
op-allocateBudgetop-enforceBudgetTracking
Part of #1. Base: RFC 003 branch.
Test plan
budget-policy.schema.jsonand OpenAPI Accounting tagMade with Cursor