Skip to content

RFC 004: Budget allocate and enforce - #6

Open
brettin wants to merge 1 commit into
rfc/003-run-status-vocabularyfrom
rfc/004-budgets-allocate-enforce
Open

RFC 004: Budget allocate and enforce#6
brettin wants to merge 1 commit into
rfc/003-run-status-vocabularyfrom
rfc/004-budgets-allocate-enforce

Conversation

@brettin

@brettin brettin commented Jun 4, 2026

Copy link
Copy Markdown
Collaborator

Summary

Adds RFC 004 for allocateBudget / enforceBudget request-response shapes and budget-bound routing.

Gap IDs

  • op-allocateBudget
  • op-enforceBudget

Tracking

Part of #1. Base: RFC 003 branch.

Test plan

  • Review against budget-policy.schema.json and OpenAPI Accounting tag

Made with Cursor

Proposes split allocate request/response schemas and enforce semantics.

Co-authored-by: Cursor <cursoragent@cursor.com>

@brettin brettin left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Test - checking if review posting is permitted

brettin

This comment was marked as duplicate.

brettin

This comment was marked as duplicate.

@brettin brettin left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review: RFC 004 — Budget allocate and enforce

Overall this RFC correctly names a real and consequential divergence between the spec single-shape BudgetPolicy and the platform allocate/enforce split, but as written it is a stub — three problem bullets and one one-line proposal — that leaves most of the actual contract work undone. A few observations and suggestions below.

brettin

This comment was marked as duplicate.

brettin

This comment was marked as duplicate.

brettin

This comment was marked as duplicate.

brettin

This comment was marked as duplicate.

brettin

This comment was marked as duplicate.

brettin

This comment was marked as duplicate.

brettin

This comment was marked as duplicate.

brettin

This comment was marked as duplicate.

@brettin brettin left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review: RFC 004 — Budget allocate and enforce

Overall this RFC correctly names a real and consequential divergence between the spec's single-shape BudgetPolicy and the platform's allocate/enforce split, but as written it is a stub that leaves most of the actual contract work undone. A few observations and suggestions below.

Note: Apologies for the noise — earlier in this PR's history my tooling fragmented this review across many comments and a couple of probe messages. This is the consolidated version; the fragments above can be ignored.


✅ What works well

  1. Gap is real and well-scoped. The spec at contracts/openapi/gmp-core-v3.yaml:586-620 uses budget-policy.schema.json as both the allocateBudget request body and the BudgetPolicyCreated response, with budgetId marked required — meaning a client must invent the id before the server has done anything. The platform implementation in aria-server/aria_server/api/routes/budgets.py (AllocateBudgetRequest, BudgetResponse, allocate_budget) correctly treats the id as server-generated and adds lifecycle fields (status, createdAt, expiresAt, usedTokens / usedToolCalls / usedCostUsd) that have no home in the spec schema.
  2. Enforce divergence accurately identified. Spec enforceBudget accepts the tier-based ModelRoutingPolicy (defaultTier in {cheap_fast, balanced, expensive_smart}, fallbackTiers, budgetAware), while aria-client/aria_client/models/requests.py:EnforceBudgetRequest and the server route ship {budgetId, fallbackModel, routingRules} — a fundamentally different shape. The client documents the split via SpecTierRoutingPolicy vs. ModelRoutingPolicy aliases in models/common.py.
  3. TTL/expiry surfaced. Calling out ttlSeconds matters: it is load-bearing in the implementation (expires_at = now + timedelta(seconds=ttl_seconds), default 86400) but absent from budget-policy.schema.json.

🔍 Suggestions

  1. Write the actual schemas, not just point at the gap. The RFC 002/003 pattern is to propose concrete request/response shapes; this RFC should propose AllocateBudgetRequest (no budgetId, with ttlSeconds, optional caps) and AllocateBudgetResponse / Budget (with status, createdAt, expiresAt, usedTokens, usedToolCalls, usedCostUsd) as JSON Schema sketches.
  2. Reconcile the two enforce models — don't pick one. Spec tier routing and platform budgetId+fallbackModel answer different questions (policy template vs. runtime binding). Propose either (a) keep both as distinct operations (enforceBudget = tier policy, new bindBudgetRouting = runtime), or (b) a union shape with oneOf. Today the silent override breaks conformance.
  3. Cover enforcement-point taxonomy. The real enforcement lives in aria-server/aria_server/services/mag.py:check_budget (pre-call, on tokens/cost) and _raise_if_tool_calls_exceeded (pre/post on tool calls), raising BudgetExceededError consumed in services/run_executor.py. The RFC should document where enforcement happens (per LLM call, per tool call, per run) and the hard_stop / degrade_model / alert_only enum from budget-policy.schema.json — the implementation only does hard_stop today.
  4. Specify accounting units precisely. Spec field is maxSpendUsd; platform field is maxCostUsd. Spec scope enum is {task, tenant, user}; platform leaves scope as a free string. These are silent ABI breaks; flag them explicitly.
  5. Tie to UsageMeter / usage-meter.schema.json. Allocate/enforce are meaningless without an accounting event model. The spec UsageMeter (modelTokens, toolCalls, externalSpendUsd, recordedAt) and the platform's richer UsageMeterResponse (inputTokens / outputTokens / totalTokens, provider, model, per-budget aggregation in GET /v3/usage/tasks/{taskId}) diverge along the same axes as BudgetPolicy; this RFC should treat them together.

❓ Open questions

  1. What happens on overrun mid-step — does an in-flight tool call complete (post-check) or get cancelled? _raise_if_tool_calls_exceeded with when=post suggests post-only enforcement for tool counts; is that intentional?
  2. Do allocations survive server restart? Budget is persisted in SQLAlchemy but expires_at is wall-clock; is there a sweeper, or do expired budgets linger until next check_budget?
  3. How do nested or child runs inherit a parent budget — single budgetId shared, or sub-allocation? run.budget_id is a nullable scalar today.
  4. Should enforceBudget be idempotent / replace prior policy, or append? The current route returns 202 with status=accepted without persisting the routing rules (the code comment says "in production this would update the MAG config").
  5. What is the failure mode for spec clients that POST a tier-shaped ModelRoutingPolicy to the platform /v3/budgets/enforce — 422, or silent partial accept?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants