Skip to content

part of #4206: model_max_class ceiling doesn't cover raw model-string / model_class=None call sites #4324

Description

@tya5

[e2e-coder] — part of #4206.

#4318 (#4206 T1) wires the ②bounding `model_max_class` ceiling into
`recorded_acompletion`, enforced when a caller passes a resolved
`model_class` (one of the 3 standard tiers). Two real gaps in coverage,
both principled limitations rather than bugs in the T1 implementation
(lead-coder review on #4318):

  1. A raw model string bypasses the ceiling entirely. When `--router-model`
    (or any future equivalent) is set to a literal LiteLLM string rather than a
    class name, `resolve_purpose_class`'s `explicit is not None` branch
    returns it unchanged. `model_class_exceeds_ceiling`'s own contract says a
    name outside the 3 standard tiers is "not comparable on this axis" → never
    a violation, so the ceiling is silently a no-op for that call. This isn't
    fixable by tightening the predicate — a raw string has no declared class to
    compare, by construction.

  2. `model_class=None` call sites are permanently outside the axis.
    `dev/dogfood/interpretation.py` and `dev/dogfood/verifiers/reply.py` both
    pass `model_class=None` because they use a fixed literal model string
    (never resolved via `class_for_purpose`) — these are real-cost dogfood
    calls, not auxiliary/free ones, so "outside the axis" is worth being
    explicit about even though it's the same principled reason as fix(mcp): serialize concurrent send_to_agent_impl per agent #1 above.
    (Compaction's `model_class=None` is different and already correctly
    scoped — compaction model is frozen at session construction and does not follow a runtime model switch; force_close does #3785, compaction always follows the session's active model, not
    a class.)

Possible directions (not scoped here, for whoever picks this up):

  • A richer per-model cost declaration (mapping a raw model string to an
    implied tier) so the ceiling can still apply to an explicit `--router-model`
    override.
  • Deciding whether dogfood's real-cost calls should be threaded onto a class
    at all, or whether a separate budget-style cap (not this axis) is the
    right tool for that surface.

No code changes proposed here — this is a scope-boundary record, not a task
assignment.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions