Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 9 additions & 9 deletions .env.public
Original file line number Diff line number Diff line change
Expand Up @@ -20,13 +20,13 @@ LLM_RELAY_HISTORY_RAW=0
LLM_RELAY_LANG=en

# ── Turn counter thresholds (CC) ─────────────────────────────────
# Zone A — absolute token count, aligned with Zone B (50/70/90/100% of 665K).
# 665K reflects the Opus 4.7 client-side auto-compact trigger observed around
# 650-670K cumulative context. Operators should hand off to a new session
# before reaching Red (600K) to avoid compaction-induced continuity loss.
LLM_TOKEN_A_YELLOW=332000
LLM_TOKEN_A_ORANGE=465000
LLM_TOKEN_A_RED=600000
LLM_TOKEN_A_HARD=665000
# Zone A — absolute token count, aligned with Zone B (50/70/90/100% of 1M).
# 1M reflects the Opus 4.7 [1m] model window. Client-side auto-compact
# triggers around 95% (~950K). Operators should hand off to a new session
# before reaching Red (900K) to avoid compaction-induced continuity loss.
LLM_TOKEN_A_YELLOW=500000
LLM_TOKEN_A_ORANGE=700000
LLM_TOKEN_A_RED=900000
LLM_TOKEN_A_HARD=1000000
# Zone B — context window ceiling (for ratio calculation)
LLM_TOKEN_CEILING=665000
LLM_TOKEN_CEILING=1000000
2 changes: 1 addition & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ All notable changes to llm-relay are documented here.
- **Incremental composition cache** (`composition.py`, #16): `analyze_session_composition` and `analyze_session_composition_per_turn` now fold delta turns into cached state instead of re-walking the full session on every new turn. Previously the cache invalidated whenever any new turn arrived, forcing an O(n²) replay; on long-running sessions this caused `/api/v1/turns` to balloon to tens of seconds and daemon RSS to climb above 10 GB within ~30 min of normal multi-agent traffic. Cache now keys on `session_id` alone with `(max_turn_processed, accumulated, totals, …)` state; new turns trigger a `WHERE turn_number > max_turn_processed` fetch. Compaction events (`storage_mode="full"`) reset accumulated state and re-absorb the snapshot. Exception during fold falls back to a full rebuild so an incremental error can't poison the cache.

### Changed
- **Claude Code zone defaults rescaled to 665K ceiling**: Yellow 332K / Orange 465K / Red 600K / Hard 665K (`_zones.py`, `tui.py`, `display.py`, `.env.public`). Rationale: Claude Code's client-side auto-compact (re-introduced in v2.1.139) triggers around 650-670K cumulative context; the new defaults let operators hand off to a new session before compaction degrades context continuity. Override via `LLM_TOKEN_A_*` / `LLM_TOKEN_CEILING` env vars if a different ceiling is needed (e.g. `500000` for public deployments without 1M entitlement).
- **Claude Code zone defaults restored to 1M model window**: Yellow 500K / Orange 700K / Red 900K / Hard 1M (`_zones.py`, `tui.py`, `display.py`, `.env.public`). Reverts the 665K rescale that was applied on the basis of a single 5/13 observation of auto-compact firing around 650K. Subsequent multi-hundred-K sessions did not reproduce that trigger point — Opus 4.7 [1m] reaches ~95% (~950K) before client-side auto-compact engages, so the 665K ceiling was unnecessarily conservative. Stock deployments without 1M context entitlement can override via `LLM_TOKEN_CEILING=200000`.
- **Red-zone message split for CC vs Codex** (`i18n.py`): new keys `zone.abs.red.cc` and `zone.ratio.red.cc` carry an explicit "Auto-compact imminent — hand off to a new session" guideline used by CC paths. Codex paths keep the existing "Session rotation required" wording (Codex has no client-side compaction; the 400K hard limit semantics differ).
- **`duplicate_reads` no longer accumulates across compaction** (`composition.py`): when a `storage_mode="full"` turn arrives, read counts reset together with the accumulated context. Previously reads from pre-compaction turns were counted as duplicates of post-compaction reads, inflating `duplicate_read_count` and triggering spurious `duplicate_read_warning` flips. Visible in `/api/v1/turns`, `/api/v1/display`, dashboard Context Health, and TUI duplicate-file listings.

Expand Down
29 changes: 16 additions & 13 deletions src/llm_relay/api/_zones.py
Original file line number Diff line number Diff line change
Expand Up @@ -23,17 +23,20 @@
# ── Claude Code zone classification ──

# Cached at module load — avoids repeated os.getenv in hot path.
# Zone A defaults aligned with Zone B ratios of the practical 665K ceiling:
# Yellow 332K (50%) / Orange 465K (70%) / Red 600K (90%) / Hard 665K (100%).
# Rationale: Claude Code's client-side auto-compact (re-introduced in v2.1.139)
# triggers around 650-670K cumulative context. Operators rely on these zones to
# hand off to a new session before compaction degrades context continuity.
# Override via LLM_TOKEN_A_*/LLM_TOKEN_CEILING if the exact threshold is confirmed.
_CACHED_TOKEN_A_YELLOW = int(os.getenv("LLM_TOKEN_A_YELLOW", "332000"))
_CACHED_TOKEN_A_ORANGE = int(os.getenv("LLM_TOKEN_A_ORANGE", "465000"))
_CACHED_TOKEN_A_RED = int(os.getenv("LLM_TOKEN_A_RED", "600000"))
_CACHED_TOKEN_A_HARD = int(os.getenv("LLM_TOKEN_A_HARD", "665000"))
_CACHED_TOKEN_CEILING = int(os.getenv("LLM_TOKEN_CEILING", "665000"))
# Zone A defaults aligned with Zone B ratios of the 1M model window:
# Yellow 500K (50%) / Orange 700K (70%) / Red 900K (90%) / Hard 1M (100%).
# Rationale: Claude Code on Opus 4.7 [1m] supports a 1M context window.
# Client-side auto-compact triggers around 95% (~950K), not the 650-670K
# observed in a single 5/13 session — that observation has not reproduced
# on subsequent multi-hundred-K sessions. Operators rely on these zones
# to hand off before the auto-compact trigger degrades context continuity.
# Override via LLM_TOKEN_A_*/LLM_TOKEN_CEILING for deployments without
# 1M context entitlement (e.g. set LLM_TOKEN_CEILING=200000 for stock 200K).
_CACHED_TOKEN_A_YELLOW = int(os.getenv("LLM_TOKEN_A_YELLOW", "500000"))
_CACHED_TOKEN_A_ORANGE = int(os.getenv("LLM_TOKEN_A_ORANGE", "700000"))
_CACHED_TOKEN_A_RED = int(os.getenv("LLM_TOKEN_A_RED", "900000"))
_CACHED_TOKEN_A_HARD = int(os.getenv("LLM_TOKEN_A_HARD", "1000000"))
_CACHED_TOKEN_CEILING = int(os.getenv("LLM_TOKEN_CEILING", "1000000"))


def _classify_zone(turns: int) -> tuple:
Expand Down Expand Up @@ -79,8 +82,8 @@ def _classify_zone_absolute(tokens: int) -> tuple:
def _classify_zone_ratio(tokens: int, ceiling: Optional[int] = None) -> tuple:
"""Zone B -- ratio-of-ceiling classification (50/70/90/100%).

Env: LLM_TOKEN_CEILING (default 665K — Opus 4.7 client-side auto-compact ceiling.
Override to 500K for public deployments without 1M context entitlement.)
Env: LLM_TOKEN_CEILING (default 1M — Opus 4.7 [1m] model window.
Override to 200K for stock deployments without 1M context entitlement.)
Returns (zone, zone_label, next_threshold, message).
"""
if ceiling is None:
Expand Down
2 changes: 1 addition & 1 deletion src/llm_relay/api/display.py
Original file line number Diff line number Diff line change
Expand Up @@ -1084,7 +1084,7 @@ def discover_external_cli_sessions(
official_context_window = _OPENAI_CODEX_OFFICIAL_CONTEXT_WINDOW
official_max_output = _OPENAI_CODEX_OFFICIAL_MAX_OUTPUT
else:
display_ceiling = int(os.getenv("LLM_TOKEN_CEILING", "665000"))
display_ceiling = int(os.getenv("LLM_TOKEN_CEILING", "1000000"))
official_context_window = 0
official_max_output = 0
try:
Expand Down
2 changes: 1 addition & 1 deletion src/llm_relay/detect/tui.py
Original file line number Diff line number Diff line change
Expand Up @@ -83,7 +83,7 @@ def _render_session_panel(s: Dict[str, Any]) -> Panel:
# ── Token metrics ──
current = s.get("current_ctx", 0)
peak = s.get("peak_ctx", 0)
ceiling = s.get("ceiling", 665_000)
ceiling = s.get("ceiling", 1_000_000)
recent = s.get("recent_peak", 0)
cumul = s.get("cumul_unique", 0)
zone = s.get("zone", "green")
Expand Down
Loading