[codex] Clarify dashboard health and quota usage by Pigbibi · Pull Request #11 · QuantStrategyLab/AIAuditBridge

Pigbibi · 2026-07-03T23:27:44Z

Summary

Add structured health degradation reasons so the dashboard can explain why a service is degraded instead of only showing the status label.
Split quota status into API key and Codex internal-estimate buckets, including calls, estimated cost, and budget progress.
Update dashboard copy to make clear the quota panel is an internal estimate, not live OpenAI/Anthropic/Codex account balance.
Add regression tests for health degradation reasons and API key vs Codex quota summary.

Root Cause

The previous dashboard rendered degraded without exposing the threshold that triggered it. The quota panel also showed only repo-level estimated cost, which made API-key and Codex usage look like one inaccurate budget line.

Validation

python3 -m unittest tests.test_quota tests.test_health -v
node --test cloudflare/ai-gateway-dash/tests/index.test.mjs
python3 -m ruff check service/quota.py service/health.py tests/test_quota.py tests/test_health.py
python3 -m unittest discover -s tests -v — 208 tests OK
git diff --check

Notes

The dashboard still cannot read live provider account quota directly; it now labels these values as internal estimates to avoid implying live balance accuracy.

cursor · 2026-07-03T23:27:58Z

Bugbot is not enabled for your account, so this pull request was not reviewed.

Enable Bugbot in the Cursor dashboard to get automatic reviews on future PRs.

github-actions · 2026-07-03T23:30:36Z

🤖 Codex PR Review

✅ Merge allowed: 2 finding(s) reported but none are blocking

ℹ️ Other Findings

1. 🟡 [MEDIUM] Reliability in `scripts/deploy_codex_audit_service.sh`

write_admin_env_file_if_needed() 在当前部署环境里没有 OPENAI_ADMIN_KEY 和 ANTHROPIC_ADMIN_KEY 时会直接删除现有的 ${ADMIN_ENV_FILE}。这意味着一次普通重部署如果没有重新导出这两个 secret，就会把之前可用的 Admin API 配置静默清掉，/v1/ai/quota 的 OpenAI/Anthropic 账户用量会突然消失。 (line 196)

Suggestion: 仅在显式传入“清除 Admin key”开关时才删除该文件；如果本次部署未提供新 key，则保留现有 admin.env，避免无意中关闭线上配额快照。

2. 🟡 [MEDIUM] Reliability in `service/feedback.py`

影子审计状态从内存改成磁盘持久化后，_write_shadow_disagreement() / path.unlink() 的文件操作没有任何错误处理。CODEX_AUDIT_SERVICE_JOB_DIR 一旦出现权限、磁盘满、损坏或临时 I/O 异常，record_shadow_disagreement() 现在会把异常直接抛到请求链路上，把原本应该成功的业务请求变成 500。 (line 339)

Suggestion: 把影子审计落盘改成 best-effort：捕获 OSError/PermissionError/JSONDecodeError，记录告警并回退到非持久化行为，避免让遥测持久化失败阻断主请求。

Review by Codex PR Review bot • PR

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: d5f1d214f9

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Sign in to view

            if model == "codex-cli":
                record.codex_calls += 1
+            else:
+                record.api_calls += 1


Co-Authored-By: Codex <noreply@openai.com>

cursor · 2026-07-03T23:33:53Z

Bugbot is not enabled for your account, so this pull request was not reviewed.

Enable Bugbot in the Cursor dashboard to get automatic reviews on future PRs.

Co-Authored-By: Codex <noreply@openai.com>

cursor · 2026-07-03T23:49:00Z

Bugbot is not enabled for your account, so this pull request was not reviewed.

Enable Bugbot in the Cursor dashboard to get automatic reviews on future PRs.

Co-Authored-By: Codex <noreply@openai.com>

cursor · 2026-07-04T00:14:57Z

Bugbot is not enabled for your account, so this pull request was not reviewed.

Enable Bugbot in the Cursor dashboard to get automatic reviews on future PRs.

Co-Authored-By: Codex <noreply@openai.com>

cursor · 2026-07-04T00:44:09Z

Bugbot is not enabled for your account, so this pull request was not reviewed.

Enable Bugbot in the Cursor dashboard to get automatic reviews on future PRs.

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 36a3dd3588

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Sign in to view

            return {
-                "repos": {r: self.status(r) for r in self._records},
+                "repos": repos,
+                "summary": self._summary_from_statuses(repos),


Sign in to view

+function healthReasonText(r){const path=r.path||"endpoint";if(r.reason==="error_rate")return path+" 错误率 "+fmtPct(r.value||0)+" ≥ "+fmtPct(r.threshold||0);if(r.reason==="p95_latency_ms")return path+" P95 "+fmtNum(r.value||0)+" ms ≥ "+fmtNum(r.threshold||0)+" ms";return path+" "+(r.reason||"degraded")}
+function renderHealth(h){setStatus(h.status||"unknown");setUpdated("health-updated");const target=document.getElementById("health"),endpoints=h.endpoints||[],reasons=h.degradation_reasons||[],total=endpoints.reduce((s,i)=>s+(Number(i.total)||0),0),errors=endpoints.reduce((s,i)=>s+(Number(i.errors)||0),0),p95=Math.max(0,...endpoints.map(i=>Number(i.p95_ms)||0)),details=el("div","");append(details,statRow("接口数量",endpoints.length,"info"),statRow("总请求",fmtNum(total),""),statRow("错误",fmtNum(errors),errors?"err":"ok"),statRow("最高 P95",fmtNum(p95)+" ms",p95>30000?"err":p95>10000?"warn":"ok"));if(reasons.length){reasons.slice(0,3).forEach((reason,i)=>append(details,statRow(i?"降级原因 "+(i+1):"降级原因",healthReasonText(reason),clsStatus(reason.severity||h.status))));if(reasons.length>3)append(details,statRow("更多原因","+"+fmtNum(reasons.length-3),"warn"))}else if(h.last_error&&h.last_error.message){append(details,statRow("最近错误",String(h.last_error.message).slice(0,80),"warn"))}clear(target);append(target,append(el("div","metric-line"),el("div","big ",statusText(h.status||"unknown")),el("span","delta "+clsStatus(h.status||"unknown"),fmtNum(Math.round((h.uptime_seconds||0)/3600))+" h")),details)}
+function quotaTotalRow(label,d){const row=el("div","stat-row"),used=Number(d&&d.total_cost_usd)||0;append(row,el("span","stat-label",label),el("span","stat-value info",fmtUSD(used)));return row}
+function codexAccountRow(d){const row=el("div","stat-row"),r=d&&d.rate_limits||{},p=r.primary||{},s=r.secondary||{},credits=r.credits||{},parts=[];if(p.used_percent!==null&&p.used_percent!==undefined)parts.push((p.window_duration_mins?Math.round(p.window_duration_mins/60)+"h":"主窗口")+" "+p.used_percent+"%");if(s.used_percent!==null&&s.used_percent!==undefined)parts.push((s.window_duration_mins?Math.round(s.window_duration_mins/1440)+"d":"次窗口")+" "+s.used_percent+"%");if(r.plan_type)parts.push(String(r.plan_type));if(credits.balance!==null&&credits.balance!==undefined)parts.push("credits "+credits.balance);append(row,el("span","stat-label","Codex 账户"),el("span","stat-value "+(Number(p.used_percent)>80?"err":Number(p.used_percent)>50?"warn":"ok"),parts.join(" · ")||"实时可用"));return row}


Sign in to view

+    try:
+        timeout = timeout_seconds if timeout_seconds is not None else float(os.environ.get("CODEX_AUDIT_SERVICE_CODEX_ACCOUNT_TIMEOUT_SECONDS", "8"))
+        deadline = time.monotonic() + max(1.0, timeout)
+        proc = subprocess.Popen(


chatgpt-codex-connector · 2026-07-04T00:50:39Z

+        tokens_input = int(d.get("tokens_input", 0))
+        tokens_output = int(d.get("tokens_output", 0))
+        has_split_api_tokens = "api_key_tokens_input" in d or "api_key_tokens_output" in d
+        legacy_api_activity = api_key_cost_usd > 0 or (not has_split_api_tokens and (tokens_input > 0 or tokens_output > 0))


Avoid treating Codex history as API-key usage

When an older persisted record already has codex_calls and token counts but its total cost is fully accounted for by the inferred Codex cost, this condition still marks it as legacy API activity solely because tokens are present. After a service restart, those Codex-only tokens are copied into api_key_tokens_* and set api_calls_incomplete, so the new API Key estimate can show usage that was actually Codex usage; only use the token fallback when there is residual API-key cost or no Codex-only evidence.

Useful? React with 👍 / 👎.

Co-Authored-By: Codex <noreply@openai.com>

cursor · 2026-07-04T00:52:07Z

Bugbot is not enabled for your account, so this pull request was not reviewed.

Enable Bugbot in the Cursor dashboard to get automatic reviews on future PRs.

Co-Authored-By: Codex <noreply@openai.com>

cursor · 2026-07-04T00:58:01Z

Bugbot is not enabled for your account, so this pull request was not reviewed.

Enable Bugbot in the Cursor dashboard to get automatic reviews on future PRs.

cursor · 2026-07-04T03:32:26Z

Bugbot is not enabled for your account, so this pull request was not reviewed.

Enable Bugbot in the Cursor dashboard to get automatic reviews on future PRs.

cursor · 2026-07-04T03:40:59Z

Bugbot is not enabled for your account, so this pull request was not reviewed.

Enable Bugbot in the Cursor dashboard to get automatic reviews on future PRs.

cursor · 2026-07-04T03:49:00Z

Bugbot is not enabled for your account, so this pull request was not reviewed.

Enable Bugbot in the Cursor dashboard to get automatic reviews on future PRs.

cursor · 2026-07-04T03:55:58Z

Bugbot is not enabled for your account, so this pull request was not reviewed.

Enable Bugbot in the Cursor dashboard to get automatic reviews on future PRs.

cursor · 2026-07-04T04:05:16Z

Bugbot is not enabled for your account, so this pull request was not reviewed.

Enable Bugbot in the Cursor dashboard to get automatic reviews on future PRs.

cursor · 2026-07-04T04:11:17Z

Bugbot is not enabled for your account, so this pull request was not reviewed.

Enable Bugbot in the Cursor dashboard to get automatic reviews on future PRs.

cursor · 2026-07-04T04:44:26Z

Bugbot is not enabled for your account, so this pull request was not reviewed.

Enable Bugbot in the Cursor dashboard to get automatic reviews on future PRs.

Co-Authored-By: Codex <noreply@openai.com>

cursor · 2026-07-04T04:51:49Z

Bugbot is not enabled for your account, so this pull request was not reviewed.

Enable Bugbot in the Cursor dashboard to get automatic reviews on future PRs.

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 75853fef0d

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Sign in to view

+    usage = None
+    costs = None
+    try:
+        usage = _sum_usage(_request_json("/organizations/usage_report/messages", usage_params, admin_key, timeout))


Co-Authored-By: Codex <noreply@openai.com>

cursor · 2026-07-04T11:49:29Z

Bugbot is not enabled for your account, so this pull request was not reviewed.

Enable Bugbot in the Cursor dashboard to get automatic reviews on future PRs.

Co-Authored-By: Codex <noreply@openai.com>

cursor · 2026-07-04T11:54:01Z

Bugbot is not enabled for your account, so this pull request was not reviewed.

Enable Bugbot in the Cursor dashboard to get automatic reviews on future PRs.

Co-Authored-By: Codex <noreply@openai.com>

cursor · 2026-07-04T12:01:43Z

Bugbot is not enabled for your account, so this pull request was not reviewed.

Enable Bugbot in the Cursor dashboard to get automatic reviews on future PRs.

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 05c3932ceb

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Sign in to view

+        }
+        with ThreadPoolExecutor(max_workers=len(readers), thread_name_prefix="quota-account") as executor:
+            futures = {executor.submit(reader): name for name, reader in readers.items()}
+            done, _ = wait(futures)


cursor · 2026-07-04T12:25:26Z

Bugbot is not enabled for your account, so this pull request was not reviewed.

Enable Bugbot in the Cursor dashboard to get automatic reviews on future PRs.

cursor · 2026-07-04T12:28:01Z

Bugbot is not enabled for your account, so this pull request was not reviewed.

Enable Bugbot in the Cursor dashboard to get automatic reviews on future PRs.

cursor · 2026-07-04T12:36:12Z

Bugbot is not enabled for your account, so this pull request was not reviewed.

Enable Bugbot in the Cursor dashboard to get automatic reviews on future PRs.

cursor · 2026-07-04T12:43:42Z

Bugbot is not enabled for your account, so this pull request was not reviewed.

Enable Bugbot in the Cursor dashboard to get automatic reviews on future PRs.

Co-Authored-By: Codex <noreply@openai.com>

cursor · 2026-07-04T12:50:59Z

Bugbot is not enabled for your account, so this pull request was not reviewed.

Enable Bugbot in the Cursor dashboard to get automatic reviews on future PRs.

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 656b8a8119

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Sign in to view

+            value = amount.get("value") if isinstance(amount, dict) else amount
+            by_currency[currency] = by_currency.get(currency, 0.0) + _num(value)


Pigbibi marked this pull request as ready for review July 3, 2026 23:27

chatgpt-codex-connector Bot reviewed Jul 3, 2026

View reviewed changes

Comment thread service/quota.py

if model == "codex-cli":

record.codex_calls += 1

else:

record.api_calls += 1

This comment was marked as off-topic.

Sign in to view

Clarify dashboard health and quota usage

7e16c25

Co-Authored-By: Codex <noreply@openai.com>

Pigbibi force-pushed the codex/fix-dashboard-health-quota branch from d5f1d21 to 7e16c25 Compare July 3, 2026 23:33

Clarify quota dashboard breakdown

afc1370

Co-Authored-By: Codex <noreply@openai.com>

Address quota dashboard review feedback

73a6441

Co-Authored-By: Codex <noreply@openai.com>

Read Codex quota from app server

36a3dd3

Co-Authored-By: Codex <noreply@openai.com>

chatgpt-codex-connector Bot reviewed Jul 4, 2026

View reviewed changes

Address Codex quota review feedback

857fea3

Co-Authored-By: Codex <noreply@openai.com>

Sanitize Codex quota app-server environment

7003a12

Co-Authored-By: Codex <noreply@openai.com>

Pigbibi force-pushed the codex/fix-dashboard-health-quota branch from d1b3d0c to 46b5832 Compare July 4, 2026 03:40

Pigbibi force-pushed the codex/fix-dashboard-health-quota branch from 46b5832 to 25644c8 Compare July 4, 2026 03:48

Pigbibi force-pushed the codex/fix-dashboard-health-quota branch from 25644c8 to e1c95ee Compare July 4, 2026 03:55

Pigbibi force-pushed the codex/fix-dashboard-health-quota branch from e1c95ee to a147462 Compare July 4, 2026 04:05

Pigbibi force-pushed the codex/fix-dashboard-health-quota branch from a147462 to 014eeb3 Compare July 4, 2026 04:11

Pigbibi force-pushed the codex/fix-dashboard-health-quota branch from 014eeb3 to 7f70405 Compare July 4, 2026 04:17

Pigbibi force-pushed the codex/fix-dashboard-health-quota branch from 30a0a1f to 34ad959 Compare July 4, 2026 04:44

Add provider admin usage snapshots

75853fe

Co-Authored-By: Codex <noreply@openai.com>

Pigbibi force-pushed the codex/fix-dashboard-health-quota branch from 34ad959 to 75853fe Compare July 4, 2026 04:51

chatgpt-codex-connector Bot reviewed Jul 4, 2026

View reviewed changes

Comment thread service/anthropic_admin_usage.py Outdated

usage = None

costs = None

try:

usage = _sum_usage(_request_json("/organizations/usage_report/messages", usage_params, admin_key, timeout))

This comment was marked as off-topic.

Sign in to view

Pigbibi temporarily deployed to codex-vps-ops July 4, 2026 11:18 — with GitHub Actions Inactive

Pigbibi temporarily deployed to codex-vps-ops July 4, 2026 11:21 — with GitHub Actions Inactive

Pigbibi temporarily deployed to codex-vps-ops July 4, 2026 11:41 — with GitHub Actions Inactive

Polish dashboard usage card

ee5a509

Co-Authored-By: Codex <noreply@openai.com>

Clarify dashboard report empty states

bc14fb7

Co-Authored-By: Codex <noreply@openai.com>

Avoid marking background Codex jobs unhealthy

05c3932

Co-Authored-By: Codex <noreply@openai.com>

Pigbibi temporarily deployed to codex-vps-ops July 4, 2026 12:01 — with GitHub Actions Inactive

chatgpt-codex-connector Bot reviewed Jul 4, 2026

View reviewed changes

Comment thread service/quota.py

}

with ThreadPoolExecutor(max_workers=len(readers), thread_name_prefix="quota-account") as executor:

futures = {executor.submit(reader): name for name, reader in readers.items()}

done, _ = wait(futures)

This comment was marked as off-topic.

Sign in to view

Pigbibi force-pushed the codex/fix-dashboard-health-quota branch from eea7ce9 to 0d4223e Compare July 4, 2026 12:27

Pigbibi force-pushed the codex/fix-dashboard-health-quota branch from 0d4223e to c014aea Compare July 4, 2026 12:36

Pigbibi force-pushed the codex/fix-dashboard-health-quota branch from c014aea to cc0e752 Compare July 4, 2026 12:43

feat: harden autonomy and review gates

656b8a8

Co-Authored-By: Codex <noreply@openai.com>

Pigbibi force-pushed the codex/fix-dashboard-health-quota branch from cc0e752 to 656b8a8 Compare July 4, 2026 12:50

chatgpt-codex-connector Bot reviewed Jul 4, 2026

View reviewed changes

Comment thread service/anthropic_admin_usage.py

Comment on lines +127 to +128

value = amount.get("value") if isinstance(amount, dict) else amount

by_currency[currency] = by_currency.get(currency, 0.0) + _num(value)

This comment was marked as off-topic.

Sign in to view

Pigbibi merged commit 4fe8e56 into main Jul 4, 2026
6 checks passed

Pigbibi deleted the codex/fix-dashboard-health-quota branch July 5, 2026 00:56

		value = amount.get("value") if isinstance(amount, dict) else amount
		by_currency[currency] = by_currency.get(currency, 0.0) + _num(value)

Uh oh!

Conversation

Pigbibi commented Jul 3, 2026

Summary

Root Cause

Validation

Notes

Uh oh!

cursor Bot commented Jul 3, 2026

Uh oh!

github-actions Bot commented Jul 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🤖 Codex PR Review

ℹ️ Other Findings

1. 🟡 [MEDIUM] Reliability in scripts/deploy_codex_audit_service.sh

2. 🟡 [MEDIUM] Reliability in service/feedback.py

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

This comment was marked as off-topic.

Uh oh!

cursor Bot commented Jul 3, 2026

Uh oh!

cursor Bot commented Jul 3, 2026

Uh oh!

cursor Bot commented Jul 4, 2026

Uh oh!

cursor Bot commented Jul 4, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

This comment was marked as off-topic.

Uh oh!

This comment was marked as off-topic.

Uh oh!

This comment was marked as off-topic.

Uh oh!

chatgpt-codex-connector Bot Jul 4, 2026

Choose a reason for hiding this comment

Uh oh!

cursor Bot commented Jul 4, 2026

Uh oh!

cursor Bot commented Jul 4, 2026

Uh oh!

cursor Bot commented Jul 4, 2026

Uh oh!

cursor Bot commented Jul 4, 2026

Uh oh!

cursor Bot commented Jul 4, 2026

Uh oh!

cursor Bot commented Jul 4, 2026

Uh oh!

cursor Bot commented Jul 4, 2026

Uh oh!

cursor Bot commented Jul 4, 2026

Uh oh!

cursor Bot commented Jul 4, 2026

Uh oh!

cursor Bot commented Jul 4, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

This comment was marked as off-topic.

Uh oh!

cursor Bot commented Jul 4, 2026

Uh oh!

cursor Bot commented Jul 4, 2026

Uh oh!

cursor Bot commented Jul 4, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

github-actions Bot commented Jul 3, 2026 •

edited

Loading

1. 🟡 [MEDIUM] Reliability in `scripts/deploy_codex_audit_service.sh`

2. 🟡 [MEDIUM] Reliability in `service/feedback.py`