Skip to content

Link structured evidence to coverage goals - #7

Merged
chriswu727 merged 1 commit into
mainfrom
agent/structured-coverage-evidence
Jul 21, 2026
Merged

Link structured evidence to coverage goals#7
chriswu727 merged 1 commit into
mainfrom
agent/structured-coverage-evidence

Conversation

@chriswu727

Copy link
Copy Markdown
Owner

What changed

  • open a bounded evidence window when a coverage goal becomes in progress
  • automatically associate terminal goals with tested URL paths, value-redacted actions, screenshots, persistence checks, bugs, and observations
  • retain explicit omitted counts when a compact evidence view reaches its safety cap
  • render linked evidence and verification verdicts in HTML while preserving the stable structure in JSON
  • document the workflow and the sensitive-data boundary in README, AGENTS, CHANGELOG, and PRIVACY

Why

The 0.6 coverage ledger required a narrative explanation but still trusted the agent to summarize what it had actually exercised. Structured references make goal completion auditable against session artifacts without exposing typed or pasted values in action references.

Impact

Coverage reports now answer both whether a goal was marked complete and what session evidence supports that state. Existing goal fields and sessions without explicit goals remain compatible. The core MCP profile remains at 30 tools and about 33,518 estimated context characters.

Validation

  • python -m pytest -q — 244 passed
  • python -m argus.bench --target all — 34/34 recall and 3/3 FP resistance
  • targeted coverage/replay/release tests — 47 passed
  • uv build — wheel and sdist built successfully

@chriswu727
chriswu727 marked this pull request as ready for review July 21, 2026 23:52

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 43a72a92ba

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread argus/mcp_server.py
description = "field input (value omitted)"
s.coverage_action_trace.append({
"tool": tool,
"description": _short(_redact(description), 120),

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Redact dropped file names from coverage actions

When a goal evidence window includes drop_file, the existing drop_file trace records the local basename (for example, 2026-layoffs.xlsx onto Upload) as the action description, and this new generic copier writes that description directly into coverage.evidence_refs.actions for both JSON and HTML reports. That exposes local or customer data in the coverage ledger even though these references are meant to contain value-redacted actions; sanitize drop_file the way upload_file only records a count.

AGENTS.md reference: AGENTS.md:L56-L56

Useful? React with 👍 / 👎.

@chriswu727
chriswu727 merged commit 60f4b7c into main Jul 21, 2026
2 checks passed
@chriswu727
chriswu727 deleted the agent/structured-coverage-evidence branch July 21, 2026 23:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant