Skip to content

feat(contradictions): advisory scanner for conflicting approved claims#405

Open
jsdevninja wants to merge 1 commit into
vouchdev:mainfrom
jsdevninja:feat/contradiction-scan
Open

feat(contradictions): advisory scanner for conflicting approved claims#405
jsdevninja wants to merge 1 commit into
vouchdev:mainfrom
jsdevninja:feat/contradiction-scan

Conversation

@jsdevninja

@jsdevninja jsdevninja commented Jul 6, 2026

Copy link
Copy Markdown
Contributor

closes #314

summary

  • adds src/vouch/contradictions.py: a read-only heuristic scanner that groups approved claims by shared Claim.entities and flags same-topic pairs whose text disagrees in polarity (one asserts, the other negates) as candidate contradictions.
  • adds vouch contradict-scan (cli-only, matching how vouch dedup ships without a dedicated kb.* method): --threshold, --entity, --limit, --dry-run/--no-dry-run (default dry-run).
  • dry-run only prints candidate pairs with a score and the shared entity. without --dry-run, each surviving pair files exactly one pending contradicts relation proposal via proposals.propose_relation, visible in kb.list_pending / vouch pending.
  • a pair already cross-linked via Claim.contradicts, already joined by an approved contradicts relation edge, or already carrying a pending contradicts proposal is skipped, so repeat scans don't pile up duplicates.
  • the scanner itself never sets ClaimStatus.CONTESTED, never writes a Relation, and never touches Claim.contradicts — all business logic lives in contradictions.py, storage.py is untouched.

design note (worth a second pair of eyes)

approving the resulting relation proposal currently lands a plain Relation(relation=CONTRADICTS) edge — it does not flip both claims to CONTESTED or cross-link Claim.contradicts. that full symmetric transition still only happens via the pre-existing, deliberately-manual lifecycle.contradict (kb.contradict), whose own docstring says it intentionally bypasses the proposal queue and warns against refactoring it for stricter gating. i kept proposals.approve() untouched rather than special-casing relation == "contradicts" there, since that's shared code for every relation kind and the issue's acceptance checklist only requires the proposal to land as a CONTRADICTS edge on approval, not the status flip. happy to wire that up instead if the intended reading of "the actual write still flows through lifecycle.contradict semantics on approval" was that kb.approve itself should trigger the full transition.

testing

  • tests/test_contradictions.py (15 tests): candidate scoring/grouping, entity/threshold/limit filters, inactive-claim exclusion, all three duplicate-skip paths (contradicts field / approved edge / pending proposal), dry-run writes nothing, non-dry-run files exactly one proposal per pair and never mutates claim status or writes a relation directly, repeat scans don't duplicate, and a full scan → propose → approve integration test landing a CONTRADICTS edge.
  • manually exercised vouch contradict-scan end-to-end against a scratch kb (dry-run, --no-dry-run, re-scan dedup, vouch approve) — matches the automated coverage.
  • ruff check src tests clean.
  • mypy src — same 3 pre-existing errors as main (sandbox.py os.getuid/os.getgid on non-posix, missing jinja2 stubs); none in the touched files.
  • pytest tests/ -q --ignore=tests/embeddings — ran locally on windows. two files (test_http_server.py, test_http_server_mcp.py) hang in this sandbox because they bind a real loopback socket, unrelated to this diff, so i excluded them locally; everything else passed except 8 pre-existing windows-only failures (posix os.getuid, unprivileged symlinks, / vs \ path assertions) untouched by this change. tests/test_contradictions.py is green.

test plan

  • vouch contradict-scan lists candidate pairs, writes nothing in --dry-run (default)
  • without --dry-run, each surviving pair produces exactly one pending contradicts proposal visible in kb.list_pending
  • scanner never sets CONTESTED, never writes a Relation, never touches Claim.contradicts
  • no duplicate proposals across repeat scans
  • --threshold, --entity, --limit, --dry-run/--no-dry-run behave as documented
  • scoring/grouping lives in src/vouch/contradictions.py, storage.py unchanged
  • cli-only for this first cut (no kb.* mcp/jsonl method), per the issue's stated allowance

Summary by CodeRabbit

  • New Features

    • Added a new command to scan approved claims for potential contradictions.
    • Results can be reviewed in preview mode or submitted as pending proposals for approval.
    • Added filtering options for similarity threshold, entity scope, and result limits.
  • Bug Fixes

    • Avoids duplicate contradiction suggestions when a pair has already been linked or previously proposed.
    • Keeps scans read-only in preview mode and does not change claim status automatically.
  • Tests

    • Added coverage for candidate detection, proposal creation, idempotency, filtering, and approval flow.

walk approved claims grouped by shared entity, flag same-topic pairs
that disagree in polarity, and file a pending `contradicts` relation
proposal per surviving pair via proposals.propose_relation. read-only
and advisory: the scanner never sets ClaimStatus.CONTESTED, never
writes a Relation, and never touches Claim.contradicts — those only
land once a human runs `vouch approve` (or the pre-existing manual
lifecycle.contradict). scoring/grouping lives in the new
src/vouch/contradictions.py, not storage.py.

`vouch contradict-scan` mirrors `vouch dedup`'s cli-only shape:
--threshold, --entity, --limit, and --dry-run/--no-dry-run (default
dry-run). a pair already cross-linked via Claim.contradicts, joined
by an approved contradicts edge, or with a pending contradicts
proposal is skipped so repeat scans don't duplicate proposals.

closes vouchdev#314
@github-actions github-actions Bot added docs documentation, specs, examples, and repo guidance cli command line interface tests tests and fixtures size: M 200-499 changed non-doc lines labels Jul 6, 2026
@coderabbitai

coderabbitai Bot commented Jul 6, 2026

Copy link
Copy Markdown

Review Change Stack

📝 Walkthrough

Walkthrough

Adds a new advisory vouch contradict-scan feature: a scoring module (src/vouch/contradictions.py) that groups approved claims by shared entity and flags likely polarity conflicts, a CLI command exposing it, a changelog entry, and a test suite covering discovery, filtering, dry-run behavior, proposal creation, and scan-to-approve flow.

Changes

Contradiction Scan Feature

Layer / File(s) Summary
Text utilities, scoring model, and entity grouping
src/vouch/contradictions.py
Adds tokenization/negation helpers, Jaccard similarity scoring, the frozen Candidate dataclass, and entity-based claim grouping.
Candidate discovery and scan/proposal logic
src/vouch/contradictions.py
Adds _already_flagged_pairs, find_candidates (pairing, scoring, threshold filtering, dedup), and scan (dry-run vs. proposal filing via propose_relation, ProposalError handling, limit).
CLI command wiring
src/vouch/cli.py
Imports contradictions and adds the contradict-scan command with --threshold, --entity, --dry-run/--no-dry-run, and --limit options, printing scan results.
Tests and changelog
tests/test_contradictions.py, CHANGELOG.md
Adds fixtures and tests for find_candidates/scan (discovery, filters, skip conditions, idempotency, limit), an end-to-end scan→approve→relation test, and a changelog entry describing the feature.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Sequence Diagram(s)

sequenceDiagram
  participant User
  participant CLI as cli.py contradict_scan
  participant Scanner as contradictions.scan
  participant Finder as find_candidates
  participant Store as KBStore
  participant Proposals as proposals.propose_relation

  User->>CLI: vouch contradict-scan --threshold --entity --limit
  CLI->>Scanner: scan(store, threshold, entity, dry_run, limit)
  Scanner->>Finder: find_candidates(store, threshold, entity)
  Finder->>Store: read approved claims, entities, contradicts, relations, proposals
  Finder-->>Scanner: sorted Candidate list
  alt dry_run is False
    Scanner->>Proposals: propose_relation(contradicts, claim_a, claim_b)
    Proposals-->>Scanner: proposal_id or ProposalError
  end
  Scanner-->>CLI: rows with claim ids, entity, score, proposal_id
  CLI-->>User: printed candidate summaries
Loading

Related issues: #314

Suggested labels: enhancement, cli

Suggested reviewers: vouchdev maintainers familiar with proposals.py and lifecycle.py

🐰 A scanner hops through claims so wise,
Spots when two facts tell contrary lies,
It never flips a status on its own,
Just whispers "check this" in a proposal tone,
Then waits for human eyes.

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title is concise and matches the main change: an advisory contradiction scanner for approved claims.
Linked Issues check ✅ Passed The PR satisfies #314 with a read-only scanner, proposal flow, CLI flags, duplicate skipping, and tests.
Out of Scope Changes check ✅ Passed All changes stay within the contradiction-scanner feature; changelog, CLI, logic, and tests are directly relevant.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@CHANGELOG.md`:
- Line 16: Capitalize the word after the period in the changelog sentence so it
reads as a new sentence; update the text containing `--entity`, `--limit`, and
the following `scoring` phrase to start with an uppercase letter.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 66e8ef38-6b6c-401c-87d9-3049d7e67505

📥 Commits

Reviewing files that changed from the base of the PR and between 36f035e and 1364e17.

📒 Files selected for processing (4)
  • CHANGELOG.md
  • src/vouch/cli.py
  • src/vouch/contradictions.py
  • tests/test_contradictions.py

Comment thread CHANGELOG.md
@jsdevninja

Copy link
Copy Markdown
Contributor Author

@plind-junior Would you review please?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cli command line interface docs documentation, specs, examples, and repo guidance size: M 200-499 changed non-doc lines tests tests and fixtures

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat: contradiction finder — scan the kb and propose contradicts links for conflicting claims

1 participant