Skip to content

Bug: terminal summary counts suppressed findings, disagreeing with JSON summary and exit code #429

Description

@Alberto-Codes

Description

The terminal summary line counts suppressed findings toward its total and its per-category breakdown. A file whose findings are entirely suppressed reports a non-zero finding count while printing no findings and exiting 0. The JSON summary reports the same run as total: 0, so the two output formats disagree about the same data.

Steps to Reproduce

  1. Create b.py:
# docvet: ignore-file
"""Module."""
from __future__ import annotations


def foo():
    """Do a thing."""
    raise ValueError("x")
  1. docvet enrichment b.py
  2. docvet --format json enrichment b.py

Expected Behavior

The summary line reflects active findings only, consistent with the exit code and the JSON summary:

Vetted 1 files [enrichment] — no findings.

Actual Behavior

$ docvet enrichment b.py
Vetted 1 files [enrichment] — 4 findings (1 required, 3 recommended). (0.0s)
$ echo $?
0

Nothing is listed under the summary, and the exit code is 0. Meanwhile:

$ docvet --format json enrichment b.py | jq .summary
{"total": 0, "by_category": {"required": 0, "recommended": 0, "scaffold": 0}, "files_checked": 1}

The mismatch also shows up in the partial-suppression case. For a file with one suppressed missing-raises and one active, the summary reports 2 required while only one required finding is active and printed.

Environment

  • Python: 3.12
  • OS: Linux
  • docvet version: 1.15.1 (reproduced on main at 257310e)

Possible Cause

Story 32.4 wired suppression as a post-filter in _output_and_exit(). Task 3.1 filters findings_by_check before flattening "so both output and exit code exclude suppressed findings" — and both do, correctly. The terminal summary line appears to be computed from the pre-filter collection, or from a count captured before the filter runs, while format_json builds its summary from the post-filter list.

Likely a one-line fix in the summary construction in src/docvet/cli/_output.py. Worth a regression test asserting the terminal summary and the JSON summary.total agree for the same input — none of story 32.4's ten ACs covered the summary line, which is how this slipped through.

Found while verifying story 32.4's acceptance criteria (#426).

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions