Skip to content

test: unit tests for the least-covered modules (53% → 96% coverage) - #28

Open
devin-ai-integration[bot] wants to merge 1 commit into
mainfrom
devin/1784930981-test-coverage
Open

test: unit tests for the least-covered modules (53% → 96% coverage)#28
devin-ai-integration[bot] wants to merge 1 commit into
mainfrom
devin/1784930981-test-coverage

Conversation

@devin-ai-integration

Copy link
Copy Markdown

Summary

Measured pytest --cov=src on main (53% overall) and added unit tests for the modules at the bottom of the table. No source changes — tests only, plus a README refresh of the test-suite blurb.

module before after
src/cli.py 0% 89%
src/engine/models.py 0% 100%
src/report_generator.py 13% 99%
src/config.py 57% 100%
src/analyzer.py 62% 95%
src/code_fetcher.py 69% 98%
src/parser.py 72% 99%
src/spec_fetcher.py 81% 99%
total 53% 96%

385 passing / 2 skipped (was 149 / 2). Everything network- or LLM-facing is stubbed, so the new tests make no API calls.

New files:

  • tests/test_cli.py — Click CliRunner over every command plus the _run_analysis pipeline helpers: analyzer selection per provider, the AZURE_AI_VERIFY_DEPLOYMENT split (Opus primary / Sonnet skeptic), fork-diff spec assembly, result re-ordering back to source file order, and the error/abort paths.
  • tests/test_report_generator.py — summary aggregation (severity counts, overall-status precedence, verification tallies), narrative wording, and JSON/Markdown/HTML output for both the single-client and cross-client differential reports, including HTML escaping of model-supplied text and the no-rich fallbacks.
  • tests/test_config.py — YAML loading, config discovery in cwd/parent/package dir, env-var precedence, and the Azure config assembly (temperature/anthropic_version omitted when unset; azure_verify_config swaps only the deployment and must not mutate azure_config).
  • tests/test_analyzer.py — prompt building, the _parse_json_response repair ladder (fenced, prose-wrapped, truncated object, truncated issue list, unrecoverable), the Gemini/OpenAI backends, and the Azure 429/529 retry logic (Retry-After honoured, non-numeric header falls back to exponential backoff, exhausted retries surface as status="ERROR").
  • tests/test_code_fetcher.py / tests/test_spec_fetcher.py — cache hit/miss and URL construction, per-file HTTP failures being inlined as # Error fetching file: ... rather than aborting a run, fork-to-fork diffing (no predecessor / identical forks / fetch failure all return None), fetch_eip_spec diff-vs-full mode selection, and section extraction.
  • tests/test_parser.py — paths the per-EIP suites miss: Python classes/docstrings, the generic fallback, Rust tuple structs and multi-line fn signatures, brace-language functions outside a class, tree-sitter parsing plus its regex fallback when the dependency is unavailable, and find_function / extract_comments.
  • tests/test_engine_api.py — asserts the ScanResult/Summary/Finding TypedDicts match what scan_path actually returns.

One behaviour worth flagging, discovered while writing tests/test_parser.py: the regex Python parser's class-end heuristic never terminates a class at a following top-level def, so a module like

class A:
    def m(self): ...

def top_level(): ...        # swallowed into A's block

yields a single class A block covering the rest of the file (and find_function("m") returns None). The tests document current behaviour rather than encode a fix; happy to follow up separately if you want the heuristic corrected. This only affects Python sources, which no client mapping currently targets.

Link to Devin session: https://app.devin.ai/sessions/e71267fa8f3f49cf97945823440d01b8
Requested by: @Fosurero

…analyzer, fetchers, parser)

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@Fosurero Fosurero self-assigned this Jul 24, 2026
@devin-ai-integration

Copy link
Copy Markdown
Author

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment, CI, and merge conflict monitoring

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants