test: unit tests for the least-covered modules (53% → 96% coverage) - #28
Open
devin-ai-integration[bot] wants to merge 1 commit into
Open
test: unit tests for the least-covered modules (53% → 96% coverage)#28devin-ai-integration[bot] wants to merge 1 commit into
devin-ai-integration[bot] wants to merge 1 commit into
Conversation
…analyzer, fetchers, parser) Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Author
🤖 Devin AI EngineerI'll be helping with this pull request! Here's what you should know: ✅ I will automatically:
Note: I can only respond to comments from users who have write access to this repository. ⚙️ Control Options:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Measured
pytest --cov=srconmain(53% overall) and added unit tests for the modules at the bottom of the table. No source changes — tests only, plus a README refresh of the test-suite blurb.src/cli.pysrc/engine/models.pysrc/report_generator.pysrc/config.pysrc/analyzer.pysrc/code_fetcher.pysrc/parser.pysrc/spec_fetcher.py385 passing / 2 skipped (was 149 / 2). Everything network- or LLM-facing is stubbed, so the new tests make no API calls.
New files:
tests/test_cli.py— ClickCliRunnerover every command plus the_run_analysispipeline helpers: analyzer selection per provider, theAZURE_AI_VERIFY_DEPLOYMENTsplit (Opus primary / Sonnet skeptic), fork-diff spec assembly, result re-ordering back to source file order, and the error/abort paths.tests/test_report_generator.py— summary aggregation (severity counts, overall-status precedence, verification tallies), narrative wording, and JSON/Markdown/HTML output for both the single-client and cross-client differential reports, including HTML escaping of model-supplied text and the no-richfallbacks.tests/test_config.py— YAML loading, config discovery in cwd/parent/package dir, env-var precedence, and the Azure config assembly (temperature/anthropic_versionomitted when unset;azure_verify_configswaps only the deployment and must not mutateazure_config).tests/test_analyzer.py— prompt building, the_parse_json_responserepair ladder (fenced, prose-wrapped, truncated object, truncated issue list, unrecoverable), the Gemini/OpenAI backends, and the Azure 429/529 retry logic (Retry-Afterhonoured, non-numeric header falls back to exponential backoff, exhausted retries surface asstatus="ERROR").tests/test_code_fetcher.py/tests/test_spec_fetcher.py— cache hit/miss and URL construction, per-file HTTP failures being inlined as# Error fetching file: ...rather than aborting a run, fork-to-fork diffing (no predecessor / identical forks / fetch failure all returnNone),fetch_eip_specdiff-vs-full mode selection, and section extraction.tests/test_parser.py— paths the per-EIP suites miss: Python classes/docstrings, the generic fallback, Rust tuple structs and multi-linefnsignatures, brace-language functions outside a class, tree-sitter parsing plus its regex fallback when the dependency is unavailable, andfind_function/extract_comments.tests/test_engine_api.py— asserts theScanResult/Summary/FindingTypedDicts match whatscan_pathactually returns.One behaviour worth flagging, discovered while writing
tests/test_parser.py: the regex Python parser's class-end heuristic never terminates a class at a following top-leveldef, so a module likeyields a single
class Ablock covering the rest of the file (andfind_function("m")returnsNone). The tests document current behaviour rather than encode a fix; happy to follow up separately if you want the heuristic corrected. This only affects Python sources, which no client mapping currently targets.Link to Devin session: https://app.devin.ai/sessions/e71267fa8f3f49cf97945823440d01b8
Requested by: @Fosurero