Skip to content

feat: code quality assessor enhancements (ADR A.6, A.7)#491

Merged
jwm4 merged 2 commits into
mainfrom
feat/code-quality-assessor-enhancements-462
Jun 11, 2026
Merged

feat: code quality assessor enhancements (ADR A.6, A.7)#491
jwm4 merged 2 commits into
mainfrom
feat/code-quality-assessor-enhancements-462

Conversation

@jwm4

@jwm4 jwm4 commented Jun 5, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Add Python type checker strict mode detection to TypeAnnotationsAssessor: checks mypy.ini, .mypy.ini, setup.cfg, pyproject.toml [tool.mypy], pyrightconfig.json, and pyproject.toml [tool.pyright] for strict mode config, awarding +15 bonus points (capped at 100)
  • Add naming consistency check to StandardLayoutAssessor: scans directories for mixed file naming conventions (snake_case vs camelCase vs PascalCase vs kebab-case) and reports as evidence without affecting score
  • Update docs/attributes.md with strict mode bonus and naming consistency documentation

Implements Proposals A.6 and A.7 from the accepted ADR. Fifth of six implementation PRs.

Self-score change: 75.1 Gold (unchanged, since agentready does not have strict mode configured and naming consistency is evidence-only).

A.6: Python strict mode detection

The ADR insight: "Strict mode prevents new violations; coverage measures current state. Both matter." The existing Python assessment scores on annotation coverage (proportional to 80% threshold). This adds a +15 bonus when strict mode is configured, matching the enforcement bonus pattern from PR #484.

Config file Strict indicator
mypy.ini / .mypy.ini strict = true or disallow_untyped_defs = true
setup.cfg [mypy] Same as above
pyproject.toml [tool.mypy] strict = true or disallow_untyped_defs = true
pyrightconfig.json typeCheckingMode: "strict" (supports JSONC)
pyproject.toml [tool.pyright] typeCheckingMode = "strict"

A.7: Naming consistency

The etirelli/ai-scaffolding skill check 2.5 looks for mixed naming conventions in the same directory. This adds a similar check as evidence (no score impact). Directories with fewer than 3 classifiable files are skipped to avoid false positives.

Related issues

  1. Remove redundant assessors, realign tiers, rebalance weights (Remove redundant assessors, realign tiers, rebalance weights (ADR C.1-4, E.1-2) #458) - merged in refactor: remove redundant assessors, realign tiers, rebalance weights #464
  2. Context file assessor improvements (Context file assessor improvements (ADR A.1, A.9, A.4) #459) - merged in feat: context file assessor improvements (ADR A.1, A.9, A.4) #477
  3. Test assessor enhancements (Test assessor enhancements (ADR A.2, A.3) #460) - merged in feat: test assessor enhancements for command documentation and organization (ADR A.2, A.3) #481
  4. Enforcement and intent assessor improvements (Enforcement and intent assessor improvements (ADR A.5, A.8) #461) - merged in feat: enforcement and intent assessor improvements (ADR A.5, A.8) #484
  5. This PR - Code quality assessor enhancements (Code quality assessor enhancements (ADR A.6, A.7) #462)
  6. New assessors for architectural boundaries and threat models (New assessors for architectural boundaries and threat models (ADR B.1, B.2) #463)

Test plan

  • black . && isort . && ruff check . passes
  • pytest tests/unit/ passes (1173 passed, 17 skipped)
  • agentready assess . runs successfully (75.1/100 Gold, unchanged)
  • 16 new tests for Python strict mode detection
  • 12 new tests for naming consistency (5 integration + 7 classifier unit tests)
  • All existing tests pass unchanged

Closes #462

Posted by Bill Murdock with assistance from Claude Code.

Summary by CodeRabbit

  • New Features

    • Type-annotation assessment gains a Strict-mode bonus (+15 pts) when strict type-checking is detected.
    • Layout assessment adds naming-consistency evidence (reports mixed vs. consistent conventions); evidence-only—no score impact; directories with fewer than 3 classifiable files are skipped.
  • Documentation

    • Updated scoring docs for Type Annotations and Standard Project Layout.
  • Tests

    • Added tests covering strict-mode detection, robustness to malformed configs, and naming-consistency behavior.

Add Python type checker strict mode detection to TypeAnnotationsAssessor
(+15 bonus points for mypy/pyright strict config) and naming consistency
checks to StandardLayoutAssessor (evidence only, flags mixed camelCase/
snake_case/PascalCase/kebab-case within directories).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Jun 5, 2026

Copy link
Copy Markdown

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Enterprise

Run ID: d79e7330-f344-4a98-8c10-956e63838b3d

📥 Commits

Reviewing files that changed from the base of the PR and between 45a4369 and 0eaca74.

📒 Files selected for processing (2)
  • src/agentready/assessors/structure.py
  • tests/unit/test_assessors_structure.py

📝 Walkthrough

Walkthrough

Implements Python strict-mode detection (+15 evidence bonus) and adds evidence-only naming-consistency checks to layout assessment; both changes include docs and unit tests.

Changes

Python strict mode detection (ADR A.6)

Layer / File(s) Summary
Strict mode detection and score bonus
src/agentready/assessors/code_quality.py
Adds configparser/tomllib imports, collects Python type evidence, and implements _check_python_strict_mode() to detect strict mypy/pyright settings (INI/TOML/pyrightconfig.json JSONC) and apply a fixed +15 score bonus when present.
Strict mode configuration tests & helpers
tests/unit/test_assessors_code_quality.py
Adds repo fixtures and tests verifying detection across mypy.ini, .mypy.ini, setup.cfg [mypy], pyproject.toml ([tool.mypy]/[tool.pyright]), pyrightconfig.json (with/without comments), handling of malformed configs, score capping at 100, and effect on low-coverage repos.
Docs: attribute update
docs/attributes.md
Documents the new “Strict mode bonus” (+15 pts) for type_annotations and enumerates the config files checked.

Naming convention consistency evidence (ADR A.7)

Layer / File(s) Summary
Naming convention classifier and check
src/agentready/assessors/structure.py
Adds _classify_naming_convention() and _check_naming_consistency(); classifies filename stems, groups by parent directory, skips hidden/common non-code paths, ignores certain stems, reports mixed conventions for directories with ≥3 classified files, and appends evidence to Python and Go layout assessments (no score impact).
Naming consistency tests
tests/unit/test_assessors_structure.py
Adds classifier unit tests and integration tests asserting consistent vs mixed conventions, neutral single-word handling, directory-size skipping, and that the check is evidence-only.
Docs: attribute update
docs/attributes.md
Adds an evidence-only “naming consistency” note to standard_layout, stating mixed conventions reduce glob-ability and directories with fewer than 3 classifiable files are skipped (no score impact).

Suggested labels

released

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 79.49% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title follows Conventional Commits format with 'feat' type and includes ADR references (A.6, A.7) that align with the main code quality assessor enhancements.
Linked Issues check ✅ Passed All objectives from #462 are met: strict-mode detection for mypy/pyright configurations with +15 bonus, naming consistency evidence-only check, comprehensive tests, and docs updates.
Out of Scope Changes check ✅ Passed All changes are directly scoped to ADR A.6 and A.7 requirements: code quality assessor enhancements, documentation, and corresponding tests.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/code-quality-assessor-enhancements-462
✨ Simplify code
  • Create PR with simplified code
  • Commit simplified code in branch feat/code-quality-assessor-enhancements-462

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@github-actions

github-actions Bot commented Jun 5, 2026

Copy link
Copy Markdown
Contributor

📈 Test Coverage Report

Branch Coverage
This PR 74.1%
Main 73.8%
Diff ✅ +0.3%

Coverage calculated from unit tests only

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/agentready/assessors/structure.py`:
- Around line 440-454: The _classify_naming_convention static method can
IndexError on empty strings because it accesses name[0]; add a guard at the top
of _classify_naming_convention to return None if not name (empty string) before
any indexing, then proceed with existing checks (snake_case, kebab-case,
PascalCase, camelCase) unchanged; update any callers/tests if they expect
different behavior for empty stems.

In `@tests/unit/test_assessors_structure.py`:
- Around line 1055-1086: Add a unit test method named test_empty_string_neutral
to the same TestCase as the other naming-convention tests that calls
StandardLayoutAssessor._classify_naming_convention("") and asserts the result is
None; this ensures the classifier gracefully handles empty stems and prevents
regressions after you add an empty-string guard in _classify_naming_convention.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Enterprise

Run ID: 906f9af6-edd0-4b97-8d91-09203d6e4204

📥 Commits

Reviewing files that changed from the base of the PR and between 0fd1975 and 45a4369.

📒 Files selected for processing (5)
  • docs/attributes.md
  • src/agentready/assessors/code_quality.py
  • src/agentready/assessors/structure.py
  • tests/unit/test_assessors_code_quality.py
  • tests/unit/test_assessors_structure.py

Comment thread src/agentready/assessors/structure.py
Comment thread tests/unit/test_assessors_structure.py
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@jwm4 jwm4 merged commit 3ec84ef into main Jun 11, 2026
6 checks passed
@jwm4 jwm4 deleted the feat/code-quality-assessor-enhancements-462 branch June 11, 2026 12:41
github-actions Bot pushed a commit that referenced this pull request Jun 11, 2026
# [2.47.0](v2.46.3...v2.47.0) (2026-06-11)

### Features

* code quality assessor enhancements (ADR A.6, A.7) ([#491](#491)) ([3ec84ef](3ec84ef))
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Code quality assessor enhancements (ADR A.6, A.7)

1 participant