Exclude non-Azure controls from confidence/sovereignty stats; split Review & Edit UI - #57
Merged
Merged
Conversation
…view UI Three related accuracy issues in the mapping review flow: 1. Average Confidence (and High/Low Confidence counts) averaged in every C_Process/D_MicrosoftAttestation control's confidence_score, which is a fixed 0.0 placeholder - Azure Policy mapping is never attempted for these controls, so 0.0 isn't a low score, it's "not applicable". Mixing them with real A/B confidence judgements dragged a batch that matched its enforceable controls well down to a misleading headline number (e.g. 30% average confidence on a batch where 22 of 27 policy-eligible controls were actually high-confidence). Added coverage.confidence_eligible() / frontend utils/coverage.py and scoped every average-confidence computation (ai_mapping_service.py, validator.py, and all three frontend pages) to A/B (+ legacy unclassified) controls only. 2. C/D controls were showing a full Sovereignty Mapping card (Level: L1, Target Archetype: sovereign_root) even when the AI's own reasoning said "no sovereignty requirement applies" - the prompt asks for a sovereignty verdict on every control, defaulting to L1/sovereign_root with an empty objectives list when none applies. Next to a control that constructs no initiative entry, that default read as a second, contradictory verdict. Added coverage.clear_moot_sovereignty(), which nulls the sovereignty field for C/D controls with no real sovereignty_objectives, while preserving the one legitimate exception (a procedural objective like SO-2 Customer Lockbox, which has no policy but genuine sovereignty relevance). 3. Split Review & Edit into two tabs: "Policy Mappings" (A_AzurePolicy / B_AzureConfig - confidence, sovereignty, Azure Policy candidates, exactly as before) and "Manual Register" (C_Process / D_MicrosoftAttestation - read-only, shows coverage_reason, responsibility, and for D controls the grounded attestation citation or an explicit attestation-gap warning). No Azure Policy initiative entry is constructed for Manual Register controls, so they no longer carry a confidence score or sovereignty verdict that never applied to them. Verified: full backend suite (639 passed) and frontend suite (846 passed, 14 pre-existing failures identical to baseline) both regression-clean. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 322e5e54-ab2e-43b0-956c-5ec01cd955e7
warrendt
force-pushed
the
warrendt-fuzzy-potato
branch
from
August 10, 2026 16:47
981a192 to
3298268
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes three accuracy issues surfaced during live testing
1. Misleading Average Confidence.
C_Process/D_MicrosoftAttestationcontrols carry a fixed0.0confidence_scoreplaceholder — Azure Policy mapping is never attempted for them, so0.0isn't a low score, it's "not applicable." Averaging it in with real A/B confidence judgements dragged a batch that matched its enforceable controls well (22 of 27 high-confidence) down to a misleading 30% headline number. Addedcoverage.confidence_eligible()(backend) /utils/coverage.py(frontend mirror) and scoped every average-confidence computation to A/B (+ legacy unclassified) controls only —ai_mapping_service.py,validator.py, and all three frontend pages.2. Sovereignty Mapping shown on process/attestation controls with no real sovereignty tie. A due-diligence/risk-assessment control (clearly
C_Process) was still rendering a full "Level: L1 — Global, Target Archetype: sovereign_root" card even though the AI's own reasoning said "no sovereignty requirement applies." The prompt asks for a sovereignty verdict on every control, defaulting to L1/sovereign_root with an empty objectives list when none applies — next to a control that constructs no initiative entry, that default reads as a contradictory second verdict. Addedcoverage.clear_moot_sovereignty(), which nulls the sovereignty field for C/D controls with no realsovereignty_objectives, while preserving the one legitimate exception (a procedural objective like SO-2 Customer Lockbox — no policy, but genuine sovereignty relevance).3. Split Review & Edit into two tabs. "Policy Mappings" (A_AzurePolicy / B_AzureConfig — confidence, sovereignty, Azure Policy candidates, unchanged UX) and "Manual Register" (C_Process / D_MicrosoftAttestation — read-only, shows
coverage_reason,responsibility, and for D controls the grounded attestation citation or an explicit attestation-gap warning). No Azure Policy initiative entry is constructed for Manual Register controls, so they no longer carry a confidence score or sovereignty verdict that never applied to them — matching how the Export page's manual register already worked.Verification
Full backend suite: 639 passed. Full frontend suite: 846 passed, 14 pre-existing failures identical (same test names) to the pre-change baseline — confirmed via
git stashdiff, no regressions introduced.Co-authored-by: Copilot App 223556219+Copilot@users.noreply.github.com
Copilot-Session: 322e5e54-ab2e-43b0-956c-5ec01cd955e7