fix(metrics): metric calibration — fidelity improvements across all dimensions - #17
Merged
Conversation
When Allure summary is unavailable, latest_build_passing now falls back to the default branch's latest terminal check-run. Coverage and stability are now marked unmeasurable (None) instead of forcing zero when no Allure source is present. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…ants Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…ck-scalar commands Handle list-item key syntax in matrix.include entries, strip inline YAML comments before scalar checks, and recognize block-scalar run commands so substrate compatibility signals are preserved across common workflow styles. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…nt heredoc tokens Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…ource Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…ree check Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…efaults.run Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Replaces semi-fake POC metric logic with deterministic, well-tested, production-grade detectors across all five dimensions. All scorers are now measurability-aware: unmeasurable signals return
Noneand propagate toUNRATEDat the engine level rather than forcing false zeros/bronze.Changes by dimension
test_verification
latest_build_passingnow falls back to GitHub default-branch check-runs when no Allure URL is set — fixes the saml-integrator false negativecoverage_pct/stability_pctreturnNone(unmeasurable) instead of0when Allure is absentdocumentation
docs-checks / vale,docs-checks / linkcheck,docs-checks / docs build)Quickstart,Getting help)substrate_compat
defaults.run:mapping blockssecurity_ssdlc
canonical_repo_automation_registerednow checks git tree ofcanonical/canonical-repo-automationfor a per-repo config file instead of code search — scoped to canonical/* owner onlysupport_engagement
avg_triage_days,avg_pr_review_days,response_coverage_ratereturnNonewhentotal_items < MIN_SAMPLE (5)— separates insufficient data from measured zeroEngine changes
compute_leaf_applicabilityandcompute_productnow checkrequired_metrics_for_scoringfrom dimensions.yaml; anyNonevalue returnsINSUFFICIENT_DATA/UNRATEDConfig/docs
config/dimensions.yaml: addsrequired_metrics_for_scoringlists for all dimensionsconfig/schemas/dimensions.schema.json: schema updated for new fielddocs/local-scoring.md: documents measurability vs unrated semanticsdocs/superpowers/artifacts/2026-07-23-metric-calibration-audit-postfix.json: before/after snapshotTest coverage
required_metrics_for_scoringentries are valid declared outputs