Skip to content

repo-finder: signal grading ignores the $maint fallback list, dropping de facto maintainers #32

Description

@imkp1

Summary

Signal grading reads $mca, which selects maintainer comments by authorAssociation alone. authorAssociation reports public org membership only. A private org member, or a lead who was never added to the org, comments as CONTRIBUTOR, so their confirmation is discarded and the issue grades none.

Correction to the original report

I first filed this claiming $mca should use the $maint fallback list, and that dropping it was an oversight. That was wrong, and I have rewritten the issue.

The $mca / $mc split is deliberate. It is stated in the code, in the rubric prose in agents/repo-finder.md, and asserted by a test fixture with the comment "The distinction PR #25 drew must survive this change". The reasoning is right: the 3c union is top-25 contributors by commit count, deliberately wide so the fortress check errs toward skipping, and reading endorsement off it would grade any prolific committer as a maintainer.

So the one line fix I proposed would have deleted a tested, intentional decision. Sorry for the noise.

The actual gap

There is no tier between the two:

  • association: too strict, misses private org members and de facto leads
  • the 3c union: too loose to read as endorsement

Someone who authored most of a repo's recent merged PRs sits in the gap. They have real authority, and they earn no signal.

Evidence

A graph database repo. The project's most active maintainer authored 15 of the last 20 merged PRs and had commits 2 days before the run, but comments as CONTRIBUTOR. On the issue they proposed a concrete implementation and the reporter agreed. It graded none, and the issue fell below the 8 point bar.

The failure is silent. Nothing in the output distinguishes "no maintainer commented" from "the maintainer commented and we did not count them".

Proposed fix

Add a third set, strictly narrower than the union: authors of >= 5 of the last 20 merged PRs, bots excluded. Grade on association ∪ de facto. Leave $mca and maintainer_comment_assoc association only, and leave the union ungraded, so PR #25's distinction holds.

Threshold measured, not picked. Across 18 Go and AI repos, per author share of the last 20 merged PRs, bots removed:

n=15  1 author
n=11  1   (bot, excluded)
n= 9  1
n= 8  1
n= 6  1
n= 5  2
n= 4  2   (both bots, excluded)
n= 3  13 authors   <- ordinary contributors
n= 2  14
n= 1  112

Clean gap between 3 and 5 once bots are removed. All 6 non-bot authors at n>=5 are insiders whose association under-reports: two work for the vendor that owns the repo they contribute to, two more are employed on the project they lead, and one is the graph database lead above. None are public org members, which is exactly the case PR #25 called out. A cutoff of 3 admits 23 authors and re-creates the commit rank error PR #25 fixed.

Reuses the merged PR list already fetched in 3a, so no extra API calls.

PR to follow.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions