Summary
The $confirms and $invites regexes in scripts/orchestrator/triage_filter.sh match phrasing without any notion of who a comment is addressed to. A maintainer endorsing another contributor's offer to do the work grades identically to a maintainer confirming the defect for us. The two mean opposite things for whether the issue is available.
Code
Line 104-105:
| ("reproduc|confirmed|i see the same|can confirm|this is (valid|a bug|indeed)"
+ "|good catch|you.re right|nice find|makes sense to me") as $confirms
makes sense to me is the clearest case, but the whole set has the property. All of these are natural replies to a proposed plan, not only to a bug report.
Impact
confirms and invites_pr carry a large enough bonus to lift a claimed or declined issue over the 8 point bar on its own. The result is a shortlisted issue that someone else is already working on, which is worse than an empty shortlist because the wasted work is only discovered after the clone.
Observed cases
Both from a single run on 2026-07-29, on large well known Go repos.
- An outside contributor posted an implementation plan. A maintainer replied "I think it makes sense." Graded
confirms. The comment endorses the contributor's plan, so the issue is already taken.
- A maintainer answered "False flag" to each reported CVE. Graded
neutral rather than declined, so the issue survived the gate.
In the same run 5 of 6 positive grades were wrong on hand verification. The other three were a duplicate redirect, a support redirect, and an issue with the fix already submitted.
Possible directions
Ranked by cost, not by preference. Happy to take direction on which is wanted before opening a PR.
- Grade only maintainer comments that have no non-maintainer comment between them and the issue body, so a reply to a plan is excluded structurally. Cheap, but drops legitimate late confirmations.
- Check whether the maintainer comment is a GitHub reply to a non-maintainer comment, and skip those. Needs the reply parent in the fetched JSON.
- Weaken the specific phrases that read as agreement with a proposal rather than with a report.
makes sense to me and you.re right are the two that carry the most false positives. Smallest change, does not address the general case.
Also worth noting the inverse. One issue contained "feel free to propose PRs", which $invites matches, and the grade still came back none. I have not tracked down why, and it may be a separate bug. Mentioning it here rather than filing a third issue on an unverified observation.
Summary
The
$confirmsand$invitesregexes inscripts/orchestrator/triage_filter.shmatch phrasing without any notion of who a comment is addressed to. A maintainer endorsing another contributor's offer to do the work grades identically to a maintainer confirming the defect for us. The two mean opposite things for whether the issue is available.Code
Line 104-105:
makes sense to meis the clearest case, but the whole set has the property. All of these are natural replies to a proposed plan, not only to a bug report.Impact
confirmsandinvites_prcarry a large enough bonus to lift a claimed or declined issue over the 8 point bar on its own. The result is a shortlisted issue that someone else is already working on, which is worse than an empty shortlist because the wasted work is only discovered after the clone.Observed cases
Both from a single run on 2026-07-29, on large well known Go repos.
confirms. The comment endorses the contributor's plan, so the issue is already taken.neutralrather thandeclined, so the issue survived the gate.In the same run 5 of 6 positive grades were wrong on hand verification. The other three were a duplicate redirect, a support redirect, and an issue with the fix already submitted.
Possible directions
Ranked by cost, not by preference. Happy to take direction on which is wanted before opening a PR.
makes sense to meandyou.re rightare the two that carry the most false positives. Smallest change, does not address the general case.Also worth noting the inverse. One issue contained "feel free to propose PRs", which
$invitesmatches, and the grade still came backnone. I have not tracked down why, and it may be a separate bug. Mentioning it here rather than filing a third issue on an unverified observation.