Skip to content

repo-finder: $confirms matches phrasing without an addressee, so endorsing another contributor's plan grades as confirmation #33

Description

@imkp1

Summary

The $confirms and $invites regexes in scripts/orchestrator/triage_filter.sh match phrasing without any notion of who a comment is addressed to. A maintainer endorsing another contributor's offer to do the work grades identically to a maintainer confirming the defect for us. The two mean opposite things for whether the issue is available.

Code

Line 104-105:

| ("reproduc|confirmed|i see the same|can confirm|this is (valid|a bug|indeed)"
+ "|good catch|you.re right|nice find|makes sense to me")               as $confirms

makes sense to me is the clearest case, but the whole set has the property. All of these are natural replies to a proposed plan, not only to a bug report.

Impact

confirms and invites_pr carry a large enough bonus to lift a claimed or declined issue over the 8 point bar on its own. The result is a shortlisted issue that someone else is already working on, which is worse than an empty shortlist because the wasted work is only discovered after the clone.

Observed cases

Both from a single run on 2026-07-29, on large well known Go repos.

  • An outside contributor posted an implementation plan. A maintainer replied "I think it makes sense." Graded confirms. The comment endorses the contributor's plan, so the issue is already taken.
  • A maintainer answered "False flag" to each reported CVE. Graded neutral rather than declined, so the issue survived the gate.

In the same run 5 of 6 positive grades were wrong on hand verification. The other three were a duplicate redirect, a support redirect, and an issue with the fix already submitted.

Possible directions

Ranked by cost, not by preference. Happy to take direction on which is wanted before opening a PR.

  1. Grade only maintainer comments that have no non-maintainer comment between them and the issue body, so a reply to a plan is excluded structurally. Cheap, but drops legitimate late confirmations.
  2. Check whether the maintainer comment is a GitHub reply to a non-maintainer comment, and skip those. Needs the reply parent in the fetched JSON.
  3. Weaken the specific phrases that read as agreement with a proposal rather than with a report. makes sense to me and you.re right are the two that carry the most false positives. Smallest change, does not address the general case.

Also worth noting the inverse. One issue contained "feel free to propose PRs", which $invites matches, and the grade still came back none. I have not tracked down why, and it may be a separate bug. Mentioning it here rather than filing a third issue on an unverified observation.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions