[feature] Flag invalid pull requests that do not reference validated issues

[nemesifier]

Is your feature request related to a problem? Please describe.

OpenWISP maintainers are receiving an increasing number of unsolicited, low-quality, or AI-assisted pull requests that do not address problems previously reviewed or accepted by maintainers.

The cost of generating code has dropped significantly, but the cost of reviewing code remains high. Pull requests that do not solve a validated issue can shift the burden of problem definition, design review, testing, and validation onto maintainers.

OpenWISP already documents a policy against AI spam and unsolicited contributions:
https://openwisp.io/docs/dev/general/code-of-conduct.html

However, the current GitHub automation only assigns linked issues when pull requests reference them. It does not verify whether external pull requests reference issues that have been validated by maintainers.

Describe the solution you'd like

Extend the existing autoassignment and stale PR automation to flag non-trivial pull requests opened by external contributors when they do not reference a validated issue.

The implementation should reuse and extend the existing workflow/action code:

• .github/workflows/bot-autoassign-pr-issue-link.yml
• .github/workflows/bot-autoassign-stale-pr.yml
• .github/actions/bot-autoassign/issue_assignment_bot.py
• .github/actions/bot-autoassign/stale_pr_bot.py
• docs/developer/reusable-github-utils.rst

A pull request should proceed normally when the author has one of the GitHub author_association values already treated as maintainer-like by the stale PR bot:

• OWNER
• MEMBER
• COLLABORATOR

For external contributors, a pull request should proceed normally only when it references at least one valid issue that has been validated by maintainers.

An issue is considered valid for external contributors when all of the following are true:

• it is open;
• it has at least one label, except invalid or wontfix;
• it belongs to either the same repository or another repository in the openwisp organization;
• it is assigned to one of the GitHub Projects listed in the contributing guidelines:
  • OpenWISP Contributor's Board (view)
  • OpenWISP Priorities for next releases (view)

When an external contributor opens or updates a pull request that does not reference a validated issue, the bot should:

• apply the standard GitHub invalid label to the pull request;
• post a single explanatory comment;
• explain why the pull request was flagged;
• explain how to rectify the issue;
• refer to Contributing Guidelines and Anti AI Spam Policy for further information.
• explain that the pull request will be closed automatically after 24 hours if not resolved.

The explanatory comment should use a hidden marker to avoid duplicate comments and to measure the 24-hour window.

The scheduled stale PR workflow should re-check pull requests labeled invalid. If the pull request now references a validated issue, the bot should remove invalid automatically. If the pull request still does not reference a validated issue after 24 hours from the bot warning comment, the bot should close it automatically. If the bot cannot verify whether the linked issue is assigned to one of the required projects because of a GitHub API or permission error, the workflow should fail and be retried as usual.

Draft pull requests and documentation-only pull requests should not be exempt. Organization members and repository collaborators remain free to open pull requests without creating issues first.

The PR issue-link workflow may need to run on additional pull request events such as edited and ready_for_review, so contributors can fix the PR description without waiting for the scheduled job.

Proposed automated comment

<!-- bot:invalid_unvalidated_issue -->

Hi @USERNAME,

Thank you for your interest in contributing to OpenWISP.

This pull request has been flagged because external contributors must target an issue validated by maintainers before requesting review.

Please link this pull request to a validated issue by adding `Fixes #ISSUE_NUMBER`, `Closes #ISSUE_NUMBER`, or `Related to #ISSUE_NUMBER` to the pull request description. The issue may be in this repository or another OpenWISP repository.

If there is no validated issue yet, please open one first and wait for maintainer validation before continuing with this pull request.

An issue is considered validated when it is open, has an appropriate label other than `invalid` or `wontfix`, and is assigned to one of the OpenWISP contributor project boards mentioned in the contributing guidelines.

Please see the OpenWISP policy on unsolicited and AI-assisted contributions:
https://openwisp.io/docs/dev/general/code-of-conduct.html

If this is not resolved within 24 hours, this pull request will be closed automatically. Thank you for your understanding.

Describe alternatives you've considered

One alternative is to create a new workflow dedicated to enforcing this policy. This is not preferred because the existing autoassignment and stale PR workflows already perform most of the required work:

• parsing linked issues from PR descriptions;
• assigning linked issues;
• managing stale PR state;
• posting bot comments;
• removing labels when PR state changes;
• running daily scheduled checks.

Reusing the existing bot package keeps the behavior centralized and easier to reuse across OpenWISP repositories.

Another alternative is to create a dedicated label such as needs-triaged-issue. This is more descriptive, but it requires creating and documenting a new label. The standard GitHub invalid label already exists by default and is already available in this repository, so using it is simpler.

Another alternative is to rely only on labels. This is simpler, but it would still allow contributors to pick random labeled issues that are not intended for beginner or occasional contributors. Requiring the issue to be assigned to one of the OpenWISP contributor project boards makes the automation consistent with the contributing guidelines and helps ensure that external contributors work on issues maintainers have made available for contribution.

Additional context

The documentation for the reusable GitHub workflows should be updated in docs/developer/reusable-github-utils.rst to explain this behavior.

The documentation should clarify that:

• external contributors must reference a validated issue before opening a pull request;
• OWNER, MEMBER, and COLLABORATOR authors are exempt;
• an issue is validated when it is open, has at least one label other than invalid or wontfix, and is assigned to one of the GitHub Projects listed in the contributing guidelines;
• validated issues may be referenced from the same repository or from another repository in the openwisp organization;
• the supported project boards are:
  • OpenWISP Contributor's Board (view)
  • OpenWISP Priorities for next releases (view)
• if the bot cannot verify whether the linked issue is assigned to one of these projects because of a GitHub API or permission error, the workflow should fail and be retried as usual;
• the workflow relies on the standard GitHub labels invalid and wontfix;
• draft PRs are not exempt;
• documentation-only PRs are not exempt;
• maintainers can validate an issue by applying an appropriate label;
• once a linked issue is validated, the scheduled workflow removes invalid from the PR automatically;
• unresolved PRs are closed automatically after 24 hours;
• maintainers should not remove invalid manually unless they also validate the linked issue.

This behavior will also be documented in the contributing guidelines.

[prathmeshkulkarni-coder]

This looks interesting ; some problems regarding issue

1. bot will fail on depend bot's PR
   The spec exempts OWNER/MEMBER/COLLABORATOR via author_association, but bot accounts like dependabot[bot] typically don't fall into any of these. Dependabot's PRs never reference an issue (there's nothing to link for a version bump), so without an explicit carve-out they'd get flagged invalid and auto-closed after 24 hours , silently breaking dependency updates. Suggest an explicit allowlist/actor check (e.g., github.actor == 'dependabot[bot]') alongside the author_association check.

2. The spec says a PR should proceed when it references "at least one valid issue," which implies OR logic when multiple issues are linked. That's reasonable, but it also means a contributor could satisfy the bot by linking to any validated issue on the contributor boards not necessarily the one their PR actually addresses. Since the bot only validates the issue's state (open/labeled/on board) and not whether the PR's content actually relates to it, this could let low-effort PRs pass by referencing an unrelated but validated issue, undermining the anti-spam intent of the whole feature. Worth clarifying whether this is an accepted trade-off or whether additional relevance-checking (even just requiring the primary Fixes # reference to match, rather than any Related to #) is warranted.

3. The spec says maintainers "should not remove invalid manually unless they also validate the linked issue or intentionally accept the PR outside the normal process" but this is only a policy statement, not something the bot enforces. If a maintainer manually removes invalid to accept a PR outside the normal flow, the next scheduled run will re-check the linked issue against the bot's own criteria and silently re-apply invalid, overriding the maintainer's decision without warning. Suggest an explicit override label (e.g., maintainer-approved or bot-exempt) that the bot checks first and skips entirely if present, giving maintainers a clean way to bypass automation instead of relying on an unenforced instruction.

[nemesifier]

Thanks for following up, see my comments below.

1. bot will fail on depend bot's PR
   The spec exempts OWNER/MEMBER/COLLABORATOR via author_association, but bot accounts like dependabot[bot] typically don't fall into any of these. Dependabot's PRs never reference an issue (there's nothing to link for a version bump), so without an explicit carve-out they'd get flagged invalid and auto-closed after 24 hours , silently breaking dependency updates. Suggest an explicit allowlist/actor check (e.g., github.actor == 'dependabot[bot]') alongside the author_association check.

Let's add an exclude list, if possible we should keep this exclude list in the reusable github actions logic in openwisp-utils so we can easily change it once for all repos if needed. This needs to be verified.

2. The spec says a PR should proceed when it references "at least one valid issue," which implies OR logic when multiple issues are linked. That's reasonable, but it also means a contributor could satisfy the bot by linking to any validated issue on the contributor boards not necessarily the one their PR actually addresses. Since the bot only validates the issue's state (open/labeled/on board) and not whether the PR's content actually relates to it, this could let low-effort PRs pass by referencing an unrelated but validated issue, undermining the anti-spam intent of the whole feature. Worth clarifying whether this is an accepted trade-off or whether additional relevance-checking (even just requiring the primary Fixes # reference to match, rather than any Related to #) is warranted.

Relevance checking can only be done by a human or AI. Coderabbit already covers part of this. Maintainers will have to keep checking. If we spot someone that tries to evade these checks by referencing unrelated issues just to pass this validation, we shall ban them for good.

3. The spec says maintainers "should not remove invalid manually unless they also validate the linked issue or intentionally accept the PR outside the normal process" but this is only a policy statement, not something the bot enforces. If a maintainer manually removes invalid to accept a PR outside the normal flow, the next scheduled run will re-check the linked issue against the bot's own criteria and silently re-apply invalid, overriding the maintainer's decision without warning. Suggest an explicit override label (e.g., maintainer-approved or bot-exempt) that the bot checks first and skips entirely if present, giving maintainers a clean way to bypass automation instead of relying on an unenforced instruction.

Removing the invalid label is not enough: the pull-request must be linked to a valid issue, hence the right solution is to open an issue, validate it and refer the PR to it, I removed "or intentionally accept the PR outside the normal process".

[openwisp-companion]

This issue has been automatically assigned to @prathmeshkulkarni-coder who opened PR #715 to address it. 🎯