Skip to content

feat: add native incident.io Alertmanager receiver - #16522

Open
LukoJy3D wants to merge 1 commit into
grafana:mainfrom
LukoJy3D:feat/alertmanager-incidentio-notifier
Open

feat: add native incident.io Alertmanager receiver#16522
LukoJy3D wants to merge 1 commit into
grafana:mainfrom
LukoJy3D:feat/alertmanager-incidentio-notifier

Conversation

@LukoJy3D

@LukoJy3D LukoJy3D commented Sep 4, 2026

Copy link
Copy Markdown

What this PR does

Mimir already parsed incidentio_configs but never built a notifier, so those receivers silently dropped notifications.

Upstream Alertmanager added a native incident.io receiver in 0.29.0. Mimir started embedding a new enough library in 3.1.0 (Alertmanager v0.31.1; 3.0.x was still v0.28.1). From 3.1.0 onward Mimir unmarshals incidentio_configs, so mimirtool alertmanager get and the v2 alerts API show the receiver as selected.

buildReceiverIntegrations never constructed a notifier from IncidentioConfigs, so those receivers have been a silent no-op since 3.1.0: no integration="incidentio" notification metrics or logs, while Slack/webhook_configs on the same alert still fire.

This wires the existing notify/incidentio notifier (same pattern as MS Teams V2), adds incidentio to allowedIntegrationNames, and rejects url_file / alert_source_token_file so tenant configs stay consistent with Mimir’s *_file secret policy. Inline url + alert_source_token works.

receivers:
  - name: incidentio-notifications
    incidentio_configs:
      - url: https://api.incident.io/v2/alert_events/alertmanager/<source-id>
        alert_source_token: <token>

Until this ships, the working workaround is webhook_configs with Bearer auth to the same URL, yet with high volume of alerts it can easily get rate limited from incident.io side

Checklist

  • Tests updated.
  • Documentation added.
  • CHANGELOG.md updated - the order of entries should be [CHANGE], [FEATURE], [ENHANCEMENT], [BUGFIX]. If changelog entry is not needed, please add the changelog-not-needed label to the PR.
  • about-versioning.md updated with experimental features.

Mimir already parsed incidentio_configs but never built a notifier, so those receivers silently dropped notifications.

Co-authored-by: Cursor <cursoragent@cursor.com>
@LukoJy3D
LukoJy3D requested review from a team as code owners September 4, 2026 09:45
@cla-assistant

cla-assistant Bot commented Sep 4, 2026

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant