Support source authentication using client certificates

[Jan-Schuppik]

This PR adds support for authenticating sources via TLS client certificates. A source can now be identified by its certificate's Subject as an alternative to username/password Basic Auth, with automatic fallback to Basic Auth when no certificate is found.

Changes

• Add a nullable unique client_certificate_subject column to the source table and relax the existing check constraint to accept either a username/password pair or a client certificate CN as the source identity. Migration scripts are provided for both MySQL and PostgreSQL.
• Extend sourceFromAuthOrAbort in the listener to resolve a source from the TLS client certificate's Subject before falling back to Basic Auth. The 401 error message is extended to mention client certificate auth when the request arrives over TLS.
• Add TestSourceFromAuthOrAbort covering four scenarios: a matching Subject authenticates the correct source, an unknown Subject returns 401, a Certificate and Basic Auth together returns 400, and TLS active with no client certificate presented also falls back to Basic Auth.

Test plan

• go test ./internal/listener/... passes
• Configure a source with client_certificate_subject set, send a TLS request with a matching client certificate, and confirm the source is resolved without Basic Auth credentials
• Send a TLS request with an unrecognised subject and no Basic Auth — confirm 401 with a WWW-Authenticate header mentioning client certificates
• Send a TLS request with client certificate and Basic Auth credentials — confirm 400
• Send a TLS request without an client certificate but valid Basic Auth credentials — confirm the Basic Auth source is resolved correctly
• Apply the migration on both MySQL and PostgreSQL and verify the schema matches

Depends on:

• add CRL-based client certificate revocation check icinga-go-library#219

resolves #411

[oxzi]

Otherwise, looks fine to me.

[oxzi]

This PR still misses documentation. Both that this feature is available as well as how to use it.

[oxzi]

First and foremost: I have successfully tested the PR with a demo CA setup, as one can read in the collapsed details block below.

Toggle demo run details

Create demo CA and certificates:

$ # Create CA

$ openssl genpkey -algorithm rsa -out ca.key

$ openssl req -new -x509 -days 1 -key ca.key -sha512 -out ca.crt
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [AU]:DE
State or Province Name (full name) [Some-State]:Franconia
Locality Name (eg, city) []:Nuremberg
Organization Name (eg, company) [Internet Widgits Pty Ltd]:Icinga GmbH
Organizational Unit Name (eg, section) []:Icinga Notifications Manufactory
Common Name (e.g. server FQDN or YOUR name) []:
Email Address []:

$ # Create server cert

$ openssl genpkey -algorithm rsa -out server.key

$ openssl req -new -key server.key -out server.csr
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [AU]:DE
State or Province Name (full name) [Some-State]:Franconia
Locality Name (eg, city) []:Nuremberg
Organization Name (eg, company) [Internet Widgits Pty Ltd]:Icinga GmbH
Organizational Unit Name (eg, section) []:
Common Name (e.g. server FQDN or YOUR name) []:example.com
Email Address []:

Please enter the following 'extra' attributes
to be sent with your certificate request
A challenge password []:
An optional company name []:

$ openssl x509 -req -days 1 -in server.csr -CA ca.crt -CAkey ca.key -CAcreateserial -out server.crt
Certificate request self-signature ok
subject=C=DE, ST=Franconia, L=Nuremberg, O=Icinga GmbH, CN=example.com

$ # Generate client cert

$ cat client_cert.ext
keyUsage = digitalSignature
extendedKeyUsage = clientAuth

$ openssl genpkey -algorithm rsa -out client.key

$ openssl req -new -key client.key -out client.csr
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [AU]:DE
State or Province Name (full name) [Some-State]:Franconia
Locality Name (eg, city) []:Nuremberg
Organization Name (eg, company) [Internet Widgits Pty Ltd]:Icinga GmbH
Organizational Unit Name (eg, section) []:
Common Name (e.g. server FQDN or YOUR name) []:icingadb
Email Address []:

Please enter the following 'extra' attributes
to be sent with your certificate request
A challenge password []:
An optional company name []:

$ openssl x509 -req -days 1 -in client.csr -CA ca.crt -CAkey ca.key -CAcreateserial -extfile client_cert.ext -out client.crt
Certificate request self-signature ok
subject=C=DE, ST=Franconia, L=Nuremberg, O=Icinga GmbH, CN=icingadb

$ # Cleanup and status

$ rm *.csr

$ for crt in *.crt; do echo ">>> $crt"; openssl x509 -in "$crt" -noout -issuer -subject; done
>>> ca.crt
issuer=C=DE, ST=Franconia, L=Nuremberg, O=Icinga GmbH, OU=Icinga Notifications Manufactory
subject=C=DE, ST=Franconia, L=Nuremberg, O=Icinga GmbH, OU=Icinga Notifications Manufactory
>>> client.crt
issuer=C=DE, ST=Franconia, L=Nuremberg, O=Icinga GmbH, OU=Icinga Notifications Manufactory
subject=C=DE, ST=Franconia, L=Nuremberg, O=Icinga GmbH, CN=icingadb
>>> server.crt
issuer=C=DE, ST=Franconia, L=Nuremberg, O=Icinga GmbH, OU=Icinga Notifications Manufactory
subject=C=DE, ST=Franconia, L=Nuremberg, O=Icinga GmbH, CN=example.com

Monkey-patched Icinga Notifications Dockerfile to add necessary server certs and CA:

diff --git a/Dockerfile b/Dockerfile
index f04a576..fe2a9ff 100644
--- a/Dockerfile
+++ b/Dockerfile
@@ -15,11 +15,16 @@ FROM docker.io/library/alpine

 COPY --from=build /target /

+COPY server.crt /
+COPY server.key /
+COPY ca.crt /
+
 RUN apk add tzdata

 ARG username=notifications
 RUN addgroup -g 1000 $username
 RUN adduser -u 1000 -H -D -G $username $username
+RUN chown notifications:notifications /ca.crt /server.crt /server.key
 USER $username

 EXPOSE 5680

Launch container with additional environment variables:

      ICINGA_NOTIFICATIONS_LISTENER_TLS: 1
      ICINGA_NOTIFICATIONS_LISTENER_CERT: /server.crt
      ICINGA_NOTIFICATIONS_LISTENER_KEY: /server.key
      ICINGA_NOTIFICATIONS_LISTENER_CA: /ca.crt

Manually populate source.client_certificate_cn as Web is not there yet:

mysql> select * from source;
+----+----------+----------+-------------------+--------------------------------------------------------------+-----------------------+---------------+---------+--------+
| id | type     | name     | listener_username | listener_password_hash                                       | client_certificate_cn | changed_at    | deleted | locked |
+----+----------+----------+-------------------+--------------------------------------------------------------+-----------------------+---------------+---------+--------+
|  1 | icingadb | icingadb | icingadb          | $2y$12$SpHOoIV7/9bO.tM4W/YyVuAgN56H25kfAIGdZnyw2dl2.afuVRj2G | NULL                  | 1783326692179 | n       | n      |
+----+----------+----------+-------------------+--------------------------------------------------------------+-----------------------+---------------+---------+--------+
1 row in set (0.001 sec)

mysql> update source set client_certificate_cn = 'icingadb', changed_at = 1783326692180;
Query OK, 1 row affected (0.006 sec)
Rows matched: 1  Changed: 1  Warnings: 0

Finally, curl it:

$ curl -v --resolve example.com:5680:127.0.0.1 --cacert ca.crt --cert client.crt --key client.key -H 'X-Icinga-Reject-If-Relations-Incomplete: true' -d '@-' 'https://example.com:5680/process-event' <<EOF
{
  "name": "dummy-809: random fortune",
  "url": "http://localhost/icingaweb2/icingadb/service?name=random%20fortune&host.name=dummy-809",
  "tags": {
    "host": "dummy-809",
    "service": "random fortune"
  },
  "type": "state",
  "severity": "crit",
  "message": "Something went somewhere very wrong.",
  "complete_relations": [
    "host",
    "services",
    "hostgroups",
    "servicegroups"
  ],
  "relations": {
    "host": {
      "name": "dummy-809",
      "display_name": "My Dummy Host",
      "vars": {
        "os": "linux"
      }
    },
    "services": [
      {
        "name": "random fortune",
        "display_name": "Random Fortune Service",
        "vars": {
          "env": "production",
          "team": "devops"
        }
      }
    ],
    "hostgroups": [
      {
        "name": "linux-servers",
        "display_name": "Linux Servers"
      }
    ],
    "servicegroups": [
      {
        "name": "production-services",
        "display_name": "Production Services"
      }
    ]
  }
}
EOF
* Added example.com:5680:127.0.0.1 to DNS cache
* Hostname example.com was found in DNS cache
* Host example.com:5680 was resolved.
* IPv6: (none)
* IPv4: 127.0.0.1
*   Trying 127.0.0.1:5680...
* ALPN: curl offers h2,http/1.1
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* SSL Trust Anchors:
*   CAfile: ca.crt
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Request CERT (13):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Certificate (11):
* TLSv1.3 (OUT), TLS handshake, CERT verify (15):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_128_GCM_SHA256 / X25519MLKEM768 / RSASSA-PSS
* ALPN: server accepted h2
* Server certificate:
*   subject: C=DE; ST=Franconia; L=Nuremberg; O=Icinga GmbH; CN=example.com
*   start date: Jul  6 08:10:06 2026 GMT
*   expire date: Jul  7 08:10:06 2026 GMT
*   issuer: C=DE; ST=Franconia; L=Nuremberg; O=Icinga GmbH; OU=Icinga Notifications Manufactory
*   Certificate level 0: Public key type RSA (2048/112 Bits/secBits), signed using sha256WithRSAEncryption
*   Certificate level 1: Public key type RSA (2048/112 Bits/secBits), signed using sha512WithRSAEncryption
*  common name: example.com (matched)
* OpenSSL verify result: 0
* SSL certificate verified via OpenSSL.
* Established connection to example.com (127.0.0.1 port 5680) from 127.0.0.1 port 50302
* using HTTP/2
* [HTTP/2] [1] OPENED stream for https://example.com:5680/process-event
* [HTTP/2] [1] [:method: POST]
* [HTTP/2] [1] [:scheme: https]
* [HTTP/2] [1] [:authority: example.com:5680]
* [HTTP/2] [1] [:path: /process-event]
* [HTTP/2] [1] [user-agent: curl/8.20.0]
* [HTTP/2] [1] [accept: */*]
* [HTTP/2] [1] [x-icinga-reject-if-relations-incomplete: true]
* [HTTP/2] [1] [content-length: 970]
* [HTTP/2] [1] [content-type: application/x-www-form-urlencoded]
> POST /process-event HTTP/2
> Host: example.com:5680
> User-Agent: curl/8.20.0
> Accept: */*
> X-Icinga-Reject-If-Relations-Incomplete: true
> Content-Length: 970
> Content-Type: application/x-www-form-urlencoded
>
* upload completely sent off: 970 bytes
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
< HTTP/2 400
< content-type: text/plain; charset=utf-8
< server: icinga-notifications/0.2.0-g1863867-dirty
< x-content-type-options: nosniff
< content-length: 65
< date: Mon, 06 Jul 2026 08:46:46 GMT
<
invalid event: at least one of 'incident' or 'muted' must be set
* Connection #0 to host example.com:5680 left intact

Verify it in the logs:

$ docker logs icinga-dev-notifications-1 2>&1 | grep 'Source is authenticated'
2026-07-06T08:46:46.615Z        DEBUG   listener        Source is authenticated via a TLS client certificate    {"source_name": "icingadb", "common name": "icingadb"}

(As the testing was quite excessive, I will urge for integration tests, similar to those for Icinga DB.)

Furthermore, I have added some small comments. Please address those.

With the reworked authentication flow, I have now suggested a few small changes to increase debugability in the Listener.sourceFromAuthOrAbort method. As there are now multiple branches, this would enable an administrator to trace authentication failures and successes.

In the end, please squash your commits. Then, this PR would be mergable for me :)

[oxzi]

Suggested change

	- **TCP with TLS:** If the request arrives with a verified TLS client certificate, the source is identified by the
	certificate's Subject CN. If no matching source is found, the request falls back to HTTP Basic Authentication.
	- **TCP with TLS:** If the request arrives with a TLS client certificate signed by the CA, the source is identified by the
	certificate's Subject CN. If no matching source is found, the request falls back to HTTP Basic Authentication.

[oxzi]

Suggested change

	zap.String("common name", clientCert.Subject.CommonName),
	zap.String("common_name", clientCert.Subject.CommonName),

[oxzi]

While this is not part of this PR, could you please move this debug message down to the success case where src is set (currently line 299) and add keys similar to those of this PR's success debug message: source_name and username.

[oxzi]

Please add a debug log with the error as a zap field here.

[oxzi]

Same as https://github.com/Icinga/icinga-notifications/pull/454/changes#r3527115246.

[oxzi]

Please prefix each error message passed to errFunc with the current branch: unix socket, tls cert, or basic auth.

Suggested change

	return errFunc("unix socket: no value found in context")

[oxzi]

Please also add a comment about curl(1) flags for client certificates after the comment about --unix-socket. Feel free to take inspiration from the shell listing in my main review comment.