Repository navigation
Add Ansible-managed Caddy installation and internal TLS for bell webhooks #96
Description
Create an idempotent Ansible solution that installs Caddy on Raspberry Pi bell nodes and exposes the School Bell webhook securely over HTTPS.
Architecture
Use the following model:
- Assign every bell a stable IP address through a DHCP reservation.
- Use a unique hostname for every bell, such as
pibell-vito-01.local. - Use hostnames, not IP addresses, in HTTPS webhook URLs.
- Resolve hostnames through managed local DNS where available.
- Provide an Ansible-managed
/etc/hostsfallback when local DNS is unavailable or too slow. - Maintain one internal certificate authority for the bell environment.
- Distribute only the public root CA certificate to participating nodes.
- Issue a unique leaf certificate and private key for every bell.
- Terminate HTTPS in Caddy and proxy to School Bell over loopback.
Do not share one leaf certificate or private key between bell nodes. Never distribute the CA private key to managed nodes.
Inventory
Support deployment through inventory hosts and groups, including selection by school or bell group.
Example:
[school_bells]
pibell-main ansible_host=192.0.2.41 caddy_hostname=pibell-main.local
pibell-yard ansible_host=192.0.2.42 caddy_hostname=pibell-yard.local
[site_bells:children]
school_bells
Activity
psmsmets commentedon Sep 12, 2026
Local HTTPS webhook setup with Caddy
This procedure was validated with two Raspberry Pi devices:
pibell-main: sending nodepibell-yard: receiving node running the webhookpibell-yard.school-bell.internal: HTTPS hostname192.0.2.42: example fixed address of the receiving nodeUse real addresses from the target network. The
192.0.2.0/24range is reserved for documentation.1. Provide stable addressing
Assign every bell a fixed DHCP reservation. Certificates and remote webhook configuration should use hostnames, not IP addresses.
Example:
Use
.internalfor private network names. Avoid.local, because it is reserved for mDNS, and avoid public suffixes such as.dev.2. Create proper DNS records
In Pi-hole, open:
Create an A-record:
Do not create a CNAME pointing to an IP address. This incorrect configuration:
must instead return:
When multiple DNS servers are advertised through DHCP, ensure they all return the same internal records. Do not advertise a public resolver as a secondary DNS server, because it cannot resolve private names and may be selected independently by clients.
3. Install DNS diagnostic tools
Install the BIND client utilities on each Raspberry Pi:
This installs tools such as
digandnslookup; it does not install the BIND DNS server.Verify DNS directly against each internal resolver:
Verify normal system resolution:
Both must resolve to the fixed address:
digqueries DNS directly, while applications such ascurluse the system resolver. Ifdigworks butgetentdoes not, inspect:grep '^hosts:' /etc/nsswitch.conf cat /etc/resolv.confThe
hosts:entry must includedns.4. Bind the School Bell webhook to loopback
Configure the receiving node so that its built-in HTTP webhook is not exposed directly to the network:
{ "webhook": { "enabled": true, "host": "127.0.0.1", "port": 8081, "token": "replace-with-a-long-random-token", "rate_limit": 10, "rate_window": 60 } }Validate the School Bell configuration and restart the service as appropriate.
Check the backend without activating the bell:
Expected result:
This confirms that the webhook is running. The endpoint only accepts
POST.5. Install Caddy on the receiving node
Install Caddy using its official Debian/Raspberry Pi OS installation procedure:
Back up the default configuration:
Configure
/etc/caddy/Caddyfile:This configuration:
Validate and load the configuration:
Verify that Caddy is listening on HTTPS:
Inspect errors when necessary:
6. Verify HTTPS locally
Caddy's public root certificate is normally stored at:
Install it in the receiving node's system trust store:
Test using normal DNS resolution:
Expected result:
For temporary testing before DNS is operational, preserve hostname validation with:
Do not use
curl -k, because that disables certificate verification.7. Export the public CA certificate
On the receiving node, export only the public root certificate to a location readable by the SSH user:
Adjust the account and home directory where necessary.
Record its SHA-256 fingerprint:
Never export or copy private keys from Caddy's data directory.
8. Trust the CA on the sending node
Copy the public certificate over an authenticated SSH connection:
Verify its fingerprint on the sending node:
Confirm that it matches the fingerprint shown on the receiving node.
Install it in the system trust store:
Verify the complete connection without
--resolve,--cacertor-k:Expected result:
This confirms that DNS resolution, CA trust, hostname validation, TLS and the reverse proxy all work.
9. Test an authenticated webhook request
Perform an explicit test before enabling automatic remote triggers:
Expected result:
The configured signal should activate on the receiving node.
Relevant failure responses include:
400: invalid payload or WAVE key;401: incorrect bearer token;409: another signal is already active;429: rate limit exceeded;502: Caddy cannot reach127.0.0.1:8081;10. Configure the remote trigger
Configure the sending node to use the final HTTPS hostname:
{ "transport": "webhook", "url": "https://pibell-yard.school-bell.internal/bell", "auth": { "type": "bearer", "token": "replace-with-a-long-random-token" }, "timeout": 5 }Restart School Bell after validating the configuration.
Trigger the sending node manually and confirm that the receiving node activates.
11. Monitor the end-to-end test
On the receiving node:
In another terminal, monitor Caddy:
A successful request should produce the School Bell webhook lifecycle events, including:
The sending node should report a successful
manual_remote_trigger.12. Network controls
Permit only the required flows:
Do not expose port
8081outside the receiving host. The built-in webhook remains plain HTTP and should stay bound to127.0.0.1.Because the reverse proxy connects over loopback, all proxied requests appear to the built-in webhook as the same client. Its rate limit is therefore shared by all clients using that proxy.
13. Requirements for later Ansible automation
The optional Caddy automation should:
caddyandbind9-dnsutils;caddy_hostname;127.0.0.1;digand application-level verification withgetent;/etc/hostsentries only as a controlled fallback;For a two-node manual deployment, trusting the receiving node's Caddy CA is sufficient. For a larger managed deployment, define one controlled internal CA and let it issue a separate certificate for every receiving node. Never copy one server certificate between devices, and never distribute the CA private key to managed nodes.
psmsmets commentedon Sep 12, 2026
Additional findings from manual validation
The complete HTTPS webhook flow was validated successfully between two Raspberry Pi nodes. The following findings should be incorporated into the documentation and optional Ansible automation.
Use a private hostname
The
.localsuffix caused conflicts with mDNS. A hostname below the private.internalnamespace worked correctly:Use the same hostname consistently in:
Create an A-record, not a CNAME
The first Pi-hole record was accidentally created as:
Although
digreturned a value, applications using the system resolver could not resolve the hostname because the IP address was incorrectly treated as a CNAME target.Create the record under:
The correct response is:
Verify DNS using:
When multiple DNS servers are advertised through DHCP, ensure that they all return the same internal records. Do not advertise a public resolver as a secondary DNS server because it cannot resolve private hostnames and may be selected independently by clients.
Install DNS diagnostic tools
Install the BIND client utilities on the Raspberry Pi nodes:
This provides
digandnslookup; it does not install the BIND DNS server.Useful diagnostic commands include:
The optional Ansible Caddy playbook should install
bind9-dnsutilsso DNS can be diagnosed immediately after deployment.Keep the built-in webhook on loopback
The School Bell webhook should remain bound to:
Caddy accepts external HTTPS traffic on port 443 and proxies it locally over HTTP:
Do not expose port 8081 directly to the network.
Remove the backend port from remote URLs
After introducing Caddy, remote webhook URLs must use the standard HTTPS port:
The following URL is incorrect:
Port 8081 is the plain HTTP backend and does not provide TLS. Existing remote webhook configurations must therefore remove
:8081when migrating to Caddy.Distribute only Caddy's public root certificate
Caddy's public root certificate is normally stored at:
Copy only this public certificate to sending nodes. Never copy private keys from Caddy's data directory.
Before installing the certificate, compare its SHA-256 fingerprint on the receiving and sending nodes:
Install the verified root certificate on a Debian or Raspberry Pi OS sending node:
After installation, the following command should complete TLS verification and return HTTP status 405:
Do not use
curl -k, because it disables certificate verification.Python Requests does not automatically use the installed local CA
Installing Caddy's root certificate in the operating-system trust store was sufficient for
curl, but School Bell initially failed with:School Bell uses Python
requestsfrom its virtual environment. In the tested environment, it used its own CA bundle instead of the operating-system bundle.The connection succeeded when tested with the system CA bundle:
The expected result is:
The verified persistent workaround is a systemd drop-in:
Drop-in contents:
Apply and verify it with:
The documentation and optional Ansible automation should:
/usr/local/share/ca-certificates/;update-ca-certificates;REQUESTS_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt;certifibundle directly, because package updates may overwrite that change.End-to-end validation and latency
The HTTPS remote trigger was successfully validated between two Raspberry Pi nodes.
Observed event sequence:
The receiving bell started approximately 150 ms after the local trigger. This includes DNS resolution, TCP connection establishment, the TLS handshake, Caddy proxying and webhook processing.
The local bell was not delayed while the remote HTTPS request was performed. The remote action ran concurrently and completed successfully.
Observed completion times remained approximately aligned:
The measured difference of roughly 140–150 ms is acceptable for this use case. No connection-persistence or latency optimization is currently required.
Automation requirements identified during validation
The optional Ansible Caddy automation should:
bind9-dnsutils;digandgetent;/etc/hostsentries only as an optional DNS fallback;psmsmets commentedon Sep 12, 2026
End-to-end validation and latency
The HTTPS remote trigger was successfully validated between two Raspberry Pi nodes.
Observed event sequence:
The receiving bell started approximately 150 ms after the local trigger. This includes DNS resolution, TCP connection establishment, the TLS handshake, Caddy proxying and webhook processing.
The local bell was not delayed while the remote HTTPS request was performed. The remote action ran concurrently and completed successfully.
Observed completion times remained approximately aligned:
The measured difference of roughly 140–150 ms is acceptable for this use case. No connection-persistence or latency optimization is currently required.