-
Notifications
You must be signed in to change notification settings - Fork 12
Troubleshooting
A grab-bag of "things break in this specific way" notes. Add to it when you debug something.
# Service-level log (most things)
sudo journalctl -u rustguac -f
# guacd log (protocol-translation issues)
sudo journalctl -u rustguac | grep guacd
# HAProxy log (reverse-proxy issues, including connection drops)
sudo journalctl -u haproxy -f
# OIDC issues
sudo journalctl -u rustguac | grep -i oidcFor more verbose logs, set RUST_LOG=rustguac=debug (or RUST_LOG=debug for the firehose) in the systemd unit's Environment= directive and restart.
Rectangular black blocks aligned to text-row boundaries on a fast-scrolling page over RDP usually means an H.264 GFX dropped-reference-frame failure: a P-frame referenced a slice the decoder didn't get, so the affected region paints black until the next IDR keyframe.
Test by toggling the per-entry enable_h264 flag off; if the bars vanish, it's a codec/network issue rather than a rustguac bug. Reasons it happens:
- Lossy network between rustguac and the user pushes the H.264 sync gate to drop a frame.
- xrdp's H.264 encoder doesn't emit IDR keyframes often enough during heavy scrolls.
- MTU issues fragmenting H.264 NAL units.
The H.264 sync gating logic landed in v1.5.1 to fix unbounded stream lag (#93). It trades occasional dropped reference frames for bounded latency. If H.264 off works, you've found the tradeoff line.
A mid-path TCP drop kills the WebSocket but the Guacamole client doesn't notice because upstream Client.js doesn't subscribe to tunnel.onerror / onstatechange. Tab looks frozen, mouse moves locally, clicks vanish, no "Session Ended" overlay. Fixed in v1.6.6. Upgrade.
The session on the rustguac side is gone. When the WebSocket closed, complete_session ran and the guacd stream was dropped; there's nothing to reconnect to. v1.6.6's Reconnect button now relaunches the original Connections entry instead of trying to re-attach to the dead session ID.
True hot-reconnect of the existing session (preserving guacd-level state across a network drop) is on the roadmap as "persistent / reconnectable sessions", tracked by issue #63.
Pre-v1.6.5: this was the silent Kerberos / DNS-KDC lookup hitting RDP entries with no auth_pkg set. Default was negotiate, FreeRDP tries Kerberos first, the lookup fails after a long timeout. v1.6.5 flipped the default to NTLM. If you're still seeing it on v1.6.5+, check the entry's auth_pkg setting and the [rdp].default_auth_pkg config knob.
The issuer_url in your rustguac config doesn't match what the IdP returns from its discovery endpoint. The most common cause is a trailing-slash mismatch. v1.6.6 wraps this error to be specific about which side is which; on older versions you have to read the raw openidconnect Debug output.
Check both halves of the comparison the error reports. If they're equal except for a trailing /, fix whichever side is wrong. Some IdPs are picky about this and serve different metadata for https://idp.example.com/ vs https://idp.example.com.
The redirect_uri registered with your IdP must exactly match what rustguac sends. rustguac sends <external_url>/auth/oidc/callback. If you're behind a reverse proxy that rewrites the host header, make sure rustguac's external_url config matches the public URL the user typed into their browser.
Check journalctl -u rustguac for the bollard / Docker error. Common causes:
-
rustguacuser not in thedockergroup. Add it (sudo usermod -aG docker rustguac) and restart the service. - Container image not in
[vdi].allowed_images. The image you're trying to spawn must be in the allowlist. - xrdp not listening when rustguac tries to connect. xrdp binds the port before fully initialising; the code waits 3 seconds after TCP accept. If your image is slow to start xrdp, increase
[vdi].ready_timeout_secs.
xrdp needs TLS certs. Most images use ssl-cert-snakeoil from the Debian ssl-cert package. If the xrdp user can't read /etc/xrdp/key.pem, xrdp falls back to "classic RDP security" and FreeRDP drops the connection with a MAC checksum error.
The repo's contrib/setup-xrdp-gfx.sh (Step 9, added in v1.6.6) covers this for bench VMs. For VDI images, make sure the xrdp user is in the ssl-cert group inside the image:
RUN usermod -aG ssl-cert xrdpChromium needs specific flags on a headless box: --in-process-gpu, --use-gl=angle, --use-angle=swiftshader, --disable-gpu-*, --disable-dev-shm-usage. These are baked into src/browser.rs.
If you're running web sessions in a container or unusual environment and you see "trap int3" in the logs, that's Chromium's crashpad failing because the rustguac user doesn't have a real home directory. Make sure the user has a writable $HOME.
HAProxy is telling rustguac "your client went away mid-stream." Almost always benign: a user closed the tab, the laptop went to sleep, a corporate proxy timed the connection out. If you're seeing it for every session ending around the same elapsed time (e.g. exactly 60 minutes), check your reverse proxy's HTTP timeout (timeout client / timeout tunnel in HAProxy).
Per-entry max_recordings caps the count. Global [recording].max_disk percent threshold + cleanup_interval_secs reaper protects you from runaway growth. If you set neither, recordings grow unbounded.
Two SQLite tables in <data_path>/rustguac.sqlite:
-
addressbook_audit_log: connections create/update/delete (added in v1.6.4). Headline only: action, user, IP, target path, count metadata. Never logs entry field values. -
token_audit_log: API key mints, share token mints, shadow-token uses.
Both are visible in the admin UI. Both are pruned on the same retention window (configurable in [admin]).
sudo systemctl stop rustguac
sudo mv /opt/rustguac/data/rustguac.sqlite{,.broken}
sudo /opt/rustguac/bin/rustguac --config /opt/rustguac/config.toml add-admin --email you@example.com
sudo systemctl start rustguacConnections are in Vault, so they survive. Sessions in flight at the moment of the rebuild are lost. Token / connections audit logs are in the DB so they're gone unless you recover from backup.
If you spent more than ten minutes debugging something, add it here so the next person doesn't.
Background
Deployment
Security
Help