Home Assistant causes issues with NGINX's zstd compression

[jalil-salame]

Describe the bug

When using home-assistant and nginx with zstd compression, nginx's service worker becomes unresposive once a request is made to home-assistant.

Steps To Reproduce

Steps to reproduce the behavior:

1. Enable home-assistant with nginx as a reverse proxy:

   {
     services = {
       home-assistant = {
         enable = true;
         config = {
           default_config = { };
           http = {
             server_host = [ "127.0.0.1" ];
             trusted_proxies = [ "127.0.0.1" ];
             use_x_forwarded_for = true;
           };
         };
       };
       nginx = {
         enable = true;
         recommendedProxySettings = true;
         recommendedZstdSettings = true;
         virtualHosts."example.org" = {
           extraConfig = # nginx
             ''
               proxy_buffering off;
             '';
           locations."/" = {
             proxyPass = "http://127.0.0.1:8123";
             proxyWebsockets = true;
           };
         };
       };
     };
   }

2. Make a request to home-assistant
3. Watch the nginx process go up to 100% CPU consumption.
4. Restarting nginx fixes it temporarily but a new request to home-assistant causes the spinloop to happen again.
5. Add zstd off; to home-assistant's virtualHosts.extraConfig and watch the problem go away.

Expected behavior

I wanted a warning or some documentation indicating that zstd and home-assistant did not work well together.

Additional context

• Core dumping the nginx worker process shows it is stuck during zstd compresssion.
• I came across this when I installed home-assistant on a server that had harmonia on it, but harmonia as of a recent commit no longer needs zstd compression.

Metadata

❯ nix run nixpkgs#nix-info -- -m
 - system: `"x86_64-linux"`
 - host os: `Linux 6.6.60, NixOS, 24.11 (Vicuna), 24.11.20241123.0c58267`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Lix, like Nix) 2.91.1
System type: x86_64-linux
Additional system types: i686-linux, x86_64-v1-linux, x86_64-v2-linux, x86_64-v3-linux
Features: gc, signed-caches
System configuration file: /etc/nix/nix.conf
User configuration files: /home/jalil/.config/nix/nix.conf:/etc/xdg/nix/nix.conf:/home/jalil/.nix-profile/etc/xdg/nix/nix.conf:/nix/profile/etc/xdg/nix/nix.conf:/home/jalil/.local/state/nix/profile/etc/xdg/nix/nix.conf:/etc/profiles/per-user/jalil/etc/xdg/nix/nix.conf:/nix/var/nix/profiles/default/etc/xdg/nix/nix.conf:/run/current-system/sw/etc/xdg/nix/nix.conf
Store directory: /nix/store
State directory: /nix/var/nix
Data directory: /nix/store/j6f86lyxvw2iig2vxzgd4aib0r6jf0z7-lix-2.91.1/share`
 - nixpkgs: `/nix/store/fnbr9s78sy4islw83vsaf0hgf88hih5v-source`

Notify maintainers

• nginx: @fpletz @RaitoBezarius @dasJ @Conni2461
• home-assistant: @fabaff @mweinelt @Mic92

---

Note for maintainers: Please tag this issue in your PR.

---

Add a 👍 reaction to issues you find important.

[mweinelt]

Not a Home Assistant issue, but I've seen the issue with harmonia as well, which now moved away from recommending nginx's zstd integration.

[jalil-salame]

Been running harmonia on stable for ~1 year and yesterday (when I set up home-assistant) is the first time this happened, that's why I assumed home-assistant was related to it.

[Mic92]

In harmonia it would be also slower to use nginx's zstd compression. Because it will not use multiple cores if you serve a single client unless you also disable http 2.

[SuperSandro2000]

Maybe tokers/zstd-nginx-module#23 ?

[Mic92]

Ah please just update the module. There was an ordering issue in the middleware layer that breaks some other middleware functions in nginx. tokers/zstd-nginx-module@f4ba115 has the fix

[mweinelt]

Please retry with the updated zstd module and report back. Will close until further reply.