Stamus dashboard is empty

[ArtisBergs]

I have installed the next gen Clear NDR version and everything is functioning except the hunting dashboard shows no data. I do see the data in Arkime though. What could be the cause? I have also installed the Selks version and its dashboard is working just fine.

[ArtisBergs]

after server reboot:
/var/run/suricata/suricata-command.socket
Failed to start suri_reloader
please help me fix it.

[angolo40]

Hello, can u please share more info about your environment? Are u on Debian? Vm or physical?

[ArtisBergs]

I use a Hyper-V virtual machine. And I have tried both the latest stable Debian and Ubuntu. Both performed exactly the same with a bug of suri-reloader container not starting up after first reboot.

[dvk99]

+1, ESXI 6.5, Ubuntu 24lts, after installation and before first reboot it was worked OK, after reboot Stamus dashboard is empty, but show last incidents before reboot.

[angolo40]

Hello,

please provide any relevant information during troubleshooting steps in README.

this will help me to better understand the problem.

[Ragevn]

+1, ESXI 6.5, Ubuntu 24lts, after installation and before first reboot it was worked OK, after reboot Stamus dashboard is empty, but show last incidents before reboot.

after server reboot: /var/run/suricata/suricata-command.socket Failed to start suri_reloader please help me fix it.

i am facing same problem on ubuntu 22.04 and debian 13.2 .

[Ragevn]

i tried on debian 12 and it still happen, dashboards are empty:

[Ragevn]

[marcenone]

+1, ESXI 6.5, Ubuntu 24lts, after installation and before first reboot it was worked OK, after reboot Stamus dashboard is empty, but show last incidents before reboot.

after server reboot: /var/run/suricata/suricata-command.socket Failed to start suri_reloader please help me fix it.

i am facing same problem on ubuntu 22.04 and debian 13.2 .

The problem lies in the synchronization of service startup and shutdown (Race Condition).
​
During startup: Fluentd launches faster than OpenSearch (which is a heavy, Java-based service that requires time to initialize).Fluentd panics because it cannot detect the database.

During shutdown: It appears that when the environment is stopping (e.g., docker-compose down), the OpenSearch container is killed before Fluentd has a chance to flush the final logs from its buffer.

​Recommended Solution: Set "Healthcheck" in Docker Compose
​If you are using Docker Compose, ensure that Fluentd waits until OpenSearch is fully ready ("healthy"), rather than just "started".

In my situation the following code solved problem with empty dashboard.
​Example of a correct configuration in docker-compose.yml (edit the following stack file)

nano /root/NDR/config/events-stack/events-stack.compose.yaml

    fluentd:
        image: ghcr.io/stamusnetworks/stamus-images/fluentd:1.16
        # build:
        #     context: .
        #     dockerfile: Dockerfile.fluentd
        user: root
        container_name: config-fluentd-xxxxxxx
        command: fluentd -c /fluentd/etc/fluent.conf
        configs:
            - source: fluentd-config
              target: /fluentd/etc/fluent.conf
            - source: fluentd-template
              target: /fluentd/etc/template.json
        volumes:
            - fluentd-pos:/fluentd/log/td-agent
            - /root/NDR/config/containers-data/suricata/logs:/var/log/suricata:ro
        networks:
            network:
        **depends_on:
            opensearch:
              condition: service_healthy  # Key line!**

If this solution solves the problem I recommend adding this permanently to docker compose stack by developers.

[Ragevn]

+1, ESXI 6.5, Ubuntu 24lts, after installation and before first reboot it was worked OK, after reboot Stamus dashboard is empty, but show last incidents before reboot.

after server reboot: /var/run/suricata/suricata-command.socket Failed to start suri_reloader please help me fix it.

i am facing same problem on ubuntu 22.04 and debian 13.2 .

The problem lies in the synchronization of service startup and shutdown (Race Condition).​During startup: Fluentd launches faster than OpenSearch (which is a heavy, Java-based service that requires time to initialize).Fluentd panics because it cannot detect the database.

During shutdown: It appears that when the environment is stopping (e.g., docker-compose down), the OpenSearch container is killed before Fluentd has a chance to flush the final logs from its buffer.

​Recommended Solution: Set "Healthcheck" in Docker Compose​If you are using Docker Compose, ensure that Fluentd waits until OpenSearch is fully ready ("healthy"), rather than just "started".

In my situation the following code solved problem with empty dashboard.​Example of a correct configuration in docker-compose.yml (edit the following stack file)

nano /root/NDR/config/events-stack/events-stack.compose.yaml

    fluentd:
        image: ghcr.io/stamusnetworks/stamus-images/fluentd:1.16
        # build:
        #     context: .
        #     dockerfile: Dockerfile.fluentd
        user: root
        container_name: config-fluentd-xxxxxxx
        command: fluentd -c /fluentd/etc/fluent.conf
        configs:
            - source: fluentd-config
              target: /fluentd/etc/fluent.conf
            - source: fluentd-template
              target: /fluentd/etc/template.json
        volumes:
            - fluentd-pos:/fluentd/log/td-agent
            - /root/NDR/config/containers-data/suricata/logs:/var/log/suricata:ro
        networks:
            network:
        **depends_on:
            opensearch:
              condition: service_healthy  # Key line!**

If this solution solves the problem I recommend adding this permanently to docker compose stack by developers.

hi, i added depends_on in yaml but still same problem when i reboot

ed2e6f7f7490 ghcr.io/stamusnetworks/scirius:clear-ndr-1.0.0 "/start-suri-reloade…" 16 minutes ago Restarting (1) 24 seconds ago config-suri-reloader-6canminjXSPMvacp

root@yj:~# docker logs config-suri-reloader-6canminjXSPMvacp
/var/run/suricata/suricata-command.socket
/var/run/suricata/suricata-command.socket
Failed to start suri_reloader
/var/run/suricata/suricata-command.socket
Failed to start suri_reloader
/var/run/suricata/suricata-command.socket
Failed to start suri_reloader
/var/run/suricata/suricata-command.socket
Failed to start suri_reloader
/var/run/suricata/suricata-command.socket
Failed to start suri_reloader

[marcenone]

+1, ESXI 6.5, Ubuntu 24lts, after installation and before first reboot it was worked OK, after reboot Stamus dashboard is empty, but show last incidents before reboot.

after server reboot: /var/run/suricata/suricata-command.socket Failed to start suri_reloader please help me fix it.

i am facing same problem on ubuntu 22.04 and debian 13.2 .

The problem lies in the synchronization of service startup and shutdown (Race Condition).​During startup: Fluentd launches faster than OpenSearch (which is a heavy, Java-based service that requires time to initialize).Fluentd panics because it cannot detect the database.
During shutdown: It appears that when the environment is stopping (e.g., docker-compose down), the OpenSearch container is killed before Fluentd has a chance to flush the final logs from its buffer.
​Recommended Solution: Set "Healthcheck" in Docker Compose​If you are using Docker Compose, ensure that Fluentd waits until OpenSearch is fully ready ("healthy"), rather than just "started".
In my situation the following code solved problem with empty dashboard.​Example of a correct configuration in docker-compose.yml (edit the following stack file)

nano /root/NDR/config/events-stack/events-stack.compose.yaml

    fluentd:
        image: ghcr.io/stamusnetworks/stamus-images/fluentd:1.16
        # build:
        #     context: .
        #     dockerfile: Dockerfile.fluentd
        user: root
        container_name: config-fluentd-xxxxxxx
        command: fluentd -c /fluentd/etc/fluent.conf
        configs:
            - source: fluentd-config
              target: /fluentd/etc/fluent.conf
            - source: fluentd-template
              target: /fluentd/etc/template.json
        volumes:
            - fluentd-pos:/fluentd/log/td-agent
            - /root/NDR/config/containers-data/suricata/logs:/var/log/suricata:ro
        networks:
            network:
        **depends_on:
            opensearch:
              condition: service_healthy  # Key line!**

If this solution solves the problem I recommend adding this permanently to docker compose stack by developers.

hi, i added depends_on in yaml but still same problem when i reboot

ed2e6f7f7490 ghcr.io/stamusnetworks/scirius:clear-ndr-1.0.0 "/start-suri-reloade…" 16 minutes ago Restarting (1) 24 seconds ago config-suri-reloader-6canminjXSPMvacp

root@yj:~# docker logs config-suri-reloader-6canminjXSPMvacp /var/run/suricata/suricata-command.socket /var/run/suricata/suricata-command.socket Failed to start suri_reloader /var/run/suricata/suricata-command.socket Failed to start suri_reloader /var/run/suricata/suricata-command.socket Failed to start suri_reloader /var/run/suricata/suricata-command.socket Failed to start suri_reloader /var/run/suricata/suricata-command.socket Failed to start suri_reloader

+1, ESXI 6.5, Ubuntu 24lts, after installation and before first reboot it was worked OK, after reboot Stamus dashboard is empty, but show last incidents before reboot.

after server reboot: /var/run/suricata/suricata-command.socket Failed to start suri_reloader please help me fix it.

i am facing same problem on ubuntu 22.04 and debian 13.2 .

The problem lies in the synchronization of service startup and shutdown (Race Condition).​During startup: Fluentd launches faster than OpenSearch (which is a heavy, Java-based service that requires time to initialize).Fluentd panics because it cannot detect the database.
During shutdown: It appears that when the environment is stopping (e.g., docker-compose down), the OpenSearch container is killed before Fluentd has a chance to flush the final logs from its buffer.
​Recommended Solution: Set "Healthcheck" in Docker Compose​If you are using Docker Compose, ensure that Fluentd waits until OpenSearch is fully ready ("healthy"), rather than just "started".
In my situation the following code solved problem with empty dashboard.​Example of a correct configuration in docker-compose.yml (edit the following stack file)

nano /root/NDR/config/events-stack/events-stack.compose.yaml

    fluentd:
        image: ghcr.io/stamusnetworks/stamus-images/fluentd:1.16
        # build:
        #     context: .
        #     dockerfile: Dockerfile.fluentd
        user: root
        container_name: config-fluentd-xxxxxxx
        command: fluentd -c /fluentd/etc/fluent.conf
        configs:
            - source: fluentd-config
              target: /fluentd/etc/fluent.conf
            - source: fluentd-template
              target: /fluentd/etc/template.json
        volumes:
            - fluentd-pos:/fluentd/log/td-agent
            - /root/NDR/config/containers-data/suricata/logs:/var/log/suricata:ro
        networks:
            network:
        **depends_on:
            opensearch:
              condition: service_healthy  # Key line!**

If this solution solves the problem I recommend adding this permanently to docker compose stack by developers.

hi, i added depends_on in yaml but still same problem when i reboot

ed2e6f7f7490 ghcr.io/stamusnetworks/scirius:clear-ndr-1.0.0 "/start-suri-reloade…" 16 minutes ago Restarting (1) 24 seconds ago config-suri-reloader-6canminjXSPMvacp

root@yj:~# docker logs config-suri-reloader-6canminjXSPMvacp /var/run/suricata/suricata-command.socket /var/run/suricata/suricata-command.socket Failed to start suri_reloader /var/run/suricata/suricata-command.socket Failed to start suri_reloader /var/run/suricata/suricata-command.socket Failed to start suri_reloader /var/run/suricata/suricata-command.socket Failed to start suri_reloader /var/run/suricata/suricata-command.socket Failed to start suri_reloader

There is clearly a problem with permissions when suri_reloader tries to communicate with docker socket created by suricata. I have simply added this to scirius docker compose (/root/NDR/config/scirius/scirius.compose.yml) - add group_add: - 998 somewhere in suri-reloader:

suri-reloader:
        image: ghcr.io/stamusnetworks/scirius:clear-ndr-1.0.0
        container_name: config-suri-reloader-V0ttr4iNzQy2DdLt
        entrypoint: /start-suri-reloader.sh
        restart: unless-stopped
        environment:
            - SURICATA_UNIX_SOCKET=/var/run/suricata/suricata-command.socket
        **group_add:
            - 998**
        healthcheck:

Than small modification in /root/NDR/config/containers-data/suricata/etc/suricata.yaml (to ensure correct permissions on docker socket)

unix-command:
    enabled: auto
    owner: root
    group: root
    mode: 0660

Of course, after this modification, you need to recreate the containers:

cd /root/NDR/config
docker compose down
docker compose up -d

The explanation is that the reloader lacks permission to communicate with the Docker socket created by Suricata. This code in Docker Compose ensures it has the correct permissions. However, this is not the cleanest solution because Docker might mark the container as unhealthy. Ultimately we should wait for developers to accept solution or create their own. I assume it would require modifying either the healthcheck command or the start-suri-reloader.sh script – but I'm too lazy to do that right now :) Nevertheless, the reloader should work now. Let me know if this helps.

[Ragevn]

@marcenone i think i followed correctly but it only works when i recreate the containers, when i reboot it still same result :
config-suri-reloader-xxxxxx keep restarting