Systemd service dont working

[norz3n]

Greetings!

The daemon for systemd doesn't work for me. It won't even start. The program itself works without issue when started via the terminal.

Here is the debugging from journalctl:

апр 26 07:36:14 workstation systemd[1]: Started Daemon that prevents eviction of executables/libraries from memory.
апр 26 07:36:14 workstation (prelockd)[551]: prelockd.service: Failed to set up mount namespacing: /run/systemd/unit-root/var/lib/prelockd: No such file or directory
апр 26 07:36:14 workstation (prelockd)[551]: prelockd.service: Failed at step NAMESPACE spawning /usr/bin/prelockd: No such file or directory
апр 26 07:36:14 workstation systemd[1]: prelockd.service: Main process exited, code=exited, status=226/NAMESPACE
апр 26 07:36:14 workstation systemd[1]: prelockd.service: Failed with result 'exit-code'.
апр 26 07:39:10 workstation systemd[1]: Started Daemon that prevents eviction of executables/libraries from memory.
апр 26 07:39:10 workstation (prelockd)[3598]: prelockd.service: Failed to set up mount namespacing: /run/systemd/unit-root/var/lib/prelockd: No such file or directory
апр 26 07:39:10 workstation (prelockd)[3598]: prelockd.service: Failed at step NAMESPACE spawning /usr/bin/prelockd: No such file or directory
апр 26 07:39:10 workstation systemd[1]: prelockd.service: Main process exited, code=exited, status=226/NAMESPACE
апр 26 07:39:10 workstation systemd[1]: prelockd.service: Failed with result 'exit-code'.
апр 26 07:50:17 workstation systemd[1]: Started Daemon that prevents eviction of executables/libraries from memory.
апр 26 07:50:17 workstation (prelockd)[5041]: prelockd.service: Failed to set up mount namespacing: /run/systemd/unit-root/var/lib/prelockd: No such file or directory
апр 26 07:50:17 workstation (prelockd)[5041]: prelockd.service: Failed at step NAMESPACE spawning /usr/bin/prelockd: No such file or directory
апр 26 07:50:17 workstation systemd[1]: prelockd.service: Main process exited, code=exited, status=226/NAMESPACE
апр 26 07:50:17 workstation systemd[1]: prelockd.service: Failed with result 'exit-code'.

But the same command in the daemon works fine.

sudo prelockd -c /etc/prelockd.conf
Starting prelockd with config /etc/prelockd.conf
currently locked 187.4M, 627 files

Here are the specifications of my system from inxi:

System:
  Host: workstation Kernel: 6.2.12-zen1-1-zen arch: x86_64 bits: 64
    Desktop: GNOME v: 43.3 Distro: Hybrid Linux
Machine:
  Type: Laptop System: Hewlett-Packard product: HP 620 v: F.20
    serial: <superuser required>
  Mobo: Hewlett-Packard model: 1526 v: KBC Version 71.0E
    serial: <superuser required> BIOS: Hewlett-Packard v: 68PVI Ver. F.20
    date: 12/12/2011
Battery:
  ID-1: BAT0 charge: 14.2 Wh (94.0%) condition: 15.1/47.5 Wh (31.7%)
CPU:
  Info: dual core Intel Core2 Duo T6570 [MCP] speed (MHz): avg: 1306
    min/max: 1200/2101
Graphics:
  Device-1: Intel Mobile 4 Series Integrated Graphics driver: i915 v: kernel
  Device-2: Chicony HP Laptop Integrated Webcam [2 MP Fixed] type: USB
    driver: uvcvideo
  Display: wayland server: X.org v: 1.21.1.8 with: Xwayland v: 22.1.8
    compositor: gnome-shell driver: X: loaded: modesetting unloaded: fbdev,vesa
    dri: crocus gpu: i915 resolution: 1366x768
  API: OpenGL v: 2.1 Mesa 23.0.3 renderer: Mesa Mobile Intel GM45 Express
    (CTG)
Network:
  Device-1: Realtek RTL8191SEvA Wireless LAN driver: rtl8192se
  Device-2: Realtek RTL810xE PCI Express Fast Ethernet driver: r8169
Drives:
  Local Storage: total: 119.24 GiB used: 86.71 GiB (72.7%)
Info:
  Processes: 231 Uptime: 32m Memory: 3.73 GiB used: 2.53 GiB (68.0%)
  Shell: Zsh inxi: 3.3.26

[hakavlad]

Hi!

апр 26 07:36:14 workstation (prelockd)[551]: prelockd.service: Failed at step NAMESPACE spawning /usr/bin/prelockd: No such file or directory

Seems to be the problem here:

RestrictNamespaces=yes

see https://github.com/hakavlad/prelockd/blob/master/prelockd.service.in

please test the corrected file (prelockd.service without this line).
Let me know if you need more information.

[hakavlad]

Are you sure you need prelockd?

Your kernel is already using the patch https://www.phoronix.com/news/MGLRU-LPC-2022

[patrakov]

Regarding the question whether prelockd is still needed: I am also an Arch Linux user, and with the default MGLRU settings, it is still possible to freeze the box. I have not yet tested whether prelockd helps, but now that I found it, I will test it, too.

My objection to your testing methodology is that you use "tail /dev/zero" as a simulation of a memory leak, because it provides overly optimistic expectations whether the OOM killer is able to fix the problem and how long it takes. In fact, while supporting certain clusterware, I found an interesting edge case. A component of the cluster has a memory leak. However, the nature of the leak is such that it only happens when the cluster is operational, from the viewpoint of that component. But, under near-OOM conditions, the internal cluster connectivity check times out, and the leak effectively stops. Thus, the OOM killer never comes.

Here is a Python script with approximately the same behavior (a leak stops by itself under near-OOM conditions) that you can use for testing:

#!/usr/bin/python3

import time

leave_free_kb = 75000  # To be fine-tuned for a particular system
chunks = []

dev_urandom = open("/dev/urandom", "rb")

# Safeguard so that the script exits by itself eventually
for _ in range(90):
    time.sleep(1)
    mem_available = 0
    with open("/proc/meminfo", "r") as meminfo:
        for line in meminfo:
            parts = line.split()
            if parts[0] == 'MemAvailable:' and parts[2] == 'kB':
                mem_available = int(parts[1])
                break

    print(f"MemAvailable: {mem_available} kB")
    claim_this_time = mem_available - leave_free_kb
    claim_this_time = int(claim_this_time // 8 * 4)
    if claim_this_time <= 0:
        print("The system doesn't have a lot of memory available, not eating")
        continue

    print(f"Eating {claim_this_time} kB of memory...")
    chunks.append(dev_urandom.read(1024 * claim_this_time))

print("Exiting due to the safeguard")

[patrakov]

Also, my objection to MGLRU is that it protects file-backed memory that was accessed in the past X milliseconds. When troubleshooting a server, the first thing that needs to be done is to ssh there. Which is a problem, because, if a server is not under attack, sshd code is likely not accessed in the past X milliseconds, and we are back to square one.

[hakavlad]

My objection to your testing methodology is that you use "tail /dev/zero" as a simulation of a memory leak

See https://github.com/hakavlad/file-starve

This script reduces available memory, but not to zero. It can be useful for modeling memory leaks and studying the state of the system during leakage.

[hakavlad]

MGLRU is that it protects file-backed memory that was accessed in the past X milliseconds

Did you try to set hight min_ttl_ms values? see https://github.com/hakavlad/mg-lru-helper

Which is a problem, because

Have you observed a problem in practice or is it just a theory? (when using MGLRU with high min_ttl_ms, for example)

[hakavlad]

btw prelockd may have non-optimal default settings

[hakavlad]

Did you fix prelockd's systemd service?

[patrakov]

For me it works out of the box if I install prelockd from the AUR. The difference is that the empty /var/lib/prelockd directory is included in the package.

I confirm that with MGLRU with the default Arch Linux settings, with my script as a malicious workload, prelockd converts a hard freeze into a sluggish system that still can be accessed via ssh. I have not yet tested whether it is possible to tune MGLRU to achieve the same.