Sandboxing systemd Services on Debian 13

Most daemons on a stock Debian box run with far more access than they need. A web server that only has to read /var/www and write its own logs can usually also read home directories, write to /etc, load kernel modules if it runs as root, and make any syscall it likes. If someone gets remote code execution in that process, all of that access comes with it.

systemd already ships a sandbox. It's configured per unit with directives like ProtectSystem=, PrivateTmp= and SystemCallFilter=, and systemd-analyze security tells you how exposed each service is. This post walks through sandboxing systemd services on Debian 13 (trixie, systemd 257): first scoring units, then hardening nginx with a drop-in, then debugging it when it breaks. The last part covers what you should not copy blindly.

Scoring units with systemd-analyze security

Run it with no arguments to get one line per loaded service:

systemd-analyze security --no-pager

Pass a unit name to get the full breakdown for one service:

systemd-analyze security --no-pager nginx.service

The detailed view lists every check with a short description and how much it adds to exposure. The last line gives the overall score from 0.0 to 10.0. Higher means less sandboxing. The labels come from a fixed table in systemd's analyze-security.c:

Two caveats from the man page are worth knowing before you start chasing numbers:

• It only checks sandboxing that systemd itself applies. If a daemon drops privileges, chroots or uses seccomp on its own, the score won't show it.
• A high score does not mean a service is vulnerable. It means systemd isn't containing it. Use the score to decide what to work on first, not as a finding.

For scripting, --json=pretty or --json=short returns the same data in a machine-readable form, and --offline=yes analyzes unit files given by path (optionally below --root= or inside an --image=) without loading them into the running system. That is handy for scoring a unit before the package is even installed.

Picking a target

Start with services that listen on the network and parse untrusted input: web servers, reverse proxies, mail daemons and your own apps. On trixie, the nginx unit from the nginx-common package contains no sandboxing directives at all. It's a plain Type=forking unit with PIDFile=/run/nginx.pid and an ExecStartPre that runs nginx -t. Scored offline with systemd 257.13 (the trixie version), the stock unit comes out at 9.4 UNSAFE, which is what you'd expect from a unit with nothing set. Older write-ups quote 9.6; the exact number depends on the systemd version. Record your own baseline before you change anything:

systemd-analyze security --no-pager nginx.service > /root/nginx-security-before.txt
tail -n 1 /root/nginx-security-before.txt

Hardening nginx with a systemd drop-in

Never edit /lib/systemd/system/nginx.service directly, because the next package upgrade overwrites it. Use a drop-in instead. systemctl edit creates the file under /etc/systemd/system/nginx.service.d/ and runs daemon-reload for you when you save. With --drop-in= you can give the file a meaningful name instead of override.conf:

systemctl edit --drop-in=hardening nginx.service

Here is a starting point for a plain nginx serving static files and proxying to local backends:

[Service]
# Filesystem
ProtectSystem=strict
ReadWritePaths=/run /var/log/nginx /var/lib/nginx
ProtectHome=yes
PrivateTmp=yes
PrivateDevices=yes

# Privileges
NoNewPrivileges=yes
CapabilityBoundingSet=CAP_NET_BIND_SERVICE CAP_SETUID CAP_SETGID CAP_SETPCAP CAP_CHOWN CAP_DAC_OVERRIDE
RestrictSUIDSGID=yes

# Kernel and system interfaces
ProtectKernelTunables=yes
ProtectKernelModules=yes
ProtectKernelLogs=yes
ProtectControlGroups=yes
ProtectClock=yes
ProtectHostname=yes

# Network, namespaces, misc
RestrictAddressFamilies=AF_UNIX AF_INET AF_INET6
RestrictNamespaces=yes
RestrictRealtime=yes
LockPersonality=yes
MemoryDenyWriteExecute=yes

# Syscalls
SystemCallArchitectures=native
SystemCallFilter=@system-service
SystemCallFilter=~@mount @reboot @clock
SystemCallErrorNumber=EPERM

Then validate the config, restart, and confirm that nginx actually serves:

systemctl cat nginx.service
systemctl restart nginx.service
systemctl status --no-pager nginx.service
curl -sI http://127.0.0.1/
systemd-analyze security --no-pager nginx.service > /root/nginx-security-after.txt
diff /root/nginx-security-before.txt /root/nginx-security-after.txt

Scored the same way (systemd-analyze security --offline=yes on the trixie unit plus this drop-in, systemd 257.13), the result is 2.7 OK, down from 9.4. The score only measures the sandbox, so still test real traffic afterwards. Your exact number depends on which directives you keep. Some checks, such as User=, PrivateNetwork= and IPAddressDeny=, stay red for nginx because the master process has to start as root and has to talk to the network.

What each directive buys you

Why these particular paths and capabilities

ReadWritePaths= is where most breakage comes from, so each entry here has a reason. /run is needed because the unit writes /run/nginx.pid. You can't allow-list a file that doesn't exist yet, so the whole directory goes in. /var/log/nginx is for the logs. /var/lib/nginx is where Debian's build puts client body and proxy temp files. If you leave it out, nginx starts fine and then fails only when a request body is too big for the memory buffer, which is the worst kind of bug to find in production. If you use proxy_cache_path or fastcgi_cache_path, add those directories too.

On capabilities: the master process runs as root, binds ports 80/443 (CAP_NET_BIND_SERVICE) and switches workers to www-data (CAP_SETUID, CAP_SETGID). CAP_CHOWN and CAP_DAC_OVERRIDE let root create and chown temp directories and open log files owned by www-data. Without CAP_DAC_OVERRIDE, root is just another uid with no special file access. The drop-in also keeps CAP_SETPCAP; the whole set matches the Linux Audit nginx profile. If you use transparent proxying, you'll need CAP_NET_RAW or CAP_NET_ADMIN as well.

For syscalls, systemd's documentation recommends SystemCallFilter=@system-service with SystemCallErrorNumber=EPERM as a reasonable baseline for most services. @system-service is an allow-list. The second ~ line is belt and braces: @mount, @reboot and @clock are not part of @system-service on systemd 257, but the line keeps them blocked if the allow-list ever changes. With the error number set, a blocked call returns EPERM instead of killing the process, which makes debugging much easier. Check what a set actually contains on your systemd version before you trust it:

systemd-analyze syscall-filter @system-service
systemd-analyze syscall-filter @privileged

Don't add ~@privileged to nginx without reading that output first. The master needs to chown and change uid, and @privileged can take those away.

Debugging a service that breaks

Sandboxing failures tend to look like ordinary errors, so you have to know where to look. Work through these in order.

1. Read the unit's journal and the app's own log

journalctl -u nginx.service -b --no-pager -n 50
tail -n 50 /var/log/nginx/error.log

The errno usually tells you which directive is responsible:

• (30: Read-only file system): ProtectSystem=. Add the path to ReadWritePaths=.
• (13: Permission denied) on a path that exists: ProtectHome=, InaccessiblePaths=, or a missing CAP_DAC_OVERRIDE.
• (1: Operation not permitted): a missing capability or, with SystemCallErrorNumber=EPERM, a filtered syscall.
• Address family not supported by protocol: RestrictAddressFamilies=.

2. Check for seccomp kills

Without SystemCallErrorNumber=, a filtered syscall kills the process with SIGSYS. systemctl status then shows the main process exiting with status=31/SYS. The kernel normally logs seccomp kills as audit records that include the syscall number:

journalctl -k -b --no-pager | grep -i 'type=1326'

Map the number to a name with ausyscall from the auditd package, then find the group it belongs to in the systemd-analyze syscall-filter output. To collect the calls a filter would block without taking the service down, temporarily comment out the SystemCallFilter= lines and set SystemCallLog=~@system-service. That logs every syscall outside the allow-list (as the same audit records) instead of blocking it. Note that SystemCallErrorNumber= only takes an errno name or kill; systemd 257 rejects log there with "Failed to parse error number".

3. Bisect the drop-in

If the logs don't point anywhere, comment out half the directives, run systemctl daemon-reload and restart. Repeat until you find the one that breaks it. It's crude, but with around 25 lines it only takes a few rounds.

4. Reproduce outside the unit

systemd-run takes the same properties with -p, so you can get a shell inside a sandbox and poke at it:

systemd-run --pty -p ProtectSystem=strict -p ProtectHome=yes -p PrivateTmp=yes /bin/bash

5. Roll back quickly

systemctl revert nginx.service
systemctl restart nginx.service

systemctl revert deletes the drop-ins in /etc and brings back the vendor unit. Keep a copy of your drop-in somewhere else before you run it. If you want to try something that won't survive a reboot, systemctl edit --runtime writes the drop-in to /run instead.

What not to apply blindly

Hardening snippets get copied between blog posts a lot. These are the settings most likely to cause problems when copied without thought:

• MemoryDenyWriteExecute=yes breaks anything with a JIT: Node.js, Java, PHP with opcache JIT enabled, and PCRE JIT (pcre_jit on; in nginx). Some runtimes fall back to slower code quietly instead of crashing.
• PrivateNetwork=yes and IPAddressDeny=any improve the score a lot and cut a network daemon off completely. They make sense for batch jobs, not for listeners.
• ProtectHome=yes on anything that serves or processes user data under /home. Use read-only, or ReadOnlyPaths=/BindPaths= for the specific directories it needs.
• UMask=0077, which some profiles include, can create logs that the adm group and your log shippers can no longer read.
• DynamicUser=yes on an existing service: files the old static user owned won't match the new UID.
• sshd: it has to create sessions for every user, so ProtectHome=, ProtectSystem=strict or NoNewPrivileges= in its unit will affect every login and can lock you out. Harden sshd through its own configuration, as described in hardening SSH on Debian 13 with OpenSSH 10, and keep a second session open whenever you touch it.
• Firewall managers such as fail2ban need AF_NETLINK and CAP_NET_ADMIN to change nftables sets. If you remove those, bans stop working while the service still reports active; the only trace is the action errors in /var/log/fail2ban.log. If you sandbox it, test that a ban actually shows up in nft list ruleset (setup described in Fail2ban on Debian 13 with nftables).

A lower score is not the goal in itself. A drop-in that scores 1.5 but makes the service fail under real traffic is worse than one that scores 3.0 and holds up. The sandbox also doesn't replace application-level basics like TLS config, request limits and keeping packages updated. Those are covered in web server security basics for self-hosters.

Takeaways

• Run systemd-analyze security and start with network-facing units in the UNSAFE/EXPOSED range.
• Save the baseline output before changing anything and diff it afterwards.
• Put hardening in a named drop-in (systemctl edit --drop-in=hardening) and never edit vendor units.
• With ProtectSystem=strict, list every writable path explicitly: PID file, logs, temp dirs and caches.
• Build CapabilityBoundingSet= from what the daemon actually does, not from a template.
• Use SystemCallFilter=@system-service with SystemCallErrorNumber=EPERM so failures show up as errors instead of crashes.
• Test real traffic, including large uploads and cache misses, not just systemctl status.
• Know your rollback: systemctl revert, or --runtime drop-ins while experimenting.
• Be careful with MemoryDenyWriteExecute, PrivateNetwork, ProtectHome and anything on sshd or firewall daemons.

Sources

• systemd-analyze(1) — Debian trixie manpages
• systemd.exec(5) — Debian trixie manpages
• systemctl(1) — Debian trixie manpages
• systemd v257 NEWS
• systemd v257 source: src/analyze/analyze-security.c
• Debian package: systemd in trixie
• Debian nginx packaging: nginx.service (trixie branch)
• Linux Audit: Nginx hardening profile for systemd
• Linux Audit: Hardening nginx with systemd security features
• DATAZONE: Systemd Security – Hardening and Securing Linux Services