File integrity checkers tell you that /etc/sudoers.d/ changed overnight. They can't tell you which login session did it, through which binary, or what else that session ran as root. The Linux audit subsystem records exactly that, and auditd on Debian 13 is the userspace daemon that writes those records to disk. Most guides hand you the full STIG rule set, a few hundred lines that bury the events you care about and fill /var/log/audit with noise.
This post goes the other way. It builds a small rule set for admin and security events and gives a verification command for each rule. It also covers searching the logs with ausearch and aureport, and tuning auditd so it can't fill the disk or stall the box.
What auditd is for (and when to use it)
The kernel audit subsystem hooks syscalls and file accesses that match your rules, then emits records. auditd receives them and writes them to /var/log/audit/audit.log. Each record carries the auid (login UID). The auid is set when you log in and survives sudo and su, so a root action can be traced back to the human who started the session.
Use auditd when you need attribution and a timeline: who edited sshd_config, who loaded a kernel module, what an admin session ran as root. Don't use it as a general process accounting tool for every execve on a busy host. That's where the volume problems start.
Installing auditd on Debian 13
Trixie ships auditd 4.0.2; apt policy auditd shows the exact Debian revision. Remote forwarding lives in the separate audispd-plugins package (same version). The syslog plugin audisp-syslog is part of auditd itself.
sudo apt update
sudo apt install auditd
sudo systemctl status auditd audit-rules
sudo auditctl -s
Debian splits startup into two units. audit-rules.service runs augenrules --load, and auditd.service requires it, so a broken rule file stops auditd from starting. The package installs /etc/audit/rules.d/audit.rules, which holds only control settings: -D, -b 8192, --backlog_wait_time 60000 and -f 1. There are no watch rules yet.
Processes that start before auditd aren't auditable unless the kernel boots with audit=1. With audit=1, the kernel buffers at most audit_backlog_limit records (kernel default 64) until auditd takes over, so raise that limit too:
GRUB_CMDLINE_LINUX="audit=1 audit_backlog_limit=8192"
Warning: a broken kernel command line can leave the machine unbootable. Keep console access before you reboot.
sudo update-grub
sudo reboot
A small, high-signal auditd rule set
auditctl(8) marks the old -w path watch syntax as deprecated for performance reasons. The rules below use the syscall form with -F path= (one file) or -F dir= (a recursive subtree) and -F perm=wa (writes and attribute changes). The examples assume amd64. Every rule appears twice, once per ABI. A rule with only arch=b64 never matches a syscall made through the 32-bit interface, which an amd64 kernel with IA32 emulation accepts even from a 64-bit process. Without the b32 twin, such a write to /etc/sudoers.d/ leaves no record.
## Accounts and privilege
-a always,exit -F arch=b64 -F path=/etc/passwd -F perm=wa -k identity
-a always,exit -F arch=b32 -F path=/etc/passwd -F perm=wa -k identity
-a always,exit -F arch=b64 -F path=/etc/shadow -F perm=wa -k identity
-a always,exit -F arch=b32 -F path=/etc/shadow -F perm=wa -k identity
-a always,exit -F arch=b64 -F path=/etc/group -F perm=wa -k identity
-a always,exit -F arch=b32 -F path=/etc/group -F perm=wa -k identity
-a always,exit -F arch=b64 -F path=/etc/gshadow -F perm=wa -k identity
-a always,exit -F arch=b32 -F path=/etc/gshadow -F perm=wa -k identity
-a always,exit -F arch=b64 -F path=/etc/sudoers -F perm=wa -k sudoers
-a always,exit -F arch=b32 -F path=/etc/sudoers -F perm=wa -k sudoers
-a always,exit -F arch=b64 -F dir=/etc/sudoers.d/ -F perm=wa -k sudoers
-a always,exit -F arch=b32 -F dir=/etc/sudoers.d/ -F perm=wa -k sudoers
## SSH daemon configuration
-a always,exit -F arch=b64 -F path=/etc/ssh/sshd_config -F perm=wa -k sshd
-a always,exit -F arch=b32 -F path=/etc/ssh/sshd_config -F perm=wa -k sshd
-a always,exit -F arch=b64 -F dir=/etc/ssh/sshd_config.d/ -F perm=wa -k sshd
-a always,exit -F arch=b32 -F dir=/etc/ssh/sshd_config.d/ -F perm=wa -k sshd
## Scheduled jobs (drop lines for paths you don't have)
-a always,exit -F arch=b64 -F path=/etc/crontab -F perm=wa -k cron
-a always,exit -F arch=b32 -F path=/etc/crontab -F perm=wa -k cron
-a always,exit -F arch=b64 -F dir=/etc/cron.d/ -F perm=wa -k cron
-a always,exit -F arch=b32 -F dir=/etc/cron.d/ -F perm=wa -k cron
-a always,exit -F arch=b64 -F dir=/etc/cron.daily/ -F perm=wa -k cron
-a always,exit -F arch=b32 -F dir=/etc/cron.daily/ -F perm=wa -k cron
-a always,exit -F arch=b64 -F dir=/var/spool/cron/crontabs/ -F perm=wa -k cron
-a always,exit -F arch=b32 -F dir=/var/spool/cron/crontabs/ -F perm=wa -k cron
## systemd units and timers
-a always,exit -F arch=b64 -F dir=/etc/systemd/system/ -F perm=wa -k systemd
-a always,exit -F arch=b32 -F dir=/etc/systemd/system/ -F perm=wa -k systemd
-a always,exit -F arch=b64 -F dir=/usr/lib/systemd/system/ -F perm=wa -k systemd
-a always,exit -F arch=b32 -F dir=/usr/lib/systemd/system/ -F perm=wa -k systemd
## The audit configuration itself
-a always,exit -F arch=b64 -F dir=/etc/audit/ -F perm=wa -k audit-config
-a always,exit -F arch=b32 -F dir=/etc/audit/ -F perm=wa -k audit-config
## Kernel module load/unload
-a always,exit -F arch=b64 -S init_module,finit_module,delete_module -k modules
-a always,exit -F arch=b32 -S init_module,finit_module,delete_module -k modules
## Commands run as root by a human login session
-a always,exit -F arch=b64 -S execve,execveat -F euid=0 -F auid>=1000 -F auid!=unset -k root-exec
-a always,exit -F arch=b32 -S execve,execveat -F euid=0 -F auid>=1000 -F auid!=unset -k root-exec
The root-exec filter keeps volume down. auid>=1000 with auid!=unset matches only sessions started by a real user (Debian's first regular UID is 1000). Root commands from systemd services have an unset auid and are skipped. Root's cron jobs are skipped too, but for a different reason: pam_loginuid in /etc/pam.d/cron gives them auid=root (0), which is below 1000. audit.rules(7) documents unset, -1 and 4294967295 as equivalent. Add the other cron.* directories the same way if you use them. The /usr/lib/systemd/system/ rule fires on every package upgrade that ships unit files. An upgrade you run through sudo carries your auid. One started by a systemd timer, such as unattended-upgrades, has an unset auid and dpkg as the executable.
Build, load and check:
sudo augenrules --check
sudo augenrules --load
sudo auditctl -l
sudo auditctl -s
augenrules merges every *.rules file in /etc/audit/rules.d/ in natural sort order. It places the last -D, -b and -f first and the last -e at the very end. Read the --load output for errors. A path that doesn't exist on this host is the usual cause.
Verify each rule
Run these from your normal account through sudo so the auid is set. Each pair triggers a rule and then searches for it. The temporary files are empty, have no .conf suffix where an include glob would pick them up, and are removed right away.
sudo touch /etc/passwd
sudo ausearch -k identity -ts recent -i
sudo touch /etc/sudoers.d/zz-audit-test
sudo rm /etc/sudoers.d/zz-audit-test
sudo ausearch -k sudoers -ts recent -i
sudo touch /etc/ssh/sshd_config.d/audit-test
sudo rm /etc/ssh/sshd_config.d/audit-test
sudo ausearch -k sshd -ts recent -i
sudo touch /etc/cron.d/audit-test
sudo rm /etc/cron.d/audit-test
sudo ausearch -k cron -ts recent -i
sudo touch /etc/systemd/system/audit-test.service
sudo rm /etc/systemd/system/audit-test.service
sudo ausearch -k systemd -ts recent -i
sudo touch /etc/audit/auditd.conf
sudo ausearch -k audit-config -ts recent -i
sudo modprobe dummy
sudo modprobe -r dummy
sudo ausearch -k modules -ts recent -i
sudo id
sudo ausearch -k root-exec -ts recent -i
The dummy module creates a throwaway dummy0 interface, which disappears when you unload it. A hit looks like this (trimmed):
----
type=PROCTITLE msg=audit(10/06/2026 09:14:02.381:1873) : proctitle=touch /etc/passwd
type=PATH msg=audit(10/06/2026 09:14:02.381:1873) : item=0 name=/etc/passwd nametype=NORMAL ...
type=SYSCALL msg=audit(10/06/2026 09:14:02.381:1873) : arch=x86_64 syscall=openat success=yes exit=3 ... auid=admin uid=root euid=root ... ses=4 comm=touch exe=/usr/bin/touch key=identity
Searching with ausearch and aureport
-i turns numeric UIDs and syscalls into names. -ts accepts keywords such as recent, today, yesterday, this-week and boot. The Debian auditd.conf sets log_group = adm, so members of adm can read the logs without root.
sudo aureport -k --summary -ts today
sudo aureport -x --summary -ts today
sudo aureport -m -i
sudo aureport -au --failed -ts this-week
sudo aureport -n
sudo ausearch -k root-exec -ts today --format csv
Key Summary Report
===========================
total key
===========================
214 root-exec
9 systemd
3 identity
1 sudoers
aureport -x --summary lists executables by event count, which is where you find what makes your execve rule noisy. -m reports account modifications, -au authentication attempts and -n anomalies such as segfaults and promiscuous NICs. For scripted polling, ausearch --checkpoint FILE resumes where the last run stopped.
Triage flow with AIDE
If you already run AIDE for file integrity monitoring on Debian 13, use the two tools together. AIDE tells you what changed, and auditd tells you who changed it.
- The AIDE report flags, for example,
/etc/sudoers.d/backupas added. - Find the event:
sudo ausearch -f /etc/sudoers.d/backup -i. Note theauid,exeandsesfields. - Replay the whole session:
sudo ausearch --session 4 -k root-exec -ishows every root command that login ran. - Check the login itself:
sudo aureport -l -i -ts yesterdayandaulastshow where the session came from. - If the auid is unset, the change came from a service or a process that started before auditd. A cron job shows the job owner instead, so
auid=rootwithout a matching root login points at root's crontab or/etc/cron.d/. Check theexeand the units under thesystemdandcronkeys.
Options that matter: backlog, disk and immutability
Kernel backlog
The shipped -b 8192 is the queue of records waiting for auditd. When the queue is full, the kernel holds event-generating tasks for up to --backlog_wait_time before it drops records. A flood of events can therefore slow the whole machine. -f 1 (printk) only logs the failure. -f 2 panics the kernel, so don't use it on a server unless losing an event really is worse than losing the host. Watch the counters:
sudo auditctl -s | grep -E 'backlog|lost'
If lost keeps growing, fix the rule that floods the queue before you raise -b.
Log rotation and disk-full actions
Let auditd rotate its own log; don't add a logrotate rule for audit.log. The Debian defaults in /etc/audit/auditd.conf:
| Setting | Debian 13 default | Effect |
|---|---|---|
max_log_file | 8 (MB) | Size per log file |
num_logs | 5 | Files kept when rotating |
max_log_file_action | ROTATE | Rotate at size limit |
space_left / _action | 75 MB / SYSLOG | Warn in syslog |
admin_space_left / _action | 50 MB / SUSPEND | Stop writing audit records |
disk_full_action | SUSPEND | Stop writing audit records |
disk_error_action | SUSPEND | Stop writing audit records |
40 MB of history is short when root-exec is active. I use:
max_log_file = 50
num_logs = 10
max_log_file_action = ROTATE
space_left = 10%
space_left_action = SYSLOG
admin_space_left = 5%
admin_space_left_action = SUSPEND
disk_full_action = SUSPEND
The percentage form is supported for space_left and admin_space_left. Rotation caps auditd at about 500 MB, but other writers on the same filesystem can still fill it. A separate /var/log/audit filesystem avoids that (see No Space Left on Device but df Shows Free Space for the general diagnosis). Warning: SINGLE and HALT are valid actions and will drop the server to single-user mode or power it off when the disk fills. Choose them only if compliance requires it. Apply config changes with a SIGHUP; SIGUSR1 forces a rotation. --kill-whom=main signals only auditd itself. The default signals its plugins as well, and audisp-syslog exits on SIGUSR1:
sudo systemctl kill --kill-whom=main --signal=SIGHUP auditd
sudo systemctl kill --kill-whom=main --signal=SIGUSR1 auditd
Immutable mode
-e 2 locks the audit configuration until reboot, so an attacker with root can't quietly delete your rules. You can't change them either.
-e 2
Warning: add this only after the rule set has run cleanly for a while. Once it's loaded, augenrules --load and auditctl changes are refused, and fixing a noisy rule means editing the files and rebooting. auditctl -s shows enabled 2 when the lock is active.
Forwarding events off-host
Logs that stay on a compromised host can be edited by whoever compromised it. There are three options:
- Syslog plugin: in
/etc/audit/plugins.d/syslog.confsetactive = yesandargs = LOG_INFO LOG_LOCAL6, then shiplocal6with your existing rsyslog/journald forwarder. The man page warns that the optionalinterpretargument can mislead naive parsers, because attackers control file and process names. - audisp-remote: install
audispd-plugins, setremote_serverin/etc/audit/audisp-remote.confandactive = yesinplugins.d/au-remote.conf. The default TCP transport is cleartext; useKRB5or a VPN. Usemode = forwardto queue events to disk while the collector is down. Theauditd.serviceunit comments explain how to overrideAfter=/Before=so auditd waits fornetwork-online.target. - journald:
systemd-journald-audit.socketlets journald collect kernel audit records as well. Check it withsystemctl is-enabled systemd-journald-audit.socket. If you'd rather not store everything twice, disable that socket.Audit=injournald.confonly controls whether journald switches kernel auditing on, not whether it collects the records.
Pitfalls, limitations and alternatives
- Noisy execve: an unfiltered
-S execverule logs every fork-exec of every service. Keep the auid filter, and checkaureport -x --summaryweekly. - Locking yourself out of changes: with
-e 2, a bad rule stays until reboot. Test first, lock last. - Silent gaps:
SUSPENDkeeps the box running but stops recording. Alert on thespace_leftsyslog message and on a growinglostcounter. - Path rules see paths, not intent: they record the write but not the content. Use AIDE or config management for diffs.
- Kernel or root compromise: a rootkit can blind auditd. Off-host forwarding protects the records written before that point.
| Tool | Strength | Use it when |
|---|---|---|
| auditd | Syscall-level attribution (auid, session) | You need who/what/when on a host |
| AIDE | Periodic hash comparison of files | You need to know what changed, including offline edits |
| osquery | SQL over OS state; process_events via kernel audit | Fleet queries. Its docs say auditd must not run alongside its process auditing. |
| Falco | eBPF/kernel-module syscall stream, YAML rules, real-time alerts | Containers and Kubernetes runtime detection |
auditd cheat sheet
| Task | Command |
|---|---|
| Rebuild and load rules | sudo augenrules --load |
| Check if rules changed | sudo augenrules --check |
| List loaded rules | sudo auditctl -l |
| Status, backlog, lost | sudo auditctl -s |
| Search by key | sudo ausearch -k sshd -ts today -i |
| Search by file | sudo ausearch -f /etc/shadow -i |
| Whole login session | sudo ausearch --session 4 -i |
| Events per key | sudo aureport -k --summary |
| Noisiest executables | sudo aureport -x --summary |
| Failed auth | sudo aureport -au --failed |
| Reload auditd.conf | sudo systemctl kill --kill-whom=main --signal=SIGHUP auditd |
| Rotate now | sudo systemctl kill --kill-whom=main --signal=SIGUSR1 auditd |
Takeaways
- Install
auditd(4.0.2 on trixie) and boot withaudit=1 audit_backlog_limit=8192. - Keep rules to identity, sudoers, sshd, cron, systemd units, the audit config, module loads and root commands from human sessions.
- Verify every rule with a trigger plus
ausearch -kbefore trusting it. - Raise
max_log_file/num_logs, keepSUSPENDunless you needHALT, and alert onlost. - Forward events off-host. Pair auditd with AIDE and with a hardened SSH setup, since SSH logins are what set the auid.
- Add
-e 2last, and expect to reboot for every later rule change.
Sources
- Debian packages: auditd in trixie
- Debian packages: audispd-plugins in trixie
- auditd.conf(5) – Debian trixie manpages
- auditctl(8) – Debian trixie manpages
- augenrules(8) – Debian trixie manpages
- audit.rules(7) – Debian trixie manpages
- ausearch(8) – Debian trixie manpages
- aureport(8) – Debian trixie manpages
- auditd(8) – Debian trixie manpages
- audisp-syslog(8) – Debian trixie manpages
- audisp-remote.conf(5) – Debian trixie manpages
- Debian audit 1:4.0.2-2+deb13u1 source: init.d/auditd.conf
- Debian audit 1:4.0.2-2+deb13u1 source: rules/10-base-config.rules
- Debian audit 1:4.0.2-2+deb13u1 source: debian/auditd.install
- Debian audit 1:4.0.2-2+deb13u1 source: debian/audispd-plugins.install
- Debian audit 1:4.0.2-2+deb13u1 source: syslog.conf plugin
- Debian audit 1:4.0.2-2+deb13u1 source: auditd.service.in
- Debian audit 1:4.0.2-2+deb13u1 source: audit-rules.service
- journald.conf(5) – Debian trixie manpages
- systemd-journald-audit.socket – Debian trixie manpages
- The kernel's command-line parameters
- osquery: Process auditing
- osquery documentation
- Falco documentation
Comments