systemd-journald is the kernel-integrated logging daemon that collects structured binary log entries from every process, the kernel, syslog socket, native API callers, and the audit subsystem into a single, indexed, compressed journal. journalctl is the primary query tool: it supports powerful filtering by time, unit, user, PID, priority, transport, and arbitrary journal fields, and outputs results in human-readable or machine-parseable formats. This cheat sheet covers architecture, configuration, all major journalctl options, field schema, remote forwarding, namespaces, the sd_journal API, modern integrations (OpenTelemetry, Vector, Loki, Fluent Bit), and operational best practices.
What This Cheat Sheet Covers
This topic spans 21 focused tables and 188 indexed concepts, 105 flashcards. Below is a complete table-by-table outline of this topic, spanning foundational concepts through advanced details.
A jump-to index of every table row in this cheat sheet.
An interactive map of every table and concept in this topic.
Table 1: Architecture and Data Flow
Core components of the systemd journal ecosystem — the daemon, its storage locations, and the sources that feed it.
| Architecture | Example | Description | |
|---|---|---|---|
systemctl status systemd-journald | • Kernel service collecting logs from kernel, user processes, syslog socket, native protocol, audit subsystem • Stores data in binary .journal files; supports indexed, compressed, optionally sealed storage | ||
/var/log/journal/<machine-id>/ | • Created automatically when directory exists; survives reboots • Contains system.journal, system@*.journal, and per-user user-UID.journal files | ||
/run/log/journal/<machine-id>/ | • Stored in tmpfs; lost on reboot • Default when /var/log/journal/ does not exist | ||
file -b /var/log/journal/*/system.journal | • Structured binary format with hash tables, object arrays, field index • Not human-readable directly — always use journalctl to query | ||
journalctl _TRANSPORT=kernel | • kernel (kmsg), syslog (/run/systemd/journal/syslog), journal (native AF_UNIX socket), stdout (service stdout/stderr), audit (Linux audit subsystem), driver (journald internal messages) | ||
StandardOutput=journal | • Services with StandardOutput=journal or StandardOutput=inherit send their stdout to journald automatically• Each line becomes a structured log entry with _SYSTEMD_UNIT= set | ||
SplitMode=uid | • Default uid: separate journal files per user UID for non-root users• none: single combined system journal for all users | ||
cat /etc/machine-id | • 128-bit unique host identifier stored in /etc/machine-id• Used as directory name under /var/log/journal/ and in _MACHINE_ID= journal field |