A Guide to systemd journal Maintenance [With Examples]
Systemd comes with many built-in features to manage the system logs. In this guide, we explain how you can manage system journals, logs and take action on them such as rotating, archiving, and clear logs. We also explain the manual systems journal clean method and using config file changes.
If your Linux distribution supports systemd, then it collects logs from all processes, applications of the system every second which starts from the boot. All these logging events are managed by
journald daemon of systemd. The journald collects all the logs (info, warnings, errors, etc) and stores them as binary data in the disk files.
As the logs remain in the disk and every second it is collected, it takes up huge disk space; especially for older systems, servers. For example, in one of my test systems which are running for around one year, the log file size is in GBs.
If you manage multiple systems, servers, it is always recommended to properly manage journald logs for efficient operation. Let’s take a look at how you can manage the log files.
The systemd journal Maintenance
Using the journalctl utility of systemd, you can query these logs, perform various operations on them. For example, viewing the log files from different boots, check for last warnings, errors from a specific process or applications. If you are unaware of these, I would suggest you quickly go through this tutorial – “use journalctl to View and Analyze Systemd Logs [With Examples]” before you follow this guide.
Where are the physical journal log files?
The systemd’s journald daemon collects logs from every boot. That means, it classifies the log files as per the boot.
The logs are stored as binary in the path
/var/log/journal with a folder as machine id.
Also, remember that based on system configuration, runtime journal files are stored at
/run/log/journal/. And these are removed in each boot.
Can I manually delete the log files?
You can, but don’t do it. Instead, follow the below instructions to clear the log files to free up disk space using journalctl utilities.
How much disk space is used by systemd log files?
Open up a terminal and run the below command.
This should provide you how much is actually used by the log files in your system.
If you have a graphical desktop environment, you can open the file manager and browse to the path
/var/log/journal and check the properties.
systemd journal clean process
The effective way of clearing the log files should be done by
journald.conf configuration file. Ideally, you should not manually delete the log files even if the journalctl provides utility to do that.
Let’s take a look at how you can delete it manually, then I will explain the configuration changes in
journald.conf so that you do not need to manually delete the files from time to time; Instead, the systemd takes care of it automatically based on your configuration.
First, you have to
rotate the log files. Rotating is a way of marking the current active log files as an archive and create a fresh logfile from this moment. The flush switch asks the journal daemon to flush any log data stored in
/var/log/journal/, if persistent storage is enabled.
Then, after flush and rotate, you need to run journalctl with
vacuum-files switches to force systemd to clear the logs.
sudo journalctl --flush --rotate
sudo journalctl --vacuum-time=1s
The above set of commands removes all archived journal log files until the last second. This effectively clears everything. So, careful while running the command.
ou can also provide the following suffixes as per your need following the number.
- s: seconds
- m: minutes
- h: hours
sudo journalctl --flush --rotate
sudo journalctl --vacuum-size=400M
This clears all archived journal log files and retains the last 400MB files. Remember this switch applies to only archived log files only, not on active journal files. You can also use suffixes as below.
- K: KB
- M: MB
- G: GB
sudo journalctl --flush --rotate
sudo journalctl --vacuum-files=2
The vacuum-files switch clears all the journal files below the number specified. So, in the above example, only the last 2 journal files are kept and everything else is removed. Again, this only works on the archived files.
You can combine the switches if you want, but I would recommend not to. However, make sure to run with
--rotate switch first.
Automatic delete using config files
While the above methods are good and easy to use, but it is recommended that you control the journal log file cleanup process using the journald configuration files which present at
The systemd provides many parameters for you to effectively manage the log files. By combining these parameters you can effectively limit the disk space used by the journal files. Let’s take a look.
|SystemMaxUse||Specifies the maximum disk space that can be used by the journal in persistent storage||SystemMaxUse=500M|
|SystemKeepFree||Specifies the amount of space that the journal should leave free when adding journal entries to persistent storage.||SystemKeepFree=100M|
|SystemMaxFileSize||Controls how large individual journal files can grow to in persistent storage before being rotated.||SystemMaxFileSize=100M|
|RuntimeMaxUse||Specifies the maximum disk space that can be used in volatile storage (within the /run filesystem).||RuntimeMaxUse=100M|
|RuntimeKeepFree||Specifies the amount of space to be set aside for other uses when writing data to volatile storage (within the /run filesystem).||RuntimeMaxUse=100M|
|RuntimeMaxFileSize||Specifies the amount of space that an individual journal file can take up in volatile storage (within the /run filesystem) before being rotated.||RuntimeMaxFileSize=200M|
If you add these values in a running system in
/etc/systemd/journald.conf file, then you have to restart the journald after updating the file. To restart use the following command.
sudo systemctl restart systemd-journald
Verification of log files
It is wiser to check the integrity of the log files after you clean up the files. To do that run the below command. The command shows the PASS, FAIL against the journal file.
I hope this guide helps you to understand the basics of the systemd journal management process. With these, you can manage the disk space used by the log files in your system or servers by limiting the space, clearing old log files. These are just the guideline commands, you can combine these in multiple ways to achieve your system demands.