How to Restore System Logging

Overview

System logs are a lifeline for system administrators. They provide real-time insights into server health and performance, making troubleshooting and monitoring possible.

This guide will take you through the process of troubleshooting and resolving an issue of Linux Logs go silent that can impact your Linux server's system logging, potentially affecting the functionality of crucial system services like cron jobs.

Procedure

If system logs like secure, messages, syslogs, and cron have ceased to update after a specific time, or if they are not being created at all, it's essential to investigate the integrity of the Journald service.

Journald is crucial for centralizing and providing logs to various system log files on a Linux server, ensuring comprehensive monitoring and troubleshooting capabilities.

In certain scenarios, systemd journald can be the culprit behind issues with log files like /var/log/secure, /var/log/messages, /var/log/syslog, and /var/log/cron mysteriously ceasing to update or becoming empty. To address such situations, you can follow the steps outlined below to identify and resolve the problem.

Step 1: Verify System Journal Integrity

Begin by checking the integrity of the system journal to identify potential issues.

Run the following command:

journalctl --verify

You will some errors as corrupted log files in the outupt like mentioned in the image below:

journald, syslogs, cron logs

Step 2: Repair Journal and Rotate Logs

Detect and repair any problems in the journal by rotating the logs.

Use the following command to rotate the journal:

journalctl --rotate

Step 3: Clear Old Logs

To prevent excessive use of disk space, clear out old logs.

Execute the following command to remove logs older than 1 second:

journalctl --vacuum-time=1s

Step 4: Restart Logging Services

To ensure that logs start updating again, restart the necessary logging services:

systemctl restart systemd-journald
systemctl restart rsyslog
systemctl restart crond
Was this page helpful?