Linux Server / Boot, systemd & Service Recovery
Find failed systemd units without assuming every failure is current impact
Turn systemd failed-unit output into a current incident finding with the exact unit, result code and bounded journal evidence.
Start hands-on practice · เริ่มภาคปฏิบัติ ↓Practical guide · คู่มือลงมือทำ
Download worksheet · ดาวน์โหลดขั้นตอนDetailed practical guide
Prepare your workspace and inputs
Use an authorized host or disposable VM. Replace example.service with the unit reported by list-units. Record the incident start time in UTC before querying logs.
Platform context: Linux with systemd; systemctl and journalctl. Read permission for the affected unit journal may require the host-approved support role.
Scenario to reproduce or investigate
A failed unit remains listed after a one-time job failed, while the user-facing service is healthy. Conversely, a restart loop can produce a fresh failure that needs action. Clear evidence before clearing counters.
- Actual target and platform release
- Exact inputs or substituted values
- Baseline result
- Per-step expected and observed output
- Failure encountered and corrective action
- Final acceptance and recovery state
-
Separate failed state from installed-but-inactive services
List failed units currently known to systemd, then inspect only the relevant unit. Capture the output before using reset-failed or restarting anything. An installed disabled service is not automatically a failed service.
Command, configuration or input
systemctl --failed --no-pager systemctl status example.service --no-pager --fullExpected result · ผลที่ควรเห็นThe record contains the exact unit name and active state, plus any displayed exit or signal result. An empty failed list does not by itself prove the application is healthy.
If it fails · เมื่อไม่ผ่านIf the unit is not found, verify its exact instance name and whether it is a user service. Do not substitute a similarly named production unit.
-
Collect machine-readable result and the incident journal
Read the effective execution result and a bounded journal interval. Use the real incident start instead of the example time. ExecMainStatus alone needs context: an exit code, a termination signal and a start-limit result lead to different next checks.
Command, configuration or input
systemctl show example.service -p ActiveState -p SubState -p Result -p ExecMainCode -p ExecMainStatus -p FragmentPath journalctl -u example.service --since "2026-11-04 09:00:00 UTC" --until "2026-11-04 09:15:00 UTC" --no-pager -o short-isoExpected result · ผลที่ควรเห็นThe unit result and timestamps agree with the journal sequence, including the first failure rather than only the last restart attempt.
If it fails · เมื่อไม่ผ่านIf logs are unavailable, record whether the cause is retention, permission or a different journal namespace. Do not describe missing logs as evidence that no failure occurred.
-
Check service impact before choosing the next diagnostic
Compare the unit result with its supported service health check and recent client symptoms. A failed backup timer child, a critical API service and an intentionally stopped test unit have different impact. Assign the next diagnostic from the first causal log entry, such as missing executable, denied file access or dependency failure; do not restart to discover the cause.
Command, configuration or input
Record: unit | Result | first causal log | current client symptom | next evidence check | ownerExpected result · ผลที่ควรเห็นThe incident finding identifies whether the failure is currently relevant and names the specific evidence needed next.
If it fails · เมื่อไม่ผ่านIf unit state and application behavior disagree, investigate supervisors, sockets, multiple instances and the actual listening process before declaring recovery.
Check the result before marking this passed
The unit identity, exact result, first causal event and current application impact are recorded. No restart or reset is presented as a fix without evidence of the underlying cause.
Record the observed result and evidence location in the downloaded worksheet. Mark unresolved or untested conditions explicitly.
Recovery and cleanup
The procedure is read-only. If a later approved correction changes a unit or drop-in, preserve its previous file and restore it through the host change process if the service health check regresses.
Apply to actual work and known limits
Use the bounded evidence in a support ticket and the specific failure class to choose the corrective runbook. These commands do not test a real fix and do not prove availability of an upstream or downstream service.
Background, explanations and original notes
Problem
A Linux server can boot successfully while a required service or mount failed. Start with the actual failed-unit list.
Start with this check
Confirm the host, shell and access role. Replace sample names, paths, domains and interfaces with approved targets. These are diagnostic commands; assess policy and workload impact before making changes.
systemctl --failed --no-pagerRead the result
On a systemd host, this lists units in failed state. A retained failure can be historical, while a running service can still be unhealthy at the application layer.
Resolve and verify
Match each relevant unit with service ownership and incident time, then inspect its status and journal. Repair the specific dependency and validate the application's real task.
Before changing production
Do not reset all failed states or restart every unit before preserving the error evidence.