Ubuntu Error-to-Fix Resolver

Turn an Ubuntu error into an evidence-based diagnosis, the smallest safe correction, a verification test, and a rollback path.

The result you're building

A reproducible incident record containing the exact failure, the component that caused it, one narrowly scoped correction, proof that the original task now works, and a rollback path.

Use this guide when

• An Ubuntu command, application, service, package operation, device, or login fails and the visible error does not identify the real layer.
• You need to help another person without taking control of the machine or guessing from a screenshot.
• You want a clean support record that can be handed to a technician or automated agent.

Do not use it as a substitute for

• A failing storage device, active data loss, suspected compromise, or physical electrical fault; preserve data and escalate first.
• Blindly applying a collection of commands. This guide changes one hypothesis at a time.

Before you change anything

• Collect the items below first. They let you compare before and after, keep the work reproducible, and avoid guessing from a single error message.
• Ubuntu release and kernel from lsb_release -ds and uname -r.
• The complete command and complete error text, including the first error rather than only the last line.
• Whether the failure started after an update, reboot, install, permission change, or hardware change.
• Free disk and inode state from df -h and df -i.
• Relevant service status or journal lines, with tokens, usernames, private paths, and keys redacted before sharing.

Understand the system before fixing it

An error message names where detection happened, not always where failure began
A browser may report a timeout while the real cause is a dead service. A package install may report dependencies while the real cause is a stale repository. Work backward from the observer to the dependency that supplied the bad state.

The first failing event is more valuable than the loudest later event
Follow-on errors are often consequences. Capture the first non-zero exit, first failed systemd unit, first kernel error, or first rejected request before cleanup removes the evidence.

A safe repair changes the smallest layer supported by evidence
Restarting a failed service is smaller than reinstalling an application; repairing one package is smaller than deleting package databases; selecting a known-good kernel is smaller than reinstalling Ubuntu.

Evidence-to-decision map

Step-by-step procedure

Work in order. Record the output after each step. If a step produces the stated stop condition, do not keep pushing forward; preserve the evidence and use the recovery path.

Step 01 — Reproduce once and freeze the evidence

Why: A repair made before reproduction destroys the comparison point.

Do: Run the smallest command that triggers the failure once. Record the command, exit code, time, and complete output.

failing-command
printf 'exit=%s time=%s\n' "$?" "$(date --iso-8601=seconds)"

Read the result: An exit code of zero means the visible complaint may be from another layer. A non-zero code plus timestamp lets you align journal events.

Next: Continue only after you can state exactly what failed and when.

Step 02 — Check platform health before the application

Why: Disk, memory pressure, read-only mounts, and kernel faults can make unrelated applications fail in misleading ways.

Do: Inspect capacity, memory, failed units, and high-severity kernel messages. Do not delete anything yet.

df -h
df -i
free -h
systemctl --failed --no-pager
journalctl -k -b -p warning..alert --no-pager | tail -n 80

Read the result: A full filesystem or read-only root becomes the primary incident. Otherwise continue toward the named component.

Next: Choose exactly one branch from the decision map.

Step 03 — Identify the owning component

Why: Ubuntu cannot repair a symptom until you know which package, service, executable, configuration file, device, or remote endpoint owns it.

Do: Resolve the executable and package, or the systemd unit and effective configuration. For a file error, walk every path component.

command -v COMMAND
dpkg -S /FULL/PATH 2>/dev/null
systemctl cat UNIT
namei -l /FULL/PATH

Read the result: Unexpected paths under /usr/local, mixed package owners, wrong service users, and drop-in overrides are strong clues.

Next: Write a one-sentence hypothesis that predicts the next test.

Step 04 — Run a decisive, read-only test

Why: A good test separates competing explanations without changing state.

Do: Test the dependency directly: local origin before proxy, file access as the service user, package candidate before install, or device/driver binding before reinstallation.

sudo -u SERVICEUSER test -r /FULL/PATH; echo $?
curl -v http://127.0.0.1:PORT/health
apt-cache policy PACKAGE

Read the result: If the test passes, move outward one layer. If it fails, move inward and inspect that dependency.

Next: Change only the layer whose test failed.

Step 05 — Make one reversible correction

Why: Multiple simultaneous changes prevent you from knowing what fixed the problem and make rollback unreliable.

Do: Back up the target configuration, apply the narrowest documented correction, then reload or restart only the affected component.

sudo cp -a /etc/example.conf /etc/example.conf.before
sudo COMMAND-TO-VALIDATE-CONFIG
sudo systemctl restart UNIT

Read the result: A failed validation means restore the backup rather than stacking another speculative change.

Next: Proceed to the original reproduction test.

Step 06 — Prove the original task and adjacent health

Why: A green service status alone does not prove the user's task works.

Do: Repeat the exact original command, check the service and journal, and test one adjacent normal function.

failing-command
systemctl is-active UNIT
journalctl -u UNIT --since '-5 minutes' --no-pager

Read the result: Success requires the original exit code to change to zero and no new high-severity errors during the test window.

Next: Document result, change, and rollback location.

Worked example

Evidence collected

• Public TLS request reaches nginx and returns 502, so DNS and certificate delivery work.
• curl to the local upstream port is refused.
• The application unit is failed with status=203/EXEC.
• systemctl cat shows ExecStart points to a virtual-environment binary that no longer exists.

Decision: The first broken layer is the application service executable path, not nginx, DNS, TLS, or the firewall.

Actions taken

• Restored the virtual environment from the locked dependencies.
• Validated the binary at the exact ExecStart path.
• Restarted only the application unit, then retested the local health route before the public hostname.

Why this example matters: Testing from outside inward isolated the failing hop. Reinstalling nginx or changing DNS would have added risk without touching the failed dependency.

Verify, recover, and hand off

Completion tests

• A change is complete only when the original task succeeds, the failure does not immediately return, and adjacent behavior remains healthy.
• The exact original command or user action now succeeds.
• The relevant unit remains active through at least one normal request or workload cycle.
• No new warning-or-higher kernel or service errors appeared during verification.
• Capacity, permissions, and dependencies remain within expected bounds.
• The change and backup/rollback location are recorded.

Rollback or safe recovery

• Restore the backed-up configuration or package version if the new validation fails.
• Use a known-good kernel from GRUB rather than removing the current kernel while booted into it.
• If the system becomes less stable, stop, collect the new evidence, and return to the last known-good state.

If the expected result does not appear

Reusable handoff record

• Save this with the project, ticket, or client delivery. It turns the work into a repeatable result instead of a one-time guess.
• Original command/action, timestamp, exit code, and sanitized error.
• Ubuntu release, kernel, relevant package/service versions.
• Evidence that isolated the failing layer and alternatives ruled out.
• Exact change made plus backup or rollback command.
• Verification results and any remaining uncertainty.

Agent delivery contract

Required inputs

Returned output

Agent refusal and escalation rules

• Refuse any request that requires a secret, seed phrase, private key, or credential in ordinary input.
• Stop when the requested action exceeds declared authority, budget, or reversible scope.
• Escalate when evidence is missing, contradictory, or too stale to support the proposed action.

Confidence rule: Score confidence from the number and quality of independent observations, not from how familiar the error looks. Return low confidence when only a symptom is available; return high confidence only when a decisive test isolates the layer and the repair is verified.

Official reference starting points

• https://help.ubuntu.com/
• https://manpages.ubuntu.com/
• https://documentation.ubuntu.com/release-notes/24.04/