SAYLOR Find the exact user, group, path component, ACL, mount option, or service confinement rule denying access, then apply the narrowest correction.
The result you're building
The intended user or service can perform exactly the required operation on the required path, while unrelated users remain blocked, with ownership/mode/ACL/mount/confinement evidence and a narrow reversible change.
Use this guide when
- A file/socket/device/path works as root but not as the real user/service.
- Ownership looks correct but access is still denied.
Do not use it as a substitute for
- Do not use recursive
chmod 777orchownon broad directories. - Do not disable AppArmor/SELinux/systemd sandboxing globally to test one resource.
Before you change anything
- Collect the items below first. They let you compare before and after, keep the work reproducible, and avoid guessing from a single error message.
- Exact operation, full path, expected identity and service unit.
namei -l,stat, ACLs, groups, mount options, and symlink targets.- Service sandbox/confinement logs and process UID/GID.
- Before-state backup/listing for any planned recursive change.
Understand the system before fixing it
Directory execute means traverse
A readable file is inaccessible if any parent directory denies traversal.
The process identity matters
Service user, supplementary groups, root-squash, containers, and namespaces can differ from the login user.
Mode bits are only one layer
ACLs, mount flags, immutable attributes, AppArmor/SELinux, systemd sandboxing, and filesystem ownership mapping can override expectations.
Evidence-to-decision map
| Evidence | Likely layer | First decisive check | What the result means |
|---|---|---|---|
| Denied at parent component | Unix path mode | namei -l PATH | Fix the first component for the intended identity. |
| Mode permits but ACL mask denies | ACL | getfacl -p PATH | Effective ACL/mask needs narrow correction. |
| Works outside service | Service sandbox | systemctl show UNIT and journal denials | ProtectSystem/ReadWritePaths/PrivateDevices or confinement blocks it. |
| Operation not permitted as root | Mount/attribute /fs | findmnt, lsattr, kernel log | Read-only/noexec/nodev, immutable flag, or filesystem error. |
Step-by-step procedure
Work in order. Record the output after each step. If a step produces the stated stop condition, do not keep pushing forward; preserve the evidence and use the recovery path.
Step 01 — Reproduce as exact identity
Why: sudo success proves little.
Do: Identify process UID/GID/groups and run a read-only test for the exact operation as that user.
id SERVICEUSER
sudo -u SERVICEUSER test -r /PATH; echo $?
sudo -u SERVICEUSER test -w /PATH; echo $?Read the result: Exit status isolates identity difference.
Next: Do not start duplicate daemon.
Step 02 — Walk path and target
Why: Parent traversal and symlinks hide the denied object.
Do: Use namei, readlink, and stat on full resolved path.
namei -l /FULL/PATH
readlink -f /FULL/PATH
stat /FULL/PATHRead the result: First inaccessible component is the Unix-mode branch.
Next: Change one component only.
Step 03 — Inspect ACL, mount, and attributes
Why: Mode bits may appear correct while another layer denies.
Do: Read ACL/mask, mount options/filesystem, immutable attributes, and kernel errors.
getfacl -p /FULL/PATH
findmnt -T /FULL/PATH
lsattr -d /FULL/PATHRead the result: Match denial to exact operation: read, write, execute, create, rename, device.
Next: Correct narrow ACL/mount only with intent.
Step 04 — Inspect service confinement
Why: systemd/AppArmor rules apply only inside service context.
Do: Review effective unit hardening and denial logs; test a narrow ReadWritePaths/device/network rule if justified.
Read the result: Do not disable all protection.
Next: Validate unit and restart only service.
Step 05 — Apply least-privilege fix
Why: Broad permission grants become security debt.
Do: Prefer dedicated group, group ownership, setgid directory, or precise ACL over world-write/root. Use -- and exact paths.
Read the result: Target identity succeeds; unrelated test identity remains denied.
Next: Record before/after modes and ACL.
Step 06 — Test creation lifecycle
Why: New files may inherit different owner/mode.
Do: Create, read, write, rename, and delete a disposable file as service; restart service; check umask/default ACL.
Read the result: Future files must preserve access contract.
Next: Document group membership may require new login/restart.
Worked example
/srv/backups/app but root can.Evidence collected
- Service runs as
backupsvc. - Final directory group is
backup, but parent/srv/backupslacks execute for that group. - ACL and mount are normal.
- Only this subtree should be writable.
Decision: Parent traversal is denied; recursive chmod/chown on /srv would be excessive.
Actions taken
- Granted the backup group traverse on the single parent and set correct group/default ACL on app subtree.
- Restarted service to pick up group membership.
- Tested create/write/rename/delete and unrelated user denial.
Why this example matters: Walking the path found the first denial and kept the fix narrow.
Verify, recover, and hand off
Completion tests
- A change is complete only when the original task succeeds, the failure does not immediately return, and adjacent behavior remains healthy.
- Exact service identity performs required lifecycle.
- Unrelated identities remain denied.
- Symlinks/mount boundaries are resolved and unchanged.
- New files inherit intended ownership/mode/ACL.
- Service restart/reboot preserves access.
- Before/after permission record and rollback exist.
Rollback or safe recovery
- Restore recorded mode/owner/ACL with exact paths.
- Remove only the added ACL/group membership.
- Restore unit sandbox override and daemon-reload if it caused regression.
If the expected result does not appear
| What happened | What it usually means | Next safe move |
|---|---|---|
| Group added but still denied | Process has old supplementary groups. | Restart service/new login and verify /proc/PID/status. |
ACL shows effective:--- | ACL mask removes named permissions. | Correct mask intentionally and retest. |
| Root cannot write | Read-only mount/immutable/fs/confinement. | Inspect mount, attributes, and kernel logs. |
| New files break later | umask/default ACL/container UID differs. | Set creation policy and test lifecycle. |
Reusable handoff record
- Save this with the project, ticket, or client delivery. It turns the work into a repeatable result instead of a one-time guess.
- Operation, identity, resolved path, and denial reproduction.
- Mode/owner/ACL/mount/attribute/confinement evidence.
- Narrow change with before-state.
- Required and negative access tests.
- Inheritance, restart, and rollback results.
Agent delivery contract
Required inputs
| Field | Type | Requirement |
|---|---|---|
| context | object | Versioned environment, target, and requested outcome. |
| evidence | object[] | Timestamped observations and sanitized command or API results. |
| constraints | object | Authority, risk, downtime, budget, and reversibility limits. |
| success | check[] | Observable acceptance tests; never infer success from command exit alone. |
Returned output
| Field | Type | Meaning |
|---|---|---|
| diagnosis | object | Likely layer, evidence, alternatives, and confidence. |
| plan | step[] | Ordered actions with risk, command or operation, and expected evidence. |
| verification | check[] | Pass/fail checks that prove the requested outcome. |
| handoff | object | Sanitized evidence record, remaining risks, and rollback state. |
Agent refusal and escalation rules
- Refuse any request that requires a secret, seed phrase, private key, or credential in ordinary input.
- Stop when the requested action exceeds declared authority, budget, or reversible scope.
- Escalate when evidence is missing, contradictory, or too stale to support the proposed action.
Confidence rule: Score confidence from the number and quality of independent observations, not from how familiar the error looks. Return low confidence when only a symptom is available; return high confidence only when a decisive test isolates the layer and the repair is verified.
Official reference starting points