The result you are building

Finished Result:

A project-local Python environment tied to the intended interpreter, installed reproducibly from declared dependencies, passing import and application tests without altering Ubuntu's system Python.

Use this guide when

Imports fail, pip installs to the wrong interpreter, venv paths moved, native extensions fail, or Ubuntu

reports externally managed environment.

A project works only after global sudo pip installs.

Do not use it as a substitute for

Do not use sudo pip against Ubuntu's system Python or bypass PEP 668 globally.
Do not copy/move an existing venv between paths or Python versions and expect it to remain valid.

Before you change anything

Collect the items below first. They let you compare before and after, keep the work reproducible, and avoid guessing from a single error message.

Project path, Python requirement, lock/requirements/pyproject files.
which/type, python -VV, python -m pip -V, and failing import traceback.
Ubuntu package/native build dependencies and architecture.
Current venv location and whether it can be recreated.

Stop Before Proceeding:

Stop before removing system Python packages or /usr/lib/python*. Ubuntu tools depend on them. Create a fresh project venv instead.

Understand the system before fixing it

python -m pip binds pip to that interpreter Bare pip can point elsewhere. Always pair version/install evidence with the interpreter executing the application.

venvs are disposable artifacts Source and dependency declarations are the recoverable assets. Recreate rather than surgically editing broken activation scripts.

Native wheels depend on platform ABI Compiler errors may indicate missing system headers, unsupported Python version, architecture, or no compatible wheel.

Evidence-to-decision map

  Start with the row that most closely matches the evidence. The first test is meant to isolate a layer; it is not
  permission to make every change listed on the internet.

    Evidence                  Likely layer       First decisive check               What the result means

    PEP 668 externally        System             Check whether venv is active       Create/use venv; do not break system
    managed                   boundary           and pip target                     protection.

    Module installed but      Interpreter/pat    `python -m pip show` and           Install and run use different
    import fails              h                  `sys.executable/sys.path`          interpreter/environment.

    Wheel build fails         Native             Read first compiler/header error   Install documented build dependency or
                              dependency         and supported Python versions      choose supported wheel/version.

    Venv points to old        Moved/broken       Inspect `pyvenv.cfg` and           Delete derived venv after saving
    path                      venv               shebangs                           dependencies; recreate at final path.

Step-by-step procedure

Work in order. Record the output after each step. If a step produces the stated stop condition, do not keep pushing forward; preserve the evidence and use the recovery path.

01 Identify the executing interpreter Why: Most Python environment failures are identity mismatches.

Do: Record command resolution, interpreter, pip, prefix/base prefix, and import path from the same shell/service.

type -a python3 pip pip3 python3 -VV python3 -m pip -V python3 -c 'import sys; print(sys.executable, sys.prefix, sys.base_prefix)'

Read the result: Different pip/interpreter paths prove the mismatch.

Next: Choose the project Python version before installing.

02 Preserve declarations and rebuild cleanly Why: Repairing a moved/corrupt venv produces hidden residue.

Do: Save pyproject/lock/requirements and any local package sources; rename old venv, create new one with intended Python, upgrade packaging tools only as project permits.

python3 -m venv .venv . .venv/bin/activate python -m pip -V

Read the result: New sys.prefix differs from base and paths point to project.

Next: Use locked install.

03 Install reproducibly Why: Unpinned installs can change the problem.

Do: Use project lock manager or exact requirements; capture complete first failure.

Read the result: Dependency-resolution conflicts are contract issues; build errors are platform/toolchain issues.

Next: Do not add arbitrary versions until constraint conflict is understood.

04 Resolve native build prerequisites narrowly Why: Installing a full compiler stack may be unnecessary and risky.

Do: Check package docs for supported Python/platform and required Ubuntu -dev libraries/compiler. Prefer compatible wheel when appropriate.

Read the result: The first missing header/library identifies the system dependency.

Next: Record apt packages separately from Python dependencies.

05 Test import and entrypoint Why: Successful install does not prove runtime config/import path.

Do: Run imports, package self-test, application command, and service context using explicit .venv/bin/python.

Read the result: No reliance on activated shell is required for systemd/automation.

Next: Fix packaging rather than setting broad PYTHONPATH where possible.

06 Remove ambiguity Why: Multiple environments invite recurrence.

Do: Update runbook/service/IDE to explicit interpreter, keep venv ignored, lock dependencies, and archive/delete broken venv only after success.

Read the result: Fresh recreation must pass.

Next: Never commit secrets in env files.

Worked example

Starting Problem:

pip install says a module is installed, but the app raises ModuleNotFoundError.

Evidence collected

Bare pip points to user Python 3.12.
Service ExecStart uses project venv Python 3.11.
Package is absent from venv and not declared.
System Python is PEP 668 protected.

Decision The package was installed into the wrong interpreter; system modification is unnecessary.

Actions taken

Added correct dependency to project declaration.
Recreated venv with required Python and locked install.
Ran import and service using explicit venv interpreter.

Proof Of Completion:

Fresh venv installs without global changes; import and app test pass; service restart uses same interpreter.

Why this example matters Pairing pip with the executing Python made the invisible mismatch explicit.

Verify, recover, and hand off

Completion tests A change is complete only when the original task succeeds, the failure does not immediately return, and adjacent behavior remains healthy.

Interpreter/pip/prefix paths all refer to intended venv.
Locked install succeeds without modifying system Python.
Imports and core app test pass in shell and service context.
Native dependencies and Python version are documented.
Fresh venv recreation reproduces result.

Rollback or safe recovery

Reactivate/restore old venv path only as temporary fallback if untouched.
Return dependency declaration/lockfile to prior version.
Remove new venv and recreate; do not roll back by altering system Python.

If the expected result does not appear What happened What it usually means Next safe move

Activation seems ignored Shell hash/alias/service uses explicit Use type -a and explicit .venv/bin/python. other path.

Install succeeds only with sudo Target permissions/interpreter are Recreate user-owned venv; never sudo pip. wrong.

No wheel available Python/platform/version unsupported. Use supported Python/package or documented compiler deps.

Service fails; shell works Unit environment/user/path differs. Test explicit interpreter as service user and inspect unit.

Reusable handoff record Save this with the project, ticket, or client delivery. It turns the work into a repeatable result instead of a one-time guess.

Python/venv/pip identity evidence.
Dependency declaration/lock and install command.
Ubuntu native dependencies and rationale.
Import/application/service tests.
Fresh-rebuild and rollback procedure.

Agent delivery contract

  Required inputs
    Field                        Type                Requirement

    context                      object              Versioned environment, target, and requested outcome.

    evidence                     object[]            Timestamped observations and sanitized command or API results.

    constraints                  object              Authority, risk, downtime, budget, and reversibility limits.

    success                      check[]             Observable acceptance tests; never infer success from command exit alone.

  Returned output
    Field                        Type                Meaning

    diagnosis                    object              Likely layer, evidence, alternatives, and confidence.

    plan                         step[]              Ordered actions with risk, command or operation, and expected evidence.

    verification                 check[]             Pass/fail checks that prove the requested outcome.

    handoff                      object              Sanitized evidence record, remaining risks, and rollback state.

  Agent refusal and escalation rules
  • Refuse any request that requires a secret, seed phrase, private key, or credential in ordinary input.
  • Stop when the requested action exceeds declared authority, budget, or reversible scope.
  • Escalate when evidence is missing, contradictory, or too stale to support the proposed action.

  Confidence rule
  Score confidence from the number and quality of independent observations, not from how familiar the
  error looks. Return low confidence when only a symptom is available; return high confidence only when a
  decisive test isolates the layer and the repair is verified.

Official reference starting points

https://packaging.python.org/en/latest/tutorials/installing-packages/
https://docs.python.org/3/library/venv.html
https://peps.python.org/pep-0668/

Python Environment Repair Guide