The result you are building

Finished Result:

A tested capacity and backpressure policy that admits bounded work, returns useful retry signals, prevents retry storms, protects priority operations, and degrades without losing or duplicating accepted jobs.

Use this guide when

An API calls costly models, databases, blockchains, or third-party providers.
Traffic arrives in bursts or through many agent workers.
Overload currently appears as timeouts, 5xx errors, or provider bans.

Do not use it as a substitute for

Increasing timeouts or automatic retries without measuring capacity.
Using one global limit that lets a single tenant or expensive route consume the system.

Before you change anything

Collect these items first. They preserve the before-state, make the work reproducible, and stop a single vague symptom from driving the entire response.

Per-route work units, cost, concurrency, latency, and dependency limits.
Client/tenant identity, quotas, priority classes, and fairness policy.
Queue depth/age, worker capacity, timeout, and cancellation behavior.
429/503 response schema, Retry-After, idempotency, and client backoff.
Steady, burst, retry-storm, dependency-slowdown, and recovery load tests.

Stop Before Proceeding:

Stop accepting new expensive work when completion before deadline is no longer plausible or when dependency protection thresholds fail. Reject early rather than timing out after consuming the full cost.

Understand the system before fixing it

Provenance is part of the record A value without source, observation time, transformation history, and known limitations cannot support a defensible automated decision.

Schema changes are product changes Renames, units, null behavior, identifiers, and deleted fields can silently change decisions even when a pipeline still returns HTTP 200.

Rate and concurrency are different Requests per second does not bound simultaneous expensive work. Protect CPU, memory, database connections, provider quotas, and queue age directly.

Retries are new load Clients and workers must use bounded exponential backoff with jitter and honor server guidance; otherwise a small failure becomes a synchronized storm.

Evidence-to-decision map

Start with the row that most closely matches the evidence. The first test isolates a layer; it is not permission to
make every available change.

   Evidence               Likely layer      First decisive check             What the result means

   Latency spikes         Dependency/c      Trace active work and            A downstream pool or serialized resource is the bottleneck.
   before CPU maxes       oncurrency        connection/provider saturation

   429 clients retry      Contract/client   Inspect Retry-After and retry    Server guidance is missing or clients ignore it.
   immediately                              jitter

   One tenant starves     Fairness          Break usage by                   Global first-come queue lacks isolation or weights.
   others                                   tenant/route/work unit

   Queue grows after      Admission         Compare arrival, service rate,   System accepts more work than it can drain.
   recovery                                 and queue age

   Timeout repeats side   Idempotency       Replay request key after         Retry safety is missing at the operation boundary.
   effect                                   ambiguous timeout

Step-by-step procedure

Work in order and retain the output from each step. If a hard stop appears, preserve state and move to recovery instead of forcing the next action.

01 Measure work and bottlenecks Why: A precise boundary prevents a plausible fix from solving the wrong problem.

Do: Define request cost units, critical dependencies, p50/p95 service time, connection use, memory, and external quotas by route and tenant.

Read the result: Capacity model identifies the first constrained resource.