SAYLOR The result you are building
Finished Result:
A cache contract that defines what is cacheable, binds entries to identity/version/authorization, exposes age, prevents stampedes, verifies invalidation, and fails safely when cache or origin is unhealthy.
Use this guide when
- You cache API results, sessions, pages, model outputs, market data, or database objects.
- Staleness or wrong-tenant data could change decisions.
- Origin load spikes when keys expire or cache fails.
Do not use it as a substitute for
- Caching authenticated responses under a URL-only key.
- Using flush-all as the normal invalidation or recovery method.
Before you change anything
Collect these items first. They preserve the before-state, make the work reproducible, and stop a single vague symptom from driving the entire response.
- Resource semantics, authority, sensitivity, volatility, and permitted staleness.
- Complete key dimensions: tenant/user/role/query/version/locale/authorization/data version.
- TTL, stale-while-revalidate, negative cache, tag/event invalidation, and delete behavior.
- Serializer/schema/compression/version and integrity checks.
- Hit/miss/age/eviction/stampede/leak/invalidation/failover test evidence.
Stop Before Proceeding:
Do not cache sensitive or authorization-dependent output until the key and invalidation include every dimension that can change the result. Stop serving cache when ownership or schema version is ambiguous.
Understand the system before fixing it
Observe before mutating Capture state, logs, versions, ownership, and dependency health before restarting, reinstalling, deleting, or rotating anything.
Recovery must be exercised A backup, rollback command, or spare endpoint is only a claim until a controlled restore or failover test proves it works.
A cache key is a security boundary If two requests can produce different authorized results, every relevant identity, role, permission, locale, version, and input dimension must be represented or caching must be disabled.
TTL is not complete invalidation Corrections, revocations, deletes, and security changes may require immediate tag/event invalidation plus sweeps for missed events.
Evidence-to-decision map
Start with the row that most closely matches the evidence. The first test isolates a layer; it is not permission to
make every available change.
Evidence Likely layer First decisive check What the result means
User sees another Key/authorizat Compare key dimensions and Tenant or permission context is missing from the key.
tenant's response ion request principals
Old schema crashes Serialization/v Inspect entry version and Writers/readers changed without namespaced compatibility.
reader ersion deployment overlap
Origin collapses at Stampede Plot concurrent misses for hot No single-flight, jitter, prewarm, or stale policy exists.
expiry key
Deleted item Invalidation Trace delete event/tag and Deletion did not reach all active cache paths.
remains served replica/cache layers
Cache outage makes Fallback Force cache timeout and The application treats cache error as data or bypasses safety
service wrong observe origin/fail mode limits.Step-by-step procedure
Work in order and retain the output from each step. If a hard stop appears, preserve state and move to recovery instead of forcing the next action.
01 Classify cacheable results Why: A precise boundary prevents a plausible fix from solving the wrong problem.
Do: For each route/value, define sensitivity, identity dependence, volatility, source cost, correctness requirement, and maximum age. Disable caching when safe keying is impractical.
Read the result: Every cache has an owner and explicit staleness contract.
Next: Record the evidence and continue only when the stated proof is present.
02 Design complete versioned keys Why: Symptoms are not enough; a baseline preserves the evidence needed to isolate the failing layer.
Do: Include normalized inputs, tenant/user/role or authorization digest, resource/data/schema version, locale, and output variant. Hash large sensitive components safely.
Read the result: Requests that can differ never collide; equivalent requests share deliberately.
Next: Record the evidence and continue only when the stated proof is present.
03 Define freshness and invalidation Why: Inconsistent inputs create false differences and make later comparisons unreliable.
Do: Set TTL from fact volatility, add jitter, declare stale-while-revalidate only where safe, and use tag/event invalidation for writes, revocations, corrections, and deletion.
Read the result: A changed authoritative record becomes visible within the declared window.
Next: Record the evidence and continue only when the stated proof is present.
04 Control serialization and integrity Why: A decisive test reduces trial-and-error and limits unnecessary change.
Do: Namespace formats, validate type/schema/version on read, cap size/decompression, encrypt sensitive values if caching is allowed, and reject corrupt entries.
Read the result: Old, malformed, oversized, or wrong-type entries fail as misses, not trusted data.
Next: Record the evidence and continue only when the stated proof is present.
Procedure continued 05 Prevent stampedes and cache abuse Why: The smallest reversible correction lowers the blast radius while preserving a recovery path.
Do: Use single-flight/leases, request coalescing, prewarm hot keys, bounded negative caching, quotas, and protection against attacker-controlled key explosion.
Read the result: Hot-key expiry and random-key load stay within origin capacity.
Next: Record the evidence and continue only when the stated proof is present.
06 Design fallback and failure mode Why: The happy path cannot expose replay, timeout, malformed-input, authority, or dependency failures.
Do: Set cache timeouts, circuit state, origin budgets, and explicit stale/error behavior. Never treat cache connection error as 'not found' for authorization.
Read the result: Cache loss degrades according to the contract without leaking or lying.
Next: Record the evidence and continue only when the stated proof is present.
07 Test isolation and invalidation Why: A result is not complete until it remains observable and repeatable after the immediate fix.
Do: Run cross-tenant canaries, permission changes, delete/correct, schema overlap, hot-key expiry, cache restart, replica lag, and missed-event sweep.
Read the result: No leak occurs and freshness, fallback, and recovery match published behavior.
Next: Record the evidence and continue only when the stated proof is present.
Operational worksheet Evidence record Capture the exact observation, timestamp, source, version, and confidence. Sanitize credentials and personal data before sharing the record.
- Resource semantics, authority, sensitivity, volatility, and permitted staleness.
- Complete key dimensions: tenant/user/role/query/version/locale/authorization/data version.
- TTL, stale-while-revalidate, negative cache, tag/event invalidation, and delete behavior.
- Serializer/schema/compression/version and integrity checks.
- Hit/miss/age/eviction/stampede/leak/invalidation/failover test evidence.
Acceptance scoreboard
- Cacheability decision covers sensitivity, authority, volatility, cost, and permitted age.
- Keys include every result-changing input and authorization dimension.
- Writes, revocations, corrections, and deletes propagate through active caches.
- Serializer/schema/version and integrity are validated before use.
- Stampede, random-key, negative-cache, and origin capacity controls pass.
- Cross-tenant, schema overlap, outage, restart, invalidation, and sweep tests pass.
Decision rule SHIP / AUTOMATE GATE Proceed only when every required acceptance check is supported by direct evidence, rollback is available, and the remaining risk is explicitly owned. Unknown is not a pass.
Minimum handoff record
- Versioned cache integrity and safe invalidation scope, owner, exclusions, and success criteria.
- Sanitized evidence snapshot with source, time, version, and confidence.
- Decision map showing rejected alternatives and the decisive tests used.
- Ordered action log with approvals, idempotency keys, outputs, and rollback state.
- Acceptance results, remaining risks, review date, and escalation owner.
Worked example
Starting Problem:
A SaaS API caches /profile by path and serves one customer's profile to another.
Evidence collected
- The route is authenticated.
- Cache key contains only method and URL.
- Reverse proxy caches 200 responses by default.
- Response lacks a private/no-store policy.
Decision This is a cross-user data incident. Disable affected caching, preserve logs, notify the owner, and redesign keys/policies before reactivation.
Actions taken
- Purged and disabled the unsafe cache rule.
- Added private/no-store where shared caching is not justified.
- Included tenant/user/authorization version in approved caches.
- Added cross-user canary tests.
Proof Of Completion:
Repeated alternating users never share cached identity data; permission changes invalidate immediately; origin and proxy policies agree.
Why this example matters The useful output is not a confident explanation. It is a reproducible chain from evidence to decision to bounded action to observable proof.
Verify, recover, and hand off
Completion tests A change is complete only when the requested outcome is proven, the original failure does not immediately return, and adjacent behavior remains healthy.
- Cacheability decision covers sensitivity, authority, volatility, cost, and permitted age.
- Keys include every result-changing input and authorization dimension.
- Writes, revocations, corrections, and deletes propagate through active caches.
- Serializer/schema/version and integrity are validated before use.
- Stampede, random-key, negative-cache, and origin capacity controls pass.
- Cross-tenant, schema overlap, outage, restart, invalidation, and sweep tests pass.
Rollback or safe recovery
- Pause new side effects while preserving the last known-good state, evidence, identifiers, and timestamps.
- Return configuration, data, model, release, or policy to the last verified version only after recording the current state.
- Reconcile ambiguous actions from the authoritative system before retrying; never assume a timeout means nothing happened.
- Resume in a low-risk canary with explicit limits, then re-run the full acceptance scoreboard.
If the expected result does not appear What happened What it usually means Next safe move
User sees another tenant's Tenant or permission context is missing Compare key dimensions and request principals response from the key.
Old schema crashes reader Writers/readers changed without Inspect entry version and deployment overlap namespaced compatibility.
Origin collapses at expiry No single-flight, jitter, prewarm, or Plot concurrent misses for hot key stale policy exists.
Deleted item remains served Deletion did not reach all active cache Trace delete event/tag and replica/cache layers paths.
Reusable handoff record
- Versioned cache integrity and safe invalidation scope, owner, exclusions, and success criteria.
- Sanitized evidence snapshot with source, time, version, and confidence.
- Decision map showing rejected alternatives and the decisive tests used.
- Ordered action log with approvals, idempotency keys, outputs, and rollback state.
- Acceptance results, remaining risks, review date, and escalation owner.
Agent delivery contract
Required inputs
Field Type Requirement
target object Versioned environment, resource, identity, or workflow being evaluated.
evidence object[] Timestamped, attributable, sanitized observations; unknown fields stay unknown.
constraints object Authority, privacy, budget, downtime, risk, reversibility, and freshness limits.
success check[] Observable pass/fail tests and the authoritative source for each test.
Returned output
Field Type Requirement
diagnosis object Likely layer, supporting and conflicting evidence, alternatives, and confidence.
plan step[] Ordered bounded actions with owner, risk, expected proof, and stop condition.
verification check[] Observed pass/fail/unknown results, not inferred success from command exit alone.
handoff object Sanitized evidence record, recovery state, remaining risk, and next review trigger.
Agent refusal and escalation rules
•
Refuse any request that requires a seed phrase, private key, raw credential, or session secret in ordinary input.
•
Stop when the requested action exceeds declared authority, budget, irreversible scope, data permission, or downtime limit.
•
Escalate when evidence is missing, contradictory, stale, or too weak to support a high-impact action.
•
Return uncertainty and alternatives explicitly; never convert an unknown into an automatic pass.
Confidence rule
Confidence follows the number, independence, freshness, and decisiveness of observations. Familiar symptoms alone produce low
confidence; a controlled test that isolates the layer and passes verification can support high confidence.Official reference starting points
- https://www.rfc-editor.org/rfc/rfc9111
- https://redis.io/docs/latest/operate/oss_and_stack/management/optimization/
- https://owasp.org/www-community/attacks/Cache_Poisoning