SAYLOR The result you are building
Finished Result:
A memory system that separates conversation state, user preferences, task facts, credentials, and durable knowledge; records provenance and consent; resolves conflicts; and proves isolation, correction, deletion, and recovery.
Use this guide when
- An assistant stores facts or preferences across sessions.
- Several agents or workflows share task state.
- Bad or stale memory could cause spending, messaging, access, or safety failures.
Do not use it as a substitute for
- Saving every message indefinitely as 'memory.'
- Treating retrieved text as trusted instruction or using memory as a secret store.
Before you change anything
Collect these items first. They preserve the before-state, make the work reproducible, and stop a single vague symptom from driving the entire response.
- Memory classes, owners, scopes, retention, and permitted consumers.
- Source message/event, observation time, confidence, and last validation.
- Conflict, correction, deletion, and legal-hold behavior.
- Encryption, tenant isolation, access logs, and export path.
- Poisoning, cross-user, stale-state, deletion, restore, and replay test results.
Stop Before Proceeding:
Do not use or retain memory whose owner, source, consent, scope, or sensitivity is unknown. Stop action when memory conflicts with current authenticated instructions.
Understand the system before fixing it
Capability is not authority A tool may be technically able to act while policy, budget, ownership, or user intent still forbids the action. Keep execution permission explicit and revocable.
Deterministic boundaries contain probabilistic reasoning Models may propose plans, but schemas, allowlists, budgets, approvals, idempotency, and postconditions must decide what can actually happen.
Memory is evidence, not authority A remembered preference can inform a proposal but cannot replace current authentication or approval for a sensitive action.
Deletion must reach derived indexes Removing a source row while embeddings, summaries, caches, or backups remain active is not effective deletion.
Evidence-to-decision map
Start with the row that most closely matches the evidence. The first test isolates a layer; it is not permission to
make every available change.
Evidence Likely layer First decisive check What the result means
Agent recalls Tenant Query with canary identities Storage, retrieval filters, cache keys, or indexes are not strictly
another user's detail isolation across tenants scoped.
Correction does not Derived state Trace source through A stale derivative still outranks the corrected record.
change behavior summary, embedding, and
cache
Retrieved note Instruction Mark note as data and replay Untrusted memory is being interpreted as instruction.
changes system poisoning tool selection
policy
Deleted fact Deletion/backu Search primary, index, cache, Deletion propagation or retention policy is incomplete.
reappears p export, and restore
Old preference Freshness/conf Compare timestamps, Resolution policy does not privilege current authenticated intent.
overrides current lict confidence, and direct
request instructionStep-by-step procedure
Work in order and retain the output from each step. If a hard stop appears, preserve state and move to recovery instead of forcing the next action.
01 Classify memory and its purpose Why: A precise boundary prevents a plausible fix from solving the wrong problem.
Do: Define ephemeral context, task state, durable user preference, verified fact, organizational knowledge, and prohibited secret classes. Record why each class exists.
Read the result: Every stored item has one class, owner, scope, retention rule, and allowed use.
Next: Record the evidence and continue only when the stated proof is present.
02 Attach provenance and confidence Why: Symptoms are not enough; a baseline preserves the evidence needed to isolate the failing layer.
Do: Store source event, actor, time, extraction method, confidence, validation time, and links to corrections or superseding records.
Read the result: A reviewer can trace every durable claim to its source.
Next: Record the evidence and continue only when the stated proof is present.
03 Enforce tenant and task isolation Why: Inconsistent inputs create false differences and make later comparisons unreliable.
Do: Use server-side identity scopes in every primary, vector, cache, and summary query. Test missing and manipulated tenant fields.
Read the result: Cross-tenant canaries never appear in results, logs, or exports.
Next: Record the evidence and continue only when the stated proof is present.
04 Treat retrieved memory as untrusted data Why: A decisive test reduces trial-and-error and limits unnecessary change.
Do: Separate memory from system/developer instruction, escape tool-like content, and require policy checks before acting on recalled facts.
Read the result: A poisoned note cannot alter tool policy, reveal secrets, or authorize action.
Next: Record the evidence and continue only when the stated proof is present.
Procedure continued 05 Resolve conflicts and staleness Why: The smallest reversible correction lowers the blast radius while preserving a recovery path.
Do: Prefer current authenticated instruction, keep superseded history, set expiry/revalidation triggers, and return uncertainty when claims disagree.
Read the result: The system explains which record won and why.
Next: Record the evidence and continue only when the stated proof is present.
06 Implement correction, export, and deletion Why: The happy path cannot expose replay, timeout, malformed-input, authority, or dependency failures.
Do: Propagate changes through summaries, embeddings, caches, replicas, and retention-aware backups; record completion and exceptions.
Read the result: A test identity can be corrected, exported, and removed from active retrieval.
Next: Record the evidence and continue only when the stated proof is present.
07 Run integrity and restore tests Why: A result is not complete until it remains observable and repeatable after the immediate fix.
Do: Inject cross-user, stale, contradictory, poisoned, and deleted canaries; restore a backup into isolation and confirm policy survives.
Read the result: Scheduled tests prove isolation and lifecycle behavior end to end.
Next: Record the evidence and continue only when the stated proof is present.
Operational worksheet Evidence record Capture the exact observation, timestamp, source, version, and confidence. Sanitize credentials and personal data before sharing the record.
- Memory classes, owners, scopes, retention, and permitted consumers.
- Source message/event, observation time, confidence, and last validation.
- Conflict, correction, deletion, and legal-hold behavior.
- Encryption, tenant isolation, access logs, and export path.
- Poisoning, cross-user, stale-state, deletion, restore, and replay test results.
Acceptance scoreboard
- Every durable item has owner, scope, purpose, source, time, confidence, retention, and permitted use.
- Current authenticated instruction and authoritative state outrank stale summaries.
- Cross-tenant and cross-task isolation tests pass for primary, vector, and cache paths.
- Retrieved memory cannot change policy or authorize sensitive action.
- Correction, export, and deletion reach derived indexes and caches.
- Backup restore preserves current isolation and lifecycle rules.
Decision rule SHIP / AUTOMATE GATE Proceed only when every required acceptance check is supported by direct evidence, rollback is available, and the remaining risk is explicitly owned. Unknown is not a pass.
Minimum handoff record
- Versioned agent memory and state integrity scope, owner, exclusions, and success criteria.
- Sanitized evidence snapshot with source, time, version, and confidence.
- Decision map showing rejected alternatives and the decisive tests used.
- Ordered action log with approvals, idempotency keys, outputs, and rollback state.
- Acceptance results, remaining risks, review date, and escalation owner.
Worked example
Starting Problem:
A sales agent keeps contacting a lead after the lead opted out because an old summary remains in vector search.
Evidence collected
- The CRM row shows opt-out.
- The memory summary still says follow-up permitted.
- Retrieval ranks the summary above the CRM event.
- Deletion jobs update SQL but not the vector index.
Decision The system has stale derived state and a dangerous precedence rule. Current authoritative consent must block outreach regardless of older memory.
Actions taken
- Marked consent as authoritative structured state.
- Removed the stale summary and rebuilt affected vectors.
- Added opt-out enforcement at the send boundary.
- Added correction and deletion propagation tests.
Proof Of Completion:
The opted-out identity is absent from active retrieval, send attempts fail closed, and restoring an old index cannot bypass current consent.
Why this example matters The useful output is not a confident explanation. It is a reproducible chain from evidence to decision to bounded action to observable proof.
Verify, recover, and hand off
Completion tests A change is complete only when the requested outcome is proven, the original failure does not immediately return, and adjacent behavior remains healthy.
- Every durable item has owner, scope, purpose, source, time, confidence, retention, and permitted use.
- Current authenticated instruction and authoritative state outrank stale summaries.
- Cross-tenant and cross-task isolation tests pass for primary, vector, and cache paths.
- Retrieved memory cannot change policy or authorize sensitive action.
- Correction, export, and deletion reach derived indexes and caches.
- Backup restore preserves current isolation and lifecycle rules.
Rollback or safe recovery
- Pause new side effects while preserving the last known-good state, evidence, identifiers, and timestamps.
- Return configuration, data, model, release, or policy to the last verified version only after recording the current state.
- Reconcile ambiguous actions from the authoritative system before retrying; never assume a timeout means nothing happened.
- Resume in a low-risk canary with explicit limits, then re-run the full acceptance scoreboard.
If the expected result does not appear What happened What it usually means Next safe move
Agent recalls another user's Storage, retrieval filters, cache keys, or Query with canary identities across tenants detail indexes are not strictly scoped.
Correction does not change A stale derivative still outranks the Trace source through summary, embedding, and cache behavior corrected record.
Retrieved note changes Untrusted memory is being interpreted Mark note as data and replay tool selection system policy as instruction.
Deleted fact reappears Deletion propagation or retention Search primary, index, cache, export, and restore policy is incomplete.
Reusable handoff record
- Versioned agent memory and state integrity scope, owner, exclusions, and success criteria.
- Sanitized evidence snapshot with source, time, version, and confidence.
- Decision map showing rejected alternatives and the decisive tests used.
- Ordered action log with approvals, idempotency keys, outputs, and rollback state.
- Acceptance results, remaining risks, review date, and escalation owner.
Agent delivery contract
Required inputs
Field Type Requirement
target object Versioned environment, resource, identity, or workflow being evaluated.
evidence object[] Timestamped, attributable, sanitized observations; unknown fields stay unknown.
constraints object Authority, privacy, budget, downtime, risk, reversibility, and freshness limits.
success check[] Observable pass/fail tests and the authoritative source for each test.
Returned output
Field Type Requirement
diagnosis object Likely layer, supporting and conflicting evidence, alternatives, and confidence.
plan step[] Ordered bounded actions with owner, risk, expected proof, and stop condition.
verification check[] Observed pass/fail/unknown results, not inferred success from command exit alone.
handoff object Sanitized evidence record, recovery state, remaining risk, and next review trigger.
Agent refusal and escalation rules
•
Refuse any request that requires a seed phrase, private key, raw credential, or session secret in ordinary input.
•
Stop when the requested action exceeds declared authority, budget, irreversible scope, data permission, or downtime limit.
•
Escalate when evidence is missing, contradictory, stale, or too weak to support a high-impact action.
•
Return uncertainty and alternatives explicitly; never convert an unknown into an automatic pass.
Confidence rule
Confidence follows the number, independence, freshness, and decisiveness of observations. Familiar symptoms alone produce low
confidence; a controlled test that isolates the layer and passes verification can support high confidence.Official reference starting points
- https://www.nist.gov/privacy-framework
- https://genai.owasp.org/llmrisk/llm01-prompt-injection/
- https://www.w3.org/TR/prov-overview/