SAYLOR Design agent workflows so untrusted webpages, documents, emails, tool output, and retrieved text cannot rewrite system policy or silently trigger actions.
The result you're building
An agent workflow where untrusted text is labeled as data, cannot alter authorization, cannot expose secrets, and cannot directly trigger consequential tools without independent policy, structured proposals, exact approval, and adversarial tests.
Use this guide when
- Agents read webpages, documents, email, tickets, code, or tool outputs and can act.
- Designing retrieval/browser/MCP workflows for sensitive systems.
Do not use it as a substitute for
- Do not treat system-prompt secrecy or 'ignore malicious instructions' wording as a complete defense.
- Do not let retrieved content supply credentials, tool permissions, or approval.
Before you change anything
- Collect the items below first. They let you compare before and after, keep the work reproducible, and avoid guessing from a single error message.
- Sources of untrusted content and transformation path.
- Tools, data, credentials, and side effects reachable afterward.
- Policy/approval controls outside the model.
- Adversarial corpus and logging/incident requirements.
Understand the system before fixing it
Content and authority must remain separate
External text may describe an action but cannot authorize it. Identity, policy, scope, and approval come from trusted channels.
Indirect injection crosses tool boundaries
A malicious webpage can target email, shell, wallet, cloud, or database tools available later.
Detection is useful but not sufficient
Obfuscation evolves. Least privilege, allowlists, output constraints, approvals, and isolation limit impact when detection misses.
Evidence-to-decision map
| Evidence | Likely layer | First decisive check | What the result means |
|---|---|---|---|
| Content asks to ignore policy | Direct injection | Label source and test policy gate | Treat as data; do not change authority. |
| Hidden/encoded instruction in page/document | Indirect injection | Render/extract variants and adversarial test | Sanitize/limit content and preserve source trust label. |
| Tool output requests another tool | Cross-tool escalation | Trace provenance to authorization decision | Output cannot self-authorize; require trusted plan/policy. |
| Content asks for secret/external send | Exfiltration | Canary-secret and egress test | Block secret access/egress independently. |
Step-by-step procedure
Work in order. Record the output after each step. If a step produces the stated stop condition, do not keep pushing forward; preserve the evidence and use the recovery path.
Step 01 — Map injection-to-impact chains
Why: Risk depends on what compromised reasoning can reach.
Do: For each untrusted source trace model context, memory, tools, credentials, networks, and side effects.
Read the result: Any path from content to irreversible action or secret is priority.
Next: Break chain with independent control.
Step 02 — Minimize and label content
Why: Unbounded raw content expands attack surface.
Do: Retrieve only needed fields/snippets, preserve source/provenance/trust, strip active content where appropriate, and bound size.
Read the result: Never concatenate content as privileged instructions.
Next: Keep secrets out of shared context.
Step 03 — Externalize policy and authorization
Why: A model can be manipulated to reinterpret prompt rules.
Do: Enforce identity, tenant, host/path, recipient, amount, verb, data class, and approval in deterministic code/tool server.
Read the result: Prohibited action fails even if model requests it.
Next: Default deny unknown destinations/tools.
Step 04 — Use propose-review-execute
Why: Separating stages exposes the exact intended effect.
Do: Model creates typed proposal with evidence/provenance; validator checks policy; human approves immutable high-impact details; executor performs only approved action.
Read the result: Changed proposal invalidates approval.
Next: Read-only actions remain separate from execute.
Step 05 — Constrain outputs and memory
Why: Injected text can persist and influence later tasks.
Do: Store facts with provenance/trust, not raw instruction authority; filter secrets; prevent untrusted text from editing policy/system memory.
Read the result: Memory changes require trusted source/approval.
Next: Test cross-session persistence.
Step 06 — Run adversarial evaluation
Why: Defenses must survive realistic variants.
Do: Test direct/indirect, hidden text, images/metadata, encoding, multilingual, tool-output, memory poisoning, data exfiltration, and approval spoofing.
Read the result: Measure unsafe action and secret leak, not only whether model mentions injection.
Next: Retest on model/tool changes.
Worked example
Evidence collected
- Ticket text is untrusted customer content.
- Agent has filesystem read and unrestricted HTTP upload tools.
.envcontains API keys.- System prompt only says not to reveal secrets.
Decision: This is an exfiltration chain; prompt wording is not a hard boundary.
Actions taken
- Removed secret-file access from ticket agent and restricted egress destinations.
- Labeled ticket content untrusted; upload requires typed proposal and exact approval.
- Added canary and obfuscated-upload tests.
.env; unknown URL is blocked; injected request cannot authorize upload; tests produce refusal/escalation.Why this example matters: Breaking authority and egress paths makes detection misses survivable.
Verify, recover, and hand off
Completion tests
- A change is complete only when the original task succeeds, the failure does not immediately return, and adjacent behavior remains healthy.
- Untrusted sources are identified/labeled with provenance.
- Model cannot grant itself credentials, scope, or approval.
- High-impact actions use typed proposal and bound approval.
- Secrets are outside unnecessary context and egress is constrained.
- Memory/policy cannot be rewritten by untrusted output.
- Adversarial chain tests fail closed.
Rollback or safe recovery
- Disable affected tools and clear/quarantine untrusted memory/context.
- Rotate secrets if canary/real secret exposure occurred.
- Restore prior policy/tool set and preserve sanitized incident evidence.
If the expected result does not appear
| What happened | What it usually means | Next safe move |
|---|---|---|
| Detector misses obfuscation | Content classifiers are bypassable. | Rely on authority/egress/approval controls; expand tests. |
| Agent refuses safe content | Trust label/policy too broad. | Permit narrow read/analysis while keeping execution boundary. |
| Injection persists | Memory stores raw content as trusted. | Remove entry, add provenance/trust and approval for memory writes. |
| Tool server accepts unsafe call | Policy enforced only in model. | Move validation/authorization to server/executor. |
Reusable handoff record
- Save this with the project, ticket, or client delivery. It turns the work into a repeatable result instead of a one-time guess.
- Injection-to-impact map.
- Content trust/provenance and minimization rules.
- External policy/authorization/approval controls.
- Memory/output/secret/egress boundaries.
- Adversarial results, incident and rollback plan.
Agent delivery contract
Required inputs
| Field | Type | Requirement |
|---|---|---|
| context | object | Versioned environment, target, and requested outcome. |
| evidence | object[] | Timestamped observations and sanitized command or API results. |
| constraints | object | Authority, risk, downtime, budget, and reversibility limits. |
| success | check[] | Observable acceptance tests; never infer success from command exit alone. |
Returned output
| Field | Type | Meaning |
|---|---|---|
| diagnosis | object | Likely layer, evidence, alternatives, and confidence. |
| plan | step[] | Ordered actions with risk, command or operation, and expected evidence. |
| verification | check[] | Pass/fail checks that prove the requested outcome. |
| handoff | object | Sanitized evidence record, remaining risks, and rollback state. |
Agent refusal and escalation rules
- Refuse any request that requires a secret, seed phrase, private key, or credential in ordinary input.
- Stop when the requested action exceeds declared authority, budget, or reversible scope.
- Escalate when evidence is missing, contradictory, or too stale to support the proposed action.
Confidence rule: Score confidence from the number and quality of independent observations, not from how familiar the error looks. Return low confidence when only a symptom is available; return high confidence only when a decisive test isolates the layer and the repair is verified.
Official reference starting points