Coordinate specialized agents through explicit ownership, shared state, message contracts, conflict policy, and a single auditable completion decision.
The result you're building
A multi-agent workflow where tasks have one owner, messages are attributable and idempotent, shared state is versioned, conflicting proposals are resolved under policy, and no agent can silently expand another's authority.
Use this guide when
- Several agents research, code, review, operate, or communicate in one workflow.
- Tasks can run in parallel but share files, budgets, accounts, or decisions.
- You need deterministic handoffs and conflict resolution.
Do not use it as a substitute for
- Using more agents as a substitute for clear decomposition.
- Letting agents write the same resource concurrently without version or ownership controls.
Before you change anything
- Collect these items first. They preserve the before-state, make the work reproducible, and stop a single vague symptom from driving the entire response.
- Agent identities, capabilities, owners, versions, and trust boundaries.
- Task graph, resource ownership, and allowed concurrency.
- Message schemas, correlation IDs, deadlines, and retry semantics.
- Shared-state versioning, locks, merge policy, and approval boundary.
- Conflict, duplicate, delayed-message, partial-failure, and recovery tests.
Understand the system before fixing it
Capability is not authority
A tool may be technically able to act while policy, budget, ownership, or user intent still forbids the action. Keep execution permission explicit and revocable.
Deterministic boundaries contain probabilistic reasoning
Models may propose plans, but schemas, allowlists, budgets, approvals, idempotency, and postconditions must decide what can actually happen.
One resource needs one mutation authority
Parallel research is cheap; concurrent writes to money, production, recipients, or source state require serialization or explicit conflict control.
Consensus is not automatically truth
Multiple agents can share the same flawed source or model bias. Decisions require evidence quality and policy, not vote count alone.
Evidence-to-decision map
| Evidence | Likely layer | First decisive check | What the result means |
|---|---|---|---|
| Two agents overwrite each other's work | Ownership/versioning | Compare resource owner and expected version | Concurrent mutation lacks serialization or optimistic concurrency. |
| Duplicate message creates duplicate action | Delivery semantics | Replay same correlation/idempotency key | At-least-once delivery is not contained at the consumer. |
| Agents disagree indefinitely | Resolution policy | Inspect evidence, authority, deadline, and resolver | No deterministic conflict/escalation rule exists. |
| Child task exceeds parent budget | Delegated authority | Compare child cap to reserved parent cap | Delegation expands instead of subdividing authority. |
| Coordinator reports done with failed branch | Aggregation | Evaluate required branch completion proofs | Status is based on response count rather than task contract. |
Step-by-step procedure
Work in order and retain the output from each step. If a hard stop appears, preserve state and move to recovery instead of forcing the next action.
Step 01 — Define roles and trust boundaries
Why: A precise boundary prevents a plausible fix from solving the wrong problem.
Do: Give each agent a stable identity, owner, capability version, data access, budget, and prohibited actions. Separate coordinator, executor, verifier, and approver where consequence warrants.
Read the result: Every action can be attributed to one bounded role.
Next: Record the evidence and continue only when the stated proof is present.
Step 02 — Decompose work around resources
Why: Symptoms are not enough; a baseline preserves the evidence needed to isolate the failing layer.
Do: Create tasks with one owner, explicit inputs/outputs, dependencies, deadline, and resources read or mutated. Mark safe parallel branches.
Read the result: No two active tasks have ungoverned write access to the same resource.
Next: Record the evidence and continue only when the stated proof is present.
Step 03 — Standardize messages and state
Why: Inconsistent inputs create false differences and make later comparisons unreliable.
Do: Use schemas with task ID, sender, recipient, state version, evidence refs, deadline, idempotency key, and requested response type.
Read the result: Malformed, stale, duplicate, and misrouted messages fail deterministically.
Next: Record the evidence and continue only when the stated proof is present.
Step 04 — Enforce delegated budgets and authority
Why: A decisive test reduces trial-and-error and limits unnecessary change.
Do: Reserve parent budget, subdivide it among children, and prevent children from delegating broader access or side effects.
Read the result: The sum of child maximums cannot exceed the parent's remaining authority.
Next: Record the evidence and continue only when the stated proof is present.
Step 05 — Control shared writes
Why: The smallest reversible correction lowers the blast radius while preserving a recovery path.
Do: Use single-writer ownership, locks, or optimistic concurrency with conflict detection. Preserve base, proposals, and resolution evidence.
Read the result: A stale write cannot silently replace a newer accepted state.
Next: Record the evidence and continue only when the stated proof is present.
Step 06 — Resolve conflict by evidence and policy
Why: The happy path cannot expose replay, timeout, malformed-input, authority, or dependency failures.
Do: Rank authoritative sources, freshness, test results, consequence, and role. Escalate unresolved material differences to the declared human or service owner.
Read the result: Resolution records why one proposal won; equal votes are not required.
Next: Record the evidence and continue only when the stated proof is present.
Step 07 — Verify the whole workflow
Why: A result is not complete until it remains observable and repeatable after the immediate fix.
Do: Require completion proof for every mandatory branch, reconcile costs and side effects, test delayed/duplicate/failed agents, and archive the final state graph.
Read the result: Coordinator completion matches the original acceptance contract.
Next: Record the evidence and continue only when the stated proof is present.
Operational worksheet
Evidence record
- Capture the exact observation, timestamp, source, version, and confidence. Sanitize credentials and personal data before sharing the record.
- Agent identities, capabilities, owners, versions, and trust boundaries.
- Task graph, resource ownership, and allowed concurrency.
- Message schemas, correlation IDs, deadlines, and retry semantics.
- Shared-state versioning, locks, merge policy, and approval boundary.
- Conflict, duplicate, delayed-message, partial-failure, and recovery tests.
Acceptance scoreboard
- Every agent has stable identity, owner, capabilities, data scope, and budget.
- Tasks declare resource reads/writes and only safe branches run concurrently.
- Messages carry schema version, state version, correlation, deadline, and idempotency.
- Child authority and total reserved spend cannot exceed the parent.
- Conflicting writes are detected and resolved by declared evidence policy.
- Global completion requires proof from every mandatory branch and reconciliation of side effects.
Minimum handoff record
- Versioned multi-agent coordination and conflict control scope, owner, exclusions, and success criteria.
- Sanitized evidence snapshot with source, time, version, and confidence.
- Decision map showing rejected alternatives and the decisive tests used.
- Ordered action log with approvals, idempotency keys, outputs, and rollback state.
- Acceptance results, remaining risks, review date, and escalation owner.
Worked example
Evidence collected
- Alert delivery is at least once.
- Both agents own the deploy capability.
- No incident lease or shared state version exists.
- Rollback requests use different keys and both execute.
Decision: The workflow has duplicate delivery without single-writer control. Introduce incident ownership, a lease, versioned state, and idempotent deployment action.
Actions taken
- Created one incident coordinator lease.
- Bound rollback to incident ID and release transition.
- Rejected stale state versions.
- Added duplicate and failover tests.
Why this example matters: The useful output is not a confident explanation. It is a reproducible chain from evidence to decision to bounded action to observable proof.
Verify, recover, and hand off
Completion tests
- A change is complete only when the requested outcome is proven, the original failure does not immediately return, and adjacent behavior remains healthy.
- Every agent has stable identity, owner, capabilities, data scope, and budget.
- Tasks declare resource reads/writes and only safe branches run concurrently.
- Messages carry schema version, state version, correlation, deadline, and idempotency.
- Child authority and total reserved spend cannot exceed the parent.
- Conflicting writes are detected and resolved by declared evidence policy.
- Global completion requires proof from every mandatory branch and reconciliation of side effects.
Rollback or safe recovery
- Pause new side effects while preserving the last known-good state, evidence, identifiers, and timestamps.
- Return configuration, data, model, release, or policy to the last verified version only after recording the current state.
- Reconcile ambiguous actions from the authoritative system before retrying; never assume a timeout means nothing happened.
- Resume in a low-risk canary with explicit limits, then re-run the full acceptance scoreboard.
If the expected result does not appear
| What happened | What it usually means | Next safe move |
|---|---|---|
| Two agents overwrite each other's work | Concurrent mutation lacks serialization or optimistic concurrency. | Compare resource owner and expected version |
| Duplicate message creates duplicate action | At-least-once delivery is not contained at the consumer. | Replay same correlation/idempotency key |
| Agents disagree indefinitely | No deterministic conflict/escalation rule exists. | Inspect evidence, authority, deadline, and resolver |
| Child task exceeds parent budget | Delegation expands instead of subdividing authority. | Compare child cap to reserved parent cap |
Reusable handoff record
- Versioned multi-agent coordination and conflict control scope, owner, exclusions, and success criteria.
- Sanitized evidence snapshot with source, time, version, and confidence.
- Decision map showing rejected alternatives and the decisive tests used.
- Ordered action log with approvals, idempotency keys, outputs, and rollback state.
- Acceptance results, remaining risks, review date, and escalation owner.
Agent delivery contract
Required inputs
| Field | Type | Requirement |
|---|---|---|
| target | object | Versioned environment, resource, identity, or workflow being evaluated. |
| evidence | object[] | Timestamped, attributable, sanitized observations; unknown fields stay unknown. |
| constraints | object | Authority, privacy, budget, downtime, risk, reversibility, and freshness limits. |
| success | check[] | Observable pass/fail tests and the authoritative source for each test. |
Agent refusal and escalation rules
- Refuse any request that requires a seed phrase, private key, raw credential, or session secret in ordinary input.
- Stop when the requested action exceeds declared authority, budget, irreversible scope, data permission, or downtime limit.
- Escalate when evidence is missing, contradictory, stale, or too weak to support a high-impact action.
- Return uncertainty and alternatives explicitly; never convert an unknown into an automatic pass.
Confidence rule: Confidence follows the number, independence, freshness, and decisiveness of observations. Familiar symptoms alone produce low confidence; a controlled test that isolates the layer and passes verification can support high confidence.
Official reference starting points