SAYLOR Match records that refer to the same real entity without silently merging different people, organizations, wallets, products, or addresses.
The result you're building
A versioned matching system that separates exact identifiers from probabilistic similarity, preserves source records, explains merge decisions, supports review and unmerge, and measures false merges and missed matches by risk slice.
Use this guide when
- You combine leads, customers, products, businesses, wallets, or source feeds.
- Names, addresses, domains, IDs, and formatting vary.
- Duplicate actions or wrong-person joins would cause harm.
Do not use it as a substitute for
- Merging solely on name, phone fragment, street, or fuzzy score.
- Discarding source records or making merges irreversible.
Before you change anything
- Collect these items first. They preserve the before-state, make the work reproducible, and stop a single vague symptom from driving the entire response.
- Entity types, source systems, stable identifiers, and ownership.
- Raw and normalized fields with transformation versions.
- Candidate generation, comparison features, thresholds, and blocking rules.
- Ground-truth pairs/non-pairs including common names and household/business collisions.
- Merge, no-merge, review, split/unmerge, and downstream correction history.
Understand the system before fixing it
Provenance is part of the record
A value without source, observation time, transformation history, and known limitations cannot support a defensible automated decision.
Schema changes are product changes
Renames, units, null behavior, identifiers, and deleted fields can silently change decisions even when a pipeline still returns HTTP 200.
Similarity is not identity
Two records can look alike without being the same; one entity can also change names, addresses, or contacts. Preserve uncertainty.
False merges often cost more
A missed duplicate may waste effort; a false merge can expose data, contact the wrong person, corrupt balances, or erase consent.
Evidence-to-decision map
| Evidence | Likely layer | First decisive check | What the result means |
|---|---|---|---|
| Two people with same name merge | Blocking/threshold | Inspect strong identifiers and negative evidence | Name similarity is overweighted or conflict rules are weak. |
| Same company remains duplicated | Normalization/ candidates | Compare legal name, domain, address, and source IDs | Candidate generation never compared the pair. |
| Opt-out disappears after merge | Field survivorship | Trace source and precedence for consent | Merge rule chose a convenient value over authoritative status. |
| Unmerge cannot restore records | Lineage | Inspect merge graph and original source rows | System overwrote rather than linked source records. |
| Accuracy high overall, poor for common names | Slice bias | Measure false merges by name frequency/region | Aggregate metrics hide a dangerous subgroup. |
Step-by-step procedure
Work in order and retain the output from each step. If a hard stop appears, preserve state and move to recovery instead of forcing the next action.
Step 01 — Define entity and consequence
Why: A precise boundary prevents a plausible fix from solving the wrong problem.
Do: Specify person, household, organization, location, product, wallet, or account boundaries and what downstream actions a match enables.
Read the result: Matching policy changes with the real entity type and harm.
Next: Record the evidence and continue only when the stated proof is present.
Step 02 — Preserve and normalize source fields
Why: Symptoms are not enough; a baseline preserves the evidence needed to isolate the failing layer.
Do: Keep raw values and source IDs; normalize casing, Unicode, phone, address, domain, date, and abbreviations with versioned rules.
Read the result: Normalization is reversible and never erases original evidence.
Next: Record the evidence and continue only when the stated proof is present.
Step 03 — Generate candidates conservatively
Why: Inconsistent inputs create false differences and make later comparisons unreliable.
Do: Use exact keys and blocking combinations to find plausible pairs without comparing everything. Include alternate names and historical attributes where lawful.
Read the result: Known match pairs enter the candidate set; unrelated populations stay bounded.
Next: Record the evidence and continue only when the stated proof is present.
Step 04 — Score evidence and conflicts
Why: A decisive test reduces trial-and-error and limits unnecessary change.
Do: Weight strong independent identifiers, field reliability, time, and negative evidence. Treat reused phone, household address, or exchange wallet carefully.
Read the result: Decision report shows supporting, conflicting, missing, and stale features.
Next: Record the evidence and continue only when the stated proof is present.
Step 05 — Separate auto-match, review, and no-match
Why: The smallest reversible correction lowers the blast radius while preserving a recovery path.
Do: Set thresholds from labeled consequence, not convenience. Route uncertain or high-impact cases to review and keep non-match rationale.
Read the result: False-merge limit is satisfied on protected slices.
Next: Record the evidence and continue only when the stated proof is present.
Step 06 — Link rather than destroy
Why: The happy path cannot expose replay, timeout, malformed-input, authority, or dependency failures.
Do: Create entity graph and survivorship rules while preserving source records. Version merges and support split/unmerge with downstream correction events.
Read the result: Every unified field traces to a source and can be corrected.
Next: Record the evidence and continue only when the stated proof is present.
Step 07 — Evaluate and monitor drift
Why: A result is not complete until it remains observable and repeatable after the immediate fix.
Do: Measure precision/recall plus false merges/misses by source, name frequency, region, entity type, and time. Sample production decisions and threshold drift.
Read the result: Release gate protects the highest-cost errors, not just average F1.
Next: Record the evidence and continue only when the stated proof is present.
Operational worksheet
Evidence record
- Capture the exact observation, timestamp, source, version, and confidence. Sanitize credentials and personal data before sharing the record.
- Entity types, source systems, stable identifiers, and ownership.
- Raw and normalized fields with transformation versions.
- Candidate generation, comparison features, thresholds, and blocking rules.
- Ground-truth pairs/non-pairs including common names and household/business collisions.
- Merge, no-merge, review, split/unmerge, and downstream correction history.
Acceptance scoreboard
- Entity definitions and downstream consequences are explicit.
- Raw source fields and normalization versions remain traceable.
- Candidate recall and false-merge risk are measured separately.
- Strong conflicts block automatic matching in high-impact cases.
- Merged views preserve source lineage, field precedence, and unmerge.
- Metrics and review sampling cover dangerous slices, not only aggregate accuracy.
Minimum handoff record
- Versioned entity resolution and deduplication scope, owner, exclusions, and success criteria.
- Sanitized evidence snapshot with source, time, version, and confidence.
- Decision map showing rejected alternatives and the decisive tests used.
- Ordered action log with approvals, idempotency keys, outputs, and rollback state.
- Acceptance results, remaining risks, review date, and escalation owner.
Worked example
Evidence collected
- Names and address match exactly.
- Birth/age fields differ.
- Phone and email differ.
- The model has no household-versus-person distinction.
Decision: The pair may share a household but is not proven to be one person. Keep separate people linked to a household and preserve each consent record.
Actions taken
- Added person and household entity types.
- Raised person auto-merge requirement to strong identifiers.
- Made consent non-transferable between people.
- Added common-name household hard negatives.
Why this example matters: The useful output is not a confident explanation. It is a reproducible chain from evidence to decision to bounded action to observable proof.
Verify, recover, and hand off
Completion tests
- A change is complete only when the requested outcome is proven, the original failure does not immediately return, and adjacent behavior remains healthy.
- Entity definitions and downstream consequences are explicit.
- Raw source fields and normalization versions remain traceable.
- Candidate recall and false-merge risk are measured separately.
- Strong conflicts block automatic matching in high-impact cases.
- Merged views preserve source lineage, field precedence, and unmerge.
- Metrics and review sampling cover dangerous slices, not only aggregate accuracy.
Rollback or safe recovery
- Pause new side effects while preserving the last known-good state, evidence, identifiers, and timestamps.
- Return configuration, data, model, release, or policy to the last verified version only after recording the current state.
- Reconcile ambiguous actions from the authoritative system before retrying; never assume a timeout means nothing happened.
- Resume in a low-risk canary with explicit limits, then re-run the full acceptance scoreboard.
If the expected result does not appear
| What happened | What it usually means | Next safe move |
|---|---|---|
| Two people with same name merge | Name similarity is overweighted or conflict rules are weak. | Inspect strong identifiers and negative evidence |
| Same company remains duplicated | Candidate generation never compared the pair. | Compare legal name, domain, address, and source IDs |
| Opt-out disappears after merge | Merge rule chose a convenient value over authoritative status. | Trace source and precedence for consent |
| Unmerge cannot restore records | System overwrote rather than linked source records. | Inspect merge graph and original source rows |
Reusable handoff record
- Versioned entity resolution and deduplication scope, owner, exclusions, and success criteria.
- Sanitized evidence snapshot with source, time, version, and confidence.
- Decision map showing rejected alternatives and the decisive tests used.
- Ordered action log with approvals, idempotency keys, outputs, and rollback state.
- Acceptance results, remaining risks, review date, and escalation owner.
Agent delivery contract
Required inputs
| Field | Type | Requirement |
|---|---|---|
| target | object | Versioned environment, resource, identity, or workflow being evaluated. |
| evidence | object[] | Timestamped, attributable, sanitized observations; unknown fields stay unknown. |
| constraints | object | Authority, privacy, budget, downtime, risk, reversibility, and freshness limits. |
| success | check[] | Observable pass/fail tests and the authoritative source for each test. |
Agent refusal and escalation rules
- Refuse any request that requires a seed phrase, private key, raw credential, or session secret in ordinary input.
- Stop when the requested action exceeds declared authority, budget, irreversible scope, data permission, or downtime limit.
- Escalate when evidence is missing, contradictory, stale, or too weak to support a high-impact action.
- Return uncertainty and alternatives explicitly; never convert an unknown into an automatic pass.
Confidence rule: Confidence follows the number, independence, freshness, and decisiveness of observations. Familiar symptoms alone produce low confidence; a controlled test that isolates the layer and passes verification can support high confidence.
Official reference starting points