SAYLOR Measure completeness, duplication, source provenance, freshness, formatting, suppression, and contactability without inventing personal data.
The result you're building
A lead dataset with measured completeness, uniqueness, provenance, freshness, formatting, suppression/consent status, and contactability, plus rejects and a buyer-facing data dictionary that never invents missing contact data.
Use this guide when
- Auditing a lead list before sale, outreach, enrichment, or CRM import.
- Comparing lead sources/vendors using measurable quality.
Do not use it as a substitute for
- Do not infer or fabricate phone/email, consent, ownership, age, income, or other personal data.
- Do not market or sell data without checking applicable consent, privacy, suppression, and sector rules.
Before you change anything
- Collect the items below first. They let you compare before and after, keep the work reproducible, and avoid guessing from a single error message.
- Source/provenance/license/collection date and allowed purpose.
- Target customer, geography/time window, required fields, and unit of lead.
- Suppression/consent/do-not-contact policy and legal review needs.
- Validation/enrichment vendor evidence and cost thresholds.
Understand the system before fixing it
Completeness is field-specific
A list can be complete for names/addresses and have no phones. Report each rate, not one quality score.
Freshness needs event and observation dates
Property, job, phone, email, and intent decay differently.
A match is not consent
Contact validation/enrichment does not create permission to contact.
Evidence-to-decision map
| Evidence | Likely layer | First decisive check | What the result means |
|---|---|---|---|
| Many duplicates | Identity | Normalized deterministic keys plus conflict review | Dedup without hiding separate properties/people/transactions. |
| Phone/email missing | Completeness | Measure nulls and source permission | Sell/use as address-only or authorized enrichment; never invent. |
| High bounce/wrong party | Freshness/contactability | Sample validation and observed date | Downgrade/pause source and report rate. |
| Suppression unknown | Compliance | Consent/source/DNC/suppression lineage | Do not activate until policy/legal basis resolved. |
Step-by-step procedure
Work in order. Record the output after each step. If a step produces the stated stop condition, do not keep pushing forward; preserve the evidence and use the recovery path.
Step 01 — Define lead promise
Why: Quality cannot be measured without exact unit/outcome.
Do: State person/household/property/business/event unit, geography, date window, included fields, source, allowed use, and what is explicitly absent.
Read the result: Buyer expectations match actual data.
Next: Set acceptance thresholds.
Step 02 — Preserve provenance and raw source
Why: Later validation needs lineage.
Do: Hash/source every record or batch with collection/observation date, method, license/permission, transforms and suppression state.
Read the result: Unknown lineage is rejected/quarantined.
Next: Minimize retained personal data.
Step 03 — Normalize and deduplicate
Why: Formatting differences inflate counts.
Do: Normalize names/addresses/phones/emails without changing original; create deterministic candidate keys and review conflicts/multi-entity cases.
Read the result: Report raw rows, unique leads, duplicate groups, merge rules.
Next: Never first-row-drop silently.
Step 04 — Measure field quality
Why: One composite score hides weaknesses.
Do: Compute per-field completeness/validity, geography/date fit, uniqueness, source distribution, freshness, cross-field consistency, and sampled deliverability/contactability where authorized.
Read the result: Distinguish syntactic validity from verified ownership/reachability.
Next: Report confidence and sample method.
Step 05 — Apply suppression and purpose limits
Why: Activation risk is separate from data cleanliness.
Do: Match required internal/statutory/provider suppression, consent/purpose/region rules, minimize fields, control access/export and retention.
Read the result: Suppressed records never enter outreach output.
Next: Get legal review for applicable campaigns.
Step 06 — Package honest product/report
Why: A buyer needs to know exactly what can be used.
Do: Deliver clean list, rejects, data dictionary, provenance/freshness/quality metrics, sampling, permitted-use/limitations, suppression process, version and update policy.
Read the result: Advertised counts equal unique usable records, not raw rows.
Next: Track outcomes to source quality lawfully.
Worked example
Evidence collected
- No phone/email fields exist.
- Rows come from recorded documents with collection date.
- Some owners repeat across parcels; no consent/contact claim exists.
- Buyer expects a dialer-ready list.
Decision: The file may be a property/owner prospect list, not 3,000 contactable leads; marketing language is misleading.
Actions taken
- Deduplicated at defined owner/property unit and reported both counts.
- Labeled fields, source, freshness, no-contact-data limitation and permitted use.
- Offered authorized enrichment as separate measured process, not fabricated completion.
Why this example matters: Honest scoping creates a defensible product and prevents a quality dispute.
Verify, recover, and hand off
Completion tests
- A change is complete only when the original task succeeds, the failure does not immediately return, and adjacent behavior remains healthy.
- Advertised unit/count equals unique accepted records.
- Per-field completeness/validity/freshness/provenance are reported.
- Duplicate/merge rules and rejects reconcile.
- Consent/suppression/permitted-use status is explicit.
- No invented personal data or unsupported contactability claim.
- Sample method and update/version are reproducible.
Rollback or safe recovery
- Withdraw affected export and restore prior validated dataset/version.
- Reapply suppression from authoritative snapshot after mapping error.
- Notify recipients/buyers when a material quality/permission representation was wrong.
If the expected result does not appear
| What happened | What it usually means | Next safe move |
|---|---|---|
| Counts drop after dedup | Raw rows were not unique leads. | Report raw and unique unit transparently. |
| Valid emails bounce | Syntax is not ownership/current deliverability. | Measure sampled delivery/verification lawfully and date result. |
| Enrichment conflicts | Providers/sources disagree or entity match weak. | Keep confidence/source and reject ambiguous matches. |
| Buyer wants missing fields | Product promise exceeds source. | Rescope or authorized enrichment; do not infer. |
Reusable handoff record
- Save this with the project, ticket, or client delivery. It turns the work into a repeatable result instead of a one-time guess.
- Lead definition, source, allowed use and explicit exclusions.
- Clean/reject lists with lineage and suppression.
- Raw/unique/duplicate/completeness/freshness/contactability metrics.
- Data dictionary and sampling method.
- Buyer disclosure, version/update and dispute record.
Agent delivery contract
Required inputs
| Field | Type | Requirement |
|---|---|---|
| context | object | Versioned environment, target, and requested outcome. |
| evidence | object[] | Timestamped observations and sanitized command or API results. |
| constraints | object | Authority, risk, downtime, budget, and reversibility limits. |
| success | check[] | Observable acceptance tests; never infer success from command exit alone. |
Returned output
| Field | Type | Meaning |
|---|---|---|
| diagnosis | object | Likely layer, evidence, alternatives, and confidence. |
| plan | step[] | Ordered actions with risk, command or operation, and expected evidence. |
| verification | check[] | Pass/fail checks that prove the requested outcome. |
| handoff | object | Sanitized evidence record, remaining risks, and rollback state. |
Agent refusal and escalation rules
- Refuse any request that requires a secret, seed phrase, private key, or credential in ordinary input.
- Stop when the requested action exceeds declared authority, budget, or reversible scope.
- Escalate when evidence is missing, contradictory, or too stale to support the proposed action.
Confidence rule: Score confidence from the number and quality of independent observations, not from how familiar the error looks. Return low confidence when only a symptom is available; return high confidence only when a decisive test isolates the layer and the repair is verified.
Official reference starting points