The result you are building
Finished Result:
A lead dataset with measured completeness, uniqueness, provenance, freshness, formatting, suppression/consent status, and contactability, plus rejects and a buyer-facing data dictionary that never invents missing contact data.
Use this guide when
- Auditing a lead list before sale, outreach, enrichment, or CRM import.
- Comparing lead sources/vendors using measurable quality.
Do not use it as a substitute for
- Do not infer or fabricate phone/email, consent, ownership, age, income, or other personal data.
- Do not market or sell data without checking applicable consent, privacy, suppression, and sector rules.
Before you change anything
Collect the items below first. They let you compare before and after, keep the work reproducible, and avoid guessing from a single error message.
- Source/provenance/license/collection date and allowed purpose.
- Target customer, geography/time window, required fields, and unit of lead.
- Suppression/consent/do-not-contact policy and legal review needs.
- Validation/enrichment vendor evidence and cost thresholds.
Stop Before Proceeding:
Stop use or sale when provenance/permission is unknown, suppression cannot be honored, sensitive fields exceed purpose, or the file materially misrepresents contactability/freshness.
Understand the system before fixing it
Completeness is field-specific A list can be complete for names/addresses and have no phones. Report each rate, not one quality score.
Freshness needs event and observation dates Property, job, phone, email, and intent decay differently.
A match is not consent Contact validation/enrichment does not create permission to contact.
Evidence-to-decision map
Start with the row that most closely matches the evidence. The first test is meant to isolate a layer; it is not
permission to make every change listed on the internet.
Evidence Likely layer First decisive check What the result means
Many duplicates Identity Normalized deterministic keys Dedup without hiding separate
plus conflict review properties/people/transactions.
Phone/email missing Completeness Measure nulls and source Sell/use as address-only or authorized
permission enrichment; never invent.
High bounce/wrong Freshness/cont Sample validation and observed Downgrade/pause source and report rate.
party actability date
Suppression unknown Compliance Consent/source/DNC/suppressio Do not activate until policy/legal basis
n lineage resolved.Step-by-step procedure
Work in order. Record the output after each step. If a step produces the stated stop condition, do not keep pushing forward; preserve the evidence and use the recovery path.
01 Define lead promise Why: Quality cannot be measured without exact unit/outcome.
Do: State person/household/property/business/event unit, geography, date window, included fields, source, allowed use, and what is explicitly absent.
Read the result: Buyer expectations match actual data.
Next: Set acceptance thresholds.
02 Preserve provenance and raw source Why: Later validation needs lineage.
Do: Hash/source every record or batch with collection/observation date, method, license/permission, transforms and suppression state.
Read the result: Unknown lineage is rejected/quarantined.
Next: Minimize retained personal data.
03 Normalize and deduplicate Why: Formatting differences inflate counts.
Do: Normalize names/addresses/phones/emails without changing original; create deterministic candidate keys and review conflicts/multi-entity cases.
Read the result: Report raw rows, unique leads, duplicate groups, merge rules.
Next: Never first-row-drop silently.
04 Measure field quality Why: One composite score hides weaknesses.
Do: Compute per-field completeness/validity, geography/date fit, uniqueness, source distribution, freshness, cross-field consistency, and sampled deliverability/contactability where authorized.
Read the result: Distinguish syntactic validity from verified ownership/reachability.
Next: Report confidence and sample method.
05 Apply suppression and purpose limits Why: Activation risk is separate from data cleanliness.
Do: Match required internal/statutory/provider suppression, consent/purpose/region rules, minimize fields, control access/export and retention.
Read the result: Suppressed records never enter outreach output.
Next: Get legal review for applicable campaigns.
06 Package honest product/report Why: A buyer needs to know exactly what can be used.
Do: Deliver clean list, rejects, data dictionary, provenance/freshness/quality metrics, sampling, permitted-use/limitations, suppression process, version and update policy.
Read the result: Advertised counts equal unique usable records, not raw rows.
Next: Track outcomes to source quality lawfully.
Worked example
Starting Problem:
A seller advertises 3,000 mortgage-protection leads, but file has names/addresses only.
Evidence collected
- No phone/email fields exist.
- Rows come from recorded documents with collection date.
- Some owners repeat across parcels; no consent/contact claim exists.
- Buyer expects a dialer-ready list.
Decision The file may be a property/owner prospect list, not 3,000 contactable leads; marketing language is misleading.
Actions taken
- Deduplicated at defined owner/property unit and reported both counts.
- Labeled fields, source, freshness, no-contact-data limitation and permitted use.
- Offered authorized enrichment as separate measured process, not fabricated completion.
Proof Of Completion:
Buyer-facing description matches file; unique/count metrics reconcile; no dialer-ready or consent claim is made; rejects/provenance are included.
Why this example matters Honest scoping creates a defensible product and prevents a quality dispute.
Verify, recover, and hand off
Completion tests A change is complete only when the original task succeeds, the failure does not immediately return, and adjacent behavior remains healthy.
- Advertised unit/count equals unique accepted records.
- Per-field completeness/validity/freshness/provenance are reported.
- Duplicate/merge rules and rejects reconcile.
- Consent/suppression/permitted-use status is explicit.
- No invented personal data or unsupported contactability claim.
- Sample method and update/version are reproducible.
Rollback or safe recovery
- Withdraw affected export and restore prior validated dataset/version.
- Reapply suppression from authoritative snapshot after mapping error.
- Notify recipients/buyers when a material quality/permission representation was wrong.
If the expected result does not appear What happened What it usually means Next safe move
Counts drop after dedup Raw rows were not unique leads. Report raw and unique unit transparently.
Valid emails bounce Syntax is not ownership/current Measure sampled delivery/verification lawfully deliverability. and date result.
Enrichment conflicts Providers/sources disagree or entity Keep confidence/source and reject ambiguous match weak. matches.
Buyer wants missing fields Product promise exceeds source. Rescope or authorized enrichment; do not infer.
Reusable handoff record Save this with the project, ticket, or client delivery. It turns the work into a repeatable result instead of a one-time guess.
- Lead definition, source, allowed use and explicit exclusions.
- Clean/reject lists with lineage and suppression.
- Raw/unique/duplicate/completeness/freshness/contactability metrics.
- Data dictionary and sampling method.
- Buyer disclosure, version/update and dispute record.
Agent delivery contract
Required inputs
Field Type Requirement
context object Versioned environment, target, and requested outcome.
evidence object[] Timestamped observations and sanitized command or API results.
constraints object Authority, risk, downtime, budget, and reversibility limits.
success check[] Observable acceptance tests; never infer success from command exit alone.
Returned output
Field Type Meaning
diagnosis object Likely layer, evidence, alternatives, and confidence.
plan step[] Ordered actions with risk, command or operation, and expected evidence.
verification check[] Pass/fail checks that prove the requested outcome.
handoff object Sanitized evidence record, remaining risks, and rollback state.
Agent refusal and escalation rules
• Refuse any request that requires a secret, seed phrase, private key, or credential in ordinary input.
• Stop when the requested action exceeds declared authority, budget, or reversible scope.
• Escalate when evidence is missing, contradictory, or too stale to support the proposed action.
Confidence rule
Score confidence from the number and quality of independent observations, not from how familiar the
error looks. Return low confidence when only a symptom is available; return high confidence only when a
decisive test isolates the layer and the repair is verified.Official reference starting points
- https://www.ftc.gov/business-guidance/privacy-security
- https://www.consumerfinance.gov/compliance/compliance-resources/mortgage-resources/