SAYLOR Convert PDFs, scans, office files, tables, and images into faithful text and structure while preserving pages, reading order, uncertainty, and visual evidence.
The result you're building
A document-extraction record that identifies native text versus OCR, preserves page and bounding-box provenance, detects broken reading order and tables, exposes confidence, and routes uncertain pages to visual review.
Use this guide when
- Agents read PDFs, scans, receipts, forms, filings, manuals, or mixed-layout reports.
- Page structure or tables matter to the answer.
- OCR mistakes could change money, dates, identities, instructions, or legal meaning.
Do not use it as a substitute for
- Trusting plain text extraction without rendering the pages.
- Using OCR confidence as proof of semantic correctness for high-impact fields.
Before you change anything
- Collect these items first. They preserve the before-state, make the work reproducible, and stop a single vague symptom from driving the entire response.
- Original file hash, MIME/magic type, page count, encryption/signature, and source.
- Rendered page images plus native text/object inventory.
- OCR engine, language, preprocessing, version, and confidence by region.
- Reading order, columns, tables, forms, headers/footers, and page locators.
- Ground-truth sample and field-level error measurements.
Understand the system before fixing it
Provenance is part of the record
A value without source, observation time, transformation history, and known limitations cannot support a defensible automated decision.
Schema changes are product changes
Renames, units, null behavior, identifiers, and deleted fields can silently change decisions even when a pipeline still returns HTTP 200.
Rendering and extraction answer different questions
Text extraction finds characters; page rendering reveals clipping, scans, columns, annotations, crossed-out values, signatures, and layout relationships.
Field risk is not average OCR accuracy
One wrong decimal, date, account, negative sign, or 'not' can outweigh thousands of correctly recognized words.
Evidence-to-decision map
| Evidence | Likely layer | First decisive check | What the result means |
|---|---|---|---|
| Text is empty but page looks normal | Scan/image | Inspect page objects and render | The page contains pixels rather than extractable text. |
| Sentences mix across columns | Reading order | Compare tokens and bounding boxes to page layout | Extractor linearized multi-column content incorrectly. |
| Table values lose row labels | Structure | Render and test cell associations | Plain text cannot preserve the table relationship. |
| OCR says 8 where image says 3 | Recognition | Crop critical field and compare engines/manual review | Confidence or preprocessing is insufficient for the field. |
| PDF render differs by tool | Fonts/transparency | Compare reference renderer and embedded resources | Missing font, transparency, annotation, or malformed object affects appearance. |
Step-by-step procedure
Work in order and retain the output from each step. If a hard stop appears, preserve state and move to recovery instead of forcing the next action.
Step 01 — Fingerprint and inventory the document
Why: A precise boundary prevents a plausible fix from solving the wrong problem.
Do: Record hash, source, MIME/magic, size, pages, encryption, signatures, attachments, fonts, images, annotations, forms, and existing text layers.
Read the result: Inventory matches independent page count and no content is silently skipped.
Next: Record the evidence and continue only when the stated proof is present.
Step 02 — Render every page
Why: Symptoms are not enough; a baseline preserves the evidence needed to isolate the failing layer.
Do: Use a maintained PDF renderer at a declared DPI; retain page images for review. Detect blank, rotated, clipped, oversized, or corrupted pages.
Read the result: Rendered pages are legible and correspond one-to-one with source pages.
Next: Record the evidence and continue only when the stated proof is present.
Step 03 — Select native extraction or OCR per page
Why: Inconsistent inputs create false differences and make later comparisons unreliable.
Do: Prefer reliable native text; OCR image-only regions using declared languages and preprocessing. Do not overwrite source or merge duplicate text layers blindly.
Read the result: Each span is labeled native or OCR with engine/version and coordinates.
Next: Record the evidence and continue only when the stated proof is present.
Step 04 — Reconstruct reading order and structure
Why: A decisive test reduces trial-and-error and limits unnecessary change.
Do: Detect columns, headings, lists, footnotes, tables, checkboxes, labels, and key-value regions. Preserve page and bounding-box locators.
Read the result: Sample reading order and table cells match visual review.
Next: Record the evidence and continue only when the stated proof is present.
Step 05 — Validate high-impact fields
Why: The smallest reversible correction lowers the blast radius while preserving a recovery path.
Do: Define types and rules for names, dates, currency, IDs, units, negatives, and totals. Cross-check redundant values and route low confidence to review.
Read the result: Critical fields meet stricter field-level thresholds than body text.
Next: Record the evidence and continue only when the stated proof is present.
Step 06 — Measure against ground truth
Why: The happy path cannot expose replay, timeout, malformed-input, authority, or dependency failures.
Do: Manually label representative pages and compute character/word error plus table and field accuracy by layout class, language, and scan quality.
Read the result: Reported quality includes failure slices, not a single average.
Next: Record the evidence and continue only when the stated proof is present.
Step 07 — Publish with visual traceability
Why: A result is not complete until it remains observable and repeatable after the immediate fix.
Do: Return page, locator, extraction method, confidence, source hash, limitations, and optionally page crops for cited fields. Retain correction history.
Read the result: A reviewer can jump from any important value to the exact visual evidence.
Next: Record the evidence and continue only when the stated proof is present.
Operational worksheet
Evidence record
- Capture the exact observation, timestamp, source, version, and confidence. Sanitize credentials and personal data before sharing the record.
- Original file hash, MIME/magic type, page count, encryption/signature, and source.
- Rendered page images plus native text/object inventory.
- OCR engine, language, preprocessing, version, and confidence by region.
- Reading order, columns, tables, forms, headers/footers, and page locators.
- Ground-truth sample and field-level error measurements.
Acceptance scoreboard
- Source hash, page count, render, and extraction inventory reconcile.
- Every page is visually inspectable and no page is silently omitted.
- Native and OCR spans retain page, bounding box, engine, and confidence.
- Reading order and tables pass representative visual tests.
- Critical fields use type, redundancy, and stricter confidence checks.
- Quality report includes ground truth, failure slices, corrections, and limitations.
Minimum handoff record
- Versioned document parsing, ocr, and layout quality scope, owner, exclusions, and success criteria.
- Sanitized evidence snapshot with source, time, version, and confidence.
- Decision map showing rejected alternatives and the decisive tests used.
- Ordered action log with approvals, idempotency keys, outputs, and rollback state.
- Acceptance results, remaining risks, review date, and escalation owner.
Worked example
$3,800.00 as $8,800.00 after low-resolution OCR.Evidence collected
- The PDF is an image-only scan.
- Average OCR confidence is high, but the first digit is low-confidence.
- The line-item sum equals $3,800.
- The pipeline does not validate totals or retain field crops.
Decision: The critical amount must be rejected and reviewed. Field-level validation and arithmetic evidence outweigh average OCR confidence.
Actions taken
- Rendered at higher DPI and cropped the amount region.
- Cross-checked subtotal, tax, and total.
- Added currency validation and low-confidence field review.
- Returned the page locator and crop with the corrected value.
Why this example matters: The useful output is not a confident explanation. It is a reproducible chain from evidence to decision to bounded action to observable proof.
Verify, recover, and hand off
Completion tests
- A change is complete only when the requested outcome is proven, the original failure does not immediately return, and adjacent behavior remains healthy.
- Source hash, page count, render, and extraction inventory reconcile.
- Every page is visually inspectable and no page is silently omitted.
- Native and OCR spans retain page, bounding box, engine, and confidence.
- Reading order and tables pass representative visual tests.
- Critical fields use type, redundancy, and stricter confidence checks.
- Quality report includes ground truth, failure slices, corrections, and limitations.
Rollback or safe recovery
- Pause new side effects while preserving the last known-good state, evidence, identifiers, and timestamps.
- Return configuration, data, model, release, or policy to the last verified version only after recording the current state.
- Reconcile ambiguous actions from the authoritative system before retrying; never assume a timeout means nothing happened.
- Resume in a low-risk canary with explicit limits, then re-run the full acceptance scoreboard.
If the expected result does not appear
| What happened | What it usually means | Next safe move |
|---|---|---|
| Text is empty but page looks normal | The page contains pixels rather than extractable text. | Inspect page objects and render |
| Sentences mix across columns | Extractor linearized multi-column content incorrectly. | Compare tokens and bounding boxes to page layout |
| Table values lose row labels | Plain text cannot preserve the table relationship. | Render and test cell associations |
| OCR says 8 where image says 3 | Confidence or preprocessing is insufficient for the field. | Crop critical field and compare engines/manual review |
Reusable handoff record
- Versioned document parsing, ocr, and layout quality scope, owner, exclusions, and success criteria.
- Sanitized evidence snapshot with source, time, version, and confidence.
- Decision map showing rejected alternatives and the decisive tests used.
- Ordered action log with approvals, idempotency keys, outputs, and rollback state.
- Acceptance results, remaining risks, review date, and escalation owner.
Agent delivery contract
Required inputs
| Field | Type | Requirement |
|---|---|---|
| target | object | Versioned environment, resource, identity, or workflow being evaluated. |
| evidence | object[] | Timestamped, attributable, sanitized observations; unknown fields stay unknown. |
| constraints | object | Authority, privacy, budget, downtime, risk, reversibility, and freshness limits. |
| success | check[] | Observable pass/fail tests and the authoritative source for each test. |
Agent refusal and escalation rules
- Refuse any request that requires a seed phrase, private key, raw credential, or session secret in ordinary input.
- Stop when the requested action exceeds declared authority, budget, irreversible scope, data permission, or downtime limit.
- Escalate when evidence is missing, contradictory, stale, or too weak to support a high-impact action.
- Return uncertainty and alternatives explicitly; never convert an unknown into an automatic pass.
Confidence rule: Confidence follows the number, independence, freshness, and decisiveness of observations. Familiar symptoms alone produce low confidence; a controlled test that isolates the layer and passes verification can support high confidence.
Official reference starting points