An AI answer misses a table value that is clearly visible in the PDF. Before changing the model, inspect the text your application actually extracted. The relevant value may be missing, reordered, or detached from its heading.
A PDF is a presentation format. Its visual reading order and the order returned by a parser do not always match.
Inspect extraction at the page level
Choose a small fixture set containing plain text, multiple columns, a table, a scanned page, and a page with a diagram. Use documents you are permitted to process and fictional content for public examples.
Compare the page with the extracted output. PyMuPDF's text-extraction documentation discusses reading order and different extraction approaches. A successful parser call does not prove that the resulting text is usable for your question.
Look for failures that change meaning
| Extraction issue | Possible downstream effect |
|---|---|
| Columns interleaved | Sentences join unrelated passages |
| Table headers separated | A value loses its unit or category |
| Scanned page has no OCR | Relevant text never enters retrieval |
| Repeated page furniture | Search retrieves headers instead of content |
| Missing page reference | Citations cannot be checked reliably |
Do not clean the text so aggressively that you remove meaningful footnotes, negative signs, units, or warning labels. Readability is not the only objective; preserving the source meaning matters more.
Create an extraction acceptance sheet
For each fixture, record a few known facts and where they appear. Check whether the extraction preserves them with enough surrounding context to interpret them correctly.
For a fictional pricing table, test that the amount remains attached to the correct plan and billing period. For a procedure, test that numbered steps remain ordered. For a scan, inspect words that OCR is likely to confuse.
Keep page references or coordinates when your application needs source previews. A cleaned passage without provenance is harder to review when an answer is disputed.
Separate parsing, retrieval, and answer evaluation
If extraction lost the fact, improve extraction or acknowledge the unsupported page type. If the fact exists in text but was not retrieved, investigate indexing and search. If the right passage reached the model but the answer changed its meaning, evaluate generation.
The retrieval-versus-generation article explains this diagnostic split. Changing the largest model in the pipeline is not a useful first response to every failure.
OCR and layout-aware processing add cost and complexity. Use them where fixtures demonstrate the need, and expose unsupported material instead of pretending every PDF is handled equally.
My ChalkLens project provides an offline textbook-processing context. It does not establish that every document layout or scan quality is supported.
PDF extraction questions
Is a non-empty text result enough?
No. The important content may still be missing or in the wrong order.
Should every PDF go through OCR?
Not necessarily. Evaluate searchable text and scans separately before selecting the pipeline.
What should a bug report include?
Provide a permitted fixture page, expected fact, and extracted output. Those make a document-pipeline review concrete.