RAG quality starts before retrieval.
Most RAG optimization starts downstream: a better embedding model, reranking, a larger context window, a stronger prompt, or a newer LLM. Those choices matter, but every one operates on the representation produced during ingestion.
A technical PDF carries more than text: reading order, hierarchy, row/column relationships, captions, diagrams, formulas, code, footnotes, and sometimes scanned content. If parsing interleaves columns, flattens a table, drops OCR-only text, or discards source geometry, the retriever is searching a damaged version of the document.
That makes parsing an architectural quality boundary. The useful question is not only “Can the model answer?” but also “Did the pipeline preserve the evidence, retrieve it near the top, and keep enough provenance for a human to verify it?”
Same document. Same downstream stack. Different representation.
We created a deterministic ten-page Project Aurora Mission Readiness Report to stress the ingestion layer: two-column prose, nested headings, two multi-page verification tables, a readiness chart, an architecture diagram, a rotated scanned appendix, a formula, code, footnotes, and retrieval distractors.
PyMuPDF + fixed windows
- Native text extraction
- 3 fixed token windows
- 800-token max + 100-token overlap
- No OCR
- Broad page ranges
- No bounding-box provenance
Structured document intelligence
- Typed
DoclingDocument - Layout + reading order
- OCR + table structure
- Hierarchical + Hybrid chunking
- Page/item/bounding-box provenance
- Exact evidence overlays
No reranker, keyword boost, LLM judge, manual evidence injection, or selective rerun was added. The intended variable was document representation.
A working application, not a notebook-only benchmark.
The comparison was implemented end to end so the parser output, chunks, retrieval hits, generated claims, and highlighted evidence could all be inspected interactively.
The FastAPI service persists processing runs and a single background worker claims queued work from PostgreSQL. Streamlit exposes five views: Upload & Process, Parsing Comparison, Structure & Chunks, Ask & Verify, and Evaluation.
It preserved document structure instead of collapsing the PDF immediately into text.
Our benchmark pinned Docling 2.123.1 and used a quality-focused local standard pipeline. Remote Docling services were disabled.
Detects document elements and reading order.
Recovers rows, columns, cells, and relationships.
Recovers text from scanned/rotated page content.
Classifies charts, diagrams, and other figures.
Creates clearly labeled derived descriptions locally.
Preserves machine-searchable representations of technical content.
| Source content | Searchable representation | Evidence retained |
|---|---|---|
| Paragraphs / lists | Text + contextual headings | Item, page, bbox, char span |
| Complex tables | Structured row/column serialization | Table object, crop, refs, regions |
| Pictures / charts | Source caption + labeled derived description | Crop, classification, page region |
| Formula / code | Extracted representation | Source item + region |
| OCR content | Recognized source text | OCR-backed item + bbox |
Generated picture descriptions were retrieval aids, not verbatim evidence. Source captions and derived model descriptions remained visibly distinct.
Natural document structure and embedding constraints were treated as separate problems.
HierarchicalChunker followed document elements and hierarchy for inspection. HybridChunker then made those units embedding-friendly through token-aware merging and contextualization.
The largest contextualized Hybrid chunk was only 364 tokens under an 800-token ceiling; no production split was needed. We embedded the contextualized form while preserving raw source-oriented text separately.
Mental model: HierarchicalChunker organizes the filing cabinet. HybridChunker packs related files into boxes that fit through the embedding model's door.
Docling recovered evidence the baseline literally did not contain.
The rotated scanned appendix contained a simple but decisive fact:
PyMuPDF never extracted the scanned sentence, so none of the three indexed chunks contained “15 minutes.”
RapidOCR recovered the text; the cited chunk resolved to page 8, Docling item #/texts/121, and stored bounding-box regions.
The LLM did not become smarter between runs. The evidence became available. No amount of prompt tuning can reliably retrieve content that never entered the index.
A second failure mode: the words survived, but the relationship did not.
For REQ-205, the baseline retrieved a broad chunk containing the right terms at rank one, yet the answer associated the requirement with Guidance. The accepted relation was Navigation, verified by Analysis. Docling preserved the table association and answered the multi-part question correctly.
Retrieving a chunk with the right words is not the same as retrieving a representation with the right relationships.
The biggest gain was getting relevant evidence closer to rank one.
Fourteen questions, 12 answerable and two unsupported, ran once against both indexes. Relevance required expected-page intersection and all normalized expected terms. Answer matching was deterministic; no LLM judge was used.
Recall@3 tied at 91.7%. Because the baseline had only three broad chunks, Recall@5 is a weak discriminator here; Recall@1 and MRR better expose ranking quality.
GPT-5.1 cited evidence IDs. The application owned the source location.
The answer model received evidence IDs and textual evidence. It did not generate or invent document coordinates. After citation validation, the application resolved each ID through stored metadata.
| Evidence property | Baseline | Docling |
|---|---|---|
| Citation-ID integrity | 100% | 100% |
| Page/range accuracy | 100% | 91.7% |
| Exact region provenance | 0% | 100% |
| Visual evidence overlay | No | Yes |
One correct Docling REQ-207 answer included an extra claim cited to page 9 instead of the strict expected page 5. We keep that result visible because it demonstrates the point of provenance: verification metadata does not eliminate hallucination; it makes citation problems inspectable.
Docling was stronger overall, but not on every metric.
Where Docling was stronger
- Recovered the OCR-only fact.
- Recall@1: 75% vs. 41.7%.
- MRR: .836 vs. .667.
- Answer match: 100% vs. 83.3%.
- Table accuracy: 100% vs. 75%.
- Precise provenance: 100% vs. 0%.
- Much smaller query evidence context.
Where baseline held its own
- Recall@3 tied at 91.7%.
- Table-category MRR: .750 vs. .508.
- Broad page/range score: 100% vs. 91.7%.
- Code question tied.
- Parsing was dramatically faster.
A very broad chunk can rank well because it contains many candidate answers. That does not necessarily mean its representation is better. It can also mean more unrelated context and weaker traceability.
Our experiment used only part of Docling's current capability surface.
The measured results above come only from our pinned standard pipeline. Current Docling documentation exposes additional VLM and enrichment paths that could further change how difficult documents are represented. These are opportunities, not results from Project Aurora.
Docling now exposes Granite-Docling-258M as a recommended full-page VLM conversion option producing structured DocTags. That offers a different route for visually complex pages where one end-to-end model interprets the entire page.
Why it could help RAG → fewer hand-offs between page-analysis stages and potentially stronger end-to-end visual understanding.Docling supports chart extraction for bar, pie, and line charts, with structured outputs such as CSV and optional code/summary representations.
Why it could help RAG → a chart value becomes searchable structured evidence instead of relying primarily on captions or generic image descriptions.The current model catalog includes Granite Vision 4.1 4B for table-structure recognition via OTSL, alongside TableFormer.
Why it could help RAG → another parsing strategy for difficult or highly visual tables where cell relationships are the answer.Docling provides GPU-oriented examples, including serving Granite-Docling through vLLM for higher-throughput VLM processing.
Why it could help RAG → the 32.6-minute CPU ingestion time in our experiment is not the ceiling; hardware and runtime architecture are a separate optimiz