Document AI Failure Gallery

Eleven pages that break OCR and document AI pipelines, one from each of our open synthetic benchmarks. Every page is generated, so the answer key is exact and there's no customer data anywhere.

For each one: what the page looks like, why pipelines fail on it, and an excerpt of the answer key.

A stamp lands on the customer name
01

A stamp lands on the customer name

A rotated APPROVED stamp sits on top of the bill-to name and address. OCR merges stamp ink with the text underneath, so the name comes out garbled, or the stamp text ends up inside the field.

Answer key (excerpt)

bill_to_nameJuniper Healthcare Systems Corp.
bill_to_address5984 Station Lane, Calgary, AB T2P 4Q6, Canada
total_amountCA$3,813.33

HyperAPI extract returned all 10 fields and all 5 line items correctly on this page, including both fields under the stamp.

Handwriting in boxes built for print
02

Handwriting in boxes built for print

Twelve fields filled in by hand on a printed claim form, plus margin notes ("verified ID", "see attached") that belong to no field. Digits like 1/7 and 0/6 are where extraction usually slips.

Answer key (excerpt)

claim_numberCLM-060721934
policy_numberPOL-56941089
date_of_birth10/13/1982
A fax-quality scan at 51 DPI
03

A fax-quality scan at 51 DPI

Severe degradation: scaled to about 51 effective DPI, blurred, noisy, low contrast, heavy JPEG compression, around 2,000 dust specks and a stamp on top. The answer key is unchanged from the clean page.

Answer key (excerpt)

po_numberPO-57717
date14/02/2025
supplier_nameCedar Lab Supplies Co.
A 1939-style business directory
04

A 1939-style business directory

Three dense columns of abbreviated listings in small serif type on aged paper. Read straight across the page and every listing merges with its neighbours, with no error to warn you.

Answer key (excerpt)

kinddirectory_listing
reading ordercolumn by column, top to bottom
labelsevery block with its bbox and order
Six kinds of PII on one intake form
05

Six kinds of PII on one intake form

Nine PII spans of six types (name, date of birth, address, phone, email, account number) spread across a patient intake form. A redactor that misses one leaves the page non-compliant, so the metric that matters is the leak rate.

Answer key (excerpt)

PII spans9
typesPERSON_NAME, DATE_OF_BIRTH, ADDRESS, PHONE, EMAIL, ACCOUNT_NUMBER
boxespixel boxes for every span
A balance sheet with five tables
06

A balance sheet with five tables

Five tables and 17 layout blocks on one page, with comparative-year columns. Parsers drop rows, merge adjacent tables or lose the column alignment between years.

Answer key (excerpt)

tables5, each with HTML ground truth
blocks17, with reading order
scoringCER, TEDS and Kendall tau
The title doesn't name the class
07

The title doesn't name the class

The page is titled "Aging Statement", but its class is statement of account, and the scan is degraded. A classifier that keys on the title gets it wrong.

Answer key (excerpt)

labelstatement_of_account
title_on_pageAging Statement
variantdegraded
Page 2
Page 3 (a new document)
08

Two P&L statements back to back

A 9-page bundle holding 7 documents. Pages 2 and 3 are both profit and loss statements, but they are different documents. Splitters that compare one page with the next see "same type" and merge them.

Answer key (excerpt)

documents7 in 9 pages
segmentscover_sheet 1 · profit_and_loss 2 · profit_and_loss 3-4 · credit_application 5 · nda 6-7 · profit_and_loss 8 · patient_intake_form 9
scoringboundary F1 and document F1
Invoice
09

A ledger-style invoice with 12 lines

A UK ledger layout with 12 line items in GBP. Line items are where extraction falls apart: rows split, descriptions wrap, and quantity × unit price has to match the line total.

Answer key (excerpt)

invoice_numberINV-2025-68021
line 1Calibration service · 4 × 316.16 = 1,264.64
totalGBP 7,446.28
Lab report
10

Not every document is a finance document

A pathology report from the enterprise set, which spans 31 document types. Nested results tables (test, result, units, reference range) need a schema, not a flat key-value list.

Answer key (excerpt)

lab_nameValemont Diagnostics GmbH
results[0]HbA1c · 7.5 % · ref < 5.7
results6 rows