A retrieved number is useless when its column header disappears. Table questions require the assistant to preserve the relationships that make a cell meaningful: row, column, unit, time period, and footnotes.
Evaluate those relationships before scoring whether the final response contains the expected digits. A coincidentally correct number can hide a parser or retrieval problem that will fail on the next table.
Define the evidence unit around the cell
For a table test, capture the original table location and the smallest complete evidence unit needed to interpret the value. That may include a repeated header on another page, a currency note above the table, or a qualification below it.
Keep retrieval evaluation distinct from answer evaluation. Inspect the text or structure actually supplied to the model. If row labels and headers never arrive together, changing answer instructions is unlikely to repair the missing relationship consistently.
Use a fictional pricing table with a deliberate trap
Create a controlled document where the Basic plan lists 30 units monthly and 300 yearly, while the Team plan lists 80 monthly and 800 yearly. Add a footnote that yearly prices exclude an optional onboarding service. These invented values are test fixtures, not current product prices.
Ask for the Team yearly price, then the Basic monthly price, then whether onboarding is included. Record both the selected value and its supporting row, header, unit, billing period, and footnote. A response of “800 per month” should fail even though it extracted the correct cell's digits.
Add another table with the columns reversed. Rename the plans to similar labels. Put the footnote on the next page. Ask a question that needs a simple comparison but no calculation, followed by one that requires arithmetic. Review extraction and calculation as separate failure categories.
If arithmetic is allowed, define the operation and rounding rules explicitly in the reference answer. If the source lacks the inputs, the assistant should identify the gap instead of supplying a plausible total.
Compare parsing choices against the same cases
Plain-text conversion is easy to inspect but can flatten relationships. Structured table representations preserve more context but may consume more processing and require handling irregular layouts. Image-based inspection can help with some documents, yet it still needs checks for legibility and exact values.
Do not choose a representation from a clean sample alone. Review merged cells, multi-page tables, missing headers, decimal separators, negative values, and visually similar symbols. Report performance by table type so a strong result on simple grids does not conceal a weak result on financial footnotes.
This proposed matrix extends the source-citation workflow. The Nucleus project supplies relevant portfolio context; no extraction benchmark is claimed here.
Questions about numeric retrieval
Is exact string matching enough?
No. Require the correct value with the correct unit, time period, entity, and conditions. The same number can appear in several unrelated cells.
Should every table question include calculations?
No. Start with direct lookups to isolate extraction errors. Add calculations only after the evidence relationships survive retrieval.
What should a bug report include?
Attach a redacted table image, extracted context, expected cell coordinates, and actual answer. That evidence supports a focused retrieval review.