Test AI Table Summaries for Units and Denominators
Use a worked numerical audit to catch lost scales, mismatched populations, missing footnotes and invented rates in AI table summaries.
10/24/20263 min read
“Returns fell from 8 to 5” is not a usable account fact until the reader knows eight and five of what. Percentages, thousands of units, percentages of different populations and absolute counts can look similar after a table becomes prose. Test the numerical meaning, not merely whether the model copied the visible numbers.
A table-to-text evaluation should preserve the row subject, column period, unit, denominator and footnotes. It should also distinguish observations from calculations. The worksheet below is an original synthetic test for an internal research summary, not evidence of a product's measured accuracy.
Establish the table contract
Before generating prose, record what each column means. If the table uses a scale such as thousands, attach that scale to the column definition rather than relying on a title that may disappear during extraction. If rates use different denominators, write those definitions alongside the rows.
Use fictional Alderbridge Instruments. Its supplied quarterly table has two product families. For sensors, Q1 shipments are 2.0 thousand units and returned units are 40. For probes, Q1 shipments are 500 units and returned units are 15. A footnote says returns are counted within the quarter and may relate to earlier shipments. No revenue or defect-cause data is supplied.
The table deliberately mixes scales and includes a denominator limitation. A model may calculate 40 divided by 2,000, but it must not call the result a verified defect rate for the Q1 shipment cohort. The numerator and denominator are not established as the same population.
Fill the numerical audit sheet
T1, sensor shipments: source value 2.0; scale thousand units; normalized count 2,000; acceptable wording “2,000 sensor units shipped in Q1.”
T2, sensor returns: source value 40 units; period Q1; cohort unspecified; acceptable wording “40 sensor units returned during Q1.”
T3, probe shipments: source value 500 units; no thousands multiplier; acceptable wording “500 probe units shipped.”
T4, probe returns: source value 15 units; same cohort caveat; acceptable wording preserves the return period.
T5, derived ratio: 40/2,000 = 2%; permitted only as a clearly labeled arithmetic ratio of the supplied totals, not a cohort defect rate.
T6, missing revenue: no source column; output unknown or omit; reject any monetary claim.
For an illustrative output, “Sensors had a 2% defect rate, versus 3% for probes” fails. The arithmetic ratios can be computed from the numbers, but the labels assert defect causation and aligned populations that the fixture does not establish. A corrected summary describes shipments and returns separately and retains the caveat.
Add controlled mutations
Change only the sensor column heading from thousands to units, leaving the displayed number 2.0 unchanged. The normalized shipment count should change accordingly. If the summary still says 2,000, it is relying on a previous pattern rather than the supplied unit. Use a fresh test context to avoid accidental cross-case carryover.
Next swap the row order. Numbers should remain attached to the same product families. Then add a blank cell and a zero cell. The model should not treat a missing count as zero. Finally, place the footnote immediately below the table and confirm that the extraction stage includes it in the packet presented to the summarizer.
Record parsing failure separately from reasoning failure. If the model never received the unit header because a document converter dropped it, rewriting the summary prompt alone will not repair the workflow. Keep the rendered table and extracted input available to authorized reviewers.
Release only interpretable figures
NIST AI RMF 1.0 treats measurement as context-dependent. The proposed checks make that context explicit for tabular sales research. Require a reviewer to approve calculations that could influence a customer claim or commercial decision. If the table cannot establish comparable populations, report the limitation instead of producing a persuasive ranking. A concise summary that preserves units and uncertainty is more useful than a precise-looking percentage with the wrong meaning.
If you are also choosing research access, ask about Leads Automation’s independent Sales Navigator offer at US$15/month on WhatsApp. Confirm the available plan, eligibility and terms before ordering; Leads Automation is not affiliated with LinkedIn. This is not LinkedIn’s official price or an assurance that any particular integration is included.