Evaluate AI Extraction of Conflicting Dates
Use a synthetic date-conflict fixture to distinguish proposals, revisions, metadata and unresolved milestones without silently choosing a date.
10/24/20263 min read
When an AI extractor returns one neat date from a document containing two incompatible dates, tidy formatting can conceal an unresolved problem. The right output may be a conflict record rather than a calendar value. Evaluate whether the system recognizes the kind of date it found before judging whether it copied the digits correctly.
A document may contain a publication date, a proposed milestone, a contractual deadline and a later correction. These are different fields. This guide proposes a synthetic fixture for date-conflict extraction, with an answer key that preserves uncertainty instead of rewarding decisive guesses.
Specify the date the task actually asks for
Write a narrow extraction request: “Identify the agreed equipment acceptance date for project AX4 from the supplied packet. Return unresolved if the evidence does not establish one.” Do not ask simply for “the project date.” Require the output to include date role, source identifier, evidence passage and resolution state.
Define permitted states before testing: established, explicitly superseded, genuine conflict and insufficient context. A newer file timestamp alone is not proof that the underlying agreement changed. The evaluator should distinguish source chronology from the business event chronology without supplying an unverified hierarchy of authority.
The AX4 fixture
The fictional company North Pier Instruments provides four synthetic records. Their identifiers and contents are part of the test, not real procurement evidence.
D1, approved project note dated 3 March: “Equipment acceptance is scheduled for 18 April.” No version relationship is stated.
D2, working checklist dated 7 March: “Target acceptance: 25 April.” Its status is draft, with no approver.
D3, internal recap dated 9 March: “The acceptance date is still being confirmed; do not update the shared calendar yet.”
D4, file metadata: modified 11 March. The document body repeats D1 and contains no correction notice.
The expected answer is not 18 April or 25 April. It is “Acceptance date unresolved in the supplied packet: D1 states 18 April, D2 proposes 25 April, and D3 says confirmation is outstanding.” D4 establishes a file modification time, not a revised milestone. The calendar-write action remains outside the extraction task.
Use this filled record as the artifact: project AX4; field equipment acceptance; candidates D1/18 April/agreed-note wording and D2/25 April/draft target; conflict status unresolved; clarification needed from project owner; prohibited next step automatic calendar update; retained evidence D1–D4.
Add a correction without teaching a blanket newest-wins rule
Create a second fixture version with D5: “Authorized revision to AX4: replace the 18 April acceptance date with 25 April. This supersedes the earlier acceptance schedule.” State within the synthetic setup that the named approver has authority over that milestone. Now the expected result changes to 25 April, with D5 as the basis and D1 retained as superseded history.
Keep another variant in which D5 concerns delivery, not acceptance. The result should stay unresolved for acceptance. This catches a common shortcut: finding a later date and ignoring the event it describes. A third variant can omit the year; the extractor must not silently borrow the current year from the computer clock.
Review the explanation, not only the field
A correct date with the wrong supporting passage fails traceability. An unresolved status with no candidate dates is also incomplete because a person cannot efficiently resolve it. Score event identity, date value, authority evidence, supersession evidence and uncertainty separately. Any invented agreement is a blocking defect.
For example, an illustrative output that chooses 25 April “because the checklist is newer” fails the authority and uncertainty checks even though that date later becomes correct in the second fixture. Accidental correctness does not establish sound extraction behavior. Preserve which fixture version produced the output.
NIST AI RMF 1.0 emphasizes context in risk assessment. This fixture applies that broad principle to one operational ambiguity; it is not an official NIST test. Stop automated date use when the packet lacks a defensible resolution. A reviewer should ask one concrete question about the milestone, then record the answer and source before any separately authorized downstream change.
If you are also choosing research access, ask about Leads Automation’s independent Sales Navigator offer at US$15/month on WhatsApp. Confirm the available plan, eligibility and terms before ordering; Leads Automation is not affiliated with LinkedIn. This is not LinkedIn’s official price or an assurance that any particular integration is included.