Detect Citation Drift in AI Account Notes
Audit each claim against its exact source passage, classify citation failures and repair unsupported additions before an account note is reused.
10/24/20263 min read
A citation can remain attached to a sentence after the sentence changes meaning. An editor shortens the wording, a second model adds an implication, or a source page gets replaced. The account note still looks referenced, but the cited passage no longer supports the claim. This is citation drift, and checking that a link opens will not detect it.
Audit claims at the passage level. The unit of review is a proposition tied to a specific source version, not a paragraph with several links at the end. A source may support one clause and leave the next entirely unproven.
Build a claim-to-passage table
Use fields for claim ID, exact claim text, source ID, passage locator, source version or capture date, supported scope, unsupported addition and disposition. Keep the source material within its permitted storage and reuse terms. A locator can identify a section and short excerpt; it need not reproduce an entire copyrighted document.
The disposition choices are retain, narrow, split, replace citation and remove. “Replace citation” is available only when a genuinely supporting source has been verified. It is not a request to search until something vaguely similar appears. When evidence cannot be accessed, mark the claim unverified.
Audit a synthetic account note
The fictional account Westhaven Components has three supplied passages. S1 states that it plans to open a service center in Lyon, subject to lease completion. S2 states that its existing support desk operates on weekdays. S3 is an industry article discussing why some manufacturers extend support hours during international expansion.
An illustrative AI draft says: “Westhaven is opening a Lyon center and will offer round-the-clock support to handle rising international demand.” It cites S1 and S3. Split that sentence into separately reviewable claims:
C1, “opening a Lyon center”: S1 supports a plan, not a completed commitment; narrow to “plans a Lyon service center, subject to lease completion.”
C2, “will offer round-the-clock support”: neither S1 nor S3 establishes Westhaven's schedule; remove.
C3, “rising international demand”: the packet supplies no account-level demand evidence; remove or mark a separate research question.
C4, “current desk operates weekdays”: S2 supports current hours, not future service-center hours; retain only with that scope.
The repaired note is shorter but more useful: it records the conditional center plan and existing desk hours, then asks whether the planned center changes support coverage. The question does not imply that an extension is already approved.
Classify how the citation failed
Use a small taxonomy so repeated problems can be fixed at the right stage. Wrong subject means the source concerns another company or an industry. Wrong time means a historical fact is presented as current. Missing qualifier means a condition was dropped. Unsupported extension means the source establishes A but the note adds B. Locator failure means the cited passage cannot be found in the identified version.
For C2 above, the primary defect is unsupported extension, with industry-to-company attribution as a contributing issue. Do not count every contributing label as a separate failed claim when reporting totals. Preserve both the claim count and the taxonomy counts so the numbers remain interpretable.
A broken URL is different from a false claim. The claim may be correct but presently unverifiable. Conversely, a working URL may support none of the sentence. Keep access status and evidential support in separate columns instead of treating either as a substitute for the other.
Recheck after substantive editing
NIST recommends verifying sources and citations in generated output. This audit is an original implementation choice, not a NIST certification. Rerun the affected rows when a claim changes scope, a source version changes or a summary is repurposed for a different audience. Cosmetic punctuation changes need not trigger a complete account investigation.
Finish with a release note: reviewed claims, unresolved claims, removed claims and reviewer. For Westhaven, the retained conditional statement is approved for the internal note; unsupported future coverage remains excluded. If the team later wants a customer-facing message, it must separately check current evidence, disclosure and authorization. A citation audit improves traceability; it does not authorize outreach or establish that a proposition remains true forever.
If you are also choosing research access, ask about Leads Automation’s independent Sales Navigator offer at US$15/month on WhatsApp. Confirm the available plan, eligibility and terms before ordering; Leads Automation is not affiliated with LinkedIn. This is not LinkedIn’s official price or an assurance that any particular integration is included.