How to check whether an AI hallucinated a citation

Why AI tools invent citations

Language models generate text by predicting what plausibly comes next, and a plausible-sounding reference is easy to predict: real journal names, real researchers in the field, a believable title, a well-formed DOI. Nothing in the generation process checks that the assembled reference corresponds to an actual publication. The result is a citation that passes visual inspection and fails verification — and courts, journals, and universities have now seen thousands of them.

Warning signs of a fabricated reference

• The DOI doesn't resolve, or resolves to a completely different paper.
• The exact title returns nothing on Google Scholar, Crossref, or the journal's own site.
• Almost-right details — real authors who never co-wrote together, a real journal with the wrong volume, a title blending two actual papers.
• It's suspiciously perfect — a single reference that supports the claim exactly, in the ideal venue, with round page numbers.

Pass one: verify the reference exists

1. Resolve the DOI. Paste it into doi.org. No DOI in the reference is itself a flag for recent journal articles.
2. Search the exact title in quotes on Google Scholar and Crossref; OpenAlex and PubMed (for biomedical work) are good second checks.
3. Cross-check every field of a match: authors, year, journal, volume, pages. A hit on the title with mismatched authors is a fabrication, not a typo.

Pass two: verify the source supports the claim

This is the check most people skip, and it's where AI-assisted writing fails most often. A model that has genuinely “seen” a real paper in training will still routinely misremember what it found, overstate its conclusions, or attach it to a claim it never addressed. The reference survives pass one; the citation is still wrong.

For each claim the citing sentence makes, open the source and find the passage the claim rests on. Read it in context — especially any hedges (“may,” “in this sample,” “under these conditions”) that the citing sentence dropped. If you can't find a passage that supports the claim, treat the citation as unsupported no matter how real the paper is. Our general guide to verifying citations covers this step in detail.

Checking a whole paper at once

Hand-checking two passes across a full bibliography is hours of work, which is why fabricated references keep slipping through. CiteFirm runs both passes systematically: it extracts every citation from the uploaded PDF, resolves each reference against scholarly metadata — unresolvable references, the signature of fabrication, get flagged — then splits each citing sentence into its individual claims and finds the relevant passage in each cited source so you can judge support directly. CiteFirm never declares a citation genuine or false itself; it puts the evidence in front of you and the verdict is yours, exported as a shareable verification record.

If you're reviewing someone else's work

Editors, peer reviewers, and instructors don't need to prove a paper was AI-written to act on a bad citation: an unresolvable reference or an unsupported claim is a concrete, checkable defect either way. Verify the citations, not the authorship — it's both more chartable and more useful.