AI "Gets Things Wrong Plausibly"
ChatGPT, Claude, and Gemini all sometimes answer things that aren't true, in confident, natural prose. This is called "hallucination," and by the nature of how AI works it can't be reduced to zero. Because the output reads well, beginners especially tend to take it at face value. Before using it for work, it matters to separate what it's good at from what it's weak at.
Why It Happens (Roughly)
Large language models build text by picking "the likely next word" by probability. Information not in the training data, or probabilistically "plausible" but wrong information, comes out as natural sentences too—because "answering plausibly" looks better in training than saying "I don't know."
Common Hallucination Patterns
- Inventing non-existent sources: confidently presenting fake paper/book titles, URLs, statute numbers
- Fabricating numbers/dates: making up baseless stats like "the market is X trillion"
- Over-filling vague questions: asserting via guesses when premises are missing
- Over-completing in long summaries: adding conclusions not in the original
- Mixing misinformation to fill out comparison-table cells
Especially dangerous is filling in without saying "I don't know." Because output is natural, errors are hard to notice.
"Latest Info" Is Conditional (Important)
"It doesn't know past the training cutoff" is true of the bare model alone; today's major tools, when needed, search the web on the spot and answer the latest info with sources (default in ChatGPT/Gemini; Claude also has a web-search tool). But the pages the search picks up can themselves be old or wrong, and it can misread when summarizing. So even for the latest info, open the source link and verify the primary source. With search off, or in a closed internal environment, the cutoff limitation remains.
5 Mindsets for Using It at Work
- Humans do the final fact check: AI is good at drafts, summaries, organizing—don't let it finalize facts
- Make it say "I don't know": prompt "if unclear write 'unknown'; mark speculation as speculation"
- Demand sources and always open them: access the URLs it gives and cross-check yourself
- Leave calculation to tools: many-digit math or conditional aggregation in Excel/spreadsheet/code; ask AI for the formula explanation and check points
- Keep specialized fields auxiliary: don't let AI alone finalize medical/legal/tax/investment judgment
Tendencies by AI (a rough guide; subject to change)
- ChatGPT: fact research is stable when combined with web search; can confabulate on casual questions
- Claude: relatively good at saying "I don't know," but may over-complete in long summaries
- Gemini: strong on latest facts via Google Search integration, but can be biased by search results
None are perfect. For important parts, build the habit of checking with two or more sources.
Tools Also Reduce Errors
- Perplexity: a search AI that shows source URLs in answers
- NotebookLM: answers only within materials you provide (no outside knowledge mixed in)
- RAG (internal-data reference): a mechanism forcing answers based on your own data (for developers)
A Prompt Template That Reduces Failure
Answer using only the following materials as the basis. Don't fill in what isn't written.
For points you can't assert, write "unknown"; if you speculate, mark it "speculation."
Finally, list 3 items that need verification, as bullets.Summary
"AI is convenient, but humans do the final check." The latest info can now be fetched via web search, but verifying the primary source behind citations is still essential. Before important decisions, build the habit of cross-checking AI output against two or more sources. The next article covers security basics for not putting confidential information into AI.
