You ask an AI assistant how many people live in your town. It answers "48,312," without hesitation, down to the last digit. It looks serious. Yet that number may be made up, or long out of date.
This phenomenon has a name: hallucination. It's not a rare bug. It's a direct consequence of how these tools work. Once you understand that, you know exactly when to be wary.
It doesn't know. It generates.
ChatGPT, Claude, Gemini, Grok and Le Chat are built on a large language model (LLM). That model was trained on huge amounts of text for one task: predicting the most likely continuation of a text.
When you ask a question, it doesn't open a database of verified facts and read you the right entry. It produces, word by word, the answer that looks most plausible.
That ability has become very broad: rephrasing, explaining, planning, writing code. But it has three consequences:
- The answer is plausible, not guaranteed. A text can be very well written and wrong.
- It depends on what you give it. With your documents and good context, it makes far fewer mistakes.
- Without web search, it doesn't know recent events. The model knows nothing about what happened after its training.
In short: an excellent tool, not a source of truth.
Risky situations
Hallucinations show up mostly when:
- you ask for a precise figure (population, price, statistic, date);
- you ask for references (books, articles, case law, studies): it can invent a very believable title or author;
- the topic is recent or very local;
- the topic is niche and rarely covered in the training texts;
- you steer the question: "Why is X better than Y?" The assistant tends to agree with you. This is called sycophancy.
The overly precise number trap
The more precise an answer is, the more we want to believe it. That's exactly the trap. A round number raises suspicion; "48,312" lulls it to sleep.
Precise doesn't mean true.
Habits that protect you
1. Check what matters
A number, a date, a reference, a quote: check it against a reliable source before using it. The higher the stakes, the more you check. Health, money, law: the assistant is a starting point, not the decision.
2. Ask where the information comes from
With an assistant that searches the web, ask for the sources and open them. A source that doesn't exist, or doesn't say what it's claimed to say, is a clear warning sign.
3. Give it your own material
For a document, paste the document (anonymized if needed) and ask it to work only from that. Add: "For each piece of information, say where it appears in the text. If it isn't stated, write not stated." Pointing to the location lets you check, and it reduces made-up content.
For long texts and PDFs, our guide on how to summarize a text with AI walks you through the full method, including how to check the summary.
4. The known-answer test
Ask a question on the same topic whose answer you already know. If the assistant gets wrong what you know, be wary of what you don't.
5. Ask neutral questions
Instead of "Why is this method the best?", ask "What are the pros and cons of this method?"
And what you should never paste
Checking is also about protecting. Before pasting a text into an assistant, remove passwords, health data, bank details and information about identifiable people. When in doubt, anonymize before you paste.
Going further
These habits are at the heart of the Lantern island on Ovrelia: chapter 1 explains what a model really does, and the chapter "Doubt, secrecy, verification" trains you to spot errors in real cases and ends with your own verification protocol.
Chapter 1 is free and takes 20 minutes to play: Light the lantern.
To put these habits to the test, run the homemade Claude vs. ChatGPT test with a question you know the answer to: you'll quickly see which one admits doubt. At the office, the checklist in the guide to using ChatGPT at work keeps you from sending out a mistake. And to check what you've learned, launch the friend challenge: ten timed questions, and it's up to you to beat your friends' scores.
To move from theory to practice, follow the step-by-step method to fact-check an AI answer, then train with the free game Hallucination Hunt: your job is to spot the errors hidden in AI answers.
Frequently asked questions
What is an AI hallucination?
It's a wrong answer delivered with confidence: a number, a date, a source or a quote that doesn't exist. The model generates plausible text; it doesn't check facts.
Can you completely stop ChatGPT from making things up?
No. You can reduce the risk by giving it your own material and asking it to say when it doesn't know, but anything that matters must be checked against a reliable source.
Which answers should you check first?
Precise figures, dates, names, legal or medical references, quotes and sources. The more precise and important a piece of information is, the more it deserves checking.
Is asking for the source enough?
No. An assistant can invent a source too. Open the link or find the document yourself before relying on it.