Why AI gets things wrong
A chatbot is right a lot of the time, which is exactly what makes its mistakes easy to miss. They tend to fall into four kinds, and once you know them you'll start spotting them.
1. It makes things up
When there's no strong pattern to follow, a chatbot still produces something that looks right. That includes facts, quotes and sources. In 2023, two lawyers in New York handed a court a document citing earlier cases a chatbot had suggested. Six of those cases did not exist. A judge fined the lawyers and their firm $5,000.
If professionals can be caught out, so can a school project. A made-up source looks exactly like a real one until you go looking for it.
2. It slips on counting and exact working
Researchers asked one chatbot how many times the letter D appears in "DEEPSEEK". In ten tries it said 2 or 3. The answer is 1. A chatbot writes what is likely, and a likely-looking number is not a counted one. Treat any exact number, sum or count as something to check, not something to copy.
3. It can be out of date
A chatbot learned from text up to a certain date. Unless it searches the web, it doesn't know what happened after that, and it won't always tell you so. Anything that changes, like news, records, rules or prices, needs checking against a current source.
4. It repeats the bias in what it learned from
A chatbot learns from writing made by people, and that writing carries people's stereotypes. A 2024 UNESCO study found that in stories one widely used model wrote, women were described in domestic roles four times as often as men. The model wasn't told to do that. It learned it.
Bias is harder to spot than a wrong number, because nothing in the answer is obviously false. Learning to spot it means looking beyond whether an answer is correct and asking whose perspective shaped it, what might have been left out, and what other sources have to say.
The one line to remember
Confident doesn't mean correct.
A wrong answer arrives in the same calm, tidy style as a right one. The tone tells you nothing. Checking does.
Test it on something you already know
Ask a chatbot a question you know the answer to, from a subject you're good at or a topic in your notes. Read the answer line by line, as if you were marking it. Did it get everything right? Did it add anything your notes don't say? Which of the four kinds of mistake would have been easiest to miss?
Now practise
Spotting mistakes is a skill, and it improves with practice. The practice page gives you worked answers where every sum is correct and something is still wrong.
Last reviewed: 25 September 2026