What AI actually is
When you type a question into a chatbot, nobody at the other end is looking the answer up. Something much stranger is happening. Here it is in five minutes, with no jargon.
It learned from a huge amount of text
AI is a big field, but on these pages, unless we say otherwise, "AI" means large language models (LLMs), and a chatbot is the form of LLM you're most likely to use.
Nobody wrote a chatbot a rule for every question. Instead it was trained on an enormous amount of text: books, websites, articles and more. During training it plays one game over and over. It sees some words, guesses the word that comes next, checks the real text, and adjusts itself a tiny amount. Repeat that enough times and it gets very good at the game.
Then comes a second stage. People write example answers and rank the chatbot's replies from best to worst, and it is trained again to give the kind of answer people rated highly. That is what turns a text-guesser into something that behaves like a helpful assistant.
Then it writes your answer one word at a time
When you ask something, the chatbot plays the same game. It looks at your question, scores every word that could come next, picks one, adds it to the end, and goes again. Your whole answer is built like that, word by word.
"One word at a time" can make it sound simple. It isn't. Researchers who look inside these models have found them planning ahead: when writing a rhyming poem, one picked the rhyme first and then wrote the line to land on it. So "it predicts the next word" is true, but it doesn't mean "it's not clever". It means something more useful to you: it produces what is likely.
Want to see how the prediction works in more detail? The Financial Times has a visual explainer: Generative AI exists because of the transformer.
Likely isn't the same as true
Most of the time, likely and true line up. "Paris" is the likely next word because a huge amount of text says so. That is why chatbots are so good at explaining things that have been explained thousands of times, like photosynthesis or how to find the area of a triangle.
The trouble starts when the pattern runs out. Take a fact that almost never appears in writing, like a particular person's birthday. There is no pattern to learn, but the chatbot still produces the most likely-looking answer. In one study, researchers asked a model for one of the authors' own birthday, saying "answer only if you know". In three tries it gave three different dates. All three were wrong.
It doesn't say "I don't know" as often as it should, either. The same researchers compare it to a student facing a hard exam question: a guess sometimes scores, "I don't know" never does, and the way chatbots are trained and tested has tended to reward the guess. So a wrong answer arrives sounding exactly as sure as a right one.
The one line to remember
It predicts what's likely, and likely isn't the same as true.
Watch the pattern for yourself
- Open a chatbot and start three new chats. In each one, type: "Finish this sentence: The best thing about weekends is". Compare the three answers.
- Now start three more new chats and ask each: "What is the capital of Australia?"
- Ask yourself: why did one question get a different answer each time, and the other the same answer? What does that tell you about when to double-check?
The weekend sentence has many likely endings, so you get a different one each time. The capital has one answer that a huge amount of text agrees on. When an answer changes each time you ask, that is a sign there is no strong pattern behind it, and a reason to check.
What this means for you
Trust a chatbot most where lots of writing agrees: explaining a common idea, giving examples, rewording something. Trust it least with rare facts, exact numbers, recent events and references to sources, which is exactly where "likely" and "true" come apart. Next, use what you now know to make a chatbot teach you rather than do the work for you.
Last reviewed: 25 September 2026