Free sample lesson ยท AI Foundations
There Is No Answer Database
It has no store of answers, so where do the sentences come from?
Lesson 1 of the Wondering course AI Foundations
What You Will Explore
- Prediction
- Probabilities
- Reliability
Discover how AI generates text through statistical probability rather than looking up facts in a database.
The Myth of the Database
Many people believe AI works like a search engine. They imagine a massive library where the model looks up facts and copies them out.
This is not how it works. There is no database of answers inside the system.
Instead, the model is a prediction machine. It generates text by guessing what comes next based on patterns it learned during training.
- Database View
- Prediction View
Predicting the Next Step
The model builds sentences one small chunk at a time. This process is called next-token prediction.
Imagine the phrase: "The capital of France is". The model does not check a geography table to find the answer.
It predicts "Paris" because that word followed those specific words most often in its training data. It appends the word and then predicts the next one.
- Chunk by chunk: The model builds sentences one small piece at a time.
- No table lookup: It relies on statistical frequency from training data instead of checking a table.
- Append and repeat: It appends the selected word and immediately predicts the next one.
A Map of Probabilities
At every single step, the model looks at its entire vocabulary. It assigns a percentage of likelihood to every possible word.
For the prompt "The best thing about Monday is", the distribution might look like this: "coffee" at 40%, "nothing" at 30%, and "the start" at 20%.
What you read is just one path sampled through these probabilities. The model holds many possibilities at once before choosing one.
- Coffee: Most likely continuation.
- Nothing: Second most likely option.
- The start: Third option in the distribution.
Fluency is Not Truth
Because the model predicts likely text, it is very good at being fluent. However, being plausible is not the same as being correct.
A model can answer a question it never saw in training. It simply follows the statistical trail of the words you provided.
Understanding this helps you see why AI can sound confident even when it is making a mistake. It is optimizing for patterns, not for facts.
- Confident Output: Sounds authoritative and persuasive.
- Statistical Trail: Follows patterns of words provided.
- Plausible Text: Optimizes for fluency rather than factual truth.
Each layer builds on statistical text generation
Putting It All Together
- Identify the source: Recognize that the model is generating fluent text from probabilities, not retrieving a verified fact.
- Evaluate plausibility: Do not mistake smooth phrasing for factual accuracy.
- Apply healthy skepticism: Anticipate failure modes where confident tone masks complete fabrication.
Check yourself
An AI language model generates text by _____ rather than querying a database of stored facts.
- predicting the most likely next token
- retrieving pre-written answers
Answer: predicting the most likely next token
AI language models are prediction machines that estimate word probabilities based on training patterns rather than looking up verified answers in a database.
Before a specific word is chosen during generation, what does the model produce across its vocabulary?
- A probability distribution assigning likelihood scores to all possible words
- A single definitive answer retrieved from internal training tables
- A list of factual corrections to verify external truth
- A completed sentence structure ready for immediate formatting
Answer: A probability distribution assigning likelihood scores to all possible words
At each step, the model assigns statistical weights (probabilities) to every word in its vocabulary. The final output word is just one path sampled from this probability distribution.
Keep going
This is lesson 1 of 19 in AI Foundations. Next up:
- Where The Patterns Came From
- Working Memory, Not Storage
- The Middle Gets Lost
- A Predictor Is Not An Assistant
- Why It Stops At A Date
Wondering turns any topic into short lessons you can read, watch, or listen to, with practice questions and an AI tutor.