There Is No Answer Database

The torn lyric sheet

Imagine finding a lyric sheet with half the words torn away.

You could probably fill in a good deal of it. Not because you memorised the song, but because you know how language works, how that kind of song goes, and what usually follows what.

You would be predicting, word by word, from everything visible.

That is not an analogy for how a language model works. It is what a language model does.

One chunk at a time

When you ask a question, the model does not compose an answer and then write it out.

It predicts the first small chunk. Adds it to what it has. Predicts the next chunk from the new total. Adds that. Continues until it predicts a stopping point.

Each chunk is chosen with no plan for the sentence it is in. The coherence you read emerges because every step is conditioned on everything before it, not because anything was outlined in advance.

This is why it can start an answer before it has worked out where the answer ends up.

It is holding many options at once

At each step, the model does not produce a word. It produces a probability for every word it knows.

After "The capital of France is", the probability on Paris is overwhelming. Almost everything else is near zero.

After "The best thing about Monday is", the spread is wide. Coffee, nothing, that, the, dozens of reasonable continuations, none dominant.

Then one gets picked from that distribution and the process repeats.

That spread matters more than it looks. When the distribution is sharp, answers are stable and usually right. When it is flat, the model is much closer to guessing, and it reads exactly the same to you.

Why this explains the strangeness

Two things that seem contradictory now come from the same mechanism.

It writes beautifully about things it gets wrong. Fluency comes from predicting likely text. Being correct is a separate property that prediction does not guarantee.

It answers questions nobody ever wrote down. Ask for a bedtime story about a robot dentist and you get one, because it is recombining learned patterns rather than searching for a match.

The same machine does both. If it were looking things up, it could do neither.

The consequence to carry

If you take one thing from this block:

Fluent and correct are unrelated properties.

A lookup system fails visibly. It returns nothing, or something obviously irrelevant.

A prediction system fails invisibly, producing something that reads exactly like a right answer, because reading like a right answer is the thing it was optimised for.

Everything else in this library is, in some sense, a consequence of that sentence.