Language Models

Text Becomes Numbers

A language model never sees words. It sees tokens, which are chunks of text such as "found", "ing", or a punctuation mark.

Each token is mapped to an embedding, a list of numbers that positions its meaning relative to other tokens.

So the first business reality is simple. Everything you buy or build sits on top of a numeric representation of text.

Attention Weighs Context

The transformer is the architecture that made modern models work. Its core move is attention.

Attention lets the model weigh how important every other token in the sequence is when interpreting the current one.

In "the bank raised rates", attention pulls weight toward "rates" and settles on the financial meaning of "bank".

Notice that the thickest connection runs from bank to rates, which is how the model settles on the financial meaning.

Context is computed, not looked up.

Predicting The Next Token

With context weighted, the model does one job: next-token prediction. It outputs a probability distribution over possible next tokens, picks one, and repeats.

Conversation is that loop running fast enough to feel like thought.

The context window is how much of the sequence the model can attend to at once. Beyond it, earlier detail is simply not there.

Training Versus Inference

Founders conflate these two, and vendors rarely correct them.

Training is the one-time, expensive process of learning weights from data, including post-training that shapes behavior.

Inference is what happens every time a user sends a prompt. Weights are frozen. Nothing is learned from the conversation.

Where Gains Come From

Capability improvements come from several distinct levers, not one magic breakthrough.

Scaling adds model size and compute. Better data improves what is learned. Post-training aligns behavior. Tool use connects the model to real systems.

Reasoning and test-time compute let a model spend more effort at inference, thinking longer before answering.

Why Confident Errors Persist

Hallucinations are not a bug bolted onto an otherwise truthful system.

A model trained to predict plausible next tokens will produce a fluent, confident answer whether or not the underlying fact exists in its weights.

Scale reduces the frequency. It does not remove the mechanism.

What This Means For You

Use the mechanics as a filter on vendor claims.

Ask what changed: scale, data, post-training, tool use, or test-time compute. Each has different cost and reliability implications.

Then ask about evaluation. A claim without a stated test on your own tasks is marketing, not evidence.

Confident errors are structural, so design review steps where being wrong is expensive.

A language model directly reads whole words rather than tokens.

• True
• False

Answer: False

Incorrect. A language model receives tokens, which can be word pieces such as “found,” “ing,” or punctuation marks.

Word pool: training; inference; post-training. The one-time process of learning weights from data is _____. Each response to a user prompt is _____. Behavior-shaping work after initial learning is _____.

Answer: Word pool: training; inference; post-training. The one-time process of learning weights from data is training. Each response to a user prompt is inference. Behavior-shaping work after initial learning is post-training.

Training learns model weights from data. Post-training shapes behavior after that learning, while inference uses the frozen weights to answer prompts.

In the sentence “the bank raised rates,” what allows a language model to favor the financial meaning of “bank”?

• Attention using the surrounding context
• The context window deleting ambiguous tokens
• Post-training replacing the word “bank”
• Scaling converting the sentence into a database record

Answer: Attention using the surrounding context

Attention weighs the relevance of other tokens, such as “rates,” when interpreting the current token.