Three Training Stages

Three Stages, One Pipeline

A model like the one behind your coding assistant does not arrive helpful. It is built in three passes.

Pre-training learns from internet documents. Supervised fine-tuning teaches assistant behavior. Reinforcement learning practices problem-solving.

Each stage consumes the artifact from the previous one, so the order is fixed.

The Education Analogy

The pipeline mirrors how humans are educated.

First you read textbooks. Then you study worked solutions. Finally you do practice problems.

Pre-training is the textbook reading. Supervised fine-tuning is the worked solutions. Reinforcement learning is the practice set.

The analogy is useful because it explains why the stages cannot be reordered.

Pre-training Builds A Base Model

Pre-training runs for months on thousands of GPUs and produces a base model.

That base model simulates internet documents. It is not directly useful yet.

Prompt it and it continues text in the statistical style of the web rather than answering you.

Notice how the base model extends the prompt in the statistical style of internet documents instead of replying to the user, which is why it is not directly useful yet.

Almost all of the knowledge in the final system is deposited here.

Fine-tuning Creates The Assistant

Supervised fine-tuning converts the base model into an assistant.

The training data is human-labeled conversations. The model learns the shape of a helpful reply, not new world knowledge.

This is why the same underlying weights can feel like a document simulator or a chat partner depending on the stage.

Reinforcement Learning Practices

The last stage lets the model practice.

Reinforcement learning allows the model to discover optimal problem-solving strategies through repetition rather than copying a labeled answer.

Fine-tuning shows the model what a good answer looks like. Reinforcement learning lets it find its own route to one.

This is the stage most closely tied to reasoning quality on hard tasks.

The Cost Collapse

The economics across stages are lopsided.

Pre-training costs millions of dollars and takes roughly 3 months. Post-training is roughly 3 hours.

The computational cost drops dramatically with each stage, yet every stage remains essential to the final product.

Why The Economics Shape Behavior

The cost gap decides what gets improved.

Knowledge is expensive to change because it lives in pre-training. Behavior is cheap to change because it lives in post-training.

So labs ship new personas, refusal policies, and tool-use habits far more often than new knowledge.

That asymmetry explains a lot about where model limitations sit.

A coding model learns human-labeled conversation patterns during supervised fine-tuning.

• True
• False

Answer: True

This is correct. Supervised fine-tuning teaches the model how a helpful assistant should respond using human-labeled conversations; it is not primarily for adding new world knowledge.

The training pipeline uses _____ to learn from internet documents, _____ to learn assistant behavior, and _____ to practice problem-solving.

Answer: The training pipeline uses pre-training to learn from internet documents, supervised fine-tuning to learn assistant behavior, and reinforcement learning to practice problem-solving.

The stages have distinct roles and a fixed order: internet data builds the base model, labeled conversations shape assistant behavior, and problem-solving practice improves strategies.

A base model is connected directly to a coding workflow. Which behavior should the engineer expect before supervised fine-tuning?

• It continues text in the statistical style of the web.
• It reliably follows assistant-style instructions and tool policies.
• It discovers optimal solutions through repeated problem-solving practice.
• It has recently acquired most of its world knowledge from labeled conversations.

Answer: It continues text in the statistical style of the web.

A base model mainly simulates the statistical style of internet documents. It may continue text plausibly, but it is not yet trained to act as a helpful conversational assistant.