Free sample lesson · AI 技术基础与实战

Inside ChatGPT

How do AI chatbots like ChatGPT work?

Lesson 1 of the Wondering course AI 技术基础与实战

ChatGPT Predicts Text

When you send ChatGPT a message, it generates a response by predicting what text should come next.

The system uses patterns learned from enormous amounts of text. It does not begin with a human-like intention or internal explanation.

This distinction matters when you build AI features. ChatGPT can produce useful language while lacking human-style understanding of the topic.

A message enters a pattern-based text prediction system, which produces a probable continuation.

Trace how a message becomes a continuation without depicting a human-like intention behind it.

The Lyric Sheet Mystery

Imagine finding a song lyric sheet with half the lines missing. You can complete it by studying the words that remain.

A familiar phrase may suggest several likely continuations. You choose one, then use the new ending to predict the next word.

This activity captures the central mechanism: incomplete text provides clues, and learned patterns guide each continuation.

A partly missing lyric sheet being completed one continuation at a time from the remaining word patterns.

Notice how the visible lyric fragments provide clues, while each completed continuation supplies the next clue.

A Prediction Machine

Chatbots act as powerful prediction machines. You provide text, and the system estimates which word should follow the current sequence.

The estimate depends on the words already present and the patterns associated with similar contexts.

For example, a sentence beginning with “The developer opened the” makes some continuations more likely than others. The system selects a probable next step.

How a prediction machine proceeds
  • Read sequence: Start with the words already present in the current text sequence.
  • Estimate options: Use the current words and patterns from similar contexts to estimate possible continuations.
  • Select next word: Choose a probable next word as the next step in the response.

Pattern Recognition At Scale

Pattern recognition means identifying regular relationships in observed examples. ChatGPT applies this process across massive amounts of language.

During training, it learns which words commonly appear together and how wording changes across contexts.

It does not retrieve a single rule for every sentence. Instead, it uses many learned associations to estimate likely continuations for the text in front of it.

From examples to likely continuations
  • Observed examples: Language examples provide the observations for identifying regular relationships.
  • Words together: Training captures which words commonly appear together.
  • Contexts: Wording changes across different language situations.
  • Learned associations: Many associations are learned rather than one fixed rule.
  • Likely continuations: The associations help estimate what may come next.

One Word At A Time

ChatGPT constructs a response incrementally. First, it predicts an opening word or text unit based on your prompt.

That prediction becomes part of the conversation. The system then predicts the next word using the original prompt plus the generated text.

It repeats this loop until the response is complete. This process is called iterative autocomplete.

Iterative autocomplete
  • Set context: The original prompt supplies the starting context.
  • Predict unit: The system predicts an opening word or text unit.
  • Append unit: The generated unit becomes part of the conversation.
  • Repeat: The expanded text is used to predict again until the response is complete.

Why Context Changes Outputs

The same starting words can lead to different continuations when surrounding context changes.

A request about debugging gives different likely next words than a request about songwriting. Earlier words constrain the choices available later.

For developers, precise prompts provide stronger context. Clear requirements, examples, and desired formats help guide the prediction sequence toward a more useful response.

Context shifts predictions
  • Debugging request
  • Songwriting request

Prediction Is Not Thinking

When ChatGPT responds, it is not privately pondering a question with a human-like inner voice.

It runs a sophisticated text completion algorithm that estimates probable next words from learned patterns.

A fluent answer can therefore sound deliberate without proving comprehension. The output reflects successful language prediction, even when the system produces an incorrect explanation or a plausible-sounding invention.

Does fluency prove understanding?

Use The Mechanism Wisely

Pattern recognition works well when the task has strong language patterns, such as drafting, summarizing, or transforming text.

Treat uncertain claims as outputs to check, especially when the response sounds confident. Fluency is not evidence that the underlying content is correct.

Use clear prompts to control the context, then review the result. Understanding word-by-word prediction helps you choose suitable tasks and catch misleading completions.

Choosing tasks and checking outputs
  • Good fits
  • Drafting text: Tasks with strong language patterns are suitable uses.
  • Summarizing text: The model can condense existing language patterns.
  • Transforming text: The model can reshape language into another form.
  • Check outputs
  • Uncertain claims: Check claims when their accuracy is uncertain.
  • Confident responses: Confidence in wording does not establish correctness.
  • Control and review
  • Clear prompts: Requirements, examples, and formats strengthen the context.
  • Review results: Review the result to catch misleading completions.

Use clear context, then check what the model produced.

Check yourself

Can changing the surrounding context change ChatGPT's likely continuation?

  • A debugging request and a songwriting request can produce different likely continuations.
  • The same starting words must always produce the same type of continuation.

Answer: A debugging request and a songwriting request can produce different likely continuations.

The prompt supplies the initial context, so changing it changes which learned language patterns are relevant. A debugging request and a songwriting request therefore lead to different likely continuations.

In iterative autocomplete, ChatGPT uses the _____ as context, adds each _____ to the conversation, and uses the expanded _____ to guide the next continuation.

Answer: In iterative autocomplete, ChatGPT uses the prompt as context, adds each prediction to the conversation, and uses the expanded context to guide the next continuation.

The sequence begins with the prompt, which supplies the current context. ChatGPT predicts a text unit, adds that prediction to the conversation, and uses the expanded context for the next prediction. Repeating this loop creates the response.

How does ChatGPT primarily generate a response?

  • Predicting likely next words from the current text
  • Retrieving a single stored answer for every question
  • Thinking through the topic with a human-like inner voice
  • Following one fixed rule for every sentence

Answer: Predicting likely next words from the current text

ChatGPT generates responses by estimating what text is likely to come next from the text already provided. It does not compose the response from human-like intentions or comprehension.

Keep going

This is lesson 1 of 36 in AI 技术基础与实战. Next up:

  1. How Prompts Work
  2. Probability & Randomness
  3. Temperature
  4. Training Data Scale
  5. Generalization

Start this course free

Wondering turns any topic into short lessons you can read, watch, or listen to, with practice questions and an AI tutor.