How Large Language Models (LLMs) Work

Master the mechanics of transformers and self-attention to see how modern AI processes language. You will learn exactly how these models predict and generate human-like text. * **Architectural flow** of transformer encoders and decoders * **Self-attention mechanisms** for contextual token relationships * **Matrix operations** driving parallelized sequence processing * **Probability distributions** for next-token generation logic

10 sections ยท 27 lessons

Course outline

Foundational Concepts

  1. Text to Numbers
  2. Layered Processing
  3. Tokenization
  4. Embeddings

Sequential Processing Problems

  1. Sequential Limitations

Transformer Architecture

  1. Parallel Processing
  2. Query-Key-Value
  3. Attention Weights

Self-Attention Mechanics

  1. Self-Attention
  2. Causal Masking
  3. Multi-Head Attention

Model Architectures

  1. Architecture Types
  2. Model Scale

Pre-Training Process

  1. Pre-Training
  2. Next-Word Prediction

Fine-Tuning and Alignment

  1. Fine-Tuning: Specializing Pre-Trained Models
  2. RLHF Overview
  3. RLHF Stages

Text Generation

  1. Autoregressive Generation
  2. Softmax Transformation
  3. Temperature: Controlling Output Randomness
  4. Sampling Strategies

Generation Control

  1. Creativity vs Precision

Model Limitations

  1. Hallucinations
  2. Mitigating Hallucinations
  3. Context Windows
  4. Context Management

Start learning with Wondering