LLM Fundamentals

Master the underlying architecture and mechanisms of modern AI to bridge the gap between using tools and building robust, production-ready applications for your new career. * **Core architecture** behind Transformers and attention mechanisms * **Training pipelines** including pre-training and supervised fine-tuning * **Model evaluation** using technical benchmarks and performance metrics * **Deployment strategies** for integrating models into production environments

10 sections ยท 28 lessons

Course outline

Foundational Concepts

  1. Text to Numbers
  2. Vectors: The DNA of AI Language
  3. Layered Processing
  4. Tokenization
  5. Embeddings

Sequential Processing Problems

  1. Sequential Limitations

Transformer Architecture

  1. Parallel Processing
  2. Query-Key-Value
  3. Attention Weights

Self-Attention Mechanics

  1. Self-Attention
  2. Causal Masking
  3. Multi-Head Attention

Model Architectures

  1. Architecture Types
  2. Model Scale

Pre-Training Process

  1. Pre-Training
  2. Next-Word Prediction

Fine-Tuning and Alignment

  1. Fine-Tuning: Specializing Pre-Trained Models
  2. RLHF Overview
  3. RLHF Stages

Text Generation

  1. Autoregressive Generation
  2. Softmax Transformation
  3. Temperature: Controlling Output Randomness
  4. Sampling Strategies

Generation Control

  1. Creativity vs Precision

Model Limitations

  1. Hallucinations
  2. Mitigating Hallucinations
  3. Context Windows
  4. Context Management

Start learning with Wondering