Reinforcement Learning: An Introduction

Richard Sutton and Andrew Barto provide the foundational text on reinforcement learning, defining it as "a computational approach to learning whereby an agent tries to maximize total reward when interacting with a complex, uncertain environment." The second edition significantly expands coverage to include artificial neural networks, Monte-Carlo tree search, average reward maximization, and new applications, organizing material into tabular methods (exact solutions) and approximate solution methods. Available free online from the authors, this widely-regarded text focuses on core online learning algorithms with mathematical material set off in shaded boxes for accessibility.

5 sections ยท 22 lessons

Course outline

RL Foundations

  1. RL Basics
  2. Bandits
  3. Explore vs Exploit
  4. MDPs
  5. RL Foundations

Dynamic Programming

  1. Bellman Equations
  2. Policy Evaluation
  3. Policy & Value Iteration
  4. Dynamic Programming

Model-Free Learning

  1. Monte Carlo
  2. Temporal-Difference Learning
  3. SARSA: On-Policy TD Control
  4. Q-Learning: Off-Policy TD Control
  5. Advanced TD Methods
  6. Eligibility Traces
  7. Model-Free Learning

Deep RL

  1. Function Approximation
  2. DQN
  3. Policy Gradients
  4. Actor-Critic
  5. Deep RL

Applications and Impact

  1. GPI Framework
  2. RL & Neuroscience
  3. AlphaGo
  4. Robotics Control
  5. Impact & Future
  6. Applications and Impact

Start learning with Wondering