RL Basics

Learning by Trial and Error

How does a young animal learn which foods are safe? How does a robot learn to walk without falling? Nobody hands them a manual.

They try things, notice what happens, and adjust.

This is the heart of reinforcement learning (RL): learning how to map situations to actions so as to maximize a numerical reward signal.

Notice that the robot (the agent) improves only by acting on the floor around it (the environment) and seeing what happens.

The learner is called the agent. Everything it interacts with is the environment. Good outcomes get repeated, and bad ones fade away.

The Agent-Environment Loop

RL happens over discrete time steps. Each step follows the same cycle.

First, the agent observes the state of the environment. Then it selects an action.

The environment responds. The agent receives a reward and observes the next state.

That new state starts the next step, and the cycle repeats. Over many loops, the agent gathers the experience it learns from.

Rewards Are the Only Feedback

After each action, the agent gets a reward: a single number.

A positive reward signals that something went well, so the agent leans toward actions that earned it. A negative reward works like a penalty, so the agent leans away.

Notice what is missing. Nobody tells the agent which action was correct.

The agent must experiment, comparing results across many tries to discover what works.

Thinking Beyond the Next Reward

The agent's goal is to maximize cumulative long-term reward, called the return.

The return adds up rewards over time. Grabbing the biggest reward right now can lower the total later.

Picture a chess player sacrificing a piece. The move costs something now but can set up a win.

A good RL agent learns to accept short-term losses when they lead to more reward over time.

The Policy: Situation to Action

What does the agent actually learn? A policy.

A policy is a mapping from states to actions. It answers one question: in this situation, what should I do?

For a walking robot, a policy might say: tilting left? Shift weight right. Tilting right? Shift weight left. Steady? Step forward.

Learning means improving the policy until it picks actions that maximize the return over many steps.

How RL Differs From Other Learning

Supervised learning trains on labeled examples that show the correct answer, and it learns to match those answers.

Unsupervised learning works with unlabeled data. Its aim is to find structure or patterns.

RL has no labeled correct examples. The agent learns from its own experience, getting only reward signals after its actions.

Its aim is different too: maximizing reward through interaction.

Putting It All Together

An agent observes the state of its environment and takes an action.

The environment gives back a reward. The return adds up rewards over time.

The policy maps states to actions, and learning improves it to maximize the return.

Spot RL in your own work: anywhere a system learns from feedback after acting, with no answer key, you are looking at this loop.

In reinforcement learning, a _____ is a mapping from states to actions.

• policy
• reward

Answer: policy

A policy tells the agent which action to take in each state. A reward is the numerical feedback signal received after acting, not the mapping itself.

At each time step, the agent observes the environment's _____, selects an _____, and receives a numerical _____ as feedback.

Answer: At each time step, the agent observes the environment's state, selects an action, and receives a numerical reward as feedback.

The state describes the current situation, the action is the agent's choice, and the reward is the numerical feedback the environment gives in response.

A game-playing agent can grab a small coin now, or skip it to reach a large treasure a few moves later. Which choice best fits the goal of reinforcement learning?

• Skip the coin to maximize total reward over time
• Grab the coin because immediate reward matters most
• Pick randomly since rewards are only feedback
• Grab the coin because negative rewards must be avoided

Answer: Skip the coin to maximize total reward over time

RL agents maximize return, the cumulative long-term reward. Skipping the small coin to get the larger treasure yields more total reward, even though the immediate reward is lower.