Reinforcement
learning, from scratch.
How machines learn to play, win, and repeat — built up from the very first idea. Hands-on games, real math, and the whole arc from cat-or-dog to AlphaGo.
The lectures
How machines learn to play
From supervised classification to AlphaGo. Hands-on interactives: predict-the-next-word, a 3-box bandit, and ε-greedy in action.
The math we'll need: probability
A probability primer — outcomes, expectation, conditional — then formalize the warrior dungeon as an RL problem: state, action, reward, Markov, MDP. With a chest belief-update quiz and a live warrior trap.
MDPs, Bellman, & dynamic programming
Name every part of the agent–environment loop — state, action, reward, transition, policy — bundle them as the MDP, meet values V and Q, and write the Bellman equation that ties them together. Doors & tree game inside.
Values, optimality & the Bellman equation
What makes a policy optimal? Through the doors and warrior-tree games, build up the value function and the principle of optimality, see why greedy ≠ optimal, and write the Bellman equation that every solution must satisfy.
Solving small MDPs
A slippery game reveals where randomness lives — the transition kernel — which puts an expectation in the Bellman backup. That gives policy evaluation, then policy iteration, and finally the one-line shortcut: value iteration. Live PI & VI demos inside.
Policy iteration: evaluate → improve
Score a policy, improve it, repeat. Nail the scoring step on the warrior tree two ways — vanilla roll-up from the leaves and the iterative guess-and-repeat — then spend those values to improve the policy and loop until it can't get any better.
Function approximation
Go and Atari have more states than any table can hold. Replace the table with a function you can train — a line, then a ReLU, then a small network — learn from sampled steps instead of a known model, and minimize the squared Bellman error. Ends with DQN on Atari pixels; universal-approximator playground inside.
Function approximation for policy iteration
Policy iteration, approximated. Follow AlphaGo from why a Go table can't exist, to a policy network that reads any board, to the move it missed against Lee Sedol in Game 4 — then see how the same careful-step update (PMD, TRPO, PPO) trains ChatGPT.