Reinforcement
learning, from scratch.
How machines learn to play, win, and repeat — built up from the very first idea. Hands-on games, real math, and the whole arc from cat-or-dog to AlphaGo.
The lectures
How machines learn to play
From supervised classification to AlphaGo. Hands-on interactives: predict-the-next-word, a 3-box bandit, and ε-greedy in action.
The math we'll need: probability
A probability primer — outcomes, expectation, conditional — then formalize the warrior dungeon as an RL problem: state, action, reward, Markov, MDP. With a chest belief-update quiz and a live warrior trap.
MDPs, Bellman, & dynamic programming
Name every part of the agent–environment loop — state, action, reward, transition, policy — bundle them as the MDP, meet values V and Q, and write the Bellman equation that ties them together. Doors & tree game inside.
Values, optimality & the Bellman equation
What makes a policy optimal? Through the doors and warrior-tree games, build up the value function and the principle of optimality, see why greedy ≠ optimal, and write the Bellman equation that every solution must satisfy.
Solving small MDPs
A slippery game reveals where randomness lives — the transition kernel — which puts an expectation in the Bellman backup. That gives policy evaluation, then policy iteration, and finally the one-line shortcut: value iteration. Live PI & VI demos inside.
Policy iteration: evaluate → improve
Score a policy, improve it, repeat. Nail the scoring step on the warrior tree two ways — vanilla roll-up from the leaves and the iterative guess-and-repeat — then spend those values to improve the policy and loop until it can't get any better.
Putting it together
Synthesis. The big map, method comparator (5 algorithms), 8 project starters, ~30 lines of Python Q-learning, debugging guide. Bring your laptop.
RL, looking forward
From your gridworld to the frontier. DQN, AlphaGo lineage (→ MuZero), RLHF for ChatGPT, real-world apps, open problems, safety & reward hacking.