Universal Approximator · Lecture 7

From a stubborn straight line to a neural network — bend math until it fits the world. Same idea that lets a Deep Q-Network predict the future.

🤖 DQN MODE · axes are now (state, Q-value)

Stage 1 · The Linear Fail

A straight line is the simplest model: pick a slope w and an intercept b. Try to make it hug the curve. You'll find it can't — no matter how you tilt or shift, a line stays a line. Math demands more.

Equation

MSE — high error

Sliders

target points predicted curve individual ReLUs error bars

DQN reveal

Congratulations — you just built a Deep Q-Network.

You no longer memorize a table of \(Q\)-values. You bend math to predict them. Every state goes in; a value comes out.

x →State (e.g., cart velocity, pixel grid, joint angles) y →Predicted Q-Value of taking some action

The same neurons you just stacked are exactly what makes Atari from pixels, AlphaGo, and RLHF work. Replace 2 inputs with 84×84 pixels, replace 4 ReLUs with millions, and — same equation — Bellman target and squared error.

Click "Auto-Fit" if you haven't already. Watch the curve snap. That's gradient descent in 1.5 seconds.