
Already know some of this? A 2-minute test skips the lessons you know.
Take the placement testA free 14-lesson course built from Andrej Karpathy's 2h 26m of YouTube video. 2 days at 90 minutes a day, with study notes and a quiz in every lesson. A 2-minute placement test skips what you already know.
Taught by Andrej Karpathy on YouTube. Videos play in YouTube's own player, so the creator keeps every view.
Understand the goal of building micrograd, a scalar-valued autograd engine, and how it implements backpropagation to train neural networks from scratch.
Grasp the numerical definition of a derivative as a sensitivity measure, and compute gradients for single and multi-input functions using finite differences.
Implement the core Value class with data, gradient, children, and operation tracking to construct and visualize computational graphs for forward passes.
Perform manual backpropagation on a simple expression graph by applying the chain rule stepwise from output to inputs, computing gradients for all intermediate nodes.
Execute manual backpropagation through a single neuron with tanh activation, computing gradients for weights, biases, and inputs to understand parameter influence on output.
Refactor the Value class to encapsulate local gradient logic in _backward closures for each operation (add, mul, tanh), enabling automatic gradient propagation.
Implement topological sort to order the computational graph, then define the backward() method to automatically propagate gradients from output to all inputs in correct order.
Identify and fix the bug where gradients are overwritten instead of accumulated when a Value node is used multiple times in the expression graph.
Add tanh activation with its local derivative (1 - tanh^2), and implement helper operations (neg, sub, div, pow) to increase expressiveness of the autograd engine.
Replicate the micrograd expression in PyTorch, confirming identical forward/backward results, and observe the tensor-based efficiency differences.
Construct Neuron, Layer, and MLP classes on top of micrograd Value objects, implementing forward passes with PyTorch-like API (__call__, parameters).
Define a tiny binary classification dataset and implement Mean Squared Error loss to quantify network performance as a single scalar value.
Gather all network parameters recursively, then perform manual gradient descent steps: zero gradients, backward pass, and update weights using learning rate.
Review the complete micrograd implementation (engine + nn), compare with PyTorch's tanh backward kernel, and understand how to extend PyTorch with custom autograd functions.