
Already know some of this? A 2-minute test skips the lessons you know.
Take the placement testA free 22-lesson course built from 3Blue1Brown's 3h 38m of YouTube playlist. 3 days at 90 minutes a day, with study notes and a quiz in every lesson. A 2-minute placement test skips what you already know.
Taught by 3Blue1Brown on YouTube. Videos play in YouTube's own player, so the creator keeps every view.
Understand the challenge of digit recognition and why neural networks are a powerful approach for this task.
Define a cost function using squared error to quantify how poorly the network classifies training examples and average it over the dataset.
Explain gradient descent as an iterative process of stepping downhill on the cost function landscape to find a local minimum.
Visualize hidden layer weights to reveal the network finds a local minimum that classifies well but does not detect human-interpretable edges or patterns.
Understand how backpropagation computes the desired nudges to weights and biases for one training example by propagating output-layer error signals backward through the network.
Derive the partial derivative of the cost function with respect to a weight using the chain rule through the weighted sum, activation, and cost.
Generalize the chain rule expressions to layers with multiple neurons using indices, and sum over multiple paths when computing sensitivity to previous-layer activations.
Understand that LLMs are probabilistic next-word predictors used to autocomplete dialogue scripts into chatbot responses.
Explain how large language models generate coherent text by repeatedly predicting the next token, sampling from the probability distribution, and appending it to the context.
Explain how the embedding matrix converts discrete tokens into continuous vectors where semantic relationships correspond to geometric directions in a high-dimensional space.
Explain how the softmax function converts raw logits into a valid probability distribution and how the temperature parameter controls the randomness of token sampling.
Understand how token embeddings encode semantic meaning in high-dimensional space and why transformers need attention to bake in contextual information.
Explain how query and key matrices project embeddings into a lower-dimensional space, how dot products measure relevance, and how column-wise softmax produces a normalized attention pattern.
Understand the high-level finding that factual knowledge in LLMs appears to be stored in the multi-layer perceptron (MLP) blocks of transformers.
Explain the superposition hypothesis: near-orthogonal vectors in high dimensions allow exponentially more features than dimensions, enabling scalable fact storage.
Understand how cross-entropy underlies the 2002 'Language Trees and Zipping' paper, where gzip compression distances recover language lineage trees without linguistic knowledge.
Build intuition with binary distributions, see that cross-entropy is minimized when Q=P (equaling entropy of P), and understand why this makes it a natural loss function.
Derive the pre-training loss function from first principles, showing why negative log probability (cross-entropy) is the unique function minimized only when the model matches data statistics.
Explain knowledge distillation as minimizing cross-entropy between a small student model's predictions and a large teacher model's full output distribution, yielding richer gradients than single-token targets.
Understand how CLIP uses contrastive learning on 400M image-caption pairs to align matching vectors and separate non-matching ones in a shared 512-dimensional embedding space, enabling concept arithmetic like 'hat' = 'me with hat' - 'me without hat'.
Describe how conditioning diffusion models on the diffusion timestep t allows them to learn coarse vector fields at high noise levels and fine-grained structure near t=0, and visualize the learned vector field's phase transition on a 2D spiral toy dataset.
Understand how classifier-free guidance subtracts the unconditional vector field from the class-conditioned one, scales the difference by a guidance factor α, and uses the result to steer diffusion trajectories toward specific classes like 'cat' or 'dog' on the data manifold.