
Already know some of this? A 2-minute test skips the lessons you know.
Take the placement testA free 30-lesson course built from Andrej Karpathy's 3h 31m of YouTube video. 2 days at 90 minutes a day, with study notes and a quiz in every lesson. A 2-minute placement test skips what you already know.
Taught by Andrej Karpathy on YouTube. Videos play in YouTube's own player, so the creator keeps every view.
Understand the goal of the course: to build mental models for how LLMs like ChatGPT work, covering the full training pipeline from data to deployment.
Explain how massive text datasets like FineWeb are constructed from Common Crawl through URL filtering, text extraction, language filtering, deduplication, and PII removal.
Describe how raw text is converted into token sequences using UTF-8 bytes and Byte Pair Encoding (BPE) to create a vocabulary of ~100k tokens, and demonstrate using the TikTokenzier tool.
Explain the core training loop: feeding token windows into a randomly initialized Transformer, predicting next-token probabilities, and updating weights via backpropagation to match training data statistics.
Detail the Transformer internals (embeddings, attention, MLP blocks, parameters) and the inference process: sampling tokens autoregressively to generate novel sequences statistically similar to training data.
Analyze GPT-2 as the first recognizably modern LLM stack (1.5B params, 1024 context, 100B tokens) and how modern hardware/software reduced its training cost from $40k to ~$100.
Interpret training logs: each step updates the model on 1M tokens, the loss metric decreases as predictions improve, and training runs for thousands of steps on massive compute clusters.
Define the base model output of pre-training: a lossy compression of the internet that simulates document token sequences but cannot follow instructions or answer questions directly.
Contrast pre-training with post-training: swap internet documents for curated human-assistant conversations, continue training on dialogue data to teach the model assistant behavior patterns.
Show how multi-turn conversations are encoded into 1D token sequences using special control tokens (im_start, im_end, user, assistant) via ChatML-style templates for model consumption.
Demonstrate using few-shot prompts (e.g., translation pairs) to elicit assistant-like behavior from a base model without fine-tuning, leveraging in-context pattern recognition.
Structure a system prompt mimicking a conversation transcript to coax a base model into the assistant role, and show the limitation of it continuing the dialogue indefinitely.
Summarize the 2022 InstructGPT paper: hiring human labelers (via Upwork/Scale AI) to write prompts and ideal responses, creating the SFT conversation dataset.
Inspect Llama 3's open SFT mixture: 1M conversations plus 240 hardcoded identity conversations (system/user/assistant) to fix the model's self-knowledge (name, creator, cutoff).
Compare two methods to program model identity: 1) baking identity into training conversations, 2) prepending a hidden system message in the context window at inference time.
Explain why models struggle with mental arithmetic: fixed compute per token prevents large leaps; solutions must spread reasoning across many tokens (Chain-of-Thought) to avoid errors.
Demonstrate hallucination on unknown facts (2024 election) and mitigation: training the model to refuse answers when internal 'uncertainty neurons' fire, via synthetic 'I don't know' conversations.
Describe tool-use protocol: special tokens (search_start, query, search_end) trigger external web search; retrieved text is injected into context window as working memory for grounded generation.
Contrast parameter knowledge (vague recollection) vs. context window (working memory); show that pasting source text (RAG) yields higher-quality summaries than relying on memorized weights.
Argue that asking an LLM its identity is ill-posed: it has no persistent self, just a stateless token simulator; default answers hallucinate 'ChatGPT by OpenAI' due to internet prevalence.
Motivate RL as the 'school practice' stage after pre-training (textbook reading) and SFT (worked examples); RL lets models discover solutions via trial-and-error on verifiable problems.
Detail RL loop: generate many solutions per prompt, score automatically (exact match or LLM judge), reinforce high-scoring token sequences; enables discovery of strategies beyond human imitation.
Draw parallel to AlphaGo: supervised learning caps at human expert level; RL discovers novel strategies (Move 37, 1/10k probability) that transcend human play, hinting at LLM reasoning frontiers.
Explain RL from Human Feedback (RLHF): train a separate reward model on human preference rankings (cheaper than scoring), then run RL against this neural simulator of human judgment.
Contrast standard models (GPT-4o, RLHF only) with thinking models (OpenAI o-series, Gemini 2.0 Flash Thinking) that use true RL to develop internal Chain-of-Thought on verifiable domains.
Highlight unknowns: whether reasoning strategies from math/code transfer to unverifiable domains (creative writing), and the nascent state of open-domain RL environments.
Synthesize the correct mental model: ChatGPT output is a neural simulation of an OpenAI data labeler following instructions, running a fixed compute graph per token, not a human-like mind.
Summarize LLM quirks: hallucinations, 9.11 vs 9.9 comparison failures, tokenization artifacts; treat as stochastic tools—verify, provide context (RAG), use thinking models for hard problems.
Recommend LMSYS Chatbot Arena (with caveats), AI News newsletter, X/Twitter for updates; access models via provider sites (chat.openai.com), Together.ai for open weights, LM Studio for local MacBook inference.
Trace the complete inference path: user query -> ChatML conversation format -> token sequence -> base model continuation -> SFT/RL personality overlay -> generated response tokens.