Let's build GPT: from scratch, in code, spelled out
Andrej Karpathy·2023·1 hr 57 minKarpathy builds and trains a decoder-only Transformer from first principles, following Attention Is All You Need, ending at the core of nanoGPT.
Software Is Changing (Again)
Andrej Karpathy·2025·Y Combinator AI Startup SchoolFrom Software 1.0 (code) to 2.0 (weights) to 3.0 (prompts): Karpathy's case that natural language is becoming the new programming interface, with LLMs as the new computer.
Intro to Large Language Models
Andrej Karpathy·2023A one-hour, general-audience tour of what LLMs are, how they're trained, where they're going, and the new class of security problems they create.
State of GPT
Andrej Karpathy·2023·Microsoft BuildThe pretraining → SFT → reward modeling → RLHF pipeline behind ChatGPT-class assistants, plus practical mental models for prompting and using them well.
Building the Software 2.0 Stack
Andrej Karpathy·2018·Spark+AI SummitKarpathy's argument that large chunks of conventional code are being replaced by learned weights — and what that means for the tools, infrastructure, and skills around them.
Objective-Driven AI
Yann LeCun·2024·Harvard CMSALeCun's case against autoregressive LLMs as a path to general intelligence, and his alternative agenda built around world models, JEPA, and energy-based inference.
Stop Writing Dead Programs
Jack Rusher·2022·Strange LoopJack Rusher argues that most of our languages are still, accidentally, designed around the punchcard — and shows what a living programming environment could look like instead.