Building the Software 2.0 Stack
Andrej Karpathy·2018·Spark+AI Summit·18 minKarpathy's argument that large chunks of conventional code are being replaced by learned weights — and what that means for the tools, infrastructure, and skills around them.
Software Is Changing (Again)
Andrej Karpathy·2025·Y Combinator AI Startup SchoolFrom Software 1.0 (code) to 2.0 (weights) to 3.0 (prompts): Karpathy's case that natural language is becoming the new programming interface, with LLMs as the new computer.
Let's build GPT: from scratch, in code, spelled out
Andrej Karpathy·2023Karpathy builds and trains a decoder-only Transformer from first principles, following Attention Is All You Need, ending at the core of nanoGPT.
Intro to Large Language Models
Andrej Karpathy·2023A one-hour, general-audience tour of what LLMs are, how they're trained, where they're going, and the new class of security problems they create.
State of GPT
Andrej Karpathy·2023·Microsoft BuildThe pretraining → SFT → reward modeling → RLHF pipeline behind ChatGPT-class assistants, plus practical mental models for prompting and using them well.
Objective-Driven AI
Yann LeCun·2024·Harvard CMSALeCun's case against autoregressive LLMs as a path to general intelligence, and his alternative agenda built around world models, JEPA, and energy-based inference.
The Language of the System
Rich Hickey·2017·Clojure ConjHickey on the fact that the language inside your process is rarely the language between your processes — and what that should imply for system design.