Next-Token Prediction Is Deceptively Deep

A mind-expanding conversation on how models actually work | Emmanuel Ameisen (Anthropic)

Topic 3 of 18

Published Oct 18, 2026 · Clip 5:59–11:50 (5:51)

The host marvels at needing to study AI like a brain, and Emmanuel explains we can't write rule-based software for general language tasks, which is why models are grown. He argues next-token prediction is deceptively complicated: to predict hard words (like 'the capital of the state that contains Dallas'), a model must build a model of the world. They discuss Ilya's point that predicting the next token well requires understanding the universe, and the observation that humans themselves are next-token predictors, which Emmanuel connects to his experience speaking and meditating.