How LLMs Work Under the Hood

AI is Smart Until It's Dumb: Why LLM Will Fail When You Least Expect It - Emmanuel Maggiori

Topic 6 of 19

Published Nov 10, 2025 · Clip 17:38–22:03 (4:25)

Emmanuel explains how LLMs work: a language model is a simple program predicting the next word, fed its own output to generate sentences; everything else (like web search) is a wrapper illusion. Models are trained on internet data and then refined with human feedback (RLHF), often by low-paid workers, to align outputs—but a long tail of problems persists. He explains the shift from LSTM technology, which compressed context into a tiny vector, to the transformer architecture that represents every token's meaning simultaneously, which unleashed performance much like convolutional networks did for images.