
Hidden States and Learned Abstractions

AI is Smart Until It's Dumb: Why LLM Will Fail When You Least Expect It - Emmanuel Maggiori
Topic 8 of 19
Published Nov 10, 2025 · Clip 23:46–27:38 (3:52)
Emmanuel explains that LLMs learn their own abstractions in 'hidden states' stacked in many layers, which configure themselves during training to solve the prediction problem. He counters the 'stochastic parrot' criticism: the model doesn't just copy data, it uses it to configure internal states. He illustrates with ChatGPT's early failure to answer whether an anaconda fits in a shopping mall—it had grammar representations but not size representations—and the cow-on-the-beach image recognition failure, where the model learned 'cow = cow shape + grass' as a shortcut, which also causes hallucinations.