Building Benchmarks and Evals

AI is Smart Until It's Dumb: Why LLM Will Fail When You Least Expect It - Emmanuel Maggiori

Topic 13 of 19

Published Nov 10, 2025 · Clip 45:53–50:52 (4:58)

Emmanuel stresses that anyone building an AI product needs their own benchmark to measure AI performance on their specific task, using a validation set (with selection-bias caveats) to choose among options and a separate test set for a final check you never iterate against. Benchmarks let you compare RAG vs fine-tuning and models, and matter when providers force model changes—like OpenAI removing older models at the GPT-5 launch. He warns to qualify errors, not just count them (the AI surgeon might cut the wrong leg), and shares a travel-planner case where an LLM approach was slow and wrong-headed because the problem had nothing to do with language—solved instead with Google Places API and Dijkstra's algorithm.