Chatbots, Hallucinations, and Air Canada

AI is Smart Until It's Dumb: Why LLM Will Fail When You Least Expect It - Emmanuel Maggiori

Topic 12 of 19

Published Nov 10, 2025 · Clip 38:18–45:53 (7:36)

Emmanuel explains that hallucinations can be reduced (better prompts, fine-tuning, chain-of-thought/reasoning models—which he calls 'a whole charade') but not eliminated, and they happen when least expected. He contrasts contexts where hallucinations don't matter (translating TripAdvisor reviews) with those where they do (literary translation, customer service—citing Air Canada being ordered by a judge to honor its chatbot's wrong policy). He suggests chatbots should help people find official information rather than interpret it. He also critiques solving problems with AI that should be fixed at the root—like a travel company where customers called about luggage allowances, fixable with proactive emails—and questions claims that AI eliminates boilerplate code, arguing good engineering (libraries, frameworks) already removes repetition, illustrated by asking ChatGPT for a sun-position algorithm that hallucinated usage of an existing library.