Fin's secret sauce and evaluation approach

The AI Transformation at Intercom 🤖 — with Darragh Curran

Topic 10 of 11

Published Oct 23, 2025 · Clip 44:40–48:44 (4:04)

Darragh says Fin's secret sauce is the ability to robustly experiment and iterate, requiring discipline and volume, across a RAG pipeline, knowledge management, specialized models, and prompts. He describes evaluation as layered: offline and back testing against prior versions, then live A/B tests revealing second-order effects. He stresses the stakes are real businesses where mistakes aren't acceptable, and early on most energy went into preventing hallucinations—declining to answer rather than making things up.