Introduction to AI Evals Basics

Why AI evals are the hottest new skill for product builders | Hamel Husain & Shreya Shankar

Topic 1 of 4

Published Sep 25, 2025 · Clip 0:00–9:58 (9:58)

The host introduces the rising importance of evals in AI product building, highlighting endorsements from Anthropic and OpenAI leaders. Guests Hamill and Shrea define evals as systematic measurement to improve AI applications, akin to data analytics on LLM outputs. They use a real estate agent example to illustrate how evals provide metrics beyond vibe checks for issues like incorrect advice or hallucinations.