Evaluations and Prototyping in AI Product Development

OpenAI researcher on why soft skills are the future of work | Karina Nguyen

Topic 4 of 12

Published Feb 9, 2025 · Clip 20:39–26:58 (6:20)

Karina describes evals as essential for measuring model progress, using deterministic checks, spreadsheets for behaviors, and human win-rate comparisons. Product teams create ground-truth labels from user feedback for features like Canvas triggering. Prompting serves as prototyping, enabling quick idea testing like personalized prompts at Anthropic.