
Practical advice on evals and vibes

Why PMs are suddenly valuable again | Tulsee Doshi (Head of Product, Gemini)
Topic 8 of 18
Published Nov 8, 2026 · Clip 23:08–26:29 (3:21)
Tulsi gives practical advice on the eval-versus-vibes debate. For evals, she admits the field is still figuring it out internally, but advises starting small—even with five prompts or a mini evaluation of ~20 prompts that stress-test capabilities and show signal between models. Start with yourself as the rater, define what 'good' looks like (especially hard for long multi-turn trajectories), build a set of good examples for self-clarity, then scale up. For vibes, she says stress-test models on things you're excited about, since that's where you'll see magic; people naturally test models on 3D animations, writing, or financial analysis. Find tasks where models show signs of life but aren't yet perfect under your definition, which reveals how different models handle the problem.