
Latency optimization secret sauce

How Perplexity Works 🤖 — with Denis Yarats
Topic 10 of 12
Published May 30, 2024 · Clip 37:29–40:02 (2:33)
Denis describes the latency optimization approach: inspect every stack component, go after the most expensive bottlenecks first, and build tooling to map where time goes per request. Controlling the end-to-end experience unlocks creative optimizations impossible when you only control part of the stack. The same methodology applies to quality via accurate evals, though measuring quality is harder than latency.