Balancing Experiments with Innovation

The ultimate guide to A/B testing | Ronny Kohavi (Airbnb, Microsoft, Amazon)

Topic 5 of 10

Published Jul 27, 2023 · Clip 21:08–26:38 (5:30)

Ronnie advocates testing everything, as even bug fixes surprise; impossible to over-experiment but possible to under-allocate to high-risk ideas (80% failure expected). He shares Bing's failed social integration after 100 engineer-years, emphasizing data as oracle. For startups, start building culture/platform at 10K users, full power at 200K.