Using Data in Product Decisions

Why great AI products are all about the data | Shaun Clowes (CPO at Confluent)

Topic 5 of 8

Published Dec 29, 2024 · Clip 34:52–45:49 (10:57)

Sean advises validating intuitions against data: If results feel wrong, check upstream/downstream and scope (e.g., 2% applicability meaningless). Use holdouts/cohorts to detect false positives (40% experiments fade). Data as 'compass' disproves ideas, not dictates; avoid over-analysis. Regrets being 'data-driven' initially—intuition sifts data. Tactical: Before/after events, click up for scale. At Atlassian, global holdouts aided long-term validation. Balance temporary lifts as opportunities, not failures. Ad insert for Wick Studio (ignore).