
Why LinkedIn built Kafka and learning scale

Designing Data-intensive Applications with Martin Kleppmann
Topic 2 of 14
Published Apr 22, 2026 · Clip 11:13–14:07 (2:55)
Martin explains why LinkedIn built Kafka (data integration between many event-producing systems and downstream consumers like Hadoop and the data warehouse), referencing Jay Kreps' blog post 'The Log.' He reflects on his first experience at large scale, working on Samza, hand-coding MapReduce jobs, and how understanding how data systems fit together fed directly into writing the book.