Why Modular: LLVM for AI

From Swift to Mojo and high-performance AI Engineering with Chris Lattner

Topic 7 of 14

Published Nov 5, 2025 · Clip 52:22–1:01:42 (9:20)

Chris explains that AI software today mirrors the pre-GCC era: every chip vendor builds its own vertical stack (Google XLA, Apple Metal/MLX, AMD Rockm), forcing labs like Anthropic to reimplement models three times for CUDA, TPUs, and Trainium, causing bugs and outages. He describes why Google's internal culture couldn't fix this — new chips every year pulled best engineers onto fires — and why only a VC-funded startup could tackle it. He explains the Modular bet: a two-level stack with the Mojo language plus compiler algorithms embedded in the language, giving developers compiler power without being compiler engineers. He contrasts Mojo's predictable, controllable compiler design with 'sufficiently smart compiler' leaky abstractions like loop vectorization.