Infrastructure, compliance, and latency

Building Windsurf with Varun Mohan

Topic 8 of 16

Published May 7, 2025 · Clip 27:03–33:42 (6:39)

Varun explains Windsurf's infrastructure choices: they built much in-house due to their infrastructure background and enterprise requirements, keeping dependencies tight. They achieved FedRAMP High compliance, aided by an early Dell deployment that forced operational rigor for 100M+ line codebases. He then discusses inference latency: for autocomplete they target sub-100ms time-to-first-token and hundreds of tokens per second, using speculative decoding, model parallelism, and careful batching. He explains GPU compute-vs-memory-bandwidth tradeoffs and notes that a 10ms latency increase materially affects usage; data center placement matters, though in India local network congestion, not speed of light, is the bottleneck.