
Observability and AI Systems

Observability: the present and future, with Charity Majors
Topic 10 of 14
Published Jan 22, 2025 · Clip 53:33–59:07 (5:34)
Charity identifies three intersections of AI and observability: building/training models, developing with LLMs, and the universal problem of AI-generated code of unknown origin—comparing it to Parse running unvetted user JavaScript. She critiques observability vendors bolting AI on guesswork instead of computing answers from gathered context, and argues AI observability startups are wrongly treating it as a self-contained model problem when it's actually a trace-shaped problem spanning services, models, and human feedback. Companies wanting to adopt AI agents need good observability first.