Telemetry Best Practices

#57 - Observing Your Production Systems and Yourself - Jamie Riedesel

Topic 8 of 13

Published Sep 27, 2021 · Clip 32:38–38:48 (6:10)

Jamie shares four best practices: use a structured logger instead of raw print statements (enabling live redaction of private data like emails under GDPR and pluggable shipping to Kafka, Redis, Kinesis, etc.); have a tracing system, even in CI systems like Jenkins where tracing helps detect performance regressions over time; keep private and health information out of telemetry systems to keep them broadly usable and avoid costly spill investigations and historic cleanup; and manage cardinality, the thing that will kill self-managed telemetry databases, noting SaaS providers like Sumo Logic enforce limits.