Understanding High Cardinality

#57 - Observing Your Production Systems and Yourself - Jamie Riedesel

Topic 5 of 13

Published Sep 27, 2021 · Clip 22:37–27:09 (4:32)

Jamie explains cardinality as the number of unique values in a field multiplied across fields, with timestamps as a special highly-cardinal, specially-indexed field since every query is time bound. She distinguishes cardinality from dimensionality (number of unique fields), describes how Elasticsearch handles high-cardinality fields well but suffers from many mostly-null fields, and how old time-series databases fall over when high-cardinality tags like account numbers are added. Modern storage, memory, and techniques finally enable high-cardinality observability systems.