C.A.L.L.I.N.G. Now I'm Calling You, Calling You Now - Mario Macías & Terra Tauri, Grafana Labs
Mario Macías, Terra Tauri, Grafana Labs
KubeCon + CloudNativeCon Europe 2025 · Session
This talk, "C.A.L.L.I.N.G. Now I'm Calling You, Calling You Now," presented by Mario Macías and Terra Tauri from Grafana Labs, delves into a critical operational challenge encountered when deploying a Kubernetes-native eBPF-based observability tool at scale. The speakers recount how their efforts to enrich raw eBPF network data with meaningful Kubernetes metadata inadvertently led to a severe degradation and eventual collapse of the Kubernetes API server itself. The core of the problem stemmed from a common pattern of using Kubernetes informers within a DaemonSet architecture, which, when scaled, exposed a fundamental bottleneck in how the Kubernetes API handles cluster-wide state subscriptions.
AI review
This talk by Macías and Tauri from Grafana Labs dissects a critical Kubernetes anti-pattern: DaemonSets using informers for cluster-wide state, leading to catastrophic API server overload. They provide a brutal, clear diagnosis of the N^2 scaling problem and present an elegant, highly actionable solution using a centralized gRPC-based cache. This isn't just theory; it's a hard-won lesson in distributed systems design, crucial for anyone operating Kubernetes at scale.