Sanity Checking Causal Representation Learning on a Simple Real-World System

Juan L. Gamella, Simon Bing, Jakob Runge

International Conference on Machine Learning 2025 · Oral

This talk, presented by Simon Bing and Juan L. Gamella, delves into a fundamental yet often overlooked question within the field of **Causal Representation Learning (CRL)**: Does it actually work on real-world data? CRL is a burgeoning area of machine learning that aims to uncover underlying causal relationships and latent factors from high-dimensional observational data. While promising, the vast majority of CRL algorithms are developed and validated using synthetic datasets, which can meticulously control for various assumptions. The speakers posit that this reliance on synthetic benchmarks might be masking critical challenges that these algorithms face when confronted with the complexities of the real world.

AI review

A careful empirical sanity check demonstrating that current CRL methods fail on a simple, well-characterized real physical system, with measurement noise identified as the proximate cause of failure for at least one method. The contribution is honest, the diagnostic methodology is clever, and the message is genuinely useful for the field. But this is fundamentally a negative empirical result, not a theoretical one — it tells us that assumptions are violated in practice without giving us the machinery to fix them or formal guarantees about when and why failure occurs.