A Picture is Worth 500 Labels: A Case Study of Demographic Disparities in Local Machine Learning Models for Instagram and TikTok
Jack West, Lea Thiemt, Shimaa Ahmed, Maggie Bartig, Kassem Fawaz, Suman Banerjee
IEEE Symposium on Security and Privacy 2024 · Day 1 · Continental Ballroom 5
This talk, presented by Lea Thiemt and Jack West at IEEE S&P, delves into the often-hidden world of local machine learning (ML) models embedded within popular social media applications like Instagram and TikTok. As ML models increasingly shift from cloud-based processing to on-device execution for features like real-time face filters and augmented reality, they gain the capability to infer a vast array of information about users directly from their camera feeds and images. The core curiosity driving this research was to uncover precisely what insights these vision models extract and, critically, whether these inferences exhibit demographic disparities in quality or accuracy.
AI review
This research brilliantly dissects the opaque world of on-device ML in Instagram and TikTok, revealing extensive visual inference and disturbing demographic biases. The novel methodology for analyzing obfuscated native models is a significant technical achievement, providing critical transparency into practices that impact user privacy and fairness daily.