FreqFed: A Frequency Analysis-Based Approach for Mitigating Poisoning Attacks in Federated Learning

FreqFed: A Frequency Analysis-Based Approach for Mitigating Poisoning Attacks in Federated Learning

Speakers: Hossein Fereidooni

Conference: NDSS Symposium

YouTube: (no public video)

Overview

Federated Learning (FL) has emerged as a crucial paradigm for collaborative machine learning, enabling multiple clients to jointly train a global model without sharing their sensitive local data. This decentralized approach offers significant privacy benefits, making it highly attractive for applications in healthcare, finance, and mobile computing. However, this distributed nature also introduces a critical vulnerability: poisoning attacks. These attacks, whether untargeted (aiming to degrade overall model performance) or targeted (implanting hidden backdoors), pose a severe threat to the integrity and reliability of FL models, often going unnoticed for extended periods and leading to serious security breaches.

Existing defenses against these sophisticated attacks often fall short, struggling with adaptive adversaries, making restrictive assumptions about data distributions (e.g., iid vs. non-iid), or diminishing model utility through techniques like differential privacy or direct weight clipping. Many approaches rely on direct inspection of model weights using metrics like L2 norms or cosine distances, making them susceptible to manipulation by adaptive attackers. Recognizing these limitations, Hossein Fereidooni presented FreqFed, a novel aggregation framework designed to provide a more generic, robust, and effective solution.

FreqFed pioneers a fundamentally different approach by analyzing model updates in the frequency domain rather than directly inspecting their weights. By transforming local model updates into their frequency components using the Discrete Cosine Transform (DCT) and then employing HDBSCAN clustering on these low-frequency representations, FreqFed effectively identifies and filters out malicious contributions. This innovative method untangles the defense mechanism from specific assumptions about the adversary, attack type, or client data distributions, offering a significant advancement in securing federated learning environments.

Background

[▶ Watch: Introduction to FL and poisoning attacks (0:00)]()

Federated Learning (FL) operates on a cyclical process: in each round, clients download the current global model, train it locally on their private datasets, and then upload their updated model parameters to a central server. The server aggregates these updates to produce a new global model for the next round. The FedAVG algorithm is a commonly used aggregation rule, averaging the weighted model updates from participating clients. This decentralized structure, while privacy-preserving, is inherently vulnerable to poisoning attacks.

Poisoning attacks in FL are broadly classified into two categories:

1. Untargeted Poisoning Attacks: These attacks aim to degrade the overall performance and generalization capabilities of the global model, essentially causing a denial-of-service. Examples include Label Flipping (where malicious clients intentionally mislabel data) or sending Random Updates.
2. Targeted Poisoning Attacks (or Backdoor Attacks): These are more insidious, as the adversary embeds hidden functionalities into the model. This allows them to trigger specific, malicious behaviors (e.g., misclassifying a specific input pattern to a chosen target class) based on secret triggers, often without significantly impacting the model's performance on benign tasks. Examples include Constrain-and-Scale, Edge-case PGD, and Distributed Backdoor Attacks (DBA).

Existing defenses often struggle due to several factors. Many rely on L2 norms or cosine distances of model weights for anomaly detection, which adaptive attackers can easily circumvent by manipulating weights to appear benign. Techniques like Differential Privacy (DP) add noise or clip gradients, often at the cost of reduced model utility. A major limitation is their reliance on specific assumptions about data distributions (whether data is independent and identically distributed (iid) or non-iid) or the precise nature of the attack. These assumptions limit their generality and lead to performance degradation when violated. Adaptive attacks, such as those employing Projected Gradient Descent (PGD) or multiple/distributed backdoors, are frequently designed to bypass such defenses.

FreqFed introduces two key technical tools to overcome these challenges:

• Discrete Cosine Transform (DCT): A signal processing technique that decomposes a signal into its frequency components. For model weights, which can be viewed as signals, DCT reveals underlying patterns. A crucial property of the 2D DCT is its energy compaction property, concentrating most of a signal's energy into a few low-frequency coefficients. Lower-order coefficients correlate to lower signal frequencies and broader patterns.
• HDBSCAN Clustering: Hierarchical Density-Based Spatial Clustering of Applications with Noise (HDBSCAN) is an advanced clustering algorithm. Unlike K-Means, it doesn't require a predefined number of clusters and can identify clusters of varying shapes and sizes. Crucially, it is robust to noise and outliers, making it ideal for identifying groups of similar model updates while isolating malicious ones.

The adversary model considered by FreqFed is comprehensive:

• Attackers can fully control up to k_a K/2 compromised clients, including their data, training operations, and hyperparameters. This means up to just under 50% of clients can be malicious.
• They can manipulate local model weights before submission and are assumed to know the server's aggregation operations.
• Malicious clients can add regularization terms to their loss function during local training to craft updates that evade detection, aiming to make their poisoned models indistinguishable from benign ones using common metrics (e.g., L2 norm, cosine distance).
• They can change their local training behavior round-to-round and conduct adaptive attacks by tweaking parameters or using state-of-the-art injection techniques.

By leveraging DCT and HDBSCAN, FreqFed aims to build a defense that is untangled from direct weight inspection and robust against this powerful, adaptive adversary model, without making restrictive assumptions about data or attack types.

Key Findings

[▶ Watch: Limitations of current FL poisoning defenses (1:00)]()

FreqFed's core contribution lies in its paradigm shift: interpreting model weights as signals and analyzing them in the frequency domain. This led to several key findings and successful outcomes:

1. Novel Frequency Domain Analysis for Defense: FreqFed successfully introduced the use of Discrete Cosine Transform (DCT) to analyze local model updates in the frequency domain. This novel approach allows for the identification of core frequency components that are robust to malicious manipulation, moving away from vulnerable direct weight inspection.
2. Effective Mitigation of Diverse Poisoning Attacks: The framework demonstrated accurate and effective mitigation of both untargeted poisoning attacks (such as Label Flipping, Random Updates, and Optimized PGD) and a wide array of targeted (backdoor) attacks (including Constrain-and-Scale, Edge-case PGD, Distributed Backdoor Attacks (DBA), 3DFed, Mirai Scanning, and Neurotoxin).
3. Robustness Against Adaptive Adversaries: FreqFed proved highly resilient against sophisticated adaptive attack strategies. This included adversaries attempting to manipulate frequency representations (e.g., by constraining loss in the frequency domain, injecting triggers in high-frequency bands, or trying to align with benign frequency components) and complex attacks like multiple backdoors and concentrated backdoor attacks.
4. Preservation of Model Utility: Crucially, FreqFed achieved effective attack mitigation with a negligible impact on the main task accuracy (MA) of the aggregated model. In most scenarios, MA remained at levels comparable to benign, no-attack situations, indicating that the defense does not compromise the model's utility. For example, on Cifar-10 against Label Flipping, FreqFed restored MA from 35.8% to 77.1%, nearly identical to the 77.3% MA without any attack.
5. Broad Applicability Across Domains and Data Distributions: FreqFed's effectiveness was validated across nine diverse datasets, multiple model architectures, and five distinct application domains: image classification, word prediction, IoT intrusion detection, speech verification, and graph classification. It also demonstrated remarkable robustness to varying non-iid rates (from 1.0 to 0.0), a significant advantage over many existing defenses that falter in non-iid settings.
6. Outperformance of State-of-the-Art Defenses: Comparative analysis against prominent existing defenses like Krum, AFA, Median, DP, FoolsGold, BayBFed, FLAME, DeepSight, and Auror consistently showed FreqFed's superior performance. FreqFed was often the only defense to maintain 0.0% Backdoor Accuracy (BA) and high Main Task Accuracy (MA) across a wide range of attacks, datasets, and data distributions.

Technical Deep Dive

[▶ Watch: Discrete Cosine Transform (DCT) technical explanation (2:40)]()

FreqFed's innovative design is rooted in the intuition that malicious updates, whether for untargeted degradation or backdoor injection, will manifest as detectable anomalies or shifts in the model's frequency components, particularly the low-frequency components. This intuition is supported by two observations: neural network model weights primarily concentrate energy in low-frequency DCT components, and training prioritizes learning these low frequencies first.

The FreqFed framework, as described in Algorithm 1, operates in three critical phases on the server side during each training round t:

1. Frequency Analysis of Local Models:

• After each client i performs its local training (ClientUpdate function) and sends its updated local model W_i to the server, the server immediately transforms W_i into the frequency domain.
• This is achieved using the 2D Discrete Cosine Transform (DCT), which converts the model's weights into a matrix of DCT coefficients, V_i. The model weights are treated as a signal for this transformation.

1. Model Filtering: This is the core of FreqFed's defense mechanism and involves two sub-steps:

• Low-Frequency Component Extraction: From the V_i matrix of DCT coefficients, the server extracts only the low-frequency components, denoted as F_i. The Filtering function selects coefficients V_ij where the sum of indices i + j is less than or equal to V/2 (where V is a predefined threshold related to the dimensions of V_i). These selected components are then stored as a vector F_i. This step leverages the energy compaction property of DCT and the hypothesis that significant malicious alterations will be reflected in these low-frequency patterns.
• Clustering with HDBSCAN: Once all clients' F_i vectors are extracted, the Clustering procedure begins.
• A distances_matrix is computed, storing the cosine distance (calculated as 1 - CosineSim(F_i, F_j)) between every pair of low-frequency component vectors F_i and F_j. Cosine distance is chosen because it measures the angular difference, which is robust to magnitude variations.
• HDBSCAN is then applied to this distances_matrix. HDBSCAN is particularly effective here because it can identify clusters of varying densities and shapes, and is robust to noise (outliers), which malicious updates are expected to be.
• HDBSCAN returns cluster_ids for each model update. The algorithm then identifies the max_cluster, which is the cluster containing the largest number of model updates. The fundamental assumption here is that the majority of clients are benign.
• A list B is compiled, containing the indices of all original model updates W_b that belong to this max_cluster. These are the "accepted" models.

1. Model Aggregation:

• Finally, only the original model updates W_b corresponding to the indices in the B list (i.e., those identified as benign by the clustering) are aggregated.
• The aggregation is performed using the standard FedAVG algorithm: G_t+1 = (Σ_l=1L W_bl) / L. This ensures that malicious updates, filtered out by the HDBSCAN clustering on their frequency representations, do not contribute to the global model update.

This entire process repeats for T training iterations. By operating in the frequency domain and employing density-based clustering, FreqFed achieves its goal of effectively identifying and removing malicious updates without requiring direct inspection of raw model weights, making it highly robust to various attack types, strategies, and client data distributions. The computational overhead is manageable, as the DCT and HDBSCAN operations are performed once per client per round on compressed frequency representations.

Demo / Proof of Concept

[▶ Watch: HDBSCAN Clustering for anomaly detection (4:00)]()

The talk transcript refers extensively to "Figure 1 of the paper" and "Algorithm 1 of the paper," along with numerous tables (Table I-XII) detailing experimental results. This indicates that FreqFed is a thoroughly evaluated research contribution, with its design and performance documented in an accompanying paper. However, the transcript does not mention any live demonstration of the FreqFed system, a publicly released codebase, or a specific proof-of-concept tool during the presentation. The evaluation setup describes experiments run on a server with specific GPU and CPU configurations, suggesting a robust experimental implementation rather than a public demo.

Defensive Implications

[▶ Watch: Why existing defenses fail adaptive attacks (5:50)]()

FreqFed offers several critical implications for defenders in Federated Learning environments, providing a robust pathway to enhancing model security:

1. Shift from Weight-Space to Frequency-Space Analysis: Defenders should consider adopting frequency domain analysis as a primary mechanism for detecting malicious model updates. FreqFed demonstrates that analyzing Discrete Cosine Transform (DCT) components, particularly low-frequency ones, is a powerful and resilient approach compared to traditional methods that directly inspect model weights (e.g., L2 norms, cosine similarity in weight space). This paradigm shift makes defenses more robust against adaptive attackers who can manipulate weights to appear benign.
2. Robustness Against Adaptive and Sophisticated Attacks: FreqFed's proven effectiveness against a wide array of state-of-the-art untargeted and targeted attacks, including those designed to manipulate frequency representations or create concentrated backdoor clusters, means organizations can deploy FL models with greater confidence. Defenders should prioritize defenses that have demonstrated resilience against such advanced adversarial tactics.
3. Utility Preservation is Key: A major advantage of FreqFed is its ability to mitigate attacks while maintaining Main Task Accuracy (MA) at levels comparable to benign, no-attack scenarios. This is crucial for real-world adoption, as security should not come at the cost of significant performance degradation. Defenders should seek out security mechanisms that minimize utility loss.
4. Applicability Across Diverse FL Scenarios: FreqFed's broad applicability across various application domains (image, text, IoT, speech, graphs), model architectures (CNN, LSTM, GRU, GNNs), and critically, its robustness to non-iid data distributions, makes it suitable for a wide range of real-world FL deployments. Organizations should favor defenses that are not constrained by specific data assumptions.
5. Leveraging Density-Based Clustering for Anomaly Detection: The use of HDBSCAN on low-frequency components provides an effective, unsupervised method for identifying outliers (malicious updates) without requiring prior knowledge of attack patterns or the number of malicious clients. Defenders should explore incorporating advanced clustering techniques for robust anomaly detection in aggregated model updates.
6. Awareness of Majority Assumption: While highly effective, defenders must remain cognizant of FreqFed's inherent limitation: its reliance on the assumption that the majority of clients are benign (i.e., Poisoned Model Rate (PMR) is less than 50%). For scenarios where this assumption might be violated, future research and complementary defenses would be needed. This implies careful client vetting and potentially tiered defense strategies in high-risk environments.

By integrating the principles of FreqFed, defenders can significantly enhance the security posture of their Federated Learning systems, safeguarding global models against a growing landscape of sophisticated poisoning attacks without sacrificing model performance or limiting deployment to specific data distributions.

Key Takeaways

• Novel Frequency Domain Defense: FreqFed pioneers a new approach by transforming local model updates into the frequency domain using Discrete Cosine Transform (DCT) to detect poisoning attacks, moving beyond direct weight inspection.
• Targeted and Untargeted Attack Mitigation: The framework effectively mitigates both untargeted attacks (e.g., Label Flipping, PGD) and targeted backdoor attacks (e.g., Constrain-and-Scale, DBA, Mirai Scanning) across diverse datasets and models.
• Robustness to Adaptive Adversaries: FreqFed demonstrates strong resilience against sophisticated adaptive attacks, including frequency domain manipulation, multiple backdoors, and concentrated backdoor attempts, making it a highly robust defense.
• Preserves Model Utility: Crucially, FreqFed achieves effective attack mitigation with negligible impact on the main task accuracy of the global model, maintaining performance at benign, no-attack levels.
• Broad Applicability and Non-IID Robustness: The defense is effective across various application domains (image, text, IoT, speech, graph) and is particularly robust to different non-iid data distributions, a common challenge for other FL defenses.
• Superior to State-of-the-Art: Comparative analysis shows FreqFed consistently outperforms existing state-of-the-art defenses in mitigating a wide range of attacks while preserving model utility.

About the Speaker(s)

Hossein Fereidooni is the speaker for this presentation at the NDSS Symposium. Based on the highly technical and detailed nature of the research presented in FreqFed, it can be inferred that Hossein Fereidooni is a dedicated researcher in the field of machine learning security, with a particular focus on the vulnerabilities and defenses within Federated Learning systems. Their work demonstrates expertise in signal processing techniques (like DCT), advanced clustering algorithms (HDBSCAN), and a deep understanding of adversarial machine learning tactics.