SLMIA-SR: Speaker-Level Membership Inference Attacks against Speaker Recognition Systems

SLMIA-SR: Speaker-Level Membership Inference Attacks against Speaker Recognition Systems

Speakers: Guangke Chen

Conference: NDSS Symposium

YouTube: (no public video)

Overview

In an era where voice data is increasingly prevalent across smart devices, social media, and various services, the privacy implications of Speaker Recognition Systems (SRSs) have become a critical concern. This talk, "SLMIA-SR: Speaker-Level Membership Inference Attacks against Speaker Recognition Systems," presented by Guangke Chen, introduces a groundbreaking Membership Inference Attack (MIA) specifically tailored to reveal whether a particular speaker's voice data was used in the training of an SRS. Traditional MIAs, primarily designed for other machine learning domains like image classification, have proven ineffective against the unique architectures and training paradigms of SRSs, often yielding unsatisfactory performance (less than 2% True Positive Rate at 0.1% False Positive Rate when ported).

SLMIA-SR addresses this significant gap by focusing on a more practical speaker-level inference, rather than example-level, determining if any voices of a given speaker were part of the training data, regardless of whether the inference voices themselves were present. This is crucial for auditing compliance with privacy regulations like GDPR and the AI Bill of Rights, empowering users to verify consent, and enabling developers to assess and mitigate privacy risks. The work provides a comprehensive framework, including novel feature engineering, advanced training strategies, and query reduction techniques, demonstrating its effectiveness against state-of-the-art and commercial SRSs.

Background

[▶ Watch: Introduction and motivation for speaker recognition privacy (0:00)]()

Speaker Recognition Systems (SRSs) are deep neural networks designed to identify or verify individuals based on their voice. These systems typically operate in three phases: training, enrollment, and recognition. During training, a background model learns to map voices to embeddings such that voices from the same speaker are clustered (high intra-similarity) and voices from different speakers are separated (low inter-dissimilarity). This is often achieved using verification-based losses like Angular Prototypical (AP) or Generalized End-to-End (GE2E) loss, which directly operate on embeddings, or classification-based losses like Cross-Entropy (CE), where the final classification layer is discarded after training to support an open-set recognition paradigm.

While the security of SRSs against adversarial examples, deepfakes, and dictionary attacks has been extensively studied, the privacy implications, particularly concerning the leakage of training data membership, remained largely unexplored. Membership Inference Attacks (MIAs) are a standard methodology to quantify such privacy risks. MIAs exploit the tendency of deep neural networks to "memorize" or overfit to their training data, exhibiting different behaviors (e.g., higher confidence, distinct embedding distributions) for training samples compared to unseen samples.

Prior MIA approaches, however, faced significant limitations when applied to SRSs. Existing methods, even those designed for embedding models like FaceAuditor, often relied on a small set of features, primarily focusing on intra-similarity. They also struggled with the open-set nature of speaker recognition and the practical reality that inference voices are rarely identical to training voices. Furthermore, previous threat models often assumed unrealistic scenarios, such as the adversary having auxiliary data consisting solely of non-member data. The distinct training objectives and architectural nuances of SRSs (e.g., dropping the final classification layer) rendered these traditional MIAs ineffective, underscoring the need for a specialized approach like SLMIA-SR.

Key Findings

[▶ Watch: Introducing SLMIA-SR: The first speaker-level MIA (2:00)]()

SLMIA-SR makes several pivotal contributions that significantly advance the state of the art in membership inference against speaker recognition systems:

• First Speaker-Level MIA for SRSs: SLMIA-SR is the first attack specifically designed to infer membership at the speaker level, rather than individual voice samples. This addresses a more practical and relevant privacy concern, as it determines if any voice data from a speaker was used in training, even if the specific inference voices were not.
• Comprehensive Feature Extractor: The work introduces a novel and comprehensive set of 103 diverse features, meticulously designed to characterize the differences between member and non-member speakers. These features quantify both intra-similarity (closeness of voices from the same speaker) and inter-dissimilarity (distance between voices of the target speaker and imposters), capturing a richer behavioral footprint than prior MIAs.
• Mixing Ratio Training Strategy: To enhance the attack model's generalizability, a novel mixing ratio training strategy is proposed. This strategy trains the attack model using features derived from both cases where inference voices were and were not part of the training set, allowing for robust performance even when the actual ratio of overlap is unknown.
• Performance Boosting Techniques: SLMIA-SR incorporates voice-number-dependent (VND) attack models to account for varying numbers of available inference voices, and voice chunk splitting to effectively increase the number of voices and improve feature precision, especially in limited data scenarios.
• Black-Box Query Reduction: For practical black-box attack scenarios where query budgets are constrained, two innovative techniques—enrollment voice concatenation and group enrollment—are introduced. These techniques drastically reduce the number of queries to the target SRS with minimal impact on attack effectiveness.
• Superior Performance and Real-World Applicability: Extensive experiments demonstrate that SLMIA-SR consistently and significantly outperforms all prior baseline MIAs across multiple datasets and state-of-the-art SRS architectures. Furthermore, its effectiveness against the commercial Microsoft Azure SRS confirms its real-world applicability and practicality.
• Overfitting-Utility Trade-off: The study reveals a clear relationship between the target SRS's overfitting level and SLMIA-SR's effectiveness, highlighting a non-trivial trade-off between recognition utility (lower EER) and privacy vulnerability.

Technical Deep Dive

[▶ Watch: Why existing MIA fails for speaker recognition systems (3:00)]()

The SLMIA-SR framework is structured as a binary classification task, following a three-stage pipeline common to many MIAs: shadow SRS training, attack model building, and membership inference.

1. Shadow SRS Training

Since an adversary lacks direct access to the target SRS's training data, a shadow SRS is trained as a proxy. The adversary samples an auxiliary speaker dataset (Sa), partitioning it into training speakers (Str) and non-training speakers (Sntr) for the shadow SRS. Crucially, the voices of Str are further divided into Ver (used for shadow SRS training) and Vntr,tr (not used for training). The shadow SRS is trained using Ver from Str with a chosen architecture and training algorithm. A set of imposter speakers (Sim) from Sa, with their voices (Vim), are also defined for later use in inter-feature computation. The adversary is assumed to have knowledge of the target SRS's architecture, which is relaxed in ablation studies.

2. Attack Model Building

The core idea is to train an attack model that can distinguish between member and non-member speakers based on their "behavior" within the SRS.

• Data Generation: The shadow SRS is queried with voices from both its training speakers (Str, using Ver and Vntr,tr) and non-training speakers (Sntr, using Vntr,ntr).
• Feature Extraction: The outputs (embeddings for white-box, scores for black-box) are fed into the feature extractor, which is the most critical component of SLMIA-SR.
• Labeling: Features extracted from Str are labeled "member," and those from Sntr are labeled "non-member."
• Attack Model Training: These labeled features are then used to train a binary classifier (a multilayer perceptron with one hidden layer of 64 neurons and ReLU activation, trained with Adam optimizer for 1,000 epochs) which serves as the attack model.

3. Membership Inference

To infer the membership of a target speaker:

• The target SRS is queried with the speaker's available inference voices.
• The outputs are processed by the same feature extractor.
• The resulting features are fed into the trained attack model, which then makes a binary decision: member or non-member.

Feature Extractor: The Heart of SLMIA-SR

The feature extractor is designed to capture the subtle differences in how an SRS processes voices from its training speakers versus unseen speakers. The hypothesis is that training speakers will exhibit better intra-similarity and inter-dissimilarity. The features are derived from the two fundamental objectives of SRS training:

• Intra-similarity: How close are the voice embeddings of the same speaker?
• Inter-dissimilarity: How far apart are the voice embeddings of distinct speakers?

A similarity function w(A, B) is defined: in white-box scenarios, it's the cosine similarity of centroid embeddings; in black-box, it's approximated by the recognition score.

1. Intra-Features (21 unique features): These quantify similarity within a target speaker's voices.

• Centroid-based similarity (Fc): Measures closeness between each voice and the centroid of all voices from the target speaker.
• Pairwise similarity (Fp): Measures closeness between every pair of voices from the target speaker.
• Refined pairwise similarity (Fp_prime): For each voice, computes statistics (average, negative standard deviation, max, min) of its similarities to other voices from the same speaker.
• Four statistics (average, negative standard deviation, maximum, minimum) are applied to these sets, yielding 21 unique intra-features. Negative standard deviation is used as training speakers are expected to have larger similarities.

2. Inter-Features (82 unique features): These quantify the distance between the target speaker's voices and a set of M imposter speakers' voices.

• Centroid-centroid distance (Fcc): Distance between the centroid of the target speaker's voices and the centroid of all imposter voices.
• Centroid-voice distance (Fcv): Distance between the centroid of the target speaker's voices and each individual imposter voice. A refined version (Fcv_prime) takes statistics over these distances.
• Voice-centroid distance (Fvc): Distance between each individual target speaker's voice and the centroid of all imposter voices. Two refined versions (Fvc_prime and Fvc_double_prime) take statistics.
• Voice-voice distance (Fvv): Distance between each individual target speaker's voice and each individual imposter voice. Two refined versions (Fvv_prime and Fvv_double_prime) are defined.
• The same four statistics (average, negative standard deviation, maximum, minimum) are applied to these 24 distance sets, resulting in 82 unique inter-features.

In total, 103 diverse features are designed, covering both intra-similarity and inter-dissimilarity from multiple perspectives, applicable to both white-box and black-box scenarios.

Attack Model Generalization & Performance Boost (CH-2)

• Mixing Ratio Training Strategy: The practical challenge is that the ratio r of inference voices that were actually in the training set is unknown. Training on a fixed r leads to poor generalization. SLMIA-SR trains its attack model using features from both r=1 (training voices) and r=0 (non-training voices) as member examples, alongside non-training speakers, significantly improving generalizability across different r values (Figure 11).
• Voice-Number-Dependent (VND) Attack Model: Attack performance varies significantly with the number N of inference voices (Figure 5). Instead of a single voice-number-independent (VNID) model, SLMIA-SR builds separate VND models for different N. An algorithm (Algorithm 1 in the paper) can determine an upper bound N' for which features converge, thus limiting the number of VND models needed.
• Voice Chunk Splitting: When N is limited, feature precision suffers. Mimicking how SRSs process short segments, this technique applies a sliding window to split inference voices into multiple overlapped chunks, effectively increasing the number of voices and improving feature precision (Figure 13).

Reducing Black-Box Queries (CH-3)

Black-box scenarios are constrained by query budgets. SLMIA-SR introduces:

• Enrollment Voice Concatenation: Instead of p enrollment queries for p individual voices to compute a centroid embedding, a single concatenated voice concat(a) is enrolled. This reduces p queries to 1, as the embedding of a concatenated voice is highly similar to the centroid of individual voice embeddings (Figure 6).
• Group Enrollment: Leveraging speaker identification systems that allow multiple speakers to be enrolled and queried simultaneously:
• For inter-group features (e.g., voice-centroid distance), enrolling all M imposters allows querying each of the N target speaker's voices once, reducing N * M recognition queries to N.
• For intra-group features (e.g., pairwise similarity), enrolling each of the N target speaker's voices and querying the remaining N-1 voices reduces N(N-1)/2 queries to N.
• Enrollment Template Sharing: Further optimizes by avoiding redundant enrollments for features that use the same templates (e.g., centroid-based intra-features and centroid-centroid inter-features).

These techniques can reduce total queries drastically, for example, from 2N + N*M to 1 + N + M + Q (where Q is for intra-group features), making black-box MIA much more practical.

Demo / Proof of Concept

[▶ Watch: Formal problem definition and adversarial threat model (4:00)]()

The efficacy and practicality of SLMIA-SR were demonstrated through extensive experiments across diverse datasets, state-of-the-art SRS models, and even a commercial system.

Experimental Setup:

• Datasets: Three widely-used voice datasets were utilized: VoxCeleb-2 (over 1M voices, 6,112 English speakers), LibriSpeech (0.3M voices, 2,484 English speakers), and KeSpeech (0.2M voices, 6,110 Mandarin speakers). These were carefully partitioned for shadow and target SRS training, imposters, and member/non-member voices (Table III).
• Target SRSs: Five state-of-the-art SRS architectures were evaluated: LSTM, TDNN, RawNet3, ResNetSE34V2, and VGGVox, combined with four training losses: GE2E, CE, AAM, and AP (Table IV). These models were trained in their default settings for 50 to 1500 epochs.
• Baselines: SLMIA-SR was compared against six recent baselines designed for embedding models: LRL-MIA, TKL-MIA, EncoderMI-T, EncoderMI-V, FaceAuditor-S, and FaceAuditor-P/R.
• Evaluation Metrics: Standard MIA metrics were used: Accuracy, AUROC (Area Under the Receiver Operating Characteristic curve), and critically, True Positive Rate (TPR) at low False Positive Rates (FPR) of 0.1%, 0.2%, and 1%.

Key Demonstrations and Results:

1. Overall Performance (Sufficient Inference Voices, r=0): In a setting with sufficient inference voices and no overlap with training data (r=0), SLMIA-SR consistently outperformed all baselines. For instance, on the LSTM-GE2E model with VoxCeleb-2, SLMIA-SR improved Accuracy by 19.4%, AUROC by 16.6%, TPR at 0.1% FPR by 31.9%, and TPR at 1% FPR by 50.6% compared to the best baseline (Table V). This highlights the power of its 103 comprehensive features.
2. Performance with Limited Inference Voices (N=10, with chunk splitting & VND models): Even with limited inference voices (N=10), SLMIA-SR significantly outperformed baselines, particularly in TPR. For example, on VGG-GE2E with LibriSpeech, TPR at 0.2% FPR increased from 4.8% to 46.7% (Table VI). This demonstrates the effectiveness of voice chunk splitting and VND models in practical, resource-constrained scenarios.
3. Commercial SRS Evaluation (Microsoft Azure): To showcase real-world applicability, SLMIA-SR was tested against Microsoft Azure's SRS. Using a Res-AP shadow model and only intra-features (due to query cost constraints), SLMIA-SR achieved an 85% AUROC and over 26% TPR at 0.5% FPR (Figure 8). This is a strong proof-of-concept for its effectiveness against deployed commercial systems, with potential for even higher performance with full features and techniques.
4. Query Reduction Effectiveness: The proposed query reduction techniques—enrollment voice concatenation and group enrollment—were shown to drastically reduce black-box queries (e.g., from 2N + M*K + Q*N to 1 + N + M + Q total queries) with minimal to no impact on attack performance (Table VII). This makes black-box MIAs significantly more practical.

These extensive evaluations serve as the practical demonstration of SLMIA-SR's capabilities, validating its design choices and confirming its superior performance and real-world relevance.

Defensive Implications

[▶ Watch: SLMIA-SR system design: The three-stage pipeline (6:00)]()

The findings from SLMIA-SR provide critical insights for developers and operators of Speaker Recognition Systems to build more privacy-preserving systems. Countermeasures can be broadly categorized into training and inference phase defenses, each with its own trade-offs.

Training Phase Defenses

1. Alleviating Overfitting: The ablation study (Figure 14) clearly shows that SLMIA-SR's effectiveness increases with the overfitting level of the target SRS. Therefore, techniques like early-stopping during training can reduce MIA effectiveness. However, this comes at a direct cost to the utility of the speaker recognition system, as reduced overfitting often means an increased Equal Error Rate (EER) during testing, making the SRS less accurate. Developers must balance privacy with utility.
2. Differential Privacy (DP-SGD): Applying Differential Privacy (DP-SGD) during the training process can effectively prevent SLMIA-SR. Experiments showed that with a privacy parameter of σ=0.2, the attack performed almost like a random guesser. The significant drawback, however, is a substantial sacrifice in SR utility, leading to an increase in testing EER by over 10% (Table VIII). This severe degradation in performance makes DP-SGD a challenging countermeasure for practical SRS deployments.

Inference Phase Defenses

1. Hiding Scores: Many commercial SRSs provide recognition scores (e.g., similarity scores) as part of their API output, which SLMIA-SR leverages in black-box scenarios. A straightforward defense is to only output binary "accept" or "reject" decisions, thereby preventing score-based MIAs. However, this significantly reduces the utility for third-party developers who often require granular scores to fine-tune thresholds for specific applications, potentially hindering innovation and customizability.
2. Decision-only MIA (Future Work): While hiding scores can mitigate current score-based attacks, future research might explore decision-only MIAs. These attacks could involve using adversarial techniques to find minimal perturbations that flip an SRS's decision, and then using these perturbations or other derived features for membership inference without direct score access. This highlights the ongoing arms race in privacy research.

Other Considerations

• Removing Shadow SRS Training: If an adversary can obtain a sufficient dataset of non-training speakers (e.g., through artificial voice generation), they might be able to tune a threshold or train an out-of-distribution classifier directly, potentially bypassing the need for a full shadow SRS training phase.
• Extending to Other Biometrics: Given the similar pipelines and training objectives across various biometric recognition systems (e.g., face or fingerprint recognition), the principles and techniques developed for SLMIA-SR could potentially be extended to assess privacy risks in these other modalities.

Ultimately, the choice of countermeasures involves a careful consideration of the trade-off between privacy protection and system utility. SLMIA-SR provides the necessary tools for SRS developers and regulators to quantify these risks and make informed decisions about privacy-preserving design.

Key Takeaways

• SRSs are Vulnerable to Speaker-Level MIAs: Deep neural network-based Speaker Recognition Systems are susceptible to novel speaker-level membership inference attacks, revealing whether a speaker's voice data was used in training, even if the specific inference voices were unseen.
• Comprehensive Features are Crucial: SLMIA-SR's success stems from its 103 diverse features, which effectively capture both intra-similarity and inter-dissimilarity behaviors unique to member speakers, outperforming prior MIAs that relied on limited feature sets.
• Practicality via Generalization and Query Reduction: The mixing ratio training strategy enhances attack generalizability, while voice-number-dependent models, voice chunk splitting, enrollment voice concatenation, and group enrollment make black-box attacks practical and efficient, even with limited data and query budgets.
• Overfitting is a Privacy Risk: There's a clear trade-off between an SRS's utility (lower EER) and its vulnerability to MIA; higher overfitting, while improving recognition performance, also increases privacy leakage risk.
• Defenses Come with Utility Costs: While countermeasures like early-stopping and Differential Privacy can mitigate MIAs, they often incur significant reductions in the SRS's recognition utility. Hiding recognition scores is also a defense, but it limits developer utility.
• Real-World Threat: SLMIA-SR's demonstrated effectiveness against commercial systems like Microsoft Azure confirms that speaker-level membership inference is a tangible privacy threat to deployed SRS services.

About the Speaker(s)

Guangke Chen is the presenter of the work "SLMIA-SR: Speaker-Level Membership Inference Attacks against Speaker Recognition Systems" at the NDSS Symposium. The transcript and metadata provided do not offer further biographical details about Guangke Chen beyond their name and affiliation with this research.