Towards Practical, End-to-End Formally Verified X.509 Certificate Validators with Verdict

MalGuard: Towards Real-Time, Accurate, and Actionable Detection of Malicious Packages in PyPI Ecosystem

Speakers: Xingan Gao (Yangzhou University), Xiaobing Sun (Yangzhou University), Sicong Cao (Yangzhou University), Kaifeng Huang (Tongji University), Di Wu (University of Southern Queensland), Xiaolei Liu (China Academy of Engineering Physics), Xingwei Lin (Zhejiang University), Yang Xiang (Swinburne University of Technology)

Conference: USENIX Security

YouTube: Not applicable (peer-reviewed paper, no recorded talk)

Overview

The Python Package Index (PyPI), the official third-party software repository for Python, has become an increasingly attractive target for malicious actors. With a staggering 704,102 malicious packages discovered across third-party registries by 2024—a 156% year-on-year increase—the need for robust, real-time, and accurate detection mechanisms is paramount. This paper introduces MalGuard, a novel approach designed to address the escalating threat of malicious packages within the PyPI ecosystem. MalGuard stands out by demonstrating that lightweight machine learning (ML) models, when equipped with a comprehensively curated feature set, can achieve detection effectiveness comparable to more complex large language models (LLMs), but with significantly reduced computational overhead.

MalGuard's core innovation lies in its ability to automate the extraction of sensitive API features using social network graph centrality analysis and to provide explainable outputs for identified malicious packages by integrating the LIME (Local Interpretable Model-agnostic Explanations) algorithm with ML models, further refined by LLMs. This approach tackles two critical challenges: the heavy reliance on manual feature engineering in traditional ML and the lack of actionable explanations in existing detection systems. By combining the efficiency of ML with the analytical power of graph theory and the interpretability of LLMs, MalGuard offers a practical, scalable solution for maintaining the security and integrity of the PyPI community, as evidenced by its success in identifying numerous previously unknown malicious packages in the wild.

Background

The proliferation of open-source software and its accompanying package registries, such as PyPI, has revolutionized software development. However, this convenience comes at a significant security cost. Attackers frequently exploit these platforms to distribute malware, often through sophisticated techniques like typosquatting or embedding malicious code within seemingly benign packages. Recent reports, including one from Sonatype, highlight the alarming rate at which malicious packages are infiltrating these ecosystems, making automated and effective detection a critical security imperative. Examples such as the "lumma" Windows Trojan targeting cryptocurrency wallets, distributed via typosquatting the crytic-compile package, underscore the severity and evolving nature of these threats.

Previous efforts to detect malicious packages have broadly fallen into two categories: traditional machine learning (ML) models and, more recently, large language models (LLMs). Traditional ML approaches, while often efficient, heavily rely on manually pre-defined feature sets. This manual analysis is resource-intensive, difficult to scale, and struggles to keep pace with the rapid evolution of attacker tactics and new malicious behaviors. As new malicious samples emerge, security professionals must invest considerable time and effort to analyze their characteristics and update the feature set, a process prone to missing critical indicators.

On the other hand, LLM-based approaches, such as those that fine-tune models like BERT (e.g., CEREBRO, EA4MP) or directly leverage commercial LLMs like GPT-3.5-turbo, have shown promise in understanding code semantics and identifying malicious patterns. However, their primary drawback is the substantial computational overhead and deployment cost associated with training, fine-tuning, or repeatedly querying these large models, particularly when dealing with the massive scale of the PyPI ecosystem. This makes real-time, continuous monitoring challenging and expensive. Furthermore, while these models can classify packages as malicious, they often function as "black boxes," providing little to no insight into why a package was flagged. This lack of explainability hinders administrators from quickly understanding the specific malicious behaviors, verifying detections, and taking targeted remediation actions. MalGuard was developed to bridge these gaps, offering a balanced approach that combines efficiency, accuracy, automation, and interpretability.

Key Findings

MalGuard's research yielded several significant findings, challenging conventional wisdom and offering practical advancements in malicious package detection:

• Lightweight ML Models Rival LLM Effectiveness with Comprehensive Features: An empirical study demonstrated that traditional ML models like Random Forest (RF) and Extreme Gradient Boosting (XGBoost), when trained on a sufficiently comprehensive feature set of 132 dimensions, achieved detection effectiveness (precision, recall, F1 score) comparable to, and in some cases surpassing, state-of-the-art (SOTA) LLM-based approaches such as EA4MP, CEREBRO, and GPT-3.5-turbo. For instance, the RF model achieved an accuracy only 0.5% lower and recall just 1.3% lower than GPT-3.5-turbo. Crucially, these ML models offered drastically superior time efficiency, requiring only seconds for training (0.097 to 22.85 seconds) and milliseconds for detection per package (0.0035 seconds per package), compared to hours for LLM fine-tuning (e.g., EA4MP: 30,741.67 seconds, CEREBRO: 2,439 seconds). This finding underscores the viability of efficient, lightweight solutions for real-time detection.

• Automated Feature Extraction via Graph Centrality and LLM Refinement: MalGuard successfully addresses the challenge of manually pre-defined feature sets by automating the extraction of sensitive APIs. By constructing API call graphs and applying graph centrality analysis (Closeness, Degree, Katz, Harmonic centrality), the system automatically identifies critical APIs with anomalous invocation patterns indicative of malicious activity. This automated process is further refined by leveraging GPT-3.5-turbo to filter out irrelevant APIs and create a Ground_Truth dataset linking sensitive API names to their potential malicious behaviors, significantly reducing human effort and improving feature quality.

• Explainable Outputs for Actionable Threat Intelligence: MalGuard integrates the LIME algorithm with its ML models to provide clear, human-readable explanations for detected malicious packages. By identifying the top 10 most influential non-zero features contributing to a model's decision and cross-referencing them with the LLM-generated Ground_Truth dataset, MalGuard outputs detailed insights into the specific sensitive APIs used, their file locations, invocation context, and potential malicious purposes. This enhanced explainability garnered an average user rating of 3.5 or higher from security professionals, proving its utility in verifying and understanding malicious behaviors.

• Significant Real-World Impact: In a real-world validation exercise, MalGuard monitored 64,348 newly uploaded packages to PyPI over a five-week period (December 21, 2024, to January 28, 2025). The system successfully identified 113 previously unknown malicious packages. Upon reporting these to PyPI officials, 109 of them were subsequently removed, demonstrating MalGuard's practical efficacy and its potential to actively secure the PyPI ecosystem.

• Robustness Against Adversarial Attacks: MalGuard demonstrated considerable resilience against both feature space attacks (random noise, feature value transformation) and source code level adversarial attacks (injecting benign code). Even under aggressive randomization, the worst-performing model (XGBoost) maintained an F1 score of 75.7%, while others stayed above 85%. Against source-level attacks, all models retained F1 scores above 91.1%, highlighting the robustness of centrality-based features in preserving the discriminative power of the detection system despite obfuscation attempts.

Technical Deep Dive

MalGuard's architecture is meticulously designed to achieve its goals of automated, accurate, and explainable malicious package detection. The system operates through a four-step workflow: API Call Graph Generation, Sensitive API Feature Set Extraction and Filtering, Model Training, and Explanation Output Generation.

API Call Graph Generation

The foundational step involves performing static analysis on each Python package to construct its Abstract Syntax Tree (AST). From the AST, an API Call Graph is generated. In this graph, each node represents an API function, method, or module, and an edge signifies an invocation relationship from a source node to a target node. Unlike Control Flow Graphs (CFGs), API call graphs emphasize high-level semantic invocation logic, making them particularly suitable for identifying suspicious behavior patterns.

A unique challenge in Python packages, especially smaller malicious ones, is that attackers often write code directly in the global scope, leading to a lack of direct invocation relationships between certain APIs. This can result in zero centrality values, obscuring the significance of invoked APIs. To mitigate this, MalGuard adjusts centrality calculations by adding a default value of 1 to all centrality scores, ensuring that every API's importance is captured regardless of direct invocation links in the graph.

Centrality Analysis

Inspired by its success in Android malware detection, MalGuard leverages graph centrality analysis to identify critical APIs within the call graphs. Four widely used centrality metrics are employed to comprehensively assess the importance and influence of each API:

1. Closeness Centrality (CC): Measures how central an API is based on its average shortest path distance to all other APIs in the graph. A higher closeness centrality indicates that an API can quickly interact with or influence other APIs, making it efficient in spreading information or control. The formula is:

$CC(v) = \frac{N-1}{\sum_{u \in V, u \neq v} d(v, u)}$

where $N$ is the total number of nodes, and $d(v,u)$ is the shortest path distance between nodes $v$ and $u$.

1. Degree Centrality (CD): Quantifies an API's importance based on the number of direct connections (invocations) it has with other APIs. A high degree centrality suggests an API is frequently invoked or invokes many others, making it a highly active component. The formula is:

$CD(v) = \frac{deg(v)}{N-1}$

where $deg(v)$ is the number of edges connected to node $v$.

1. Katz Centrality (CK): Extends degree centrality by considering not just immediate neighbors but also more distant connections, with a damping factor ($\alpha$) that penalizes distant interactions. This metric captures an API's influence through its entire network. The formula is:

$CK(v) = \alpha \sum_{u \in V} A_{vu} CK(u) + \beta$

where $\alpha$ is a damping factor, $A_{vu}$ is an entry in the adjacency matrix (1 if $v$ invokes $u$, else 0), and $\beta$ is a constant.

1. Harmonic Centrality (CH): Addresses limitations of closeness centrality by summing the reciprocal of distances, effectively accounting for disconnected nodes. This provides a robust measure of influence even in fragmented graphs. The formula is:

$CH(v) = \sum_{u \in V, u \neq v} \frac{1}{d(v,u)}$

Empirical results highlighted significant differences in API usage between benign and malicious packages. Malicious packages frequently invoke APIs like os.getenv, subprocess.Popen, install.run, base64.b64decode, and exec for evasion, process creation, data obfuscation, and arbitrary code execution. Benign packages, in contrast, tend to use simpler data processing APIs such as int, str, list, and print. This distinction, captured by centrality scores, forms the basis for feature extraction.

Sensitive API Feature Set Extraction and Filtering

To automate feature selection, MalGuard aggregates the centrality values of APIs with the same name across all malicious packages. The average centrality score for each API is calculated, and APIs are ranked accordingly. The top K APIs (with K empirically set to 500 for optimal effectiveness) are initially selected as the sensitive API feature set.

This initial set undergoes a crucial filtering and refinement step using GPT-3.5-turbo. Through role-based prompt engineering, each of the top 500 APIs is sent to the LLM for analysis. The LLM assesses whether an API could potentially be used for malicious purposes. Only APIs deemed potentially malicious are retained in the feature set. For these retained APIs, the LLM further analyzes and infers their possible malicious behaviors, saving these analyses in a Ground_Truth dataset in the format api_name: malicious_behavior. This process is highly efficient, requiring only a single query per sensitive API, significantly reducing the computational and economic costs typically associated with LLM-based detection. This step transforms raw centrality data into semantically rich and security-relevant features.

Model Training

With the refined sensitive API feature set, MalGuard proceeds to train ML models. For each package in the dataset, a feature vector is extracted. The feature value for each API in the sensitive API set is its corresponding centrality value within that package. These feature vectors are then used to train various ML classifiers. The paper evaluated five widely adopted ML models: Naive Bayes (NB), Extreme Gradient Boosting (XGBoost), Random Forest (RF), Support Vector Machine (SVM), and Multi-Layer Perceptron (MLP). The RF model consistently performed best among these, achieving high precision (99.6%), recall (98.4%), and F1 score (99.0%).

Explanation Output based on LIME Algorithm

A hallmark of MalGuard is its ability to provide explainable outputs, addressing a significant limitation of many black-box ML models. This is achieved by integrating the LIME (Local Interpretable Model-agnostic Explanations) algorithm into the trained ML model. LIME works by approximating the behavior of any black-box model locally around a specific instance, highlighting the features most influential in its prediction.

For a detected malicious package, LIME identifies the top 10 most influential non-zero features (i.e., sensitive APIs) that contributed to the model's classification decision. These API names are then cross-referenced with the Ground_Truth dataset (generated by GPT-3.5-turbo) to retrieve all associated potential malicious behaviors. MalGuard further enhances interpretability by locating the specific lines of code where these APIs appear and organizing the explanation output sequentially based on the APIs' invocation order within the package's files. This detailed output, as exemplified in Figure 2 of the paper, provides:

• The Sensitive API Name (e.g., requests.get, subprocess.call).
• The File Name and Code Line where the API is used (e.g., pandas-numpy-8.19.3/reqinstaller/__init__.py line 6).
• The Usage Context (global scope or specific function).
• A list of Potential Malicious Behaviors (e.g., "Performing unauthorized data extraction from a remote server," "Execute harmful system commands or shell scripts").

This comprehensive explanation empowers security researchers and administrators to quickly understand the nature of the threat, verify the model's decision, and take targeted actions, transforming a binary classification into actionable intelligence.

Demo / Proof of Concept

As this work is presented as a peer-reviewed conference paper rather than a recorded talk, a live demonstration was not part of the presentation format. However, the paper provides a robust Proof of Concept through its "Wild Truth" experiments and detailed explanation outputs, effectively demonstrating MalGuard's practical applicability and efficacy.

The most compelling proof of concept comes from MalGuard's real-world validation. The researchers deployed MalGuard to monitor the PyPI ecosystem, crawling 64,348 packages newly uploaded between December 21, 2024, and January 28, 2025. This extensive monitoring period allowed MalGuard to operate in a live environment, identifying threats as they emerged. The system successfully flagged 144 suspicious packages, and after a meticulous manual review process (involving two authors and an external security expert), 113 of these were confirmed to be previously unknown malicious packages. The subsequent reporting to PyPI officials led to the removal of 109 of these confirmed malicious packages by January 21, 2025. This tangible impact on PyPI security serves as a powerful demonstration of MalGuard's real-time detection capabilities and its ability to uncover emergent threats in the wild.

Furthermore, the paper explicitly details the explanation output generated by MalGuard for identified malicious packages, serving as a critical component of its proof of concept. For example, Figure 2 in the paper illustrates the explanation for a package named pandas-numpy-8.19.3. This output clearly identifies sensitive APIs like requests.get, subprocess.call, setup, find_packages, base64.b64decode, and exec, along with their exact file paths, line numbers, and inferred malicious behaviors (e.g., "Performing unauthorized data extraction from a remote server," "Execute harmful system commands or shell scripts," "Arbitrary code execution"). The user study, where 24 software security experts rated these explanations with an average score of 3.5 or higher, further validates that these outputs are indeed useful and help researchers understand attacker intent.

The analysis of false positives also served as a valuable demonstration. For instance, benign packages like eazure-0.1.1 were misclassified by other methods but correctly handled by MalGuard. This was attributed to MalGuard's use of API centrality values as features, which allowed it to distinguish between legitimate use of sensitive APIs (where centrality might not be anomalously high) and malicious use. This highlights the nuanced detection capability of the approach beyond simple presence-based rules. These experiments collectively serve as a robust proof of concept for MalGuard's effectiveness, efficiency, and actionable interpretability.

Defensive Implications

MalGuard offers several critical implications for defenders, ranging from PyPI maintainers to individual developers:

• Real-time, Automated Threat Detection for Package Registries: For maintainers of package registries like PyPI, MalGuard presents a highly efficient and effective solution for real-time scanning of newly uploaded packages. Its lightweight ML models, coupled with rapid feature extraction, can process packages in milliseconds, allowing for daily updates and prompt identification of emerging threats. The demonstrated success in uncovering 113 unknown malicious packages and the subsequent removal of 109 by PyPI officials underscore its practical value as a proactive security measure. Implementing MalGuard can significantly reduce the window of exposure for users downloading newly published malicious software.

• Actionable Intelligence and Faster Triage: The explainable outputs generated by MalGuard, powered by LIME and LLM-derived malicious behavior descriptions, are a game-changer for security analysts. Instead of just a binary "malicious/benign" verdict, defenders receive precise details: the sensitive APIs involved, their exact locations in the code (file and line number), their invocation order, and their potential malicious purposes. This actionable intelligence drastically reduces the manual effort required for verification and analysis, enabling faster triage of incidents, more targeted remediation, and a clearer understanding of attacker techniques.

• Reduced Reliance on Manual Feature Engineering: MalGuard's automated feature extraction via graph centrality analysis and LLM filtering significantly lessens the burden on security professionals. Historically, keeping pace with evolving malicious behaviors required continuous, labor-intensive manual analysis to update feature sets. MalGuard automates this process, making the detection system more scalable, adaptable, and less prone to human error or oversight. This allows security teams to allocate resources to more complex threat intelligence tasks.

• Robustness Against Evolving Adversarial Techniques: The demonstrated resilience of MalGuard against common adversarial attacks, including feature space manipulation and source code obfuscation, provides confidence in its long-term effectiveness. While no system is perfectly impervious, MalGuard's ability to maintain high detection accuracy even under attack suggests that its underlying centrality-based features are robust against typical evasion tactics. Defenders can rely on MalGuard to detect not only straightforward malicious packages but also those employing basic obfuscation.

• Guidance for Secure Development and Dependency Management: For developers, MalGuard's insights can serve as an educational tool. Understanding which APIs are frequently abused by attackers and how their centrality patterns differ in malicious contexts can inform more secure coding practices and dependency auditing. Developers can use such knowledge to critically evaluate third-party packages, scrutinize the use of sensitive APIs within their own projects, and prioritize security reviews based on potential risk indicators highlighted by MalGuard's principles.

In summary, MalGuard provides a comprehensive framework that enhances the speed, accuracy, and interpretability of malicious package detection, empowering defenders with the tools needed to combat the rapidly expanding threat landscape in open-source ecosystems.

Key Takeaways

• Efficiency and Effectiveness of Lightweight ML: MalGuard empirically demonstrates that traditional ML models (e.g., Random Forest, XGBoost) can achieve detection performance comparable to SOTA LLM-based approaches, but with significantly lower computational overhead (seconds for training, milliseconds for detection per package), making them ideal for real-time, high-volume scanning of PyPI.
• Automated Feature Engineering: The approach innovates by automating the extraction of sensitive API features using API call graph centrality analysis and refining this set with GPT-3.5-turbo, eliminating the laborious and time-consuming manual feature engineering typically required for ML models.
• Actionable Explainability: MalGuard integrates the LIME algorithm with ML models and leverages LLM-generated malicious behavior descriptions to produce detailed, human-readable explanations for detected malicious packages, providing insights into specific sensitive APIs, their code locations, and potential malicious purposes.
• Proven Real-World Impact: In a live deployment, MalGuard successfully identified 113 previously unknown malicious packages from 64,348 newly uploaded PyPI packages, with 109 of them subsequently removed by PyPI officials, validating its practical utility and effectiveness in securing the open-source ecosystem.
• Robustness Against Adversarial Attacks: The system exhibits strong resilience against both feature space and source code level adversarial attacks, maintaining high detection effectiveness (F1 scores above 75.7% even for the worst-performing model under aggressive attacks), indicating its reliability against evolving evasion tactics.
• Rapid Iteration Capability: The low computational cost for feature extraction and model training enables MalGuard to be updated daily with newly emerging packages, ensuring timely adaptation to the continuously evolving landscape of malicious threats.

About the Speaker(s)

The research presented in this paper, "MalGuard: Towards Real-Time, Accurate, and Actionable Detection of Malicious Packages in PyPI Ecosystem," was a collaborative effort by a team of distinguished researchers from various academic and industrial institutions. The authors include Xingan Gao, Xiaobing Sun, and Sicong Cao from Yangzhou University; Kaifeng Huang from Tongji University; Di Wu from the University of Southern Queensland; Xiaolei Liu from the China Academy of Engineering Physics; Xingwei Lin from Zhejiang University; and Yang Xiang from Swinburne University of Technology. This diverse team brings expertise in areas such as software security, machine learning, and network security, contributing to the comprehensive and innovative approach presented in MalGuard.