Saving Ourselves the ID Headache: How Purls Can Work for Models and Datasets

Saving Ourselves the ID Headache: How Purls Can Work for Models and Datasets

Speakers: Daniel Bardenstein, CEO and co-founder, Manifest

Conference: VulnCon

YouTube: https://www.youtube.com/watch?v=m4bM2pWJBUQ

Overview

In a critical address at VulnCon, Daniel Bardenstein, CEO and co-founder of Manifest Cyber, illuminated a rapidly escalating challenge in modern cybersecurity: the lack of standardized identification for Artificial Intelligence (AI) artifacts. His talk, "Saving Ourselves the ID Headache: How Purls Can Work for Models and Datasets," posited a pragmatic solution to this looming problem. Bardenstein argued that instead of inventing an entirely new identification scheme for AI, the security community should leverage and adapt existing Package URLs (PURLs), a well-established standard for software components.

The core premise of Bardenstein's presentation was that AI, fundamentally, is a subset of software. Therefore, the robust mechanisms and workflows developed over decades for software supply chain security, particularly those reliant on precise identification, should be extended to AI. The absence of such identifiers for models, datasets, and their various permutations currently cripples organizations' ability to understand their AI attack surface, respond to vulnerabilities, and maintain compliance. By advocating for PURLs, Bardenstein aims to prevent the creation of "yet another standard" that fragments efforts and delays crucial security advancements in the burgeoning AI landscape.

This talk is particularly significant for product security, vulnerability management, and PSIRT teams grappling with the integration of AI into their products and operations. It offers a clear, actionable path forward for gaining essential visibility and control over AI assets, enabling timely vulnerability response, robust software bill of materials (SBOM) generation, and effective governance. As AI vulnerabilities become more prevalent and sophisticated, Bardenstein's proposal provides a foundational element necessary to build resilient AI security postures.

Background

▶ Watch: Organizations don't know what AI models they're using (1:00)

The pervasive integration of AI into organizational workflows has introduced a significant blind spot in enterprise security. As Daniel Bardenstein highlighted, a fundamental problem persists: "people still don't know what AI systems they're actually using in their organizations." This lack of visibility is not a new phenomenon in cybersecurity; it mirrors the challenges faced with traditional software components, famously exemplified by the Log4Shell vulnerability five years ago, and more recently by the Team PCP attacks targeting libraries like Axios, Trivia Action, and Light LM. In these instances, the inability to quickly identify and locate vulnerable components led to widespread, prolonged exposure. For AI, this problem is compounded by the rapid adoption and complex, often opaque, nature of AI models and datasets.

Bardenstein emphasized that AI vulnerabilities are not a hypothetical future threat; they are "already here." These range from traditional software vulnerabilities within the AI infrastructure itself to specific AI-centric risks such as data poisoning, model bias, and hallucinations. He cited instances of child pornography found in publicly available datasets and known biases in major models, underscoring the diverse and severe nature of these issues. As adversaries increasingly target AI supply chains, the frequency and impact of these vulnerabilities are expected to rise.

The bedrock of effective vulnerability response, as Bardenstein reminded the VulnCon audience, relies entirely on identifiers. The established workflow for traditional software involves:

1. Vulnerability Announcement: An issue is announced (e.g., on NVD, OSV, or even social media).
2. Advisory Review: Security teams read advisories to understand the impacted software, versions, and exploitation details.
3. Inventory Check: Organizations cross-reference their software inventory to determine if they are using the affected component and where.
4. Impact Assessment: The severity and blast radius are assessed based on deployment context (e.g., a vulnerable MRI in an offsite clinic versus a busy emergency room).
5. Response: Appropriate containment, mitigation, and patching strategies are developed.

This entire workflow grinds to a halt for AI assets because "we don't have a standard way to identify AI assets." The foundational element – a consistent naming and identification scheme – is missing.

Bardenstein then introduced existing software identifiers as potential solutions, specifically Common Platform Enumeration (CPEs) and Package URLs (PURLs).

• CPEs are sourced from CNAs (e.g., NVD) and primarily describe vendor products (e.g., Microsoft Windows, Progress MoveIt). They are standardized and widely used but suffer from limitations: they are not directly tied to specific artifacts or how software is obtained, and their naming can be ambiguous (e.g., identifying the vendor for XZ Utils). CPEs answer "What is this product called and who made it?"
• PURLs (Package URLs) are inherited from package managers and describe how open-source packages are fetched. They are ecosystem-native, stronger for open source, and provide granular detail about the artifact's source, owner, name, and version. PURLs answer "What exactly is this thing, this artifact that we have deployed in our environment?"

The speaker highlighted a critical choice: either create a brand new identifier for AI assets or leverage an existing standard. He strongly advocated against the former, invoking the classic XKCD comic illustrating how attempts to create a "universal standard" often result in "15 competing standards." Given that "AI is a subset and necessarily is deployed into software," the logical and most efficient path is to adapt an existing, proven standard like PURLs.

Key Findings

▶ Watch: Package URLs (PURLs) are better for open source artifacts (7:00)

The talk's key findings revolve around the critical need for a standardized approach to identifying AI assets and the suitability of Package URLs (PURLs) for this purpose.

Firstly, Bardenstein unequivocally established that AI assets, including models and datasets, currently lack standardized identifiers. This deficiency is a significant impediment to effective AI security, as it prevents organizations from understanding their AI inventory, assessing their exposure to vulnerabilities, and implementing timely defensive measures. Without a consistent naming convention, the entire vulnerability management lifecycle, which is predicated on the ability to uniquely identify affected components, becomes unmanageable for AI.

Secondly, the presentation demonstrated that existing software identifiers, specifically PURLs, are well-suited to address this gap. PURLs offer a level of granularity and flexibility that is particularly beneficial for the complex and dynamic nature of AI artifacts. Unlike CPEs, which focus on vendor products, PURLs are designed to describe specific software artifacts, their origin, and how they are fetched. This "artifact-centric" approach aligns closely with how AI models and datasets are developed, shared, and deployed—often from diverse sources like Hugging Face, GitHub, or cloud provider repositories, and frequently undergoing modifications such as fine-tuning or quantization.

Thirdly, Bardenstein showed that PURLs can provide the necessary precision to describe AI models and datasets comprehensively. Through concrete examples, he illustrated how a PURL can encapsulate vital metadata: the model's owner (e.g., Meta, Anthropic), its base name (e.g., Llama, Claude), its specific version or snapshot (e.g., a commit SHA), its source or repository (e.g., Hugging Face, AWS Bedrock API), its file format (e.g., GGUF, safetensors), and even specific modifications like quantization. This rich descriptive capability is crucial for distinguishing between subtly different versions or derivations of a seemingly identical model, which can have significant security implications.

Finally, the talk underscored that applying PURLs to AI can immediately enable critical security workflows that are currently absent or severely hampered. These include efficient vulnerability tracking, automated generation of AI-specific SBOMs, robust governance and compliance reporting, and the ability to "pin" specific model versions to mitigate supply chain risks. By adopting PURLs, organizations can avoid the costly and time-consuming endeavor of creating a new, AI-specific identification standard, instead leveraging existing infrastructure and expertise from the broader software security domain.

Technical Deep Dive

▶ Watch: Simple proposal: use existing Package URLs for AI assets (14:40)

The technical core of Daniel Bardenstein's proposal lies in demonstrating how Package URLs (PURLs) can effectively model the intricate characteristics of AI artifacts. He began by deconstructing what AI models and datasets "actually look like in real life," highlighting their various identifying components:

1. Names and Major Versions: The public-facing identity of a model, such as GPT-4O, Claude Opus 4.6, or Llama 3.3.
2. Owners or Suppliers: The entity responsible for creating or distributing the model (e.g., Anthropic, Google, Cisco, or an individual on Hugging Face).
3. Snapshots (Sub-versions): Beyond major versions, models often have specific "snapshots" or sub-versions, identified by dates or unique IDs, which are critical for precision in vulnerability management. For instance, loading a model in code typically requires specifying a precise snapshot, not just a major version.
4. Sources/Locations: Where the model or dataset is fetched from significantly impacts its identity. Examples include Hugging Face, GitHub, AWS Bedrock, Google Vertex, ModelScope, or even proprietary supplier APIs. Models from different sources, even if nominally the same, can have subtle differences.
5. Formats: Models can be downloaded in various binary formats (e.g., safetensors, ONNX, GGUF), and datasets in formats like JSONL, Parquet, CSV, or TF Records. The format itself can be a software wrapper, influencing its security posture.
6. Mutations: Models are frequently modified for specific use cases. This includes quantization (shrinking the model for efficiency) or fine-tuning (adapting it for specific tasks). Identifying these mutations is crucial for lineage and blast radius assessment.

Bardenstein further illustrated that AI infrastructure can be hosted and accessed in diverse ways, each impacting how an artifact is identified:

• Directly via supplier API (e.g., Anthropic's API).
• Downloaded or cloned from repositories like Hugging Face.
• Accessed through cloud providers such as AWS Bedrock, Google Vertex, or Azure AI Studio.
• Used via chat interfaces.
• Datasets downloaded from their own websites (e.g., Microsoft's COCO dataset), Hugging Face, or Papers With Code.

The core technical proposal is to map these characteristics onto the established PURL structure, which typically follows the format: pkg:<type>/<namespace>/<name>@<version>?<qualifiers>#<subpath>.

Bardenstein presented a direct comparison between a PURL and a CPE for the same Llama model from Meta:

• PURL Example: pkg:huggingface/meta/llama-3-8b-instruct@0a1b2c3d4e5f6a7b8c9d0e1f2a3b4c5d6e7f8a9b?file_format=gguf&quantization=q4_0
• pkg:huggingface: Indicates the package manager or source is Hugging Face.
• meta: The namespace or owner (Meta).
• llama-3-8b-instruct: The model's base name.
• @0a1b2c3d4e5f6a7b8c9d0e1f2a3b4c5d6e7f8a9b: The version, represented by a specific commit SHA, offering precise versioning.
• ?file_format=gguf&quantization=q4_0: Qualifiers providing additional, crucial metadata: the file format (GGUF) and the specific quantization level (q4_0).

• CPE Example: cpe:/a:meta:llama-3-8b-instruct::::::gguf:
• This CPE provides the vendor (meta), product name (llama-3-8b-instruct), and a wildcard for version, with gguf potentially as a platform or edition. Bardenstein noted the ambiguity of asterisks and the limited space for detailed artifact-specific information.

The PURL clearly offers significantly richer, artifact-specific detail compared to the CPE. It captures the exact source, precise versioning via a commit SHA (common in open-source model repositories), and critical modification details like quantization.

Further examples demonstrated PURLs for different AI deployment scenarios:

1. Cloud API Access (Anthropic Claude):

pkg:api/anthropic/claude-3-sonnet@20240229?interface=api

• pkg:api: Indicates an API interface.
• anthropic: The supplier.
• claude-3-sonnet: The model name.
• @20240229: A specific snapshot/version date.
• ?interface=api: A qualifier to specify API usage.

1. Cloud Provider Hosting (AWS Bedrock for Anthropic Claude):

pkg:bedrock/aws/anthropic/claude-3-opus@20240229?region=us-east-1

• pkg:bedrock: Indicates AWS Bedrock as the hosting platform.
• aws/anthropic: The cloud provider (AWS) and original supplier (Anthropic).
• claude-3-opus: The model name.
• @20240229: Snapshot/version.
• ?region=us-east-1: A qualifier for the specific AWS region.

These examples highlight PURLs' flexibility in expressing the "reality of how models are provided and hosted, how they are built, how they can be customized and modified, how they are versioned." While acknowledging that PURLs aren't perfect, Bardenstein underscored their ability to abstract away complexity for AppSec and product security teams, making AI identification straightforward and automated. The "qualifiers" section of a PURL is especially powerful for capturing the nuanced metadata unique to AI artifacts, such as file formats, quantization levels, interfaces, and regions. The main challenge, as he noted, is standardizing these qualifiers across the community.

Demo / Proof of Concept

▶ Watch: Practical PURL examples for diverse AI models and sources (16:00)

The talk by Daniel Bardenstein did not include a live demonstration or a formal proof of concept in the traditional sense, such as a software tool being run or code being executed to generate PURLs for AI artifacts. Instead, the "technical deep dive" section served to illustrate the concept of how PURLs would be constructed for various AI models and datasets through concrete examples.

Bardenstein's presentation focused on laying the theoretical and practical groundwork for adopting PURLs, using hypothetical PURL strings for models like Llama and Claude to demonstrate their descriptive power. While no direct demonstration of a tool in action was provided, the discussion of "use cases" acted as a conceptual proof of how this identification scheme would function in real-world security scenarios, effectively showcasing the implications and benefits of such a system rather than its live implementation. The speaker's emphasis was on the urgent need for community agreement and subsequent tooling development, rather than presenting a fully-fledged, implemented solution.

Defensive Implications

▶ Watch: Q&A: PURL vs. AI Bill of Materials for detailed model info (30:00)

The adoption of Package URLs (PURLs) for AI artifacts carries profound defensive implications, fundamentally transforming how organizations approach AI security. Daniel Bardenstein outlined several critical use cases that would immediately benefit from this standardized identification scheme, enabling security teams to extend their existing workflows to the AI domain.

1. Vulnerability Tracking and Blast Radius Assessment: This is arguably the most immediate and impactful defensive implication. With PURLs, PSIRT, AppSec, and vulnerability management teams can finally answer the crucial question: "Are we affected?" When a vulnerability is announced in a specific model or dataset—for example, an issue found in "Llama 337B Instruct" that "only affects GGML downloads"—a PURL can precisely identify whether an organization is using that exact variant. Without PURLs, determining the blast radius for such an issue is a manual, often impossible, task. PURLs would allow for rapid, automated lookups against an inventory, saving "seconds or minutes rather than months" in incident response, as was the lesson from Log4Shell.

1. SBOM Generation for AI: The ability to generate comprehensive Software Bill of Materials (SBOMs) is vital for supply chain security. Bardenstein highlighted that existing tools, including Manifest Cyber's own, can extract model information from source code. By standardizing AI identification with PURLs, these tools can automatically generate SBOM entries for every AI model and dataset brought into a product or repository. This ensures that AI components are accounted for in the same way traditional software libraries are, providing a complete picture of an application's dependencies. This is crucial for understanding the provenance and composition of AI systems.

1. Governance and Compliance: As regulatory bodies (like the FDA for medical devices) and customers increasingly demand transparency and accountability for AI systems, robust governance becomes paramount. PURLs enable organizations to demonstrate precisely "what we're using, what we're not using, how we're using it." This ability to clearly articulate the AI assets within an organization, their versions, sources, and any modifications, is essential for meeting compliance mandates and contractual obligations, establishing trust, and managing risk effectively.

1. Pinning Models and Mitigating Supply Chain Attacks: In the wake of supply chain attacks like Team PCP, the practice of "pinning" specific versions of dependencies has gained renewed importance. PURLs facilitate the precise pinning of AI models, whether accessed via APIs or downloaded directly. If a new snapshot or version of a model is released with a critical vulnerability, PSIRT teams can quickly determine if their organization is using that specific vulnerable version or if they have pinned to an unaffected earlier version. This proactive measure significantly reduces exposure to new, unforeseen AI supply chain risks.

In essence, adopting PURLs for AI artifacts extends the well-established principles and tooling of software supply chain security to the emerging domain of AI. It moves AI security from a reactive, often chaotic, state to a proactive, structured, and manageable discipline, allowing defenders to leverage decades of experience in vulnerability management rather than starting from scratch.

Key Takeaways

• AI Security Requires Identifiers: Just like traditional software, effective security for AI models and datasets hinges on having standardized, unambiguous identifiers. Without them, core security functions like vulnerability management and inventory tracking are impossible.
• PURLs are the Optimal Existing Solution: Instead of creating a new standard, Package URLs (PURLs) offer a robust, flexible, and artifact-centric mechanism suitable for identifying AI assets. This approach leverages existing infrastructure and knowledge within the software security community.
• PURLs Provide Granular Detail: PURLs can capture critical metadata about AI artifacts, including owner, base name, precise version (e.g., commit SHA), source (e.g., Hugging Face, AWS Bedrock), file format (e.g., GGUF, safetensors), and modifications like quantization, which is crucial for accurate vulnerability assessment.
• Enables Core Security Workflows: Implementing PURLs for AI immediately empowers organizations to perform essential security tasks such as rapid vulnerability tracking, automated SBOM generation, robust governance, and effective model pinning, thereby integrating AI into existing security frameworks.
• Standardization of Qualifiers is Crucial: To maximize the utility of PURLs for AI, the community must collaborate to define and standardize the "types and qualifiers" (e.g., file_format, quantization, interface, region) that provide specific metadata unique to AI artifacts.
• Adoption and Tooling are Essential: The success of this proposal depends on widespread adoption and support from key stakeholders, including CNAs, government entities (NIST, CISA), and especially tooling vendors and open-source projects, to reduce friction and facilitate practical implementation.

About the Speaker(s)

Daniel Bardenstein is the CEO and co-founder of Manifest Cyber, a software supply chain security startup. His work at Manifest Cyber deeply immerses him and his team in the intricacies of software supply chain security on a daily basis, collaborating with major companies, organizations, and government agencies worldwide. Bardenstein's insights shared at VulnCon are drawn directly from this extensive practical experience, particularly in understanding the challenges and developing solutions for the evolving landscape of AI security for PSIRT (Product Security Incident Response Team) professionals. His expertise lies in bridging the gap between traditional software security practices and the emerging requirements for securing AI artifacts.