Developing & Deploying AI Fingerprints for Advanced Threat Detection

Developing & Deploying AI Fingerprints for Advanced Threat Detection

Speakers: Natalie Isak (Software Engineer, Microsoft); Waris Gill (Applied Scientist, Microsoft)

Conference: [un]prompted 2026 — The AI Security Practitioner Conference

Date: March 3–4, 2026, San Francisco

Watch on YouTube: https://www.youtube.com/watch?v=u7pag5p9z5o

Reading time: ~8 minutes

TL;DR

Microsoft researchers Natalie Isak and Waris Gill presented BinaryShield (referred to in the talk as "Boundary Shield"), a system that converts detected prompt injection attacks into privacy-preserving binary fingerprints and broadcasts them across all of an organization's AI services simultaneously — without sharing any raw user data. The result: a single detection anywhere in a product suite protects every service, and fingerprints match attack variants, not just exact copies.

Introduction

Large organizations deploying AI services face a problem that grows with scale: each product tends to develop its own safety stack independently. Azure AI Foundry, GitHub Copilot, and Microsoft 365 Copilot, for example, operate in separate silos with no signal flowing between them. When an adversary discovers an effective prompt injection against one service, they can spray variations of it across every other service in the organization — and each service is on its own.

The Microsoft team's solution, presented at [un]prompted 2026, is BinaryShield: a cross-service threat intelligence system that fingerprints detected attacks and broadcasts them to all other services in a privacy-preserving form. The key constraint the system had to satisfy: you cannot share raw user prompts across services. The fingerprint must contain enough information to catch related attacks, but nothing that could reconstruct the original query or identify the user who sent it.

The research underpinning the system is available as arXiv:2509.05608.

▶ Watch: The Problem — Siloed Services and Cross-Service Threats (00:00)

The Four-Step Fingerprinting Pipeline

Waris Gill walked through BinaryShield's core pipeline, which converts a raw detected prompt into a shareable fingerprint in four steps:

Step 1: PII Redaction. Before any processing, personally identifiable information is stripped from the prompt. Names, social security numbers, email addresses, and other identifying content are replaced with placeholders. The Microsoft implementation uses an open-source library called Presidio, though Isak noted that many approaches are available. This is a non-negotiable privacy baseline.

Step 2: Semantic Embedding. The redacted prompt is fed into an embedding model to produce a high-dimensional vector capturing the semantic meaning of the text. The Microsoft team used OpenAI's text-embedding-3-large model, which produces vectors of 3,072 dimensions. The output is a long list of floating-point numbers that encodes the meaning of the prompt without preserving its exact wording.

Step 3: Binary Quantization. The floating-point embedding is compressed into a binary vector — just zeros and ones. This dramatically reduces memory footprint and storage cost, and crucially, makes the pipeline more one-way: information is deliberately lost, making it significantly harder for an adversary to reverse-engineer the fingerprint back to the original prompt.

Step 4: Differential Privacy. Controlled noise is added to the binary vector by randomly flipping bits. The degree of noise is governed by an epsilon parameter. When epsilon is large, few bits change — high utility, low privacy. When epsilon is small, more bits are randomized — high privacy, low utility. The system then attaches metadata (query region, tools invoked during execution) and broadcasts the resulting fingerprint to all other services.

▶ Watch: The Four-Step Pipeline Walkthrough (02:00)

Evaluation: The Privacy-Utility Trade-off and Speed Gains

Gill presented the empirical results across two dimensions.

The first chart plotted privacy budget (epsilon) on the x-axis against threat correlation accuracy on the y-axis. The relationship is clear: at epsilon ≈ 0.5, so many bits are flipped that accuracy reaches 0%. As epsilon increases, the fingerprints achieve accuracy approaching raw dense embeddings. The trade-off is real, and the right epsilon is not a technical decision — it should be determined in collaboration with legal and privacy teams based on regulatory context.

The second chart demonstrated the performance advantage. With corpus size on the x-axis and search time on the y-axis, dense floating-point embeddings show significant overhead as data volume grows. BinaryShield is 36 times faster for threat correlation at scale. The binary format allows Hamming distance comparisons (counting differing bits between two vectors) rather than expensive floating-point cosine similarity computations.

As Gill summarized: "BinaryShield provides semantics, speed, and privacy together."

▶ Watch: Evaluation Results — Accuracy and Speed (04:00)

Live Demo: Building a Fingerprint Step by Step

Natalie Isak took over for a live code walkthrough, implementing all four steps from scratch in a Jupyter notebook built with Claude Code.

For Step 1 (PII redaction), she showed the transformation of "My name is John Smith" into a placeholder — with the same treatment applied to social security numbers, email addresses, and all other PII categories.

For Step 2 (embedding), the redacted text was fed into text-embedding-3-large, producing the characteristic long list of floating-point numbers.

For Step 3 (quantization), she compressed those floats into a binary vector and highlighted the privacy benefit: "We're intentionally losing information that's kind of encapsulated by these long floating-point numbers to make it more difficult for an adversary to reverse engineer this pipeline."

For Step 4 (differential privacy), she demonstrated the epsilon parameter's effect, then concatenated all four steps into the fingerprint generation function.

▶ Watch: Live Fingerprint Implementation (06:00)

Proving Similarity: The Hamming Distance Matrix

To validate that BinaryShield actually works as intended, Isak created a test with four prompts:

• The original "ignore all previous instructions" prompt injection
• Variant 1 (slightly modified)
• Variant 2 (further modified)
• One benign prompt: "What is the weather in Seattle?"

She then computed pairwise Hamming distances between all four fingerprints. The results were visual and intuitive: the three injection variants all showed low Hamming distances to each other — lighter colors in the matrix, indicating close neighbors in fingerprint space. The benign prompt showed high distances from all three injections — darker colors, clearly separated.

This is the key property the system requires: semantically similar prompts must produce similar fingerprints, while dissimilar prompts must be clearly distinguished. The test confirmed both.

Gill later addressed a skeptical question about whether binary quantization loses too much semantic information: "Even compressed to binary, a great deal of semantic structure is preserved in those bits. That's what we observed in our evaluations: the binary representations retain enough meaningful information to reliably detect attack variants." High-dimensional source vectors — 768 or 3,072 dimensions — preserve a lot of information even after binarization.

▶ Watch: Hamming Distance Demo and Similarity Verification (10:00)

Cross-Service Integration: One Detection Protects All

The final section addressed how to integrate BinaryShield across a product suite. Isak observed that for any organization with more than one product, safety stacks across products almost certainly diverge over time: different development velocity, different customer requirements, different regulatory environments.

The result is a fragmented situation: a pattern caught by Service Alpha might not be caught by Service Gamma. The BinaryShield registry solves this. When any service detects an attack:

1. It fingerprints the prompt using the four-step pipeline
2. It broadcasts the fingerprint to all other services via the central registry
3. All other services can now detect that attack — and small perturbations of it

The demo showed three services with different block lists. Only Service Alpha could catch "ignore all previous instructions." After Alpha fingerprinted and broadcast the detection, all three services gained the capability to catch that variant. The next time an adversary sprays the attack, every service responds — without any service having to rebuild its safety stack.

▶ Watch: Cross-Service Integration Demo (12:03)

Q&A: Open Source and Fuzzy Matching

An audience question from Slack asked whether the system would be open sourced. Isak answered: the paper is already available, and the implementation notebook was built quickly using Claude Code. Open sourcing the full implementation more formally is under consideration.

A sharper question raised the "whack-a-mole" concern: if BinaryShield only catches attacks one by one as they're detected, isn't the system perpetually reactive? Isak's answer: the fuzziness of the fingerprint is already doing proactive work. "The fingerprint doesn't just catch the exact prompt — it captures a neighborhood around it, which includes small perturbations you'd expect from an adversary tweaking their attack." Organizations can also adjust the Hamming distance threshold to capture a wider or narrower range of variants, and proactively generating synthetic attack variants using LLMs to pre-populate the registry is a recommended complement to the reactive detection loop.

▶ Watch: Q&A on Open Source and Fuzzy Matching (14:03)

Notable Quotes

"We need the ability to correlate related prompt injections through unrelated products or services without revealing any customer content. That's the goal." — Natalie Isak (06:00)

"We're intentionally losing information to make it more difficult for an adversary to reverse-engineer this pipeline and ultimately gain any private information from the users." — Natalie Isak (08:00)

"BindiShield provides semantics, speed, and privacy together." — Waris Gill (04:00)

"The fingerprint doesn't just catch the exact prompt — it captures a neighborhood around it." — Natalie Isak (18:04)

Key Takeaways

• Siloed safety stacks are a structural risk. When one service catches an attack that others don't, the adversary can rotate to the unprotected services. BinaryShield treats cross-service coordination as a first-class requirement.
• Binary fingerprints enable privacy-preserving sharing. The four-step pipeline (PII redaction → embedding → quantization → differential privacy) allows threat intelligence to flow between services without any raw user data crossing service boundaries.
• 36x faster than dense embeddings at scale. Binary fingerprints use Hamming distance for comparison, which is dramatically cheaper than floating-point similarity search as corpus size grows.
• The epsilon parameter must be calibrated with your privacy team. There's no universal right answer — the privacy-utility trade-off should be determined by your organization's regulatory requirements and risk tolerance.
• Fuzzy matching catches attack variants. Because fingerprints capture a semantic neighborhood around the original prompt, small perturbations of a detected attack are automatically caught by the same fingerprint without requiring separate detection.

Slides Reference

Slides are available as 2026-04-04-D1-S1-12-04-Developing-Deploying-AI-Fingerprints-f.pdf. The associated research paper is available at arXiv:2509.05608. Key slide topics include: the siloed services problem, BinaryShield high-level architecture, the four-step fingerprinting pipeline, accuracy vs. privacy budget evaluation chart, speed comparison chart (36x advantage), and the cross-service registry integration pattern.