If Google Uses It to Find Webpages, We Can Use It to Find Fraudsters

If Google Uses It to Find Webpages, We Can Use It to Find Fraudsters

Speakers: David Geer; Ido Ganor

Conference: Black Hat USA 2025 — August 6-7, 2025, Mandalay Bay, Las Vegas

YouTube: https://www.youtube.com/watch?v=WVHxCedkYSg

Reading time: ~7 minutes

Type: Briefing

TL;DR

David Geer and Ido Ganor demonstrate that TF-IDF — the same text-frequency algorithm that underpinned early web search — can be applied to device and behavioral fingerprints to detect fraud at scale without requiring labeled training data or expensive model retraining. In proof-of-concept deployments at a bank and a mobile network operator, the technique added up to 9% recall on top of existing detection systems at equivalent precision, with minimal compute cost.

Introduction

The fraud detection landscape is undergoing a structural shift. Agentic AI now enables automated commerce — agents booking flights, paying bills, and making purchases on behalf of users — which means the bots and automation tools historically associated with attackers are increasingly indistinguishable from legitimate user agents. At the same time, tools like WormGPT and Grok have lowered the barrier for sophisticated phishing and social engineering, accelerating both the scale and credibility of attacks.

Existing fraud detection models share a common weakness: they are trained on known fraud patterns and struggle with low-confidence decisions on novel entities. Analysts reviewing flagged cases often cannot see the "connective tissue" linking a suspicious new account to known bad actors. This talk addresses both problems with a single, interpretable, computationally cheap method borrowed directly from information retrieval: TF-IDF (Term Frequency–Inverse Document Frequency), augmented with cosine similarity clustering.

TF-IDF: The Core Algorithm

TF-IDF was designed to rank webpages by measuring how significant a term is within a document relative to how common it is across all documents. In its classic form:

• TF (Term Frequency): how often a term appears in a single document
• IDF (Inverse Document Frequency): the log-scaled inverse of how many documents contain that term — rare terms score higher

Applied to fraud, the "document" is an entity (a device ID, a user account, an IP address) and the "terms" are its attributes: operating system, GPU model, timezone, ASN, connection type, account age bucket, and so on. Attributes that appear constantly across the entire user population (equivalent to common words like "the") score low. A rarely seen combination — say, a GPU that appears in only a handful of accounts — scores high, making the entity distinctive.

▶ Watch: TF-IDF Formula Explained (10:00)

The presenters note that Google itself has moved on to BM25 and transformer-based retrieval, but for fraud detection the simplicity of classic TF-IDF is a feature, not a bug: the features being compared are deterministic categorical values (not semantically meaningful words), and the computation is cheap enough to run continuously against large entity corpora.

Cosine Similarity and the Enrichment Layer

Once each entity is represented as a TF-IDF vector, cosine similarity measures how close a new, unknown entity is to every entity already in the corpus. Rather than trying to classify the new entity in isolation, the system asks: which known entities does this new entity most resemble, and what do we know about those neighbors?

The enrichment layer is what makes the results actionable. Known fraud labels — from previous manual investigations, sandbox detonations, or existing detection rules — are attached to corpus entities. When a new entity's nearest neighbors are disproportionately fraud-labeled, a signal fires. Two tunable parameters control precision/recall tradeoffs:

1. Similarity threshold (e.g., >70% cosine similarity)
2. Minimum connected fraud labels (e.g., at least 4 labeled neighbors)

▶ Watch: Enrichment and Similarity Computation (22:01)

Crucially, the corpus can be enriched with either external fraud labels or the output of entirely separate internal detection models — meaning the technique layers on top of whatever a team already has in production rather than replacing it.

Real-World Proof-of-Concept Results

The presenters shared two anonymized customer deployments, both validated live:

Bank — New Account Fraud

• Challenge: identify fraudulent new account registrations at registration time, before any transaction history exists
• Approach: used internal detection labels (not external fraud labels) to enrich the corpus; the TF-IDF system operated independently of the bank's existing fraud engine
• Result: standalone TF-IDF achieved 60% recall on its own; layered on top of the existing system, it added 9% additional recall at the same 50% precision operating point — "money on the floor," as Ganor put it

▶ Watch: Bank POC Results (32:03)

Mobile Network Operator — SIM/Device Fraud

• Challenge: detect fraudulent device changes and account modifications in a high-volume mobile environment
• Approach: used external fraud labels; tuned to top-20 similar entities with >70% cosine similarity and at least 4 connected fraud labels
• Result: visualization showed most flagged new-week entities surrounded predominantly by red (fraud-labeled) dots from the prior corpus; false positives existed but were interpretable by analysts through the graph view

▶ Watch: Mobile Operator POC Demo (36:03)

A notable extension discussed: the same framework can be applied to ransomware detection by treating file-read/write patterns as entity attributes. One Gartner-era example cited was an organization that detected ransomware purely by observing a user's file access rate jump from 100 files/hour to 10,000 files in five minutes — exactly the kind of distributional outlier TF-IDF would flag.

Operational Characteristics

The method has several properties that make it attractive in production:

• No labeled training data required at initialization. The corpus is built from whatever telemetry the organization already has; fraud labels are additive
• Incremental updates. The corpus can be retrained weekly or daily; only the delta period needs reprocessing, not the full historical dataset
• Interpretable output. The graph visualization of which corpus entities connect to a flagged new entity provides analysts with immediate context rather than a black-box score
• Low compute cost. TF-IDF vectorization and cosine similarity are linear-algebra operations; no GPU required, no neural network inference at serving time

The presenters acknowledged the dual-use nature of the algorithm: a sophisticated attacker could apply the same technique in reverse — learning what "normal" looks like in a target organization's telemetry, then gradually adjusting their device fingerprint to blend in.

▶ Watch: Offensive Dual-Use Discussion (38:03)

Notable Quotes

"It's like money on the floor. You just do it and you're getting additional recall at the same precision that you are working from home." — Ido Ganor 28:03

"On its own, without knowing the system, using internal labeling, it was getting to 60% recall." — Ido Ganor 34:03

"A hacker or bad actor could use the same technique in reverse — learn behaviors of a good user, and over time determine where to tweak to make fraud look normal." — David Geer 38:03

Key Takeaways

• TF-IDF is a computationally cheap, interpretable method for converting device and behavioral fingerprints into anomaly signals, requiring no labeled data at initialization.
• Cosine similarity enriched with known fraud labels turns similarity into actionable verdicts, connecting new unknown entities to their nearest known-bad neighbors in a graph that analysts can inspect.
• The technique is additive, not replacement: layered on existing fraud models at a bank, it added 9% recall at no precision cost.
• Numerical features must be bucketized before applying TF-IDF; raw counts violate the categorical assumption underlying the algorithm.
• The method generalizes beyond account fraud to ransomware, insider threat, and agent-based attack scenarios, and is trivially extensible as new telemetry sources are added.

Slides: No slides PDF was available for this talk. The presenters recommended using ChatGPT or Claude to work through TF-IDF examples interactively as a learning tool.