State of the art credential stuffing

Jarrod Overson

Swiss Cyber Storm 2021 · Day 1 · Arena

Overview

In this insightful and at times "terrifying" talk, Jarrod Overson, a veteran in the field of online fraud and automation, unpacks the relentless evolution of credential stuffing attacks. He meticulously details a decade-long arms race between attackers seeking to monetize stolen credentials and defenders striving to protect user accounts and services. Overson argues that understanding this "arc" of automation is crucial, as the tactics employed by credential stuffers are not confined to website login forms but extend to any API or service exposing value, including mobile applications and SSH.

Watch on YouTube · Slides

Visual summary for State of the art credential stuffing by Jarrod Overson
Visual summary for State of the art credential stuffing by Jarrod Overson

Key moments

  1. 0:00 Introduction to credential stuffing and automation evolution
  2. 3:45 The lucrative economics driving credential stuffing attacks
  3. 8:00 Evolution of automation: early tools and proxy rotation
  4. 10:45 Defeating captchas with human-powered API services
  5. 17:00 Emulating human behavior with advanced browser automation
  6. 20:15 The ultimate attack: session hijacking and Genesis Marketplace
  7. 22:30 Key defense strategies and rethinking captchas
  8. 26:00 Q&A: Invisible captchas and privacy in fingerprinting

State of the art credential stuffing

Speakers: Jarrod Overson

Conference: Swiss Cyber Storm

YouTube: https://www.youtube.com/watch?v=XKkyvO2rQ-E

Overview

In this insightful and at times "terrifying" talk, Jarrod Overson, a veteran in the field of online fraud and automation, unpacks the relentless evolution of credential stuffing attacks. He meticulously details a decade-long arms race between attackers seeking to monetize stolen credentials and defenders striving to protect user accounts and services. Overson argues that understanding this "arc" of automation is crucial, as the tactics employed by credential stuffers are not confined to website login forms but extend to any API or service exposing value, including mobile applications and SSH.

Overson, drawing on his extensive experience at Shape Security and as the founder of Vino Technologies, highlights the primary driver behind this escalating sophistication: immense financial incentive. Criminals are highly motivated and adept at extracting value from any compromised account, making credential stuffing a highly lucrative endeavor. The talk serves as a critical wake-up call for organizations, emphasizing that traditional, static defenses are no longer adequate against an adversary that continuously adapts and innovates.

The core message resonates beyond the technical specifics: the battle against automation is a dynamic one, where attackers are constantly striving for perfect emulation of legitimate user behavior. Overson provides a historical perspective on how automation tools and defensive measures have co-evolved, culminating in today's advanced threats that can bypass even sophisticated multi-factor authentication (MFA) and sophisticated fingerprinting techniques by selling entire hijacked user sessions. For security professionals, this talk is essential for understanding the current landscape and anticipating future attack vectors.

Background

▶ Watch: Introduction to credential stuffing and automation evolution (0:00)

Credential stuffing, also known as credential replay, is the malicious act of taking previously leaked combinations of usernames and passwords and attempting to use them to log into other online services. The underlying premise of this attack is the widespread practice of password reuse by users across multiple websites and applications. When one service suffers a data breach, the exposed credentials become a valuable commodity for attackers, who then leverage automation to test these stolen pairs against a vast array of other platforms.

The primary reason this problem persists and escalates is the staggering financial incentive for criminals. As Overson explains, the value of compromised accounts can range from a few dollars for basic bulletin boards or gaming sites to over $100 for high-value accounts like banks or cryptocurrency exchanges, where direct financial transfer or money laundering is possible. This clear expectation of value, coupled with an incredibly low cost of attack, fuels the continuous evolution of these tactics.

Attackers can acquire the necessary credentials for virtually no cost. Major data breaches, such as "Collection #1" in 2019, which encompassed billions of credentials from breaches dating back to 2009, are frequently distributed freely on dark web forums. Once obtained, these credentials can be automated using readily available tools. Simple account checkers can be downloaded for free or purchased, or even built by attackers themselves. To avoid immediate detection by IP rate limiting, attackers utilize proxy lists, which can be free or purchased. While free proxies are often blocked by IP reputation systems, more sophisticated attackers leverage residential proxy networks like Illuminati, which route traffic through legitimate home networks, making it appear as normal user traffic. Services like Hola VPN, often used by consumers for geo-unblocking, inadvertently contribute to these residential proxy networks when users allow their idle computers to be used as proxy nodes.

Even bypassing common defenses like CAPTCHAs has become trivial and inexpensive. Services like 2Captcha, Anti-Captcha, and Death By Captcha offer API access to human farms that solve CAPTCHAs for fractions of a penny (e.g., $1.39 per 1000 solved CAPTCHAs, or $0.99 for "gold members"). This makes the total cost for an attacker incredibly low, often "less than two tenths of one US penny per tested credential." With success rates observed at Shape Security around 2% and accounts selling for an average of $10, a relatively small investment yields substantial returns. For example, testing 100,000 accounts could cost around $200 but yield $20,000 in profit, representing a massive return on investment that drives the relentless innovation in attack methodologies.

Key Findings

▶ Watch: Evolution of automation: early tools and proxy rotation (8:00)

Jarrod Overson's talk distills several critical findings regarding the state of credential stuffing and automation:

  • The Relentless Arc of Evolution: The most significant finding is the continuous and accelerating evolution of attack automation. Attackers consistently adapt to defensive measures, moving through predictable stages from simple HTTP requests to sophisticated human behavior emulation, always striving for perfect mimicry of legitimate user traffic. This "arc" demonstrates that no single defense is permanently effective; the arms race is ongoing.
  • High Profitability Fuels Innovation: The massive financial returns on credential stuffing attacks are the primary incentive for attackers to invest in developing and utilizing increasingly sophisticated tools and techniques. The low cost of acquiring credentials, automation tools, and bypass services ensures a substantial profit margin, making these attacks highly attractive.
  • Traditional Defenses Are Obsolete: Simple defenses like basic IP rate limiting, text-based CAPTCHAs (especially as a gate), and even standalone browser fingerprinting are largely ineffective against modern attacks. Attackers have developed readily available, cheap, and effective methods to bypass or circumvent these measures.
  • The "User as a Product" Model: A critical development is the emergence of marketplaces like Genesis Marketplace, where entire hijacked user sessions, complete with browser fingerprints, session IDs, and even malware to perpetually update the record, are sold. This allows attackers to bypass even multi-factor authentication (MFA) by appearing as the legitimate user from a low-risk context (e.g., a nearby geographic region).
  • Security Must Be Integrated into User Experience (UX): Overson stresses that optimizing solely for the best user experience for good users makes it too easy for bad users. Security and fraud teams must be involved early in the UX design process to implement subtle defenses, such as limiting or varying feedback (e.g., not confirming if an email exists during a "forgot password" flow), which can significantly hinder attackers without overtly frustrating legitimate users.
  • Captchas as Gates Are Detrimental: While CAPTCHAs or similar tests can be useful for gathering data for risk scoring, using them as a hard gate for access is counterproductive. They are easily bypassed by human-solving services and primarily serve to annoy legitimate users, eroding trust and creating friction.
  • The Need for Dynamic, Adaptive Defenses: Organizations cannot rely on static, slow-to-deploy defenses. They need a "system of dials and levers" that allows for rapid, flexible responses to evolving threats without waiting for lengthy product development cycles. This agility is crucial in an environment where attack tactics can shift quickly.

Technical Deep Dive

▶ Watch: Emulating human behavior with advanced browser automation (17:00)

The evolution of credential stuffing automation is a fascinating, decade-long arms race, characterized by attackers constantly adapting to defensive measures. Jarrod Overson meticulously charts this "arc," demonstrating how each defensive innovation spurred a more sophisticated attacking countermeasure.

The journey began with basic automation, employing command-line tools like curl and wget to send HTTP requests directly to login endpoints. Specialized tools like Sentry MBA emerged, designed specifically for hammering HTTP requests through proxies and recording successful logins. The initial defense against this wave was straightforward: IP rate limiting, which blocked excessive requests from a single IP address.

Attackers quickly countered by rotating through proxy lists, distributing their traffic across many IP addresses. While free proxy lists were initially effective, defenders responded with IP reputation services, which identified and blocked traffic from known malicious or suspicious proxy IPs. This led attackers to adopt residential proxy networks such as Illuminati, which route traffic through legitimate home networks, often facilitated by services like Hola VPN. This made it significantly harder for defenders to distinguish malicious traffic based on IP reputation, as it appeared to originate from diverse, legitimate residential IPs.

As defenses evolved, text-based CAPTCHAs (like older versions of reCAPTCHA with squiggly words) became prevalent. This defense was rapidly undermined by the rise of CAPTCHA solving services such as 2Captcha, Anti-Captcha, and Death By Captcha. These services leverage human farms to solve CAPTCHAs in real-time, providing the solutions via API calls that attackers integrate directly into their automation scripts. The cost for these services was incredibly low, making CAPTCHA bypass a trivial and affordable step.

With the web becoming more dynamic and reliant on JavaScript, defenses incorporated more advanced, client-side logic. This rendered simple HTTP request-based tools ineffective. Attackers then moved to scriptable web views like PhantomJS, a headless browser that could execute JavaScript and interact with web pages programmatically. This made malicious traffic look more like legitimate browser traffic. Defenders, however, adapted by analyzing subtle differences in how these tools interacted with the network stack. For instance, network request fingerprinting could identify PhantomJS by the unusual order of HTTP headers it sent (e.g., sending the Host header last, whereas Chrome always sends it first). Blocking PhantomJS traffic became a viable defense due to its limited legitimate use on production websites.

The next leap for attackers was the direct scripting of actual production browsers using tools like puppeteer, playwright, and selenium. These frameworks drive real Chrome, Firefox, Safari, or Edge instances, making it impossible to identify attacks based on the browser engine itself. In response, defenders implemented sophisticated browser fingerprinting techniques, similar to those used by advertisers, to identify unique browser and device characteristics. The goal was to link disparate requests to a single, underlying attacking "fingerprint."

However, attackers again had a counter-measure ready, leveraging existing anti-tracking technology. Tools like FraudFox and Anti-Detect emerged, specifically designed to spoof or randomize browser fingerprints, making it appear as if traffic originated from a multitude of unique, legitimate devices. These tools were often marketed for fraud-related activities, making their purpose clear.

Defenders then shifted to user behavior analysis, looking for patterns in mouse movements, typing speed, and click locations. Automated scripts typically click on exact coordinates (0,0 XY) or type at unnaturally consistent intervals, making it easy to identify non-human behavior. This led to the development of attacker tools like Browser Automation Studio, which could emulate human behavior by introducing realistic variations in mouse paths, typing speeds, and click locations. The goal was not to be perfectly human, but "definitely not a bot."

The arms race continued with defenders attempting to identify "lies" in the attacker's emulated data, performing device consistency checks. For example, if a browser claimed to be "mobile Safari" but reported support for the "OG Vorbis" audio format (which Safari does not), it would indicate a spoofed profile. Attackers countered this by using real browser data. Tools like Bablosoft's Fingerprint Switcher began collecting legitimate browser fingerprints from real devices and providing them via an API. Other methods involved deploying defender's own fingerprinting scripts on high-traffic forums to collect genuine user data, which could then be replayed or sold.

The latest and most alarming evolution addresses multi-factor authentication (MFA) and risk scoring. While MFA adds a significant layer of security, it often introduces friction for users, prompting companies to present it only when necessary. Attackers exploited this by moving towards session hijacking and effectively selling the "user as a product" through platforms like the Genesis Marketplace. This marketplace offers access to compromised user computers, complete with their browser fingerprints, session IDs, and a persistent malware component that continuously updates the user's record with new session data and password changes. By acquiring a full user profile and logging in from a geographically proximate region, attackers can often bypass MFA because the traffic appears to be from a known, low-risk user and device, effectively tricking the risk scoring systems. This allows attackers to maintain perpetual access to accounts without triggering additional security checks.

This relentless back-and-forth highlights that the goal for attackers is increasingly to perfectly emulate legitimate user traffic, making it indistinguishable from genuine activity.

Demo / Proof of Concept

▶ Watch: The ultimate attack: session hijacking and Genesis Marketplace (20:15)

While Jarrod Overson's talk is rich with examples of attacker tools and methodologies, it does not feature a live demonstration or proof of concept performed by the speaker. Instead, the presentation meticulously describes the functionality and impact of various tools employed by attackers, such as Sentry MBA for early automation, 2Captcha for CAPTCHA solving, Browser Automation Studio for human behavior emulation, and the Genesis Marketplace for selling hijacked user sessions. The detailed explanations and visual aids of these tools serve to illustrate the technical capabilities and evolution of credential stuffing attacks, providing a clear understanding of how these mechanisms work in practice from the attacker's perspective.

Defensive Implications

▶ Watch: Q&A: Invisible captchas and privacy in fingerprinting (26:00)

Understanding the "arc of evolution" in credential stuffing attacks provides critical insights for defenders. Overson offers several key pieces of advice for organizations looking to bolster their defenses:

  1. Integrate Security and Fraud Teams into User Experience (UX) Processes: This is paramount. Security cannot be an afterthought or solely optimized for the "best" user experience, as this often inadvertently creates vulnerabilities for bad actors. Security and fraud teams possess crucial knowledge about attacker tactics that UX designers may not. For instance, in a "forgot password" flow, revealing whether an email address exists in the database (e.g., "Sorry, that user is not in our database") provides valuable information to attackers, allowing them to prune their lists for valid accounts. Limiting, delaying, and varying feedback can significantly hinder attackers without overtly frustrating legitimate users.
  2. Reduce Your Surface Area: Organizations should strive to funnel risky points, such as login and purchase flows, into bottlenecks. Having "18 different purchase flows or login flows across 36 different apps" creates disparate defense profiles that attackers can exploit by cycling through the weakest links. Consolidating these critical pathways allows for more focused and robust defense mechanisms.
  3. Build a System of Dials and Levers: Defenders need the ability to respond quickly and flexibly to attacks, often in ways that were not initially anticipated. Waiting for standard development and iteration cycles to deploy new defenses during an active attack is not feasible. Implementing a system that allows for rapid adjustments to security parameters in critical flows, independent of the product development life cycle, is essential for agility.
  4. Rethink CAPTCHAs as Gates: Overson strongly advocates for organizations to "please stop using captchas" as a primary gate for access. While tests like CAPTCHAs can be valuable for gathering user data to inform risk scoring, relying on a successful response to the test as the sole barrier is ineffective. They are easily bypassed by cheap human-solving services and primarily serve to "really, really, really bother your good users." Instead, use them as one signal among many for risk assessment rather than a hard block.
  5. Beware of Standalone Fingerprinting: While browser fingerprinting was a useful defense for a time, attackers have developed sophisticated methods to bypass it using anti-detect tools and by collecting and replaying legitimate fingerprints. Relying solely on fingerprinting as a defense is no longer effective.
  6. Multi-Factor Authentication (MFA) Requires Context: While MFA is a strong defense, attackers using methods like the Genesis Marketplace can bypass it by hijacking entire user sessions, complete with legitimate fingerprints and session IDs. If the attacker can log in from a location close to the legitimate user, the risk scoring system may deem the login low-risk and not prompt for MFA. This highlights the need for continuous risk scoring and anomaly detection that goes beyond static MFA prompts.

Ultimately, effective defense requires a proactive, integrated, and adaptive approach that anticipates attacker evolution rather than merely reacting to it.

Key Takeaways

  • Credential stuffing is a highly lucrative and constantly evolving threat. Attackers are driven by significant financial incentives to continually innovate their automation techniques, making this a perpetual arms race.
  • Attackers are striving for perfect emulation of legitimate user behavior. From basic HTTP requests to sophisticated human-like mouse movements and typing, the goal is to bypass defenses by appearing indistinguishable from a genuine user.
  • Traditional defenses like IP blocking, simple CAPTCHAs, and standalone browser fingerprinting are largely ineffective. Attackers have readily available, low-cost services and tools to bypass these measures.
  • Security and fraud teams must be deeply integrated into the user experience (UX) design process. Subtle changes in feedback and flow can significantly hinder attackers without frustrating legitimate users, emphasizing a "security by design" approach.
  • The "user as a product" model, exemplified by Genesis Marketplace, represents a significant escalation. Attackers can purchase and maintain access to hijacked user sessions, complete with fingerprints and live updates, to bypass even multi-factor authentication (MFA).
  • Organizations need agile, adaptive defenses. A "system of dials and levers" is crucial for rapid response to evolving threats, allowing for quick adjustments without lengthy development cycles. CAPTCHAs should be reconsidered as risk assessment tools rather than hard gates.

About the Speaker(s)

Jarrod Overson is presented as a "perpetually frustrated maker of things" who is driven to fix the complexities and frustrations inherent in modern computing. His professional journey includes significant time at Napster and, notably, at Shape Security, where he specialized in combating credential stuffing and understanding the exploitation of everyday people. Drawing on this experience, he later founded Vino Technologies, aiming to generalize and apply the concepts learned in enterprise security to a broader audience. Overson is deeply knowledgeable about the evolution of automation and the tactics employed by online criminals, making him an authority on the subject of sophisticated account takeovers.

Reviews

Dr. Zero (Offensive Security Researcher) — SOLID

A competent, clear-eyed history lesson on the botting arms race from an ex-Shape Security operator. It's a solid primer on why CAPTCHAs are dead and fingerprinting is spoofable, but lacks any novel research or new bypasses.

Heather Calloway (CISO) — STRONG ACCEPT

Overson traces the decade-long arms race in credential stuffing, proving definitively that static defenses like CAPTCHAs and IP blocking are dead. The real value here is the breakdown of how attackers bypass multi-factor authentication by hijacking entire user sessions via platforms like Genesis Marketplace. If your fraud and security teams are not already integrated into UX design, this talk is your mandate to fix that.

→ Top-rated talks at Swiss Cyber Storm 2021

All talks from Swiss Cyber Storm 2021