Data Breaches: Your Casual End of Year Security Problem

Adam Black (Software Engineer)

purplecon 2024 · Day 1 · Main Room

Overview

In his purplecon talk, "Data Breaches: Your Casual End of Year Security Problem," Adam Black, a software engineer with a passion for security, delves into the pervasive yet often underestimated vulnerability of Insecure Direct Object Reference (IDOR). Black argues that while organizations meticulously secure their primary authentication mechanisms, they frequently leave "windows open" by exposing sensitive user data through easily guessable or enumerable URLs. His presentation serves as a stark reminder that data protection extends far beyond the login page, urging a deeper scrutiny of how resources are accessed and identified.

Watch on YouTube

Visual summary for Data Breaches: Your Casual End of Year Security Problem by Adam Black
Visual summary for Data Breaches: Your Casual End of Year Security Problem by Adam Black

Key moments

  1. 1:25 The 'Cat Mask' URL puzzle introduction
  2. 3:00 Discovering Shrek's private tax invoice via IDOR
  3. 4:00 What is an Insecure Direct Object Reference (IDOR)?
  4. 4:40 Exposing Bruce Wayne's Batmobile insurance details
  5. 7:40 How URL shorteners make private data guessable
  6. 10:30 Best practice: Use non-enumerable IDs like UUIDs
  7. 11:30 Implementing rate limiting and facing botnet attacks

Data Breaches: Your Casual End of Year Security Problem

Speakers: Adam Black, Software Engineer

Conference: purplecon

YouTube: https://www.youtube.com/watch?v=5F9cvL1fLQo

Overview

In his purplecon talk, "Data Breaches: Your Casual End of Year Security Problem," Adam Black, a software engineer with a passion for security, delves into the pervasive yet often underestimated vulnerability of Insecure Direct Object Reference (IDOR). Black argues that while organizations meticulously secure their primary authentication mechanisms, they frequently leave "windows open" by exposing sensitive user data through easily guessable or enumerable URLs. His presentation serves as a stark reminder that data protection extends far beyond the login page, urging a deeper scrutiny of how resources are accessed and identified.

Black's central thesis is that a casual approach to URL design and object referencing can lead to significant data breaches, often without sophisticated hacking techniques. Through a series of compelling real-world examples drawn from his personal experiences, he illustrates how simple modifications to a URL — changing a single digit or letter — can grant unauthorized access to confidential information such as tax invoices, car insurance details, or customer order histories. The talk resonates with anyone involved in web development, security, or data privacy, highlighting the critical importance of secure design principles in every aspect of an application's architecture.

This presentation is particularly timely given the increasing volume and sensitivity of personal data handled by online services. As users become more aware of privacy concerns, the responsibility on developers to implement robust access controls for all data objects becomes paramount. Black's accessible explanations and relatable anecdotes make complex security concepts understandable, underscoring why IDOR vulnerabilities are not just theoretical risks but tangible threats that can compromise user trust and lead to severe reputational and financial consequences for businesses.

Background

▶ Watch: The 'Cat Mask' URL puzzle introduction (1:25)

Adam Black opens his talk by drawing an analogy between physical and digital security, particularly relevant during the holiday season. He likens securing a home for vacation – locking the front door – to implementing robust login pages for web applications. However, he quickly pivots to the often-overlooked "windows" – the direct access points to data via URLs. Just as a burglar might bypass a locked front door by entering through an open window, an attacker can bypass authentication by directly accessing sensitive data through a guessable URL, even if the primary login mechanism is secure. This foundational metaphor sets the stage for understanding the Insecure Direct Object Reference (IDOR) vulnerability.

The problem, Black explains, often stems from a lack of vigilance regarding URL structures. He questions whether users truly scrutinize URLs beyond the domain name or the presence of HTTPS, noting that while browsers once highlighted HTTPS with a lock symbol, users have grown accustomed to it. The core issue arises when web applications use predictable, sequential, or easily enumerable identifiers for objects (like invoices, user profiles, or order numbers) in their URLs or API parameters without adequate authorization checks. This allows an attacker to simply change an ID in a URL (e.g., from invoice/101 to invoice/102) to gain unauthorized access to another user's data.

Black's personal journey into this security problem began casually. While reviewing his own tax summary, he noticed a sequential number in the download link. His curiosity, a trait he attributes to his software engineering background, prompted him to alter the number. This simple act revealed not his own past tax documents, but those of a fictional character, Shrek, complete with details about his "swamp decoration kit." This discovery underscored the ease with which such vulnerabilities can be exploited and the severe privacy implications, even for seemingly innocuous data. This talk builds upon the long-standing understanding of access control flaws, a consistent entry in the OWASP Top 10, demonstrating its continued relevance and prevalence in modern applications.

Key Findings

▶ Watch: What is an Insecure Direct Object Reference (IDOR)? (4:00)

Adam Black's talk illuminates several critical findings regarding IDOR vulnerabilities and their pervasive nature in modern web applications. The primary discovery is the alarming frequency with which these vulnerabilities appear, even in services from reputable companies. Black notes that "the soon as you see it like once sometimes you just keep seeing it over and over," indicating a systemic issue rather than isolated incidents.

One key finding is that sequential or easily guessable identifiers are the root cause of most IDOR flaws. Whether it's a simple numerical sequence (e.g., invoice/101, invoice/102), a predictable alphanumeric string (e.g., linkmonitor/H, linkmonitor/A), or a combination of common parameters (e.g., memo number and postcode), developers frequently use IDs that are trivial for an attacker to enumerate. This predictability allows an attacker to bypass intended access controls by merely altering the object's identifier in a URL or API request.

The impact of these vulnerabilities is significant and varied, ranging from severe privacy breaches to potential avenues for social engineering. Black's examples demonstrate:

  • Direct Data Exposure: Accessing other users' tax invoices, revealing personal purchases and addresses. The "Shrek's tax invoice" example vividly illustrates the privacy violation.
  • Sensitive Information Disclosure: Retrieving car insurance details, potentially revealing vehicle information, insurance costs, and expiration dates. The "Bruce Wayne has a Batman" example highlights how even anonymized data can lead to sensitive inferences (e.g., inferring the Batmobile's service date).
  • Business Logic Flaws: The "Find my order" example shows how guessable order numbers, combined with known email addresses, can expose entire order histories, including delivery addresses.
  • Risk of Social Engineering: Even when direct personal identifiers are removed (e.g., car details without the owner's name), the existence of a guessable member number could be used in conjunction with social engineering tactics to extract more information from customer service representatives.

Another critical finding is the inadequate incident response and communication from some companies when these vulnerabilities are reported. Black recounts frustrating experiences where his reports were met with skepticism, bureaucratic hurdles, or a misunderstanding of the core security issue. Companies often focus solely on the security of their login pages, failing to grasp that the "window" of direct object access is wide open. This highlights a gap in security awareness and incident handling processes within many organizations.

Finally, Black implicitly finds that curiosity and simple testing methods are often sufficient to uncover these flaws. He didn't use sophisticated tools or advanced hacking techniques; rather, he simply changed numbers and letters in URLs, mirroring how a curious user or a low-skilled attacker might probe an application. This underscores the ease of exploitation and the necessity for developers to adopt a "think like an attacker" mindset during development and testing.

Technical Deep Dive

▶ Watch: Exposing Bruce Wayne's Batmobile insurance details (4:40)

The core technical concept explored in Adam Black's talk is the Insecure Direct Object Reference (IDOR) vulnerability, categorized under Broken Access Control in the OWASP Top 10. IDOR occurs when an application provides direct access to objects based on user-supplied input, typically an identifier, without performing sufficient authorization checks to ensure the requesting user is permitted to access that specific object.

The mechanism of IDOR exploitation hinges on the predictability of object identifiers. Black illustrates this with several concrete examples:

  1. Sequential Numerical IDs:
  • Tax Invoices: Black's initial discovery involved a URL like https://stock.com/invoice/1. By changing the 1 to 0, he accessed a different invoice. This demonstrates that if an application uses a simple, incrementing integer as an object identifier, an attacker can iterate through these numbers to access other users' data. The lack of proper authorization for invoice/0 allowed unauthorized access.
  • Car Insurance: The URL contained payment?memo_number=X&postcode=Y. While memo_number might be longer, in combination with a known postcode, it could still be brute-forced, especially if the memo_number space isn't astronomically large. This is a form of parameter tampering combined with IDOR, where the attacker manipulates query parameters to reference different objects.
  1. Predictable Alphanumeric IDs / URL Shorteners:
  • LinkMonitor: Black encountered a URL like https://linkmonitor.com/H. By simply changing the H to A, he accessed another user's tax invoice. This indicates that the URL shortening service, or the application using it, was generating short codes in a sequential or easily guessable alphanumeric pattern. These short codes acted as direct object references without corresponding authorization. The problem here is exacerbated by the design of URL shorteners, which prioritize brevity, often at the expense of non-enumerability if not implemented carefully.
  1. Predictable IDs in Forms:
  • "Find My Order" Forms: Black describes a scenario where a form requests an order number and billing email address. While seemingly requiring two pieces of information, if the order number space is small and predictable (e.g., order number A, order number B), an attacker who knows any email address could potentially guess order numbers for that store and retrieve order details and shipping addresses. This is a variant where the IDOR is exposed not directly in the URL path, but through user input fields that reference objects.

Technical Countermeasures and Best Practices:

Black highlights several critical defensive strategies to mitigate IDOR:

  1. Prefer Non-Enumerable IDs:
  • UUIDs (Universally Unique Identifiers): This is Black's primary recommendation. A UUID (e.g., UUIDv4) is a 128-bit number that is globally unique and effectively random. When used as an object identifier, the probability of guessing a valid UUID is astronomically low (2^128 possible values). As Black states, "you may as well with IPv6 going you may as well go anywhere else on the internet like just like leave this website alone find find some other vulnerability." This makes brute-forcing infeasible. He cites Eventbrite as a good example, using UUIDs for event IDs.
  1. Ensure Adequate Authentication and Authorization Controls:
  • Beyond Login: A secure login page is insufficient. Every request to access a specific resource identified by an ID must be accompanied by robust authorization checks. The application must verify that the currently authenticated user is indeed authorized to view or manipulate that specific object. This means checking user roles, ownership, and permissions against the requested object ID on the server-side for every single request. Black's analogy of "securing your front door with a great lock and you leave the window wide open" perfectly encapsulates this.
  1. Rate Limiting:
  • Deterring Brute-Force: Implementing rate limiting on endpoints that accept object identifiers can significantly impede brute-force attacks. By limiting the number of requests a single IP address (or user session) can make within a given timeframe, attackers are prevented from rapidly iterating through ID spaces.
  • Challenges with Distributed Attacks: Black acknowledges the sophistication of modern attackers, mentioning a scenario with "16,000 routers that are vulnerable that like one malicious actor is using to do logins to a certain cloud provider each day." This highlights that IP-based rate limiting alone might not be sufficient against highly distributed botnets, necessitating more advanced anomaly detection and behavioral analysis.
  1. Verify Not a Bot (CAPTCHAs):
  • To combat automated brute-forcing, integrating CAPTCHAs or other bot verification mechanisms can add another layer of defense. These challenges make it harder for automated scripts to rapidly test ID variations.
  1. Developer Education and Engineering Handbooks:
  • Black stresses the importance of proactive developer education. Providing engineering handbooks or internal documentation with best practices for ID generation, access control implementation, and secure coding can guide software engineers toward building secure applications from the ground up, preventing IDORs before they are deployed.

In summary, the technical deep dive reveals that IDOR is a fundamental access control flaw stemming from predictable object identifiers and insufficient server-side authorization. The solution lies in adopting non-enumerable IDs, implementing granular authorization checks for every resource access, and employing rate limiting, backed by strong developer education.

Demo / Proof of Concept

▶ Watch: Best practice: Use non-enumerable IDs like UUIDs (10:30)

While Adam Black's presentation did not feature a live coding demonstration or a pre-recorded video of a technical exploit tool, he effectively walked the audience through a series of conceptual "proofs of concept" derived from his personal experiences. These were not theoretical exercises but actual vulnerabilities he discovered and reported, illustrating the IDOR mechanism in a relatable and impactful way.

His "demonstrations" involved:

  1. Tax Invoice Manipulation: Black described receiving an email with a download link to his tax summary, which contained a URL like https://stock.com/invoice/1. His "proof of concept" was simply changing the 1 to a 0 in the URL. This trivial modification resulted in him accessing "Shrek's tax invoice," complete with details about "miniature toy donkeys" and a "swamp decoration kit." This clearly demonstrated how sequential numerical IDs, when coupled with a lack of authorization, can expose sensitive data.
  1. Car Insurance Brute-Forcing: Black detailed a car insurance renewal URL that included parameters such as memo_number and postcode. His "proof of concept" involved the idea of brute-forcing the memo_number in conjunction with a postcode that might cover a large area. He hypothesized that by iterating through possible memo_number values, one could uncover details like "Bruce Wayne has a Batman" (registration, insurance cost, expiry date), highlighting the potential for inferring sensitive information even if direct personal names were not immediately exposed.
  1. URL Shortener Enumeration: Another example involved a URL from a service called linkmonitor, which had a short, alphanumeric identifier (e.g., https://linkmonitor.com/H). Black's "proof of concept" was to change the last letter from H to A. This simple change revealed someone else's tax invoice, demonstrating how even seemingly random-looking short URLs can be guessable if the underlying ID generation is sequential or uses a small, predictable character set.
  1. "Find My Order" Form Guessing: Black described a common "Find my order" feature requiring an order number and billing email address. His "proof of concept" here was to infer the predictable nature of order IDs within a specific store. If an attacker knew the pattern of order numbers (e.g., order number A then order number B) and possessed any valid email address associated with the store, they could potentially guess other order numbers to retrieve details like items ordered and delivery addresses.

In each of these scenarios, Black's "demo" was a straightforward, manual manipulation of an identifier within a URL or form input. The impact was immediate and unauthorized access to data that should have been protected. These practical examples served as compelling evidence of the vulnerability's existence and ease of exploitation, effectively serving the purpose of a proof of concept without requiring complex tools or live coding.

Defensive Implications

▶ Watch: Implementing rate limiting and facing botnet attacks (11:30)

Adam Black's talk provides crucial insights for defenders aiming to secure their applications against IDOR vulnerabilities. The defensive implications are multi-layered, encompassing architectural design, implementation practices, and organizational culture.

  1. Adopt Non-Enumerable Identifiers: This is the foremost recommendation. Developers should abandon sequential or easily guessable IDs for all sensitive or user-specific resources. The preferred solution is to use Universally Unique Identifiers (UUIDs), specifically UUIDv4, which are 128-bit random numbers. The vastness of the UUID space (2^128 possibilities) makes brute-forcing practically impossible, effectively eliminating the "guessable URL" problem. This applies not only to URLs but also to API parameters, form fields, and any other mechanism used to reference objects.
  1. Implement Robust Server-Side Authorization for Every Resource Access: A secure login page is not enough. Defenders must ensure that every single request to access a resource identified by an ID performs a server-side authorization check. This means verifying that the authenticated user is explicitly authorized to view, modify, or delete that specific object. This goes beyond checking if a user is logged in; it checks if the user owns or has permission for the requested resource. This is a fundamental principle of least privilege.
  1. Enforce Granular Rate Limiting: To mitigate brute-force attempts on any endpoint that accepts user-controlled identifiers, implement comprehensive rate limiting. This should limit the number of requests a single user, IP address, or session can make within a given timeframe. While Black acknowledges the challenge of distributed attacks using botnets (e.g., 16,000 compromised routers), effective rate limiting still significantly raises the bar for attackers and can deter simpler, automated scans. Consider implementing adaptive rate limiting that can detect and respond to unusual traffic patterns.
  1. Integrate Bot Detection and CAPTCHAs: For critical endpoints where ID enumeration might be attempted, or where user input directly references objects, deploying CAPTCHAs or other bot detection mechanisms can help distinguish legitimate users from automated scripts. This adds a hurdle for attackers attempting to systematically iterate through IDs.
  1. Prioritize Developer Security Education and Secure Coding Practices: Black implicitly emphasizes the need for a strong security culture within engineering teams. Organizations should invest in ongoing security training for developers, focusing on common vulnerabilities like IDOR (part of OWASP A1: Broken Access Control). Creating and maintaining an engineering handbook that outlines secure coding standards, including guidance on ID generation and access control implementation, can serve as an invaluable resource. Security champions within teams can also help disseminate best practices.
  1. Review Third-Party Service Integrations: If using URL shorteners or other third-party services that handle identifiers, defenders must scrutinize their implementation to ensure they do not reintroduce IDOR vulnerabilities. The application using the shortener must still enforce authorization on the backend, and the shortener itself should ideally generate non-guessable links if they point to sensitive data.
  1. Establish a Clear and Empathetic Security Reporting Process: Black's personal experiences highlight the frustration of reporting vulnerabilities to companies with inadequate processes. Defenders should ensure their organizations have a well-defined, accessible, and responsive channel for security researchers and concerned users to report issues. Treating reporters with respect and understanding the nature of the vulnerability are crucial for timely remediation and fostering a positive security community relationship.

In essence, defending against IDOR requires a shift from superficial security measures to a deep understanding of how data objects are referenced and accessed throughout an application's lifecycle. It's about securing not just the "front door," but every "window" that leads to sensitive information.

Key Takeaways

  • IDOR is a Pervasive and Underestimated Vulnerability: Insecure Direct Object Reference (IDOR) is a common and often overlooked flaw where applications expose sensitive data through easily guessable or enumerable identifiers in URLs or API parameters.
  • Guessable IDs are the Root Cause: Sequential numbers (e.g., invoice/1, invoice/2), predictable alphanumeric strings, or common parameter combinations (e.g., memo_number, postcode) make it trivial for attackers to iterate and gain unauthorized access.
  • UUIDs are the Gold Standard for IDs: To mitigate IDOR, use Universally Unique Identifiers (UUIDs), especially UUIDv4. Their 128-bit randomness makes brute-forcing practically impossible, providing strong protection against enumeration attacks.
  • Authentication ≠ Authorization: A secure login page is not enough. Robust server-side authorization checks are critical for every request accessing a resource. The application must verify that the authenticated user is explicitly permitted to access that specific object.
  • Rate Limiting and Bot Detection are Essential: Implement rate limiting on all endpoints that accept object identifiers to deter brute-force attacks. For higher security, integrate CAPTCHAs or other bot verification to combat automated enumeration attempts, acknowledging the challenge of distributed attacks.
  • Developer Education is Paramount: Proactive security training and clear engineering handbooks on secure coding practices (including ID generation and access control) are vital for developers to build secure applications from the ground up, preventing IDORs before they occur.

About the Speaker(s)

Adam Black is a Software Engineer with a strong passion for security. His professional background in software engineering provides him with the technical expertise to understand and dissect application vulnerabilities. Black's personal experiences, as detailed in his talk, highlight his commitment to identifying and responsibly reporting security flaws, driven by a genuine curiosity and a desire to improve data protection practices across the industry. He advocates for greater awareness and better implementation of fundamental security principles in software development.

Reviews

Dr. Zero (Offensive Security Researcher) — SOLID

A competent IDOR awareness talk that hits the basics correctly but brings nothing new to anyone who's touched web security in the last decade. Good for junior devs or a general audience; a snooze for anyone who's read the OWASP Top 10 twice.

Heather Calloway (CISO) — SOLID

A clear, accessible walkthrough of IDOR vulnerabilities using real-world examples. Solid for junior developers or teams that need a refresher on access control basics, but offers nothing new for seasoned practitioners or security leadership.

→ Top-rated talks at purplecon 2024

All talks from purplecon 2024