A Fistful of Headers: Taming the Wild Web at Scale

Cary Hooper, Wesley McElhinny

Wild West Hackin' Fest @ Mile High 2026 · Day 1

Overview

In "A Fistful of Headers: Taming the Wild Web at Scale," Cary Hooper and Wesley McElhinny of Bank of America's pentesting team deliver a critical examination of HTTP Strict Transport Security (HSTS) adoption and configuration across the internet's top 1 million domains. The talk addresses the persistent threat of TLS stripping and man-in-the-middle (MITM) attacks, showcasing HSTS as a vital browser security mechanism designed to enforce HTTPS-only connections. The speakers highlight the often-overlooked complexity of correctly implementing HSTS, particularly its preload directive, and reveal widespread misconfigurations that leave many websites vulnerable despite appearing secure.

Watch on YouTube

Visual summary for A Fistful of Headers: Taming the Wild Web at Scale by Cary Hooper, Wesley McElhinny
Visual summary for A Fistful of Headers: Taming the Wild Web at Scale by Cary Hooper, Wesley McElhinny

Key moments

  1. 0:00 Introduction and talk overview
  2. 4:00 Live demo: TLS stripping attack
  3. 8:00 Understanding HSTS preload directive
  4. 11:58 Introducing the 'News Team' analysis tool
  5. 24:00 HSTS adoption statistics and preload issues
  6. 30:00 Examples of 'weird' HSTS header misconfigurations
  7. 39:40 Q&A: HSTS support on non-standard ports

A Fistful of Headers: Taming the Wild Web at Scale

Speakers: Cary Hooper, Pentesting Team, Bank of America; Wesley McElhinny, Pentesting Team, Bank of America

Conference: Wild West Hackin' Fest @ Mile High

YouTube: https://www.youtube.com/watch?v=tfgDNqJdEi4

Overview

In "A Fistful of Headers: Taming the Wild Web at Scale," Cary Hooper and Wesley McElhinny of Bank of America's pentesting team deliver a critical examination of HTTP Strict Transport Security (HSTS) adoption and configuration across the internet's top 1 million domains. The talk addresses the persistent threat of TLS stripping and man-in-the-middle (MITM) attacks, showcasing HSTS as a vital browser security mechanism designed to enforce HTTPS-only connections. The speakers highlight the often-overlooked complexity of correctly implementing HSTS, particularly its preload directive, and reveal widespread misconfigurations that leave many websites vulnerable despite appearing secure.

The core of their research involved developing and deploying a custom, cost-effective, and highly scalable distributed execution framework dubbed "news team." This innovative tool allowed them to conduct a rapid, large-scale analysis of web headers, uncovering surprising statistics on HSTS adoption, preload eligibility, and an array of "weird stuff" – non-standard and often nonsensical directives – in production environments. Their findings underscore a significant gap between security best practices and real-world implementation, urging developers and defenders to re-evaluate their approach to web security headers.

This talk is particularly relevant for web developers, security engineers, pentesters, and system administrators. It provides a deep dive into the technical intricacies of HSTS, exposes common pitfalls, and offers actionable insights for improving the security posture of web applications at scale. By demonstrating both the offensive capabilities of TLS stripping and the defensive power of correctly configured HSTS, Hooper and McElhinny deliver a compelling argument for rigorous implementation of foundational web security controls.

Background

▶ Watch: Introduction and talk overview (0:00)

The genesis of this research lies in the enduring threat of TLS stripping, a sophisticated man-in-the-middle (MITM) attack technique first popularized by Moxie Marlinspike at Black Hat in 2009. While the original talk focused on SSL stripping, the underlying principle remains relevant for TLS. A MITM attacker, positioned between a victim and a web server (often via ARP spoofing, DNS hijacking, or rogue access points), intercepts initial unencrypted HTTP requests. The server attempts to redirect the victim to a secure HTTPS connection (e.g., a 301 redirect). However, the attacker intercepts this redirect, maintains a secure connection with the server, and downgrades the connection to HTTP when communicating with the victim. The victim perceives a functioning, albeit unencrypted, website, while the attacker can observe, capture, and even modify all traffic, including sensitive credentials. This leaves victims susceptible to credential theft, session hijacking, and website defacement.

HTTP Strict Transport Security (HSTS) emerges as a critical defense against TLS stripping. HSTS is a browser security mechanism where a web server instructs the client's browser to exclusively use HTTPS for all future connections to that domain for a specified period. This instruction is conveyed through a special HTTP response header: Strict-Transport-Security. Key directives within this header include:

  • Max-Age: Specifies the duration (in seconds) for which the browser should remember to only connect via HTTPS. A common recommendation is one year or more.
  • IncludeSubDomains: An optional directive that applies the HSTS policy to all subdomains of the current domain. Given the proliferation of subdomains, this is considered a best practice.

Once a browser receives and processes an HSTS header, it caches this policy. For the duration of the Max-Age, any subsequent attempt to access the domain via HTTP will be automatically upgraded to HTTPS internally by the browser, without ever sending an unencrypted request to the server. This prevents the initial HTTP request that TLS stripping relies upon. If the server's certificate is invalid or the connection fails for other reasons, the browser will display a hard error, preventing the user from bypassing the security warning and potentially exposing themselves.

A crucial enhancement to HSTS is the preload directive, which allows domains to be hardcoded into web browsers' HSTS lists (maintained by the Chromium project and adopted by all major browsers). This means that even the very first visit to a preloaded domain will enforce HTTPS, eliminating the "trust on first use" vulnerability where the initial connection might still be vulnerable to TLS stripping. To be eligible for preload, domains must meet strict criteria, including serving HSTS with a sufficiently long Max-Age, including subdomains, and redirecting all HTTP traffic to HTTPS on the same host. The process involves submitting the domain to hstspreload.org after ensuring all technical requirements are met. Despite its importance, the speakers found that HSTS, particularly preload, often "fails quietly and at scale," leading to a widespread but often unnoticed state of vulnerability. This observation spurred their large-scale internet scan to quantify the problem.

Key Findings

▶ Watch: Understanding HSTS preload directive (8:00)

The large-scale scan conducted by Hooper and McElhinny across the internet's top 1 million domains yielded several critical and often surprising insights into the state of HSTS adoption and configuration. The analysis, completed in a remarkable 88 minutes on their fastest run using 40 execution nodes, processed 2 million requests (HTTP and HTTPS).

Firstly, the overall adoption of HTTPS and HSTS remains lower than expected. While 66.4% of the top million domains were found to be serving HTTPS, only 25% of those HTTPS-enabled sites were actually utilizing HSTS. This indicates a significant portion of secure websites are still vulnerable to first-visit TLS stripping attacks. Furthermore, of the domains serving HSTS, only 20% included the preload directive, and a mere 22.8% of those were actually listed in the global HSTS preload JSON document shipped with browsers. This means a vast number of sites claiming preload eligibility are not truly preloaded, leaving their users exposed on the initial connection.

A particularly alarming discovery was the identification of 13,000 domains deemed "preload eligible" but not actually preloaded. These domains meet all technical requirements for preload but haven't completed the final submission step on hstspreload.org. This creates a potential "preload DDoS" vulnerability: an attacker could maliciously submit such a domain for preload. If accepted, and if the organization behind the domain has any internal HTTP services or misconfigured HTTPS, the browser's hardcoded HSTS policy would break all connections to those services, effectively taking them offline for users with updated browsers. This highlights a critical governance gap in the preload submission process.

The research also uncovered the use of HTTPS resource records (also known as SVCB/HTTPS DNS records, governed by RFC 9460). This DNS record type signals to browsers that a domain should only be accessed via HTTPS, often before an HTTP request is even attempted. While a good mitigating control, found on 150 domains, the speakers emphatically warned against relying solely on DNS for security, as DNS itself can be susceptible to MITM attacks.

Finally, the scan revealed a trove of "weird stuff" within HSTS headers, demonstrating a profound misunderstanding of the standard:

  • Content Security Policy (CSP) directives or X-Content-Type-Options: nosniff headers embedded within the Strict-Transport-Security header, signifying complete misconfiguration.
  • Nonsensical Max-Age values, including Max-Age=0 (which effectively disables HSTS and removes the entry from the browser cache, found in 4% of HSTS directives), Max-Age=-1, or astronomically large values like five million years.
  • Max-Age=0 combined with the preload directive, explicitly turning off HSTS while simultaneously attempting to be preloaded, a contradictory and ineffective setup. This was notably observed on a security company's website (SSL store).
  • A range of Max-Age distributions: 52% set to one year, 13% to two years (both recommended values), but also many shorter durations (3-6 months, often due to recommended slow rollout processes) and extremely short ones (one day, or even one second).

These findings collectively paint a picture of an internet where foundational security headers are frequently misconfigured, misunderstood, or simply not adopted, leaving a vast attack surface open to well-understood threats like TLS stripping.

Technical Deep Dive

▶ Watch: Introducing the 'News Team' analysis tool (11:58)

To execute their ambitious scan of the top 1 million domains, Cary Hooper developed a custom, distributed execution framework named "news team." The framework was designed with several core tenets: parallel job execution, ephemeral execution nodes for cost efficiency, throughput over latency, robust error handling, and the ability to run arbitrary user-defined computations at scale.

The architecture of "news team" comprises three main components:

  1. West Mant (gRPC Abstraction): This component provides a gRPC abstraction layer, leveraging Protobuff for defining the schema of communication between the client and server. gRPC, a high-performance RPC framework by Google, was chosen for its strong typing and efficient handling of concurrency. West Mant also abstracts MTLS (Mutual TLS) for secure communication within the framework.
  2. Ed Harkin (Resource Manager): Named after the manager in a news team, this component serves as the resource manager, abstracting cloud compute APIs. It handles the dynamic spinning up and tearing down of infrastructure (e.g., EC2 instances on AWS) to minimize cost. Ed Harkin also includes a health monitor that uses continuous gRPC pings to ensure nodes are live and replaces them if they fail, embodying the principle that "failures aren't exceptional anymore" at scale.
  3. Jazz Flute (User-Defined Functions): This is the "secret sauce" where the actual business logic resides. A domain expert writes a Python function (a "funk function") to define the specific task, such as collecting HSTS headers, querying DNS, or vulnerability scanning. This function is served as a server-side callback to the gRPC server, which executes it when a job arrives. Users define a Python data class for both the job request (input data) and job response (expected output data), and a shell script automatically compiles this into the Protobuff and SQL schema definitions (e.g., for SQLite).

The overall architecture involves a client-side control node that implements a producer (to feed jobs, e.g., domain names from a text file) and a consumer (to process results, e.g., saving to a SQL database). The client instantiates a gRPC client and a resource manager. The resource manager spins up ephemeral execution nodes in the cloud, which communicate via gRPC with the client. These execution nodes perform the user-defined work.

A significant design choice for the client-side control node was the use of an async architecture with a single-threaded event loop. Instead of relying on multi-threading, which can lead to resource contention, the async event loop allows the client to send off network requests and then "await" the event of their return, performing other tasks in the interim. This makes the system incredibly fast and efficient for network-latent tasks. The data flow is straightforward: the producer creates jobs, which are submitted via the gRPC client to one of several servers (using a round-robin scheme). The server invokes the user-defined callback function, which returns a job response. This response is then passed back to the client's consumer function for processing.

This robust, scalable, and cost-efficient framework enabled the researchers to collect a precise dataset, including the HSTS string, whether a redirect occurred, and the redirect string itself, for each of the 1 million domains. The resulting database, containing 2 million records, was surprisingly compact at 146 MB, demonstrating the efficiency of their data collection and storage strategy. The cost to run such a large-scale scan was estimated at a mere $1-2, largely due to the ephemeral nature of the cloud resources.

Demo / Proof of Concept

▶ Watch: Examples of 'weird' HSTS header misconfigurations (30:00)

Wesley McElhinny's initial demonstration vividly illustrated the efficacy of TLS stripping against an unprotected website and, conversely, the protective power of HSTS. His setup involved creating a hostile network on his laptop using Linux namespaces and the tool Bettercap. This allowed him to simulate a MITM attack with an isolated browser acting as the victim, enabling him to observe precisely what the victim would see (or not see).

The demo began by establishing a baseline: showing a standard, properly rendered website. He then launched the Bettercap-driven MITM attack. When the isolated victim browser attempted to access whitebytecyber.com (an unprotected site), the attacker successfully intercepted the connection. Instead of the expected secure website, the victim's browser displayed a defaced website (a simple "Bam." message), demonstrating how the attacker could not only steal credentials but also manipulate content. A pcap file containing the intercepted traffic was also generated, available for further analysis on GitHub.

Crucially, McElhinny then demonstrated the same attack against the Wild West Hackin' Fest website. Despite using the identical network, browser, tool, and attack methodology, the attack failed to strip TLS. The browser immediately enforced a secure connection, displaying the legitimate, un-defaced website. This stark contrast highlighted the protective role of HSTS. The Wild West Hackin' Fest website had correctly implemented HSTS, likely with the preload directive, meaning the browser knew to enforce HTTPS even on the first visit, thwarting the TLS stripping attempt before it could begin.

While the primary demo focused on Bettercap and TLS stripping, Cary Hooper also provided a bonus, live demonstration of the "news team" framework. This involved spinning up five EC2 instances, preparing the nodes with a specific Python version, and then showing the framework rapidly collecting results (10,000 records) from hotel Wi-Fi. This live demonstration underscored the framework's speed, scalability, and ability to manage distributed tasks efficiently, validating its capability to perform the large-scale internet scan.

Defensive Implications

▶ Watch: Q&A: HSTS support on non-standard ports (39:40)

The findings from this extensive research offer critical defensive implications for organizations and web administrators. Correctly implementing HTTP Strict Transport Security (HSTS) is no longer an optional best practice but a fundamental requirement for protecting users against well-established TLS stripping attacks.

Here are key takeaways for defenders:

  1. Implement HSTS with a Robust Max-Age: Ensure your HSTS header includes a Max-Age directive set to at least one year (e.g., Max-Age=31536000). While a slow rollout is recommended (starting with Max-Age=300 seconds for 5 minutes, then one month, then three months, before moving to a year or more), the goal should be a long-term, persistent policy. Avoid Max-Age=0 or negative values, as these effectively disable HSTS.
  2. Always Include Subdomains: The includeSubDomains directive is crucial. Given the prevalence of subdomains for various services, applying HSTS across the entire domain space ensures consistent protection and prevents attackers from targeting less-protected subdomains.
  3. Prioritize HSTS Preload: For maximum protection, aim to get your domain on the HSTS preload list. This hardcodes the HSTS policy directly into browsers, eliminating the "trust on first use" vulnerability. Before submission, ensure all subdomains and the base domain consistently redirect HTTP to HTTPS and serve a valid HSTS header with includeSubDomains and a sufficiently long Max-Age.
  4. Understand Preload Risks (and the "Preload DDoS"): Be acutely aware that once a domain is preloaded, all HTTP connections to that domain (and its subdomains, if includeSubDomains is used) will fail with hard browser errors. If your organization has internal HTTP services on the same domain or subdomains, or if there are any certificate issues, preload will break them. Thoroughly audit your environment before considering preload. This also means being vigilant about unauthorized preload submissions for your domains, as demonstrated by the "preload DDoS" concept.
  5. Do Not Rely on DNS for Security: While HTTPS resource records (SVCB/HTTPS DNS records) are a good mitigating control, they should not be seen as a replacement for HSTS. DNS can be compromised via MITM attacks, allowing an attacker to manipulate these records. For true end-to-end security, HSTS (especially with preload) is paramount. Using DNS over HTTPS (DoH) can enhance DNS security, but its widespread adoption is still a work in progress.
  6. Regularly Audit HSTS Implementation: Misconfigurations are rampant, as evidenced by the "weird stuff" found in headers. Regularly audit your HSTS headers to ensure they conform to RFC standards and your intended security posture. Developers must understand the implications of the headers they deploy, rather than simply copy-pasting directives.
  7. Consider Internal vs. External Domains: For organizations with complex internal infrastructure that might rely on HTTP for certain services, consider using a distinct internal-only domain that is not covered by HSTS, while fully preloading your external-facing domains. This allows for both robust external security and operational flexibility internally.

By adopting these defensive measures, organizations can significantly reduce their exposure to TLS stripping and other MITM attacks, providing a more secure browsing experience for their users.

Key Takeaways

  • TLS stripping remains a prevalent threat: Despite being an old attack, many websites are still vulnerable due to the lack of proper HSTS implementation, exposing users to credential theft and content manipulation.
  • HSTS adoption is insufficient and often misconfigured: Only 25% of HTTPS-enabled sites use HSTS, and a significant portion of those that do have incorrect or contradictory directives like Max-Age=0 or nonsensical entries.
  • HSTS preload is critical but underutilized: The preload directive offers the strongest defense against TLS stripping by hardcoding HTTPS enforcement into browsers, yet less than 23% of domains advertising preload are actually preloaded.
  • "Preload DDoS" is a real risk: Over 13,000 domains are eligible for preload but not yet submitted, making them vulnerable to malicious submission that could break their internal HTTP services if they are not fully HTTPS-ready.
  • Don't trust DNS for connection security: While HTTPS resource records (RFC 9460) are a good signal, they are not a substitute for HSTS, as DNS itself can be a target for MITM attacks.
  • A scalable, cost-effective framework (news team) can drive large-scale security research: The custom-built tool demonstrates how async architecture, gRPC, and ephemeral cloud resources can enable rapid, internet-wide analysis at minimal cost (estimated $1-2 for a top 1 million domain scan).

About the Speaker(s)

Cary Hooper is a member of the pentesting team at Bank of America. He is the architect and primary developer behind the "news team" distributed execution framework, which was instrumental in conducting the large-scale HSTS analysis presented in this talk. His work focuses on building tools for efficient and cost-effective security research at internet scale.

Wesley McElhinny (who goes by "Oberlin") is also on the pentesting team at Bank of America. This presentation marked his first conference talk. He demonstrated the practical implications of TLS stripping using the Bettercap tool and contributed significantly to the research on HSTS adoption and misconfigurations. He is passionate about web security and the interaction between offensive security and development teams. Both speakers highlighted that Bank of America is actively hiring for their pentesting team.

Reviews

Dr. Zero (Offensive Security Researcher) — SOLID

This is a well-executed internet-wide scan proving that most of the web still screws up HSTS. The custom gRPC scanning rig they built is slick, but at its core, this is a presentation about a 2009 attack vector. Solid methodology and clean data, but fundamentally a reminder to check your basic configs.

Heather Calloway (CISO) — SOLID

This is a clear, data-driven look at how poorly the enterprise web implements HSTS. The speakers highlight a specific, actionable risk: the "preload DDoS," where an attacker maliciously submits your eligible domain to the browser preload list, instantly breaking your legacy internal HTTP services.

→ Top-rated talks at Wild West Hackin' Fest @ Mile High 2026

All talks from Wild West Hackin' Fest @ Mile High 2026