Beyond Quick Cash: Rethinking Bug Bounties for...

Beyond Quick Cash: Rethinking Bug Bounties for...

Speakers: Jayson Grace, Farah Hawa

Conference: BSidesSF 2024

YouTube: https://www.youtube.com/watch?v=SnEG7LA0aPc

Overview

This talk, "Beyond Quick Cash: Rethinking Bug Bounties for Greater Impact," delivered by Jayson Grace and Farah Hawa at BSidesSF 2024, advocates for a significant evolution in the bug bounty industry. The speakers contend that while the market is experiencing explosive growth, many programs and hunters remain stuck in a transactional, quantity-over-quality mindset that ultimately diminishes the strategic value and return on investment for organizations. The core message is a call to shift focus from rapid, low-impact submissions to a more strategic, in-depth approach that uncovers complex, high-impact vulnerabilities.

Jayson Grace, a seasoned bug bounty hunter with over 20 CVEs and a technical lead on Meta's Purple Team, brings a wealth of experience from both sides of the bug bounty fence. Farah Hawa, who works on Meta's Bug Bounty program and previously triaged reports at Bugcrowd, complements this perspective with insights into program management and researcher experience. Together, they highlight the shortcomings of current practices, such as the over-reliance on the Common Vulnerability Scoring System (CVSS) for payouts and the prioritization of report volume, which often leads to an influx of informational or duplicate findings.

The talk is crucial for both bug bounty program managers and security researchers. For programs, it offers a blueprint for enhancing their effectiveness by focusing on cumulative risk, improving triage processes, and revamping incentive structures to reward depth and collaboration. For hunters, it provides a strategic roadmap for identifying and reporting vulnerabilities that yield greater impact and, consequently, higher rewards, moving beyond the "quick cash" mentality to become true partners in security. By presenting Meta's bug bounty program as a model, the speakers demonstrate how an impact-driven approach, coupled with transparency and collaboration, can lead to significantly more valuable discoveries and a more robust security posture.

Background

▶ Watch: Overview of the bug bounty industry's current state and market growth. (02:00)

The bug bounty industry is in a period of rapid expansion, with public programs projected to grow from $200 million in 2020 to an estimated $5.5 billion by 2027. This growth is mirrored in the hacker community, which saw a 63% increase in participants in 2020 alone, and a rise in average bounty spend from $2,000 in 2021 to $3,000 in 2022. The HackerOne community alone has surpassed $300 million in total all-time rewards, and even emerging fields like AI are quickly adopting bug bounty programs.

Despite this impressive growth, the current state of bug bounties presents several challenges. The typical bug lifecycle emphasizes speed, as hunters are motivated to submit findings quickly to avoid duplicates and secure a reward. Once submitted, triage teams assess the vulnerability's impact, often using CVSS, which can lead to a lengthy waiting period for the hunter. While there are undeniable positives, such as the strategic value derived from collaborating with a global community of diverse talent and the supportive, knowledge-sharing ecosystem among hunters, critical issues persist. Safe Harbor policies have been a "total game changer," providing legal protection that encourages hackers to delve deeper without fear of prosecution.

However, the speakers argue that many bug bounty programs struggle with a fundamental problem: they are not cheap or easy to run, and often fail to demonstrate a clear return on investment (ROI). This failure often stems from prioritizing the quantity of reports over the quality of findings, leading to an overwhelming number of informational or low-validity submissions. Consequently, triage teams become "drowning in false positives and duplicates," making the effort feel disproportionate to the benefit. From Meta's perspective, the bug bounty program serves as the "last point of catching a vulnerability," identifying issues missed by internal tooling and automation. While the industry's shift-left approach and stronger product security make it harder for newer hunters to find bugs, the speakers emphasize that bug bounty remains accessible to anyone who can find a valid bug, regardless of their formal experience or credentials.

Key Findings

▶ Watch: Proposal for capturing cumulative risk and chaining vulnerabilities beyond in... (07:00)

The central findings of the talk revolve around identifying the shortcomings of traditional bug bounty programs and proposing a strategic evolution to maximize impact and value.

1. Inadequacy of Vanilla CVSS for Cumulative Risk: The CVSS scoring system, while widely used, often fails to capture the combined impact of multiple, individually low-to-medium severity vulnerabilities. A low and a medium bug, when chained, can suddenly become critical, but this isn't reflected in standard CVSS or typical payouts, leading to hunter dissatisfaction. The key finding is the necessity to enhance CVSS with contextual elements specific to the company, such as asset value, existing security controls, and environmental conditions, and to explicitly consider the risk of chained vulnerabilities.

1. Need for a Shift in Triage Prioritization: Programs often get bogged down by the sheer volume of reports. A critical finding is that triage teams must develop a strategy to dedicate sufficient time to complex vulnerabilities and attack chains. This requires promoting a culture of collaboration with hunters, who possess the deepest understanding of their findings, and redefining what constitutes a "significant bug" based on its actual impact rather than solely on quantity or unenhanced CVSS scores.

1. Revamping Payout Structures to Incentivize Depth: The current "quick cash" model often discourages deep, time-consuming research. A key finding is that programs should offer substantially higher rewards for vulnerabilities with profound impact. This incentivizes hunters to invest the additional time and energy required to fully understand a target and uncover unique, high-value vulnerabilities. Additionally, rewarding hunters for providing unique insights, participating in exclusive events (like bug bashes), and submitting high-quality reports (with clear reproducibility, impact, and proposed mitigations) is crucial.

1. Strategic Approach for Hunters: For hunters, the findings suggest a shift from broad, superficial scanning to a more focused, in-depth methodology. This includes:

• Program Selection: Choosing programs that offer diverse targets and genuinely interest the hunter.
• Deep Target Understanding: Delving into the business and technological frameworks of the target, including operations, industry role, revenue mechanisms, tech stack (internal/external), and compliance requirements, to identify worst-case scenarios.
• Proactive Communication: Immediately engaging with triagers upon discovering a vulnerability with unexplored potential to gain authorization for further investigation.
• Structured Investigation: Providing a clear plan of investigation, including a problem statement, specific assets, and a proposed timeline.
• Thorough Documentation: Recording all research activities for reproducibility, additional context, and as a clear record in case of security incidents.

1. Meta's Program as a Model: Meta's bug bounty program serves as a practical demonstration of these principles. Key findings from Meta's approach include:

• Impact-Driven Payouts: A unique system that prioritizes issues based on what Meta as a business cares about, rather than solely CVSS.
• Researcher Experience Focus: Prioritizing collaboration with researchers on complex reports, even when extra work is needed to maximize impact.
• Rewarding Internal Discoveries: Paying researchers for the maximum impact discovered internally, even if their initial report was less severe, fostering transparency and trust.
• Incentivizing Loyalty: Programs like Hacker Plus offer multipliers and benefits (e.g., paid travel to Defcon, invites to private bounties) to top researchers, building a loyal and highly skilled community.

These findings collectively underscore the need for a paradigm shift in how bug bounties are perceived and executed, moving towards a more collaborative, strategic, and impact-focused model.

Technical Deep Dive

▶ Watch: Case study: Chained XSS and XXE vulnerabilities leading to credential comprom... (15:00)

The talk provided several compelling case studies that illustrate the technical depth and strategic thinking required for impactful bug bounty hunting, as well as Meta's approach to rewarding such discoveries.

1. Enhancing CVSS for Cumulative Risk:

The speakers highlighted a critical flaw in the standard CVSS model: its inability to adequately assess the combined impact of multiple vulnerabilities. A scenario was presented where two individually assessed bugs, one low and one medium in severity, could, when chained together, escalate to a critical impact. To address this, programs are encouraged to enhance their risk determination processes by incorporating contextual elements such as asset value, existing security controls, and environmental conditions. This could be as simple as adding a checklist item for triagers to consider the cumulative risk of all findings.

2. Jayson Grace's Internal Search Engine Case Study:

This example demonstrated chaining vulnerabilities within an internal system. Jayson discovered a stored Cross-Site Scripting (XSS) vulnerability in a newly rolled-out internal search engine. Initial payloads resulted in an "annoying popup" for an admin, manifesting in an admin endpoint that displayed queries. After socializing this with the owners and gaining additional access, he discovered an XML External Entity (XXE) vulnerability in an XML configuration upload endpoint.

The critical step was chaining these: the stored XSS was used to compromise an admin of the system. With admin privileges, a "looping logic" was employed via the XXE vulnerability to enumerate files on disk. This eventually led to the discovery of credentials, allowing full access to the system. This case perfectly illustrates how seemingly individual issues can be combined for a profound, system-compromising impact, which was initially doubted by the product owner until a full proof of concept was provided.

3. Jayson Grace's Miter Caldera XSS Case Study:

At Defcon 30, Jayson investigated Miter Caldera, an open-source framework for automating breach and attack simulations. He found a stored XSS in the operations page, where simulations are configured. After alerting the Caldera engineers, he continued his investigation and discovered additional stored XSS vulnerabilities. The Caldera developers requested a "more realistic POC" to prioritize the fix. Within two days, Jayson developed a weaponized exploit that leveraged one of the stored XSS vulnerabilities to facilitate getting reverse shells to all enrolled systems within the Caldera environment. This demonstrated the severe risk, leading the Caldera team to swiftly prioritize and develop a robust fix in just four days. The full disclosure, including the weaponized exploit, is publicly available, underscoring the value of deep, impactful research.

4. Meta's AES Engine RCE Case Study (Farah Hawa):

This case highlights Meta's commitment to collaboration and rewarding maximum impact. A researcher reported a memory corruption vulnerability in AES, Meta's open-source JavaScript engine used for rendering AR filters. The initial report demonstrated a crash by running a malicious JavaScript file, which might have only qualified for a minimum $500 bounty.

Meta's security teams investigated and identified the potential for Remote Code Execution (RCE), noting the absence of sandboxing between the JavaScript engine and the product using it. They communicated this potential to the researcher, offering time to develop a full RCE exploit. The researcher successfully delivered an RCE exploit, which Meta confirmed. The impact of this RCE was significant: it granted the same permissions as the Facebook app on the user's device, allowing access to sensitive data like access tokens, chat history files, and potentially SMS messages if the app had those permissions.

This issue, initially a crash, escalated to a one-click RCE. Meta's payout guidelines for RCE had recently increased from $45,000 to $300,000. The researcher received a $90,000 base payout for the one-click RCE, a $6,800 Hacker Plus bonus (1.75x multiplier for being in the silver league), and a $9,000 delay bonus due to the 110-day reward processing time. A CVE was issued for this vulnerability.

5. Meta's Account Takeover Case Study (Farah Hawa):

This case, submitted by a top researcher named Yousef, involved an account takeover (ATO) vulnerability affecting Facebook or Oculus accounts by stealing a first-party access token. The attack chain leveraged multiple open redirects:

facebook.com (an OAuth endpoint with response_type=token) -> Oculus -> facebook.com/shorturl (which had an open redirect vulnerability).

The critical aspect was that the access token, generated by the OAuth endpoint, was passed through this entire chain until it reached the attacker's website via the final open redirect. While some redirects in the chain were intended, the vulnerability in facebook.com/shorturl was the exploitable link. The attack required the victim to be logged in and out of some apps, but Meta did not deduct from the bounty for this prerequisite, acknowledging that login/logout CSRF (Cross-Site Request Forgery) can be used in chains even if not rewarded individually.

Yousef received a $25,000 base payout for the two-click account takeover, a $5,000 Hacker Plus bonus, an event bonus for participating in a live hacking event, a special scope bonus, and an award for the highest impact report. The vulnerability was also publicly disclosed.

These technical examples underscore the power of chaining vulnerabilities, the importance of deep investigation, and the significant rewards that can be achieved when programs and hunters collaborate effectively to uncover and fully exploit high-impact security flaws.

Demo / Proof of Concept

▶ Watch: Case study: Stored XSS in MITRE Caldera exploited to gain reverse shells on e... (16:00)

While the talk did not feature a live, interactive demonstration during the presentation, the speakers effectively conveyed the concept of "Demo / Proof of Concept" through detailed descriptions of several real-world vulnerabilities and their exploitation. These case studies served as compelling proofs of concept for the strategic, in-depth approach to bug hunting and the significant impact that can be achieved through collaboration and chained vulnerabilities.

Jayson Grace presented two personal examples:

1. Internal Search Engine Compromise: This involved chaining a stored Cross-Site Scripting (XSS) vulnerability with an XML External Entity (XXE) vulnerability. The XSS was used to compromise an administrator, and then the XXE was leveraged with looping logic to enumerate files on disk, ultimately leading to the discovery of credentials and full system access. This demonstrated a complete system compromise, moving beyond a simple XSS popup to a critical impact.
2. Miter Caldera Reverse Shells: Jayson discovered multiple stored XSS vulnerabilities in the Miter Caldera framework. When asked for a more realistic proof of concept, he developed a weaponized exploit that used one of these XSS flaws to gain reverse shells on all systems enrolled in the Caldera simulation environment. This clearly illustrated the severity of the XSS, transforming it from a mere client-side issue into a direct remote code execution capability across multiple targets.

Farah Hawa, on behalf of Meta's bug bounty program, also detailed two significant cases:

1. AES Engine RCE: An initial report of a memory corruption (crash) in Meta's AES JavaScript engine was escalated to a one-click Remote Code Execution (RCE). Through collaboration with the researcher, Meta's security teams confirmed the RCE potential due to the lack of sandboxing, and the researcher successfully developed a full exploit. This demonstrated how a seemingly low-impact crash could be escalated to a critical RCE with significant data access implications.
2. Account Takeover via Chained Open Redirects: This proof of concept involved chaining multiple open redirects to steal a first-party access token, leading to an account takeover. The sequence of redirects, starting from an OAuth endpoint and ending with a vulnerable facebook.com/shorturl, showcased how seemingly minor vulnerabilities, when combined, could lead to a critical security breach.

In each of these instances, the "proof of concept" was not a live demonstration but rather a detailed account of how the vulnerabilities were discovered, chained, escalated, and ultimately exploited to achieve maximum impact, thereby validating the speakers' arguments for a more strategic and collaborative approach to bug bounties.

Defensive Implications

▶ Watch: Case study: Memory corruption in Meta's Hermes JavaScript engine escalated to... (20:00)

The insights shared in this talk offer crucial defensive implications for organizations running bug bounty programs and their internal security teams. Moving beyond the "quick cash" mentality requires a fundamental shift in how vulnerabilities are perceived, prioritized, and addressed.

1. Rethink Risk Assessment Beyond Vanilla CVSS: Defenders must recognize the limitations of relying solely on CVSS for risk assessment. Individual low or medium severity findings can combine into critical attack chains. Organizations should enhance their risk models by incorporating contextual elements such as the asset's business value, existing security controls, and environmental conditions. A holistic view that considers cumulative risk and the potential for chained vulnerabilities is essential to accurately gauge true impact. This means adding specific checks or processes for triage teams to evaluate how multiple reported issues might interact.

1. Prioritize Quality and Impact Over Quantity: Programs often get overwhelmed by a high volume of low-impact or duplicate reports. Defenders should actively work to shift this dynamic by:

• Developing a Prioritization Strategy: Triage teams need to be empowered and given the time to conduct deep investigations into complex vulnerabilities, rather than being pressured to quickly process a large number of simple reports.
• Redefining "Significant Bug": Focus on the actual impact a vulnerability poses to the business, rather than just its perceived risk from a basic scoring system. This might involve developing new internal metrics or collaborating with software engineering teams for deeper insights.
• Incentivizing Depth: Adjusting payout structures to offer substantially higher rewards for profound impact will naturally encourage hunters to spend more time on deeper, more complex findings, ultimately providing more value to the program.

1. Foster Proactive Collaboration with Researchers: The talk repeatedly emphasized the value of collaboration. Defenders should:

• Establish Open Communication Channels: Encourage triage teams to work directly with hunters on complex bugs or attack chains. The hunter often has the most in-depth understanding of the vulnerability.
• Grant Authorized Investigation: When a hunter discovers a potential vulnerability, programs should provide clear authorization and a framework for further investigation, including a problem statement, target assets, and a timeline. This prevents hunters from operating in a grey area and ensures blue teams aren't chasing legitimate researchers.
• Reward Internal Discoveries: Meta's model of paying for the maximum impact discovered internally, even if the initial report was less severe, builds trust and encourages researchers to share potential leads, leading to more comprehensive fixes.

1. Leverage Bug Bounty Trends for Holistic Security Improvements: Beyond fixing individual bugs, organizations should analyze trends in bug bounty reports. If similar issues are reported across different applications (e.g., Messenger, WhatsApp), this indicates a systemic problem. Defenders should use this intelligence to engage product security teams and implement holistic fixes across the organization, rather than just squashing individual bugs one by one. This "shift left" approach, where security is integrated earlier in the development lifecycle, is crucial for long-term resilience.

1. Be Prepared for Weaponized Exploits: The Miter Caldera case study demonstrated how a researcher, when incentivized and authorized, can develop a weaponized exploit to prove severe impact. Defenders should be prepared to swiftly prioritize and remediate such issues, as the Caldera team did with their four-day robust fix. This highlights the importance of having agile incident response and remediation processes in place.

By adopting these defensive strategies, organizations can transform their bug bounty programs from a reactive, transactional expense into a proactive, strategic investment that significantly enhances their overall security posture and fosters a stronger, more collaborative relationship with the global hacker community.

Key Takeaways

• Strategic, In-Depth Approach: Both bug bounty programs and hunters must move beyond surface-level vulnerability identification, focusing instead on understanding interconnectedness and the potential for complex, high-impact attack chains.
• Combined Risk Assessment is Crucial: Standard CVSS is insufficient; programs need to incorporate contextual elements (asset value, controls, environment) and explicitly consider how multiple individual vulnerabilities can combine to create a critical, cumulative risk.
• Incentivize Quality over Quantity: Programs should revamp payout structures to offer substantially higher rewards for profound impact, unique insights, and high-quality reports, motivating hunters to invest more time and effort into deeper research.
• Collaboration is Paramount: Open communication and active collaboration between triage teams, product owners, and hunters are essential for fully understanding, escalating, and effectively mitigating complex vulnerabilities.
• Hunter Empowerment and Planning: Hunters should strategically select targets, deeply understand their business and technical frameworks, and proactively communicate with programs to gain authorization for structured, in-depth investigations, leading to more valuable findings.
• Meta's Program as a Model: Meta's impact-driven payout system, focus on researcher experience, and policy of rewarding for maximum internal impact discovered, serve as a blueprint for elevating the bug bounty industry towards greater transparency and effectiveness.

About the Speaker(s)

Jayson Grace is the founder and a technical lead on Meta's Purple Team. He possesses extensive experience in web application penetration testing and bug bounty hunting, dating back to 2017. To date, he has earned over 20 CVEs and has been instrumental in establishing responsible disclosure programs. His background includes both triaging and submitting bugs across various platforms. Before his career in tech, Jayson was a touring death metal vocalist.

Farah Hawa works on Meta's Bug Bounty program. Prior to joining Meta, she gained valuable experience as a triager at Bugcrowd, where she was responsible for validating bug bounty reports, and also participated as an occasional bug bounty hunter herself. Farah is deeply passionate about all aspects of cybersecurity, bug bounties, and sharing her knowledge with these communities. She actively contributes by creating cybersecurity content on her social media channels, speaking at conferences, and collaborating with hackers globally.