Shades of crisis across an insecure software supply chain

Trey Herr (Director of the Cyber Statecraft Initiative · Atlantic Council)

Swiss Cyber Storm 2021 · Day 1 · Scenario_2

Overview

In this insightful talk from Swiss Cyber Storm, Trey Herr, Director of the Cyber Statecraft Initiative at the Atlantic Council, dissects the escalating crisis within the software supply chain. The presentation, aptly titled "Shades of crisis across an insecure software supply chain," unveils a systematic analysis of software supply chain attacks, drawing from a unique dataset compiled by Herr's team since October 2019. This initiative was born out of a critical gap: the absence of a consistent, systematic database tracking such incidents, despite anecdotal evidence of their growing prevalence and impact.

Watch on YouTube · Slides

Visual summary for Shades of crisis across an insecure software supply chain by Trey Herr
Visual summary for Shades of crisis across an insecure software supply chain by Trey Herr

Key moments

  1. 1:10 The need for systematic software supply chain attack data
  2. 5:40 Four critical trends in software supply chain attacks
  3. 9:00 Massive scale and targeted exploitation in supply chain attacks
  4. 15:40 SolarWinds shows a repeating pattern of trust abuse
  5. 17:00 Defending open source, S-BOMs, and architectural security
  6. 23:20 Q&A: Who should fund open source security initiatives?
  7. 26:00 Q&A: Adversary monitoring and maintaining access post-compromise

Shades of crisis across an insecure software supply chain

Speakers: Trey Herr, Director of the Cyber Statecraft Initiative, Atlantic Council

Conference: Swiss Cyber Storm

YouTube: https://www.youtube.com/watch?v=7muWEr2G7Q0

Overview

In this insightful talk from Swiss Cyber Storm, Trey Herr, Director of the Cyber Statecraft Initiative at the Atlantic Council, dissects the escalating crisis within the software supply chain. The presentation, aptly titled "Shades of crisis across an insecure software supply chain," unveils a systematic analysis of software supply chain attacks, drawing from a unique dataset compiled by Herr's team since October 2019. This initiative was born out of a critical gap: the absence of a consistent, systematic database tracking such incidents, despite anecdotal evidence of their growing prevalence and impact.

Herr's work provides a sobering, data-backed perspective on why these attacks are not merely a persistent threat but a fundamental challenge to modern digital infrastructure. He argues that our increasing reliance on software, which often replaces traditional hardware functions, coupled with an inherent trust in code we don't build ourselves, creates a fertile ground for exploitation. Furthermore, existing policy frameworks and risk management strategies, largely inherited from hardware-centric models, are ill-equipped to address the complex, web-like nature of software dependencies.

The talk underscores the urgent need for a paradigm shift in how organizations, governments, and the open-source community approach software security. By revealing key trends, common attack vectors, and the strategic motivations behind these compromises, Herr not only quantifies the scale of the problem but also lays out actionable pathways for collective defense. His analysis serves as a critical call to action for stakeholders across the digital ecosystem to invest in open-source security, embrace transparency through tools like Software Bill of Materials (SBOMs), and fundamentally rethink architectural design principles to build more resilient software from the ground up.

Background

▶ Watch: The need for systematic software supply chain attack data (1:10)

The pervasive nature of software in modern society forms the bedrock of its vulnerability. As Herr highlights, software now scales business processes, and in many instances, is replacing hardware entirely—from virtualized network functions to the core logic of devices. The F-35 fighter jet, for example, is described not as a fighter but "a computer that happens to fly," illustrating the deep integration and reliance on software even in critical national security assets. This dependency means that software supply chains are inherently attractive targets for attackers seeking widespread impact.

A foundational problem, recognized as early as 1984 by Ken Thompson in his Turing Award speech, is the inability to fully trust code that one does not build oneself. This issue has only intensified over the past 35 years, as modern software development relies heavily on third-party components, libraries, and frameworks. Customers are often locked into a relationship of trust with vendors, with limited ability to influence or inspect the underlying code. This "channel of trust" becomes a prime target for abuse.

Compounding this challenge is a significant mismatch between existing risk management frameworks and the reality of software supply chains. Most established practices, particularly in governmental policy, were designed for hardware supply chains, which typically follow a linear progression from vendor to user. Software supply chains, however, are far more complex, resembling a "web" or "network of relationships" rather than a simple chain. This architectural discrepancy leaves a huge gap in the cybersecurity space, where dependence on untrusted software is managed by outdated models. The Atlantic Council's Cyber Statecraft Initiative recognized this void, embarking in October 2019 on a project to build the first consistent, systematic database of software supply chain attacks, providing the empirical foundation for this critical discussion.

Key Findings

▶ Watch: Massive scale and targeted exploitation in supply chain attacks (9:00)

The Atlantic Council's dataset, now in its third iteration, presents a stark picture of the software supply chain landscape. Over the last 11 years, there have been at least 161 software supply chain attacks or disclosures of vulnerabilities that could have directly led to an attack. This total comprises 44 disclosures and 117 live attacks, a number that has steadily grown from 119 events in March 2020 to 161 today, and is expected to continue increasing. Herr highlighted four critical trends from this data:

  1. Frequent and Repeated Targeting of Open Source: Open-source projects are under constant assault. In the last year alone, Herr noted almost a dozen attacks on repositories like npm (for Node.js) and PyPI (for Python). These attacks are not always sophisticated, often involving typosquatting—renaming commonly used projects with minor variations to trick users into downloading malicious versions. In some cases, these malicious packages have been downloaded tens of thousands of times. More advanced attacks on npm have even manipulated how the dependency management tool pulls packages, redirecting users to malicious versions.
  2. State-Attributed Attacks: At least 32 attacks have been positively attributed to state actors, primarily Russia and China, with the U.S., Vietnam, and India also present. Herr stresses that this number is almost certainly an undercount, as the dataset only includes publicly reported events in English-language sources, with known instances of non-disclosed vendor compromises. This trend indicates that nation-states are actively abusing trust relationships within the software supply chain to achieve strategic objectives, often targeting "hard targets."
  3. Targeting App Stores and Developer Tools: A significant 24% (almost a quarter) of all attacks target app stores and developer tools. This vector is particularly potent because compromising the tools used to build many different types of software can scale the attack's effect almost exponentially. Attackers can insert malicious functionality at a foundational level, impacting numerous downstream applications and users.
  4. Attacks on Software Updates: A full 25% (41 out of 161) of these attacks directly target software update mechanisms. This undermines the fundamental trust relationship between users and vendors. If this channel is compromised, vendors lose their ability to deliver security fixes, rendering bug bounty programs and vulnerability disclosure processes ineffective. These attacks not only have direct consequences but also weaken the entire cybersecurity ecosystem by eroding basic trust.

Beyond these trends, Herr identified several overarching characteristics of modern software supply chain attacks:

  • Efficiency: Supply chain attacks are incredibly efficient, allowing attackers to target one entity and impact millions of users. The XcodeGhost incident in 2015 saw malicious functionality inserted into a version of Apple's developer tool, which was then uploaded to a Chinese file-sharing platform. This compromised Xcode was used to build numerous legitimate apps, including WinZip, leading to 500 million users downloading compromised software before the issue was resolved.
  • Targeted Exploitation within Broad Compromise: While some attacks (often criminal) use a "shotgun approach" for financial gain, strategic actors frequently employ a more surgical method. They compromise widely used software to gain access to a broad base of customers, but then only exploit a very small, specific number of high-value targets. Examples include the Netfilter incident (where a legitimately signed malicious Microsoft driver targeted 100,000 potential victims but only exploited five) and the SolarWinds/Sunburst attack (which compromised 18,000 customers but only exploited eight U.S. government executive departments and agencies and 113 large companies).
  • Subverting Vendor Code Signing: Attackers increasingly aim to inject malicious functionality earlier in the development process, ensuring it gets signed and "blessed" by the legitimate vendor. The Asus ShadowHammer attack in 2019, attributed to the Chinese group Barium, involved using two authentic code-signing certificates to upload malicious firmware to one million laptops. The attackers specifically targeted only 600 users by hard-coding their MAC addresses into the malware, demonstrating sophisticated, targeted exploitation within a massive compromise.
  • Open Source Vulnerability as a Generational Shift: Herr emphasizes that open source itself is not "broken," and its community values are positive. However, the critical infrastructure now run on open-source code creates risks developers never contemplated. The "many eyes make bugs shallow" adage is only true if resources exist to fix and distribute those fixes. This necessitates a generational shift in how open source is secured, including funding, tools, and better security practices at repositories.

Finally, Herr contextualized the SolarWinds/Sunburst attack, discovered in December 2020, as a prime example of these trends. This years-long operation, likely by a Russian intelligence organization, compromised an Austin-based software vendor's build process in 2019, pushing a malicious update to 18,000 customers in February 2020. Despite its perceived unprecedented nature in media, Herr revealed that seven other eerily similar attacks had occurred in the preceding decade, including Sign Site Desktop and Wizard of Airport in the same year. These incidents consistently target administrative and security tools, exploit common flaws (not exotic ones), and collectively undermine trust, making future defenses significantly harder.

Technical Deep Dive

▶ Watch: SolarWinds shows a repeating pattern of trust abuse (15:40)

The talk elaborates on how these attacks manifest across the software development lifecycle, moving beyond a simplistic "waterfall" model to illustrate the interconnectedness of modern software. Herr's simplified model tracks software from system design through implementation, iterative testing, deployment, and finally to updates. The data shows attacks are detected at every stage, but certain points are particularly vulnerable due to their inherent trust mechanisms.

A significant portion of attacks target open source components and repositories. The sheer volume of open-source libraries and packages, combined with often under-resourced maintainer communities, creates an attractive attack surface. Simple yet effective attacks like typosquatting on platforms such as npm and PyPI exploit human error, tricking developers into downloading malicious look-alike packages. More sophisticated attacks have demonstrated the ability to manipulate how dependency managers pull packages, redirecting legitimate requests to malicious versions, effectively compromising the integrity of the package distribution pipeline itself.

Developer tools represent another critical attack vector. By compromising the tools used to build software, attackers achieve a multiplicative effect. The XcodeGhost incident serves as a prime example. In 2015, a malicious version of Apple's Xcode development environment was inserted with harmful functionality. This compromised version was then uploaded to a Chinese file-sharing platform, bypassing official distribution channels. Developers in regions with slower access to Apple's servers downloaded this unofficial version, unwittingly embedding malware into their applications. Legitimate applications, including popular ones like WinZip, were then built with this tainted Xcode, leading to 500 million users downloading compromised versions of software that they believed to be secure. This attack demonstrated how a single compromise at the tool level could cascade through the entire software ecosystem.

The direct targeting of software updates is particularly insidious, as it directly attacks the vendor-customer trust relationship. When an attacker compromises a vendor's update mechanism, they can push malicious code that appears to be legitimate, signed, and sourced from the trusted vendor. This was a core element of the SolarWinds/Sunburst attack. The attackers gained access to SolarWinds' network in 2019, specifically targeting the build stage of their Orion platform. They injected malicious code into the legitimate software, which was then compiled, signed, and distributed as a regular update to 18,000 customers in February 2020. This "supply chain compromise" allowed a Russian intelligence organization to leverage SolarWinds' trusted update channel to deliver their malware, SUNSPOT (the build-time implant) and SUNBURST (the delivered backdoor).

The subversion of vendor code signing further highlights the depth of these compromises. In the Asus ShadowHammer attack, the Chinese group Barium obtained two authentic code-signing certificates belonging to laptop manufacturer Asus. They used these certificates to sign malicious firmware updates, which were then pushed to approximately one million Asus laptops. Crucially, the malware contained hard-coded MAC addresses for only 600 specific targets, demonstrating a highly selective and precise exploitation strategy within a massive initial compromise. This shows attackers are able to infiltrate deeply into vendor operations to leverage their trusted authentication mechanisms.

The SolarWinds/Sunburst incident also revealed critical architectural vulnerabilities. While the initial compromise was a sophisticated supply chain attack, the subsequent exploitation leveraged a "relatively well-known flaw" in Microsoft's identity and access management (IAM) services. Microsoft's complex architecture for tying on-premises IAM to cloud services had not undergone a "rigorous security audit" of how these different services could be abused. This meant that when security assumptions were violated – for instance, regarding the interaction between ADFS, Azure AD, and various authentication protocols – attackers could pivot and steal mail, files, and other sensitive data. Herr argues that this combination flaw underscores the importance of architectural security; simply finding flaws post-build is insufficient if the foundational design introduces systemic weaknesses. The parallels with seven other similar state-backed espionage attacks, such as Sign Site Desktop and Wizard of Airport, further illustrate that these are not isolated or exotic incidents but rather recurring patterns exploiting known weaknesses in system design and operational security.

Demo / Proof of Concept

▶ Watch: Q&A: Who should fund open source security initiatives? (23:20)

This talk focuses on a data-driven analysis of software supply chain attacks and their implications, rather than demonstrating a specific tool, exploit, or proof of concept. The content is an analytical exposition of findings from the Atlantic Council's dataset, aimed at identifying trends and proposing strategic defensive measures.

Defensive Implications

▶ Watch: Q&A: Adversary monitoring and maintaining access post-compromise (26:00)

Trey Herr outlines three critical areas for action to counter the growing threat of software supply chain attacks, moving beyond simply identifying vulnerabilities to implementing systemic changes.

First and foremost, there is an urgent need to get serious about defending open source. This requires substantial investment in "dollars, time, and effort" from various stakeholders. Herr highlights the Open Source Security Foundation (OpenSSF), an initiative by the Linux Foundation, which has received millions from tech giants like Google, Microsoft, and IBM. However, he stresses that the problem is a "hundreds of millions of dollars a problem, not a tens of millions of dollars a problem," and current investment is still in the "single millions." The goal is not just to find bugs, but to provide resources to fix them, kill "whole classes of bugs," and make code maintenance and security management easier and cheaper for open-source maintainers. He calls upon large technology companies, governments (e.g., EU initiatives that focus on vulnerability discovery but need to shift to funding fixes), and the academic community to step up. Governments, in particular, should recognize open source as critical infrastructure and fund its security, potentially by maintaining widely used open-source security tools.

Second, organizations must embrace Software Bill of Materials (SBOMs), but with a clear understanding of their role. Herr describes SBOMs as a "step change" in visibility, providing customers with unprecedented access to data about the dependency graphs and provenance of the code they use. However, this increased data comes with a challenge: customers need to "figure out how to use that." The community must collaborate to develop methods for making better risk assessments from this incoming flood of information. Governments and academicians will play a crucial role in determining how changes in relationships and code provenance inform risk over time. Herr warns against two potential pitfalls: transparency measures sputtering out due to a lack of understanding or action, and resistance from major cloud vendors (Platform-as-a-Service providers often express "neutral to negative" views on SBOM disclosure), which could hinder widespread adoption. The goal is reliable information that enables customers to understand and manage product risk.

Finally, Herr emphasizes the critical importance of architecture. It is no longer sufficient to merely find flaws after software has been built; security must be foundational to the design process. The SolarWinds/Sunburst incident highlighted this, where a "relatively well-known flaw in Microsoft's identity and access management services" was exploited. Herr points out that Microsoft's architecture, with its numerous ways to tie on-premises IAM to the cloud, had not undergone a sufficiently "rigorous security audit" of how these interdependencies could be abused. When security assumptions were violated, the game was over. Defenders must think about security early in the design phase, "trying to douse this before it becomes a service used by tens of thousands of customers." This means rigorously auditing how services rely on both vendor and customer configurations, understanding potential points of failure, and designing for resilience. Secure development and application security are vital, but they must be complemented by a strong focus on secure software deployment and management architecture, acknowledging that simply moving code from one place to another does not absolve it of risk, especially within the complex "web" of modern software supply chains.

Key Takeaways

  • Escalating Threat: Software supply chain attacks are frequent, growing in number (161+ in 11 years), and highly efficient, allowing attackers to compromise one entity for widespread impact (e.g., XcodeGhost affecting 500 million users).
  • Open Source Vulnerability: Open-source projects are a primary target, facing repeated assaults including typosquatting and dependency manipulation. They require significant, collective investment (hundreds of millions, not millions) from industry, government, and academia for sustained security.
  • Strategic State-Sponsored Attacks: At least 32 state-attributed attacks (Russia, China, etc.) leverage supply chain compromises, often employing broad initial compromise followed by highly targeted exploitation against specific, high-value entities (e.g., SolarWinds targeting 8 government agencies out of 18,000 customers).
  • Undermining Trust Mechanisms: Attacks frequently target software update mechanisms (25% of incidents) and subvert vendor code signing (e.g., Asus ShadowHammer), fundamentally eroding the trust relationship between users and vendors, and hindering effective vulnerability remediation.
  • Architectural Security is Paramount: Beyond finding post-build flaws, secure architectural design from the outset is critical. Complex systems like Microsoft's IAM, when not rigorously audited for interdependency flaws, create systemic vulnerabilities that can be exploited even after sophisticated supply chain compromises.
  • Collective Responsibility for Transparency: Initiatives like Software Bill of Materials (SBOMs) offer crucial visibility into dependencies but require collective effort from customers, governments, and academics to develop effective risk assessment methodologies and overcome resistance from some cloud vendors.

About the Speaker(s)

Trey Herr is the Director of the Cyber Statecraft Initiative at the Atlantic Council, a U.S.-based policy think tank. The Atlantic Council, founded in 1961, is a non-partisan, non-profit organization primarily focused on national security and foreign policy. Herr's team of five within the initiative concentrates specifically on cybersecurity. He noted that his team was "lucky" to have started their project on understanding the frequency of software supply chain attacks in October 2019, just before the issue gained widespread prominence. Prior to his role at the Atlantic Council, Herr held positions at other notable organizations, including Microsoft, and has been associated with prestigious academic institutions such as Harvard and Stanford. He is known for backing his claims with data, a principle central to the Atlantic Council's systematic analysis of software supply chain incidents.

Reviews

Dr. Zero (Offensive Security Researcher) — STRONG ACCEPT

I usually walk out of think-tank keynotes five minutes in. Herr actually brought the receipts. He maps out 161 supply chain attacks to show exactly how states and criminals are weaponizing our dependencies, and he does it without a single vendor pitch.

Heather Calloway (CISO) — STRONG ACCEPT

Herr puts hard numbers behind what every CISO has felt for the last five years: the software supply chain is structurally compromised. This is a clear, data-driven briefing on how trust mechanisms are being weaponized against us. Worth your leadership team's time, especially when reviewing vendor concentration and procurement standards.

→ Top-rated talks at Swiss Cyber Storm 2021

All talks from Swiss Cyber Storm 2021