The Hardest Problem I've Ever Seen: Making US Elections More Trustworthy in a World of Untrustworthy Technology

The Hardest Problem I've Ever Seen: Making US Elections More Trustworthy in a World of Untrustworthy Technology

Speakers: Matt Blaze

Conference: ShmooCon

YouTube: https://www.youtube.com/watch?v=5YHcw-qj094

Overview

In this compelling ShmooCon talk, renowned computer security researcher Matt Blaze delves into the intricate and often fraught landscape of U.S. election security. Drawing from a quarter-century of experience in the field, Blaze characterizes the challenge of ensuring election trustworthiness as "the hardest problem I've ever encountered." He explores the paradoxical reality that while serious technical vulnerabilities undeniably exist within American election infrastructure, there remains no credible evidence of these flaws ever having been exploited to alter a national election outcome. This talk is a critical examination of how the nation arrived at this precarious juncture, the technological and societal shifts that have exacerbated mistrust, and the surprisingly optimistic advancements that offer a path toward more reliable and verifiable election processes.

Blaze articulates the journey from early, seemingly primitive voting methods to the complex, software-dependent systems of today, highlighting how well-intentioned reforms inadvertently introduced new vectors of risk. He meticulously dissects the core dilemma: the absolute necessity of automation in large-scale elections versus the inherent unreliability of complex software. Far from merely identifying problems, Blaze presents a powerful narrative of progress, detailing how the security community has collaboratively developed theoretical frameworks and practical auditing techniques – specifically Software Independence and Risk Limiting Audits (RLAs) – that enable trustworthy election results even with imperfect technology.

The talk serves not only as an insightful technical analysis but also as a passionate call to action. Blaze urges technologists to engage with the ongoing challenges, from refining audit methodologies to assisting under-resourced local election officials. He emphasizes that the integrity of democratic processes hinges on both robust technical solutions and a renewed public confidence, underscoring the vital role the security community can play in securing the future of U.S. elections.

Background

▶ Watch: Introducing the "hardest problem": US election trustworthiness (4:00)

The U.S. election system, as Matt Blaze explains, is characterized by its profound decentralization. While we often focus on presidential elections, the actual administration of voting—from voter registration to ballot counting—is largely a local government function, primarily managed by thousands of counties and townships across the country. These local operations are governed by state law, typically overseen by a state official, often the Secretary of State, whose responsibilities incongruously also include registering corporations. This localized, often underfunded model means election budgets compete with essential services like pothole repair and fire departments, leading to a reliance on vast armies of volunteers and temporary workers. This logistical "nightmare or miracle," as Blaze describes it, forms the backdrop for the pervasive mistrust that has plagued U.S. elections over the past two decades.

Blaze asserts that an honest conversation about election integrity must acknowledge two frustratingly coexisting realities. First, there are indeed serious technical flaws and vulnerabilities in much of the U.S. election infrastructure. If exploited, these could potentially alter election outcomes in some jurisdictions. Second, despite extensive scrutiny, no credible evidence has ever emerged to prove that these technical vulnerabilities have actually been exploited to change the outcome of a U.S. election. This creates a "weird middle space" where the potential for disaster looms, yet evidence of its occurrence is absent, akin to the early days of internet security where known vulnerabilities didn't immediately translate into widespread catastrophic exploits.

The catalyst for modern election system changes was the contentious 2000 presidential election, particularly the Florida recount. The infamous "hanging chad" incident, arising from the use of punch card ballots and Voteomatic machines (a 1960s technology using 1920s Hollerith punch card principles), exposed a critical flaw. As ballots were punched, cardboard "chads" accumulated behind the punch positions, making it progressively harder to fully punch out a selection for popular candidates. This led to ballots with incomplete punches, or "dimples," which the machines might not read, necessitating manual human interpretation. The national embarrassment over this process spurred bipartisan action.

In 2002, Congress passed the Help America Vote Act (HAVA), which took effect in 2003. This legislation aimed to replace outdated voting technology, particularly punch card systems, and mandated the adoption of accessible voting technology for people with disabilities. HAVA provided significant federal funding to states for these upgrades. However, the problem was that much of the "modern new technology" did not yet exist in a mature commercial marketplace. HAVA primarily funded two types of systems: Precinct-counted Optical Scan paper ballots and Direct Recording Electronic (DRE) voting machines. Optical scan systems involved voters marking paper ballots (like SAT tests) that were then read by a scanner, with options for assistive devices to help voters mark ballots. DREs, often touchscreens, presented the ballot on a screen and recorded selections internally, electronically, without a voter-verifiable paper trail. This shift fundamentally introduced software into the core voting process, setting the stage for the complex security challenges that followed.

Key Findings

▶ Watch: Acknowledging widespread mistrust in US election integrity (8:00)

The introduction of DREs and the increased reliance on software marked a pivotal moment, creating two significant problems: the challenge of making this technology secure and trustworthy enough for elections, and the equally critical task of fostering public confidence in the accuracy of election outcomes. Blaze notes the perverse effect that progress in improving the technology can sometimes highlight existing flaws, paradoxically reducing public confidence. The core technical problem, he states, is simply: "Is the outcome correct? Does the outcome reflect the votes? Did we get the right winner?" This seemingly straightforward question becomes extraordinarily difficult given the unique requirements of U.S. elections.

Blaze outlines why this is "the hardest problem I've ever encountered":

1. Complexity: U.S. elections are incredibly complex, with a vast number of local, state, and federal races, leading to highly varied and intricate ballots. This necessitates complex software, which is inherently unreliable.
2. No Do-overs: Election schedules are rigid, and there's generally no mechanism for holding a new election if errors are discovered. This means security protocols must prevent problems, not just detect them, as detection without correction is worse than ignorance.
3. No Trusted Third Parties: In a democracy, everyone has an interest in the outcome, making it impossible to outsource trust to an independent, neutral party.
4. Secret Ballot Requirement: Voters expect their ballot to be secret, yet simultaneously demand assurance that the overall outcome is correct and trustworthy, even when the underlying technology and administrators are not universally trusted. This creates a fundamental dilemma.

The pervasive presence of complex, potentially untrustworthy software across all facets of election management, from voter registration to ballot counting, can lead to unrecoverable and undetectable uncertainty about election outcomes. This forms the basis for an "unresolvable argument" where allegations of a "stolen" or "rigged" election, whether due to malice or accident, are difficult to definitively refute, a point tragically underscored by the January 6th Capitol riot following the 2020 election.

However, Blaze emphasizes that the security community has made "enormous progress" in addressing these challenges. The breakthrough came from a paradigm shift: instead of striving for perfect, flaw-free technology (an ultimately "losing game"), the focus moved to acknowledging inherent flaws and designing systems that can yield reliable results despite unreliable technology. This was enabled by two key concepts:

1. Software Independence: Proposed by Ron Rivest (the 'R' in RSA), this design requirement stipulates that a flaw in the software should not result in an unrecoverable error in the election outcome. This means the election's integrity should not solely depend on the flawless execution of software.
2. Risk Limiting Audits (RLAs): Developed by statistician Philip Stark at Berkeley, RLAs provide a practical method to achieve software independence. These audits involve statistically rigorous sampling and manual verification of voter-marked paper ballots. By comparing the machine's interpretation of a small, randomly selected subset of ballots with human interpretation, a measurable level of confidence can be gained that the entire election was counted correctly. This allows for quantification of the risk that an incorrect outcome is being reported, even if the underlying machines have software flaws.

This combination of theoretical design principles and practical auditing techniques has transformed the problem from seemingly "unsolvable" to one with tangible solutions, offering a pragmatic path to trustworthy elections.

Technical Deep Dive

▶ Watch: Reality 1: Serious technical flaws in election infrastructure (9:30)

The core technical challenge in modern U.S. elections stems directly from the introduction of software into processes previously handled mechanically or manually. As Blaze explains, the Help America Vote Act (HAVA) of 2002 pushed states to adopt new voting technologies, primarily Precinct-counted Optical Scan paper ballots and Direct Recording Electronic (DRE) voting machines.

Optical Scan systems generally involve voters marking a physical paper ballot, typically by filling in bubbles with a pencil, similar to standardized tests. These hand-marked ballots are then fed into a scanner, which reads the marks and stores the results electronically. There are also machine-marked optical scan ballots, where an assistive device helps the voter (e.g., via an audio interface for blind voters) to mark a paper ballot, which is then fed into the scanner. The crucial aspect of these systems is the creation of a voter-marked physical artifact – the paper ballot – which exists independently of the machine's software.

In contrast, DRE voting machines, often touchscreens, present the ballot digitally on a screen. Voters make their selections directly on the screen, and these choices are recorded internally within the machine, also electronically. Critically, many early DREs, such as the ES&S iVotronic, did not produce a voter-verifiable paper record. In these systems, the display of the ballot, the processing of voter input, and the recording of the vote are all entirely under the control of software. The final record of the vote is machine-readable only, meaning a human cannot directly inspect it to confirm their intent without relying on another piece of software.

Blaze highlights that this pervasive reliance on software is inherently problematic. "Complex software is something we simply don't know how to build at scale" without introducing unreliability. This means that if an election outcome is decided by software, and a flaw, whether accidental or malicious, is present, defending the integrity of that outcome becomes incredibly difficult, often leading to an "unresolvable argument."

The "hardest problem" is further compounded by several unique requirements of U.S. elections:

• Extreme Complexity of Ballots: The sheer number of offices and propositions voted on, from federal to hyper-local (e.g., school boards, dog catcher), means ballots are often unique even across neighboring precincts. This necessitates highly complex software to manage ballot definitions, voter eligibility, and vote tabulation, increasing the likelihood of bugs and vulnerabilities.
• Impossibility of Do-overs: Unlike many other security contexts where problems can be detected and corrected, election timelines are fixed. If a flaw is discovered post-election, a re-election is almost never feasible, particularly for major races. This mandates a system that can prevent errors or recover from them without a full restart.
• Absence of Trusted Third Parties: In an adversarial political environment where all participants have a vested interest in the outcome, there is no universally trusted, impartial entity to whom election integrity can be delegated.
• Secret Ballot Paradox: The fundamental democratic principle of a secret ballot means individual votes cannot be linked back to a voter. Yet, voters demand assurance that their vote was accurately counted and the overall outcome is correct. This creates a challenging paradox: how to verify the aggregate without compromising individual privacy.

These factors together make election security a unique and formidable challenge, exacerbated by threats ranging from random mishaps (e.g., Hurricane Sandy's impact on elections), insider threats (malicious election officials), and outsider threats (dishonest candidates, sophisticated foreign state actors).

The breakthrough in addressing these systemic issues lies in the concepts of Software Independence and Risk Limiting Audits (RLAs).

• Software Independence, as defined by Ron Rivest, means that "a flaw in the software should not result in an unrecoverable flaw in the election outcome." This shifts the focus from building perfect software (which is impossible) to designing systems that can verify results through an independent means.
• Risk Limiting Audits (RLAs), pioneered by Philip Stark, provide the practical mechanism to achieve software independence. RLAs are post-election audits that leverage the existence of voter-marked paper ballots. The process involves:

1. Random Sampling: A statistically rigorous, random sample of paper ballots is selected. The size of this sample depends on the closeness of the race; tighter races require larger samples.
2. Human Interpretation: The selected paper ballots are manually inspected by humans to determine the voter's intent, independently of the machine's initial count.
3. Comparison and Quantification: The human-verified results for the sampled ballots are compared against the machine's original tabulation for those same ballots. If discrepancies are found, the audit may expand.
4. Confidence Level: Based on the comparison, a quantifiable level of confidence is established regarding the accuracy of the overall election outcome. This allows election officials to state, for example, "there is a 95% chance that the reported winner is correct." If the audit reveals a significant discrepancy, it can trigger a full hand recount.

The power of RLAs lies in their ability to provide a measurable assurance of correctness, even if the electronic tabulation software has bugs or has been compromised. This is because the audit relies on the physical paper ballots—artifacts created by the voter, not the software—as the canonical record. Systems like DREs without a voter-verifiable paper trail cannot be subjected to RLAs, as there is no independent record to audit against the electronic tally. This is why Blaze strongly advocates for the widespread adoption of paper-based voting systems that are amenable to RLAs.

Demo / Proof of Concept

▶ Watch: Reality 2: No evidence of exploited technical vulnerabilities (10:10)

The talk by Matt Blaze, "The Hardest Problem I've Ever Seen: Making US Elections More Trustworthy in a World of Untrustworthy Technology," is primarily a high-level technical and societal analysis, historical overview, and call to action rather than a demonstration of specific tools or vulnerabilities. While Blaze mentions his involvement with the Defcon Voting Village, where voting systems are examined and reverse-engineered, the presentation itself does not feature a live demo or a proof of concept of an exploit or a new security tool. Instead, it focuses on the theoretical and practical frameworks, such as Software Independence and Risk Limiting Audits, that contribute to election security.

Defensive Implications

▶ Watch: The paradox: potential problems versus no exploitation evidence (11:30)

Matt Blaze's talk provides a clear roadmap for improving election security, emphasizing both technical and non-technical interventions. Defenders, including election officials, technologists, and concerned citizens, should consider the following actions:

1. Prioritize Software Independent Voting Systems: The most critical defensive measure is the widespread adoption of voting systems that produce voter-verifiable paper ballots. These are systems where the voter's intent is recorded on a physical piece of paper that they can review before casting. This ensures that even if the electronic tabulation software is flawed or compromised, an independent, human-readable record exists as the canonical ballot. This implicitly means phasing out DREs without paper trails.
2. Implement Risk Limiting Audits (RLAs): RLAs should become standard practice for all elections. These statistically rigorous post-election audits provide a quantifiable measure of confidence that the reported election outcome is correct. Defenders should advocate for state laws that mandate RLAs and ensure they are conducted transparently and effectively.
3. Optimize Optical Scan Systems for RLAs: While optical scan systems produce paper ballots, current implementations are not always optimal for the most efficient RLAs. There's ongoing work to improve the information reported by these systems to facilitate more streamlined and effective audits. Technologists can contribute to research and development in this area.
4. Strengthen Non-Ballot Systems: Election integrity extends beyond casting and tallying votes. The security and robustness of other critical components, such as voter registration databases, election logistics, and results reporting systems, must be enhanced. These systems are also susceptible to software flaws and attacks that can undermine trust or create chaos.
5. Engage with State and Local Election Officials: Historically, election officials have been wary of external technical input. However, this has changed, and they are now more welcoming of help. Technologists should actively offer their expertise, learn the "language" of election administration, and build collaborative relationships. This involves understanding their operational constraints, budget limitations, and legal frameworks, rather than simply dictating technical solutions. Blaze specifically advises becoming an election worker to gain firsthand experience and build trust.
6. Accelerate Obsolescence of Insecure DREs: While many insecure DREs are disappearing, some remain in use. These systems, lacking voter-verifiable paper trails, cannot be reliably audited. Efforts should continue to encourage jurisdictions to replace them with software-independent alternatives.
7. Address Underfunding and Resource Disparity: Elections, particularly at the county level, are chronically underfunded. This forces them to operate with limited resources, making it nearly impossible to defend against sophisticated adversaries like foreign state actors. Advocacy for increased and consistent funding for election security, along with treating election infrastructure as critical national infrastructure, is essential. Blaze points out the absurdity of asking county sheriffs to "repel foreign military invasions," yet expecting local election officials to defend against nation-state cyberattacks.
8. Combat Disinformation: The pervasive issue of election disinformation erodes public trust, regardless of technical security. While a larger problem, defenders must be prepared to articulate the security measures in place and counter false narratives with facts and transparent processes, leveraging the confidence gained through RLAs.
9. Proactive Engagement Between Elections: The period between presidential elections is the crucial window for implementing changes and improvements. Technologists should use this "lull" to work on audit improvements, system upgrades, and relationship building, as changes cannot be effectively made immediately before an election.

In summary, the defensive strategy is multifaceted: embrace paper-based, auditable systems; rigorously audit them; secure the entire election ecosystem; and foster strong, collaborative relationships between technical experts and election administrators, all while advocating for adequate resources and countering disinformation.

Key Takeaways

• Persistent Vulnerabilities, Emerging Solutions: U.S. election infrastructure still harbors serious technical vulnerabilities, but significant progress has been made in developing theoretical and practical solutions to achieve trustworthy outcomes despite these flaws.
• Software Independence and Risk Limiting Audits (RLAs) are Crucial: The concepts of Software Independence (systems where software flaws don't lead to unrecoverable errors) and Risk Limiting Audits (statistical post-election audits of paper ballots) are the cornerstones for ensuring election integrity.
• Paper Ballots are Essential for Trust: Systems that produce voter-verifiable paper ballots are fundamental for conducting RLAs and providing an independent, human-readable record against which electronic tallies can be audited. DREs without paper trails are inherently problematic.
• Complexity and Underfunding are Systemic Challenges: The highly decentralized, complex, and often underfunded nature of U.S. election administration, particularly at the county level, makes it uniquely susceptible to both accidental errors and sophisticated attacks, including from foreign state actors.
• Technologists Must Engage Collaboratively: The election community needs technical expertise, but engagement must be collaborative, respectful, and understanding of the operational realities and constraints faced by election officials. Simply dictating solutions is ineffective.
• Active Participation is Key: Matt Blaze strongly encourages individuals to become election workers to gain firsthand experience, build trust, and contribute directly to the integrity of the process.

About the Speaker(s)

Matt Blaze is a distinguished computer security researcher and professor, currently holding a position at Georgetown University. With a career spanning over a quarter of a century in the field of election security, Blaze is a veteran voice in the discussion surrounding the trustworthiness of democratic processes. Early in his work, he admits to finding the problem of election integrity "pretty hopeless" due to its complexity and the lack of collaboration among stakeholders. However, he has since witnessed and contributed to "enormous progress" in developing practical solutions. Blaze is also known for his involvement in initiatives like the Defcon Voting Village, where he co-runs efforts that invite security researchers to examine and reverse-engineer voting systems, contributing to vital good-faith security research. His work often bridges the gap between theoretical computer science and the practical, often messy, realities of real-world system implementation and policy.