Regulatory Failures with Ballot Marking Devices

Regulatory Failures with Ballot Marking Devices

Speakers: Marnie Mahoney, Professor of Law, University of Miami

Conference: Voting Village

YouTube: https://www.youtube.com/watch?v=y1zZtEm_rvk

Overview

Professor Marnie Mahoney, a Professor of Law at the University of Miami and a distinguished Dean's Scholar, delivered a compelling presentation at the Defcon Voting Village, critically examining the pervasive issue of regulatory failures with Ballot Marking Devices (BMDs). The talk, provocatively titled "A Recipe for Distrust," delves beyond mere technical vulnerabilities to explore the systemic issues that erode public confidence in election outcomes. Mahoney argues that the current regulatory frameworks for BMDs are fundamentally flawed, leading to systems that are not only susceptible to security risks but also inherently incapable of providing reliable public assurance of their trustworthiness.

Mahoney's central thesis highlights the urgent need for a paradigm shift in how election equipment is regulated. She contends that the widespread adoption of BMDs, particularly those relying on inscrutable codes like QR codes and barcodes, creates an unbridgeable gap between the voter's intent and the auditable record of their vote. This talk emphasizes that while technical accuracy and verifiability are crucial, the ultimate goal must be to foster public trust, which current BMD implementations, even those moving towards "human-readable" lists, consistently fail to achieve. Her research, still evolving at the time of the talk, suggests a move away from flexible legal "standards" towards more definitive "bright-line rules" to address these deep-seated problems.

The significance of this discussion is underscored by alarming statistics on public distrust. As of 2024, a Washington Post poll found that approximately one-third of Americans believe the 2020 election was unfair or rigged, despite a complete lack of legal evidence to support such claims. This environment of distrust is not only damaging to the integrity of democratic processes but also to the dedicated election officials tasked with upholding them. Mahoney's presentation serves as a critical call to action for legislators, election officials, and the public to confront the fundamental design flaws of BMDs and demand systems that are truly transparent, verifiable, and capable of building and maintaining public confidence.

Background

▶ Watch: Introduction: Regulatory failures & urgent need for trust (0:00)

The journey to the current state of Ballot Marking Devices (BMDs), and their associated regulatory shortcomings, is a complex one rooted in decisions made years ago, often under the guise of modernization or accessibility. Professor Mahoney traces this trajectory, emphasizing that the flaws now evident in BMD systems were, or should have been, "obvious" from the outset. Once these systems are certified, purchased with substantial public funds, and deployed, challenging or changing them becomes exceedingly difficult. Legal obstacles like standing and statutes of limitations, as seen in cases challenging Dominion systems in Georgia, often impede efforts to rectify problematic implementations. Judges, mindful of their judicial role, are also hesitant to order system overhauls unless legal mandates are unequivocally violated.

A significant point of concern arises from the way votes are recorded and interpreted by BMDs. Mahoney specifically highlights the use of QR codes and barcodes on printed ballots, which she refers to as "ballot artwork." These codes, while ubiquitous in daily life for tasks like package delivery or connecting to Wi-Fi, are problematic in the context of elections because they are inscrutable to the human voter. The Florida regulation, for instance, defines "ballot artwork" as information used to decode barcoding schemes, implying that the actual vote count relies on an interpretation of this encoded data rather than directly from human-readable text. This lack of transparency means that the voter cannot independently verify that their selections, as represented in the machine-generated code, accurately reflect their intent.

Mahoney further explains the concept of an "independent automated audit" in Florida, which paradoxically allows "information used to decode the ballot artwork" to be shared from the primary vote tabulation system to the audit system. This means the audit is essentially checking if the primary system can read its own code consistently, rather than verifying the code's accuracy against the voter's intent. This setup undermines the very purpose of an independent audit. The cultural acceptance of QR codes and barcodes, driven by their convenience in other sectors, has led to a lack of critical reflection on their suitability for sensitive electoral processes. Mahoney argues that if voters were presented with ballot "artwork" in a more obviously encoded form, such as Unicode emojis or invisible ink, the inherent inscrutability and lack of verifiability would have been immediately apparent and unacceptable.

The issue of BMDs was not without its early critics. In Georgia, for example, the statewide adoption of Direct Recording Electronic (DRE) systems, which often include BMD functionalities, was fiercely contested. Dr. Lee, the sole cybersecurity expert on the recommending committee, vehemently opposed their implementation due to cybersecurity and other concerns. Groups like the Coalition for Good Governance and some legislators also spoke out. Despite these warnings, the systems were widely accepted and deployed, creating the entrenched problems observed today. This historical context underscores the regulatory failures that allowed fundamentally flawed systems to become the standard, making the current efforts to secure and verify elections significantly more challenging.

Key Findings

▶ Watch: Fundamental flaws make BMDs underregulated, hard to change (4:15)

Professor Mahoney's talk unveils several critical findings that underscore the systemic failures in regulating Ballot Marking Devices (BMDs) and the profound impact on election integrity and public trust.

First, a core finding is that the flaws in BMDs are fundamental and should have been obvious from their inception. Mahoney illustrates this with vivid analogies:

• "Ballot Artwork" and Inscrutable Codes: She points out that current BMDs often rely on QR codes or barcodes which are essentially "encoded artwork" with an accompanying human-readable list. Voters cannot decode these codes, making them functionally similar to arbitrary emojis or even invisible ink. This design choice prevents independent verification by the voter, creating a "more impenetrable system" than even a grocery store scanner, where vigilant shoppers can at least see prices on a screen and a receipt.
• The Kroger's Analogy: Mahoney cites instances where grocery stores like Kroger's were fined for scanning items at higher prices than listed. Shoppers can detect these errors by watching the screen or checking receipts. In contrast, with BMDs, "the voter doesn't see anything as you get rung up as it were, when the vote is tabulated," offering no immediate feedback or receipt of the scanned code.

Second, Mahoney highlights the unreliability of human review of printed lists. Studies have consistently shown that voters often do not adequately review the printed text lists on their ballots, even when prompted.

• Sevierville, Tennessee Study: Of 87 voters, only 46 reviewed their ballot summaries, with an average review time of just 3.9 seconds for ballots containing 18 contests. The minimum time was 1 second, suggesting a mere glance.
• Ann Arbor, Michigan Study: In a sample election, when shown a ballot with an altered item on the list, over 10% of the 44 voters who saw the errors blamed themselves, rather than the system. This self-blame makes it less likely for errors to be reported effectively and investigated.
• Functional Limitations: Mahoney presents a sample Palm Beach County ballot list with 23-25 choices in a format different from how the voter encountered the ballot. Such long lists, especially for "down-ballot" races like judicial contests (which can be high-stakes, as seen in a Wisconsin Supreme Court case exceeding $100 million), are not functionally reviewable by voters.

Third, the introduction of multiple images or records by BMDs—the encoded vote, the printed list, and the scanner's interpretation—multiplies opportunities for conflict and distrust. Mahoney cites examples of discrepancies:

• DeKalb County, Georgia: A winning candidate received no votes in many precincts, attributed to scanner programming errors. Hand counts from printed lists were used, implying the list was deemed correct.
• Northampton County, Pennsylvania (2019): No votes recorded for a judicial candidate, attributed to human error in ballot formatting. The paper list was used for counting.
• Later Error in Northampton County: Both the machine and the paper list were incorrect, demonstrating that the human-readable list is not inherently infallible.

Mahoney concludes that "any of the three could be wrong," making it impossible to designate any single component (code, list, or scanner interpretation) as the authoritative "ballot of record."

Finally, a crucial finding is that human readability, while necessary, is not sufficient for trustworthy election systems. Mahoney observes a trend towards proposals and recently certified systems under Voluntary Guidelines 2.0 that eliminate codes but retain long, human-readable printed lists. This direction is also supported by recent executive orders. However, she argues that these lists are often not truly functional for voters and may still contain undetected errors. The core problem, as articulated by Alex Halderman, remains: "Anytime that you're going to put a potentially vulnerable computer between the voter and the only records of their vote, you can have problems." Mahoney's evolving research indicates that moving towards "bright-line rules" is essential to avoid reinterpretation and ensure clarity, rather than relying on general "standards" that can be undermined.

Technical Deep Dive

▶ Watch: Legal and judicial obstacles to changing adopted systems (5:50)

Professor Mahoney's technical deep dive, while framed from a legal perspective, critically dissects the underlying mechanisms of Ballot Marking Devices (BMDs) and their inherent vulnerabilities. The central technical concern revolves around the machine-generated, machine-readable components of the ballot, specifically QR codes and barcodes, and their relationship to the human-readable text.

At the core of many BMD systems is the concept of "ballot artwork." Mahoney explains that this refers to the encoded information on the ballot that the machine interprets to count votes. In systems like the Dominion system in Georgia, this artwork often takes the form of barcodes or QR codes. These codes are generated by the BMD based on the voter's selections made on a touchscreen interface. When the voter reviews their ballot, they are presented with a printed paper record that typically includes both the inscrutable code and a human-readable list of their selections.

The critical technical flaw, as Mahoney highlights, is that the actual vote count is derived from scanning and interpreting these codes, not directly from the human-readable text. This creates a fundamental problem of verifiability. As Mahoney states, an "independent automated audit" in Florida, for instance, allows "information used to decode the ballot artwork" to be shared with the audit system. This means the audit is essentially checking if the audit system can read the primary system's code in the same way, rather than independently verifying the code against the voter's intent or the human-readable text. This setup fundamentally compromises the independence and effectiveness of the audit.

Mahoney draws on expert opinions, including Alex Halderman's report on the Dominion system, to emphasize that these codes are "encoded in something inscrutable" and are not typically encrypted. This means their content is opaque to the voter, who must trust that the machine accurately translated their selections into the code. Her "entertaining illustrations" — using Unicode emojis or invisible ink as hypothetical ballot artwork — are designed to make this technical opaqueness palpable. If a ballot were encoded with random emojis, even with a corresponding list, no voter would accept it as verifiable. The only difference with barcodes and QR codes, she argues, is a "cultural artifact" of their ubiquity, masking their functional inscrutability in an election context.

The problem is compounded by the fact that BMDs introduce multiple "images" or representations of the vote:

1. The machine-readable code (barcode/QR code): This is the primary source for tabulation in many systems.
2. The human-readable list: Printed alongside the code, intended for voter review.
3. The scanner's interpretation: The process by which the precinct scanner reads the code and converts it into a digital vote count.

Mahoney points out that any of these three components can be incorrect. She provides examples from real elections:

• In DeKalb County, Georgia, programming errors for the scanner led to a winning candidate receiving zero votes in many precincts. The hand count was then made from printed lists, implicitly trusting the list over the scanner's initial output.
• In Northampton County, Pennsylvania, a 2019 error attributed to human formatting resulted in no votes for a judicial candidate, again rectified by counting from paper.
• A later error in the same county showed that both the machine's record and the paper list were incorrect, demonstrating that the "human-readable" list is not always a reliable fallback.

This multiplication of potential points of failure, coupled with the inscrutability of the primary counting mechanism (the code), means that "any of the three could be wrong" and there is no single, consistently authoritative "ballot of record" that is transparently verifiable by the voter. This is a direct challenge to the principle of software independence, which dictates that an error in software should not be able to cause an undetected error in the election outcome. With BMDs, an error in the code generation or scanner interpretation can easily go undetected by a human voter reviewing a potentially correct (but not necessarily authoritative) printed list.

Mahoney also touches upon the reinterpretation of terminology, such as in Florida where BMDs are treated as producing Mark Sense ballots or optical scan ballots. While technically the output is slid through an optical scanner, this conflation obscures the fact that a computer was placed "between the voter and the only records of their vote," a critical distinction for security and verifiability. This technical reclassification allows BMDs to fit into existing regulatory frameworks without addressing their fundamental design flaws.

Demo / Proof of Concept

▶ Watch: Uncritical acceptance of ubiquitous QR/barcodes in voting (7:55)

While Professor Mahoney's talk did not feature a live technical demonstration or a traditional proof of concept in the form of exploiting a vulnerability, she effectively utilized conceptual "illustrations" to highlight the core technical and verifiability issues of Ballot Marking Devices (BMDs). These thought experiments served as powerful demonstrations of the inherent inscrutability of machine-readable codes on ballots.

Mahoney presented three such illustrations to make the "obvious" flaws of BMDs transparent:

1. The Unicode Emoji Ballot: She imagined a ballot where the vote choices were encoded not in QR codes or barcodes, but in colorful, whimsical Unicode emojis. These emojis would be "inscrutable and more colorful" than standard barcodes, yet functionally similar in that they represent encoded data that the voter cannot easily decipher. Alongside this "artwork," there would be a simple substitution code list. Mahoney's point was that if presented with such a ballot, no voter would accept it as verifiable, even with a list of candidates printed nearby. This illustrates that the widespread acceptance of QR codes and barcodes is a "cultural artifact" rather than a critical assessment of their suitability for elections, as they are equally opaque to the human eye.

1. The Kroger's Scanner Analogy: This illustration drew a parallel between the barcodes on BMD ballots and product barcodes in a grocery store. Mahoney recounted how Kroger's and other companies have been fined for discrepancies between listed prices and scanned prices, as reported by Consumer Reports. Vigilant shoppers can detect these errors by watching the screen or reviewing their receipts. The "proof of concept" here is the daily experience of consumers revealing the potential for discrepancy between encoded information and its human-readable representation. Mahoney then contrasted this with BMDs, where "the voter doesn't see anything as you get rung up as it were, when the vote is tabulated," and crucially, receives no receipt or visible evidence of the scanned vote. This highlights the BMD's "more impenetrable system" compared to everyday transactions where verification mechanisms exist.

1. The Invisible Ink Ballot: Mahoney's favorite analogy posited a "great new high-tech ballot" using invisible ink for the vote, with a visible list printed alongside it to show "what's written in invisible ink." This invisible ink would be scanned by a machine that "will light up the code and it will save a picture of that." She argued that no one would accept such a system, as it would be "more transparent to the voter that this was ballot artwork and a list." The "proof" is intuitive: the concept of invisible ink immediately conveys a lack of transparency and direct verifiability. This powerful illustration effectively demonstrates that the relationship between the machine-readable code and the human-readable list on current BMDs is fundamentally flawed, as voters are asked to trust an opaque process.

These conceptual "demonstrations" served to demystify the technical components of BMDs, exposing their functional inscrutability to the public. By using relatable, albeit absurd, scenarios, Mahoney effectively conveyed the core message that voters are being asked to implicitly trust a system they cannot independently verify, leading directly to a "recipe for distrust."

Defensive Implications

▶ Watch: Absurd hypothetical illustrating ballot artwork problems (8:50)

Professor Mahoney's analysis of Ballot Marking Devices (BMDs) carries significant defensive implications for election officials, legislators, and cybersecurity professionals striving to secure democratic processes. Her insights call for a fundamental re-evaluation of current practices and a shift towards more robust, transparent, and verifiable election systems.

Firstly, the most critical defensive implication is the need to move beyond merely "human-readable" lists as a sufficient standard for election security. Mahoney emphatically states, "human readability is necessary but not sufficient." While the trend towards eliminating QR codes and barcodes in favor of printed text lists is a step in the right direction (as seen in proposals under Voluntary Guidelines 2.0 and recent executive orders), these lists are often functionally inadequate for voter review due to their length and format. Furthermore, as demonstrated by the Northampton County example, even printed lists can contain errors. Defenders must recognize that placing "a potentially vulnerable computer between the voter and the only records of their vote" inherently introduces risks that cannot be fully mitigated by a simple printed list. The core defense must be to ensure the voter's intent is directly captured and verifiable without relying on an opaque machine translation.

Secondly, Mahoney advocates for a shift from broad legal "standards" to more specific "bright-line rules" in election law. General standards are susceptible to reinterpretation, as seen in Florida's classification of BMD outputs as Mark Sense ballots to fit existing rules, effectively obscuring the critical role of the computer in the voting process. Bright-line rules would offer clearer, unambiguous mandates regarding system design, auditability, and verifiability, making it harder for flawed systems to gain certification or for problematic interpretations to persist. This requires careful legislative drafting, moving away from vague requirements towards concrete technical and procedural specifications.

Thirdly, the talk underscores the paramount importance of truly independent and effective Risk-Limiting Audits (RLAs). Mahoney references Philip Stark's work, noting that many states claim to conduct RLAs but are engaging in "security theater." A genuine RLA requires a trustworthy record to base the audit on. If the "ballot artwork" (codes) is the primary record, and the human-readable list is unreliable or not the authoritative source, then the foundation for a meaningful RLA is absent. Defenders must push for RLAs that can verify the voter's intent against a truly independent, human-verifiable record, rather than merely checking the consistency of machine-generated codes. This also implies that the "ballot of record" must be clearly defined and accessible for audit.

Fourthly, the issue of time for effective audits is a critical defensive consideration. Mahoney highlights Florida's extremely short two-week deadline for election certification, driven by constitutional requirements for legislative seating. Such compressed timelines create immense pressure and hinder thorough auditing processes, potentially forcing states to make decisions "in fear that the certification deadline would be expanded." Defenders must advocate for reasonable audit timelines, recognizing that robust verification and potential recounts require adequate time. States like Texas, California, and New York, with much longer deadlines, provide models for how this can be accommodated without compromising constitutional mandates.

Finally, the talk implicitly calls for a deeper understanding of software independence, contestability, and defensibility in election systems. Defenders need systems where errors, whether from software, hardware, or human input, can be detected, demonstrated, and corrected. This requires mechanisms for voters to effectively report discrepancies (without self-blame), and for those reports to be verifiably investigated. The current lack of verifiable truth for reported discrepancies ("you can't prove either way which one it was") is a severe defensive weakness, as it allows distrust to fester and prevents effective problem resolution. Building systems with inherent contestability and defensibility should be a core defensive goal.

Key Takeaways

• BMDs are a "Recipe for Distrust": Current Ballot Marking Devices (BMDs), particularly those using inscrutable codes, fundamentally erode public trust in election outcomes due to their lack of transparency and verifiability.
• Codes are Inscrutable "Ballot Artwork": QR codes and barcodes on ballots are opaque to voters, making it impossible for them to verify that their selections are accurately recorded. This is functionally equivalent to using emojis or invisible ink, which would be universally rejected.
• Human Review of Lists is Ineffective: Studies show voters rarely review printed ballot lists thoroughly, and even when errors are present, voters often blame themselves, hindering detection and reporting. Long, complex ballots exacerbate this problem.
• Multiple Records Create Discrepancies: BMDs generate multiple forms of a vote record (code, printed list, scanner interpretation), and any of these can be incorrect, making it impossible to establish an authoritative "ballot of record" and multiplying opportunities for conflict and distrust.
• Human Readability is Insufficient: While moving away from codes to human-readable lists is a positive step, it is "necessary but not sufficient." These lists may still contain errors, go undetected, and still place a vulnerable computer between the voter and their vote.
• Need for Bright-Line Rules and Time: Regulatory frameworks need to shift from general "standards" to specific "bright-line rules" to prevent reinterpretation and ensure clarity. Additionally, adequate time for robust Risk-Limiting Audits (RLAs) is crucial, as current short deadlines in some states undermine thorough verification.

About the Speaker(s)

Marnie Mahoney is a Professor of Law at the University of Miami, where she also holds the distinguished title of Dean's Scholar. Her work focuses on critical legal issues, with a particular emphasis on areas where law intersects with technology and societal trust, as demonstrated by her deep dive into the regulatory failures surrounding Ballot Marking Devices. Professor Mahoney's expertise allows her to analyze the complex interplay between technical system design, legal frameworks, and their collective impact on public confidence in democratic processes.