1. VoTeR CenterUniversity of Connecticut Pre-Election Testing and Post-Election Audit of Optical Scan Voting Terminal Memory Cards Voting Technology Research (VoTeR) Center Department of Computer Science and Engineering University of Connecticut http://voter.engr.uconn.edu Seda Davtyan, Sotiris Kentros, Aggelos Kiayias, Laurent Michel, Nicolas Nicolaou, Alexander Russell, Narasimha Shashidhar, Andrew See, Alexander A. Shvartsman Work funded by the Connecticut Secretary of the State Office Voting Technology Research (VoTeR) Center Department of Computer Science and Engineering University of Connecticut http://voter.engr.uconn.edu Seda Davtyan, Sotiris Kentros, Aggelos Kiayias, Laurent Michel, Nicolas Nicolaou, Alexander Russell, Narasimha Shashidhar, Andrew See, Alexander A. Shvartsman Work funded by the Connecticut Secretary of the State Office

2. VoTeR CenterUniversity of Connecticut Outline Motivation Introduction Goals of the Memory Card Audit AccuVote OS AV-OS Software Components Auditing Process Results and Observations Conclusion 2

3. VoTeR CenterUniversity of Connecticut Motivation In a recent primary in an unnamed state there was a mix of hand-counted and machine-counted precincts It was observed that in hand-counted precinct Candidate A was favored by the voters, while in optical-scan tabulated precincts Candidate B was favored There were sensible demographic reasons for this Nevertheless, a valid question was asked: Were the voting machines programmed correctly? The state officials did not have an answer 3

4. VoTeR CenterUniversity of Connecticut Motivation The machine in question is Premiers Accu-Vote Optical Scan tabulator Provides inherent VVPB/VVPAT Not the bleeding edge machine – relatively few attack vectors But: [Hursti05] Memory cards are easy to tamper with if removed from the tabulator [EVT07] Memory cards are easy to tamper with if sealed in the tabulator Reports by other workers and CA, CT, FL, AL,… Tests/audits of equipment/technology are necessary 4

5. VoTeR CenterUniversity of Connecticut AccuVote OS (AV-OS) AV-OS Firmware version 1.96.6 Memory cards programmed on GEMS 5

6. VoTeR CenterUniversity of Connecticut Process in Connecticut Ballot information for a district Memory cards programmed using GEMS (at LHS Associates) Cards inserted and tested at the district Cards used in the election at the district Cards shipped

7. VoTeR CenterUniversity of Connecticut Goals of the Memory Card Audit Pre-election Memory Card Audit Perform an integrity check of the contents of the memory cards Post-election Memory Card Audit Integrity check of contents State of cards consistent with election use 7

8. VoTeR CenterUniversity of Connecticut Outline Motivation Introduction Goals of the Memory Card Audit AccuVote OS AV-OS Software Components Auditing Process Results and Observations Conclusion 8

9. VoTeR CenterUniversity of Connecticut AV-OS Software Components The behavior of AV-OS is determined by two components: AV-OS Firmware Data and program on Memory Card Memory Card includes: Status Information Audit Log Ballot Description Counters Bytecode 9

10. VoTeR CenterUniversity of Connecticut Outline Motivation Introduction Goals of the Memory Card Audit AccuVote OS AV-OS Software Components Auditing Process Results and Observations Conclusion 10

11. VoTeR CenterUniversity of Connecticut Auditing Process Preparation for audit Analysis of the AV-OS firmware, development of custom firmware, a data collection and comparison tool, and analysis of the bytecode The auditing process Data collection from memory cards Analysis of the data 11

12. VoTeR CenterUniversity of Connecticut Contractual Issues Contract between Premier and State of CT Prohibits reverse engineering, de- compilation, re-assembly, etc. One exception: Contract permits modification/alteration of software/firmware to display data related to election results We used this exception to perform engineering to understand the format of memory cards and to extract this data using special purpose firmware we designed 12

13. VoTeR CenterUniversity of Connecticut Custom Firmware Custom firmware was developed to resolve major issues in using the built-in dumping procedure of AV-OS: Relying on the undocumented built-in procedure is questionable Avoid altering card contents (audit log) Ensure faithful reading of contents Speeding up memory card dumping 13

14. VoTeR CenterUniversity of Connecticut Custom Firmware Development Four main point were considered during the production of new firmware: Memory Card Access Serial Port Access Delivery of the Memory Card data Avoid any logging on the memory card (Technical details in the full paper) 14

15. VoTeR CenterUniversity of Connecticut Format of the Memory Card Epson 128K card Our analysis revealed the following formatting of the memory cards 15

16. VoTeR CenterUniversity of Connecticut Data Collection Tool The Data Collection/Comparison tool serves two purposes: Collecting the memory card dump sent using run length encoding Auditing the collected data by comparing baseline and audit data and analyzing the differences 16

17. VoTeR CenterUniversity of Connecticut Testing Methodology Testing for potential data inconsistencies and integrity problems of the memory cards requires collection of three types of data: Baseline Data Pre-Election Data Post-Election Data 17

18. VoTeR CenterUniversity of Connecticut State of the Memory Card Memory card examination focused on: Card Format (data and byte code) Card Status (set for election, etc.) Counter Status (zero / non-zero) Election Count (usage) Audit Log 18

19. VoTeR CenterUniversity of Connecticut State Diagram State transitions for a memory card 19

20. VoTeR CenterUniversity of Connecticut Outline Motivation Introduction Goals of the Memory Card Audit AccuVote OS AV-OS Software Components Auditing Process Results and Observations Conclusion 20

21. VoTeR CenterUniversity of Connecticut Results and Observations Pre-election audit performed on 522 memory cards Covers 75% of all districts 378 out of 522 memory cards were received prior to the election, the rest later Post-election audit was performed on 100 cards Partial audit en route to future broader audits 36 out of 100 memory cards were used during the election Represents > 5% of the cards used in election 21

22. VoTeR CenterUniversity of Connecticut Pre-Election Sampling Issues A few differences between the procedures followed by the poll workers and the procedures defined by SOTS were noticed: The cards were not chosen uniformly at random for the audit Instead of choosing random memory cards for each district random districts were chosen Some cards were labeled backup 22

23. VoTeR CenterUniversity of Connecticut Pre-Election Memory Card Audit Results 23

24. VoTeR CenterUniversity of Connecticut Post-Election Memory Card Audit Results 24

25. VoTeR CenterUniversity of Connecticut Conclusions The following were identified during the memory card audit Examination of memory cards revealed no incorrect ballot data or bytecode Poll workers did not follow the exact testing procedures Surprising number of cards with junk data: 3.5% in pre-election audit and 8% in post- election audit 25

26. VoTeR CenterUniversity of Connecticut References Black Box Voting http://blackboxvoting.org Jonathan Bannet, David W. Price, Algis Rudys, Justin Singer, Dan S. Wallach: Hack-a-Vote: Security Issues with Electronic Voting Systems. IEEE Security & Privacy 2(1): 32-37 (2004) Help America Vote Act (HAVA), http://www.fec. gov/hava/law_ext.txt Harri Hursti, Critical Security Issues with Diebold Optical Scan Design, Black Box Voting Project, July 4, 2005 http://www.blackboxvoting.org/BBVreport.pdfhttp://www.blackboxvoting.org/BBVreport.pdf A. Kiayias, L. Mchel, A. Russell, A.A. Shvartsman, M. Korman, A. See, N. Shashidhar and D. Walluck, Security Assessment of the Diebold Optical Scan Voting Terminal, http://voter.engr.uconn.edu/ voter/Report-OS.html A. Kiayias, L. Michel, A. Russell, N. Sashidar, A. See, and A. Shvartsman, An Authentication and Ballot Layout Attack Against an Optical Scan Voting Terminal. 2007 USENIX/ACCURATE Electronic Voting Technology Workshop (EVT 07), Augist, 2007, Boston, MA. A. Kiayias, L. Michel, A. Russel, N. Sashidar, A. See, A. Shvartsman, S. Davtyan. Tampering with Special Purpose Trusted Computing Devices: A Case Study in Optical Optical Scan E- Voting. Twenty-Third Annual Computer Security Applications Conference (ACSAC), December, 2007, Miami Beach, Fl. 26

27. VoTeR CenterUniversity of Connecticut About the UConn VoTeR Center Participation in Connecticut Voting Technology Standards Board 2005-2006 Relationship with the CT SOTS Office Advising on voting technology issues Evaluation of proposed voting equipment Development of safe use procedures Technology audits and security analysis Faculty: A. Shvartsman, A. Kiayias, L. Michel, A. Russell Research Assistants: S. Davtyan, S. Kentros, N. Nicolaou, N. Sashidhar, A. See 27