1. CYLAB/ISR SEMINAR APRIL 16, 2007 COPYRIGHT © 2007 MICHAEL I. SHAMOS What Happened to 18,000 Votes? Results of the Sarasota Source Code Audit Michael I. Shamos, Ph.D., J.D. Institute for Software Research School of Computer Science Carnegie Mellon University

2. CYLAB/ISR SEMINAR APRIL 16, 2007 COPYRIGHT © 2007 MICHAEL I. SHAMOS Outline What happened in Sarasota County? –The problem –Political events Source code review –What was done –What was found –Vote flipping –Touchscreen delay Where did the votes go? Buchanan Jennings

3. CYLAB/ISR SEMINAR APRIL 16, 2007 COPYRIGHT © 2007 MICHAEL I. SHAMOS Florida U.S. House District 13 Includes all of Sarasota, De Soto and Hardee Counties Parts of Manatee and Charlotte Counties

4. CYLAB/ISR SEMINAR APRIL 16, 2007 COPYRIGHT © 2007 MICHAEL I. SHAMOS Voting Methods in District 13 Manatee, De Soto and Hardee Counties use Diebold opscan Sarasota and Charlotte Counties use ES&S iVotronic touchscreen machines (no VVPAT), version 8.0.1.2 Touchscreen Opscan

5. CYLAB/ISR SEMINAR APRIL 16, 2007 COPYRIGHT © 2007 MICHAEL I. SHAMOS Florida U.S. House District 13 Vern Buchanan (R) beat Christine Jennings (D) by 369 votes out of 238,249 cast, a 0.15% margin In Sarasota County, 18,412 ballots showed no vote at all in that race, an undervote of 15% Jennings beat Buchanan, 65,487-58,632 in Sarasota If the 18,412 undervotes followed that percentage (52.76%-47.24%), Jennings would win by 648 votes The other counties in District 13 had an average undervote of 2.5% (range: 2.1-4.0%)

6. CYLAB/ISR SEMINAR APRIL 16, 2007 COPYRIGHT © 2007 MICHAEL I. SHAMOS What Happened? Jennings has filed suit in Florida. Can she find out? The Florida Secretary of State ordered an audit. Can he find out? Congress is investigating. Can it find out? What sort of forensic investigation is needed?

7. CYLAB/ISR SEMINAR APRIL 16, 2007 COPYRIGHT © 2007 MICHAEL I. SHAMOS A New Election? Legal scholars believe Jennings must show there was a machine malfunction to win a new election Voter “confusion” is not enough Fla. Stat. §102.168(4) lists all grounds for a contest: –(c) Receipt of a number of illegal votes or rejection of a number of legal votes sufficient to change or place in doubt the result of the election. –(e) Any other cause or allegation which … would show that a person other than the successful candidate was … elected

8. CYLAB/ISR SEMINAR APRIL 16, 2007 COPYRIGHT © 2007 MICHAEL I. SHAMOS U.S. Const. Art I, Sec. 5 “Each House shall be the judge of the elections, returns and qualifications of its own members” Election matters are referred to the Committee on House Administration (9 members: 6 Dem, 3 Rep.) Federal Contested Elections Act, 2 U.S.C. §318ff.2 U.S.C. §318ff Chairwoman Millender-McDonald: “Florida law will facilitate the evaluation of the election contest – to the extent that it provides access to relevant and critical evidence … the House may not have to get involved at all if the state court does a thorough job.” Jennings is trying to show that the court is not doing a thorough job. April 13 memorandum.memorandum

9. CYLAB/ISR SEMINAR APRIL 16, 2007 COPYRIGHT © 2007 MICHAEL I. SHAMOS ES&S iVotronic Voting System Ballot (eligible candidates) loaded from infrared device (“personal electronic ballot” – PEB) Choices (votes) recorded in 4 places: 3 on the machine, 1 on removable memory device Totals printed at polling location AND sent to county on media for tabulation AND retained in machines 1498 machines in Sarasota Allegheny County uses a later version of iVotronic: 9. Touchscreen DRE

10. CYLAB/ISR SEMINAR APRIL 16, 2007 COPYRIGHT © 2007 MICHAEL I. SHAMOS Some Possible Explanations Software error –Voters cast votes, but no votes were recorded –Unlikely, because 85% of votes were counted –Post-election testing, source code review Tampering (malicious software) –Post-election testing –Source code review Conscious voter protest –Unlikely, because of comparison demographics –Absentee (opscan) undervote in Sarasota was 2.6% Bad ballot layout – voters missed the race –Compare with Charlotte County

11. The Sarasota Ballot UNDERVOTE 1.1%

12. The Sarasota Ballot UNDERVOTE 15% UNDERVOTE 1.3%

13. The Sarasota Ballot UNDERVOTE 4.4% UNDERVOTE 5.2%

14. Sarasota Summary Page

15. CYLAB/ISR SEMINAR APRIL 16, 2007 COPYRIGHT © 2007 MICHAEL I. SHAMOS Comparison with Charlotte County Sarasota and Charlotte used the same touchscreen system In Sarasota, House and Governor were on the same screen In Charlotte, House had its own screen, but Attorney General and Governor were on the same screen Sarasota had a 13% undervote for House, but 1.3% for Governor Charlotte had a 2.4% undervote in the U.S. House race, 26% undervote for attorney general (would not have made a difference statewide). 41% undervote for Florida House District 71

16. CYLAB/ISR SEMINAR APRIL 16, 2007 COPYRIGHT © 2007 MICHAEL I. SHAMOS Ballot Comparison SARASOTA CHARLOTTE 4.4% UNDERVOTE 26% UNDERVOTE 4.4% UNDERVOTE 5.2% UNDERVOTE 0.7% UNDERVOTE

17. CYLAB/ISR SEMINAR APRIL 16, 2007 COPYRIGHT © 2007 MICHAEL I. SHAMOS The Dent Memo (Nov. 3, 2006)

18. CYLAB/ISR SEMINAR APRIL 16, 2007 COPYRIGHT © 2007 MICHAEL I. SHAMOS Timeline Nov. 3Sarasota SOE letter to precincts warning of potential to overlook the race Nov. 7Election Day Nov. 8Hell breaks loose with 15% undervote Nov. 9FL Sec’y of State announces audit Nov. 13Canvassing commission orders recount Nov. 20Canvassing commission certifies election Nov. 20Jennings sues in FL to contest election Nov. 21Voters sue for new election Dec. 5Florida forms source code task force

19. Timeline Dec. 20Jennings contests election in Congress Dec. 26Florida judge rules against source code access by Jennings Jan. 4Buchanan seated by House of Representatives Jan. 4Jennings appeals denial of source code access to Fla. Court of Appeal Jan. 4Rep. Millender-McDonald urges Court of Appeal to expedite the case Jan. 10Court tells Millender-McDonald to butt out Feb. 14House of Representatives forms Subcommittee on Elections Feb. 23Source code task force report released

20. CYLAB/ISR SEMINAR APRIL 16, 2007 COPYRIGHT © 2007 MICHAEL I. SHAMOS Secretary of State Audit 1. Review of election, procedures, results, and certification examination 2. Testing machines actually used in election and machines held aside as spares 3. Independent Source Code Review

21. CYLAB/ISR SEMINAR APRIL 16, 2007 COPYRIGHT © 2007 MICHAEL I. SHAMOS Post-Election Testing Results The machines properly recorded votes The software was certified and unaltered The internal audit trail shows the undervotes No evidence of tampering or vote-dropping No evidence of vote-flipping

22. CYLAB/ISR SEMINAR APRIL 16, 2007 COPYRIGHT © 2007 MICHAEL I. SHAMOS Source Code Task Force Florida State University was prime contractor Alec Yasinsac, FSU –Director, Security and Assurance in Information Technology Lab Ted Baker, FSU –Device drivers, hardware/software interaction Matt Bishop, UC Davis –Author: Computer Security: Art & Science Mike Burmeister, FSU Co-Director, SAIT Breno de Medeiros, FSU Information security Michael Shamos, CMU Voting systems examiner Gary Tyson, FSU Architecture and compilers David Wagner, UC Berkeley Secure software, e-voting

23. CYLAB/ISR SEMINAR APRIL 16, 2007 COPYRIGHT © 2007 MICHAEL I. SHAMOS Ground Rules Total independence from Secretary of State All source code provided Access to actual voting machines Vendor furnished documentation and briefings No confidentiality restriction for discoveries relevant to the District 13 race or any system flaws

24. CYLAB/ISR SEMINAR APRIL 16, 2007 COPYRIGHT © 2007 MICHAEL I. SHAMOS Evidence Considered Source code Machine behavior Election statistics Ballot definition files Ballot images, electronic files Election event logs Court filings, county documents Poll-worker logs of voter complaints News stories, blogs Did not review: firmware of I/O devices, 3 rd -party utility libraries

25. PEB CF TF Compact Flash Processor RAM Video Card RAM Firmware Display Data Display Data Audio Ballots Ballot Images Summary Data Ballot Images Ballot Style Removable components are pink Dashed lines are memory mappings TF – Terminal Flash Memory, PIC = Programmable Interrupt Controller, PEB – Personal Electronic Ballot Controller Touch Screen EPROM Touches PIC Interrupt iVotronic Hardware Architecture SOURCE: TASK FORCE REPORT Intel 386 EX TRIPLY REDUNDANT

26. CYLAB/ISR SEMINAR APRIL 16, 2007 COPYRIGHT © 2007 MICHAEL I. SHAMOS iVotronic Software Architecture NO operating system Low-level and machine interface code –Mostly C, some assembly language – all was available Application code –All C COTS –Very little, e.g. C libraries, driver for compact flash card

27. CYLAB/ISR SEMINAR APRIL 16, 2007 COPYRIGHT © 2007 MICHAEL I. SHAMOS iVotronic Software Properties Good –No GOTOs –No dynamic memory allocation –No multithreading –Single address space –Not object-oriented, so no fragile base class problem –After each voter, processor is reset, program reloaded from EPROM and variables re-initialized Bad –No high-level design –Limited code readability –Aging code base, numerous updates –Global variables updated by main program and interrupt handlers

28. CYLAB/ISR SEMINAR APRIL 16, 2007 COPYRIGHT © 2007 MICHAEL I. SHAMOS Technical Approach Follow the evidence Consider all proposed hypotheses We traced program execution 1. Voting machine initialization 2. Voter selections & screen review 3. Ballot image creation 4. Ballot image storage 5. Asynchronous system faults not associated with a voting phase. Used Fortify Source Code Analysis (SCA) tool from Fortify Software

29. CYLAB/ISR SEMINAR APRIL 16, 2007 COPYRIGHT © 2007 MICHAEL I. SHAMOS Unanimous Findings Complete ballot was presented to each voter All selections presented on review screens All selections recorded to terminal flash memory All flash memory selections recorded to external media No queueing or stacking of interrupts No malware No time-sensitive code No serial race effect –Race A unaffected by race B for A≠B No serial voter effect –Voter n unaffected by voters 1, …, n-1

30. CYLAB/ISR SEMINAR APRIL 16, 2007 COPYRIGHT © 2007 MICHAEL I. SHAMOS Vote Flipping Some voters reported vote-flipping Voter presses the square next to a Democrat, but the square next to the Republican gets marked (Reported widely, especially Broward County, FL) This is not caused by malware, but by miscalibrated touchscreens How do we know? The problem goes away when the screens are recalibrated.

31. CYLAB/ISR SEMINAR APRIL 16, 2007 COPYRIGHT © 2007 MICHAEL I. SHAMOS Touchscreens 1.Sensor 2.Controller 3.Software driver

32. CYLAB/ISR SEMINAR APRIL 16, 2007 COPYRIGHT © 2007 MICHAEL I. SHAMOS Resistive Touchscreens 1. Polyester Film 2. Upper Resistive Circuit Layer 3. Conductive ITO (Indium-Tin Oxide, transparent metal coating) 4. Lower Resistive Circuit Layer 5. Insulating Dots 6. Glass/Acrylic Substrate 7. Touching the overlay surface causes the (2) Upper Resistive Circuit Layer to contact the (4) Lower Resistive Circuit Layer, producing a circuit switch from the activated area. 8. Touchscreen controller measures alternating voltages between the circuit layers (7) and converts them into the digital X and Y coordinates of the activated area.

33. Resistive Touchscreens Screen is fed clock signals Touching the screen creates voltage dividers in two dimensions Transient signals from the wires must be interpreted to determine (x, y) coordinates SOURCE: RICK DOWNS Smoothing of the signal is required This is done in software by a “smoothing filter”

34. CYLAB/ISR SEMINAR APRIL 16, 2007 COPYRIGHT © 2007 MICHAEL I. SHAMOS Calibrating Touchscreens SOURCE:WWW.EMBEDDED.COM A circle on the display and in touchscreen coordinates

35. CYLAB/ISR SEMINAR APRIL 16, 2007 COPYRIGHT © 2007 MICHAEL I. SHAMOS The Smoothing Filter The iVotronic smoothing filter was slow, sometimes 3 seconds until a touch was registered Florida’s primary election was on September 5, 2006 About August 21, 2006, the Sarasota Supervisor of Elections received a letter from the vendor advising of the slow response and suggesting either: –Installing a new version with a faster filter; or –Alerting the voters to the slow response Sarasota did neither for the primary or the November election

36. The ES&S Letter (condensed)

37. CYLAB/ISR SEMINAR APRIL 16, 2007 COPYRIGHT © 2007 MICHAEL I. SHAMOS The Smoothing Filter Hypothesis It is now alleged that the smoothing filter was the cause of the undervote Theory: Voters pressed “Jennings.” This did not register immediately, so they pressed it again. This had the effect of selecting and then deselecting Jennings. Plausible but incorrect: –Interrupts (touches) are not queued. Only the last touch takes effect. If a voter touches again before the first touch registers, the second one registers, does not cancel the first. –If the effect existed, it would have affected other races in Sarasota and other jurisdictions. –If the effect were widespread (15%), it would have been observed in testing, but was not.

38. CYLAB/ISR SEMINAR APRIL 16, 2007 COPYRIGHT © 2007 MICHAEL I. SHAMOS What Caused the Undervote? Bad ballot design COMBINED WITH ineffective undervote warning WHY DO WE BELIEVE THIS? No other hypothesis is confirmed by the facts WHAT IS THE FIX? Do not allow exit from an undervoted screen without warning and express confirmation EFFECT ON PENNSYLVANIA? Vendor will not receive any new certification until all vulnerabilities and the undervote warning are repaired

39. CYLAB/ISR SEMINAR APRIL 16, 2007 COPYRIGHT © 2007 MICHAEL I. SHAMOS Aftermath Go hence, to have more talk of these sad things; Some shall be pardon'd, and some punishèd; For never was a story of more note Than this of Jennings and her undervote.

40. CYLAB/ISR SEMINAR APRIL 16, 2007 COPYRIGHT © 2007 MICHAEL I. SHAMOS Q A &