1. 12/9-10/2009 TGDC Meeting Auditing concepts David Flater National Institute of Standards and Technology http://vote.nist.gov

2. 12/9-10/2009 TGDC Meeting Auditing vs. auditing Auditability from the Software Independence perspective Establish high confidence in the authenticity and completeness of vote records Verifiability of each and every vote Seeking alternatives that preserve these properties without VVPAT Vs. general external auditing concepts Statistical sampling Material error Audit risk Now discussing the latter Page 2 12/9-10/2009 TGDC Meeting

3. Goals of election audits As identified by participants in an ASA-hosted working meeting, 2009-10: "To verify that the election outcomes implied by the reported vote totals are correct" "To provide data for process improvement: specifically, to identify and quantify various causes of discrepancies between voter intentions and the originally reported vote totals" Note careful wording An audit will not necessarily demonstrate that the reported vote totals are correct It may suffice to estimate the uncertainty of the "measurement" No impact on outcome, no "material effect" Page 3 12/9-10/2009 TGDC Meeting

4. Auditability made simple Keep good records Complete (everything you will need) Identifiable Readable (usable for audits) Authenticated Archival Secure Conformant with U.S. generally accepted principles & standards Don't throw away information Redundant information isn't (auditors check for consistency) Cannot predict what might turn out to be critically relevant Better to have and not need than to need and not have Unfortunate conflicts with privacy, secrecy, election laws—this is what makes it complicated Page 4 12/9-10/2009 TGDC Meeting

5. Auditability and Software Independence Discrepancies may be attributable to software, firmware, hardware, people, processes, configuration, calibration, documentation, … A system can be software-independent without being auditable E.g., software-free system that reports totals but does not create records for cross-checking A system can be auditable without being software-independent By definition, not software-independent → possible to have undetectable change in election outcome Possible does not mean probable, or practically doable Concept of audit risk Internal controls Page 5 12/9-10/2009 TGDC Meeting

6. Auditing and election administration Management is responsible for: Presenting results fairly Adopting sound policies Establishing and maintaining internal control Preventing and detecting fraud An auditor's ability to compensate for inadequate internal control is limited Page 6 12/9-10/2009 TGDC Meeting