P1622 Audit Use Case Neal McBurnett IEEE P1622 meeting

Audit: Compare system's reported results with the evidence Take sample of detailed results by audit unit and compare to hand counts of those audit units

Report of Vote Counts by Audit Unit

Recent progress in Auditing Growing momentum behind good audits Law in Colorado: risk limiting audits by 2014 New law in CA to pilot risk-limiting audits NM: state-wide audit starting 2010 EAC audit grant funding opportunity ; notes risk-limiting small batch audits

Principles and Best Practices for Post-Election Audits (2008) League of Women Voters Election Audit Report (2009) American Statistical Association 2010‏ statement on Risk-Limiting Audits and small-batch audit reporting

Typically, amount of work required to perform a risk-limiting vote-tabulation audit is roughly proportional to the batch sizes, with no reduction in statistical efficacy.

State-wide coordination of audits Auditable tabulation reports from counties to state Audit unit selections from state back to counties Audit discrepancy reports back to state Potential for additional selections Final audit report aggregates discrepancies

Audit results also need to be published in a standard format Include hand counts and discrepancies for all units audited, whether they were targeted or randomly selected, information needed to validate the selections (e.g. algorithms used, random seeds used as input), results of the audit

Audit unit result Audit unit id For each candidate or “under” or “over”  System tally  Hand count (multiple?)‏  Discrepancies (Hand count - system)‏  Resolution code  Resolution text description

Proof of Concept: ElectionAudits Audit Management System Open source, MIT license Supports the Best Practices and risk-limiting audits Imports standard election report file, e.g. Hart, Sequoia; EML 510 support Automates many steps of the audit Enter the data, publish with statistics Roll 15 dice, publish all the selections

Biggest challenge: getting small-batch data out of election systems Want Interoperable systems, out-of-the-box Not just Integratable via consultants See David Flater's CDF presentation

Summary Proper audits are critical Risk-limiting audits in CO, CA law Small-batch reports help enormously Coordinated state-wide audits: lots of data flow Standard, machine-readable format is needed EML looks appropriate