Resources Needed for Post-Election Audits: People, Time, and Money Robert Kibrick, Legislative Analyst Verified Voting Foundation Gail Pellerin, County Clerk Santa Cruz County, California Jaime Young Santa Cruz County Elections Dept Post Election Audit Summit Minneapolis, Minnesota October 26, 2007

Overview of Talk ● Key factors affecting costs of election audits ● California's 1% manual audit requirement ● Audit data from Santa Cruz County, CA – 1% manual audit of selected precincts – 10% manual audit of DRE VVPATs ● Why VVPAT rolls are difficult to audit ● Difficulties in predicting audit costs ● The need to research costs of manual audits

Key factors affecting cost ● Scope of audit (fixed %, tiered %, other) ● Type of voting system ● Method of manual counting ● Size & composition of manual audit board ● Number of ballots/hour board can process ● Hourly labor costs for counters, supervisors ● Costs of training, equipment, facilities

California's 1% Manual Audit Requirement – Enacted 1965 ● Purpose: check if machines are counting accurately ● Each county must manually audit: – All contests in 1% of precincts – Additional precincts to capture all contests – Other precincts at discretion of election officials ● Initially applied to ballots cast in polling places ● Ill-defined application to paperless DREs ● 2004: VVPATs required on all DREs ● 2006: VVPAT must be the record audited ● 2008: Will include 1% of absentee ballots

Nov. 7, 2006 General Election in Santa Cruz County, CA ● Area: 445 sq. miles ● Population: 249,705 ● Ballots cast: 92,236 – In-person: 50,189 -> – Absentee: 42,047 -> ● Voting Systems: – Sequoia PCOS – Sequoia AVC Edge (for accessibility) – Deployed 11/2006 ● 574 persons/sq. mile ● Reg. Voters: 142,428 ● Turnout:65% – 54.4% of ballots – 45.6% of ballots ● Mosts ballots cast are opscan paper ballots ● Absentee ballots are centrally-counted at County Elections Dept. on high- speed scanners

Parameters of 1% Manual Audit in Santa Cruz County (2006) ● Total voting precincts: 317 Total contests: 45 – 170 precincts assigned to 117 polling places ● Most had only one precinct assigned ● Some had as many as 5 precincts – 147 mail-ballot precincts (these have no polling place) ● Each of these precincts has less than 250 voters ● 4 precincts selected for 1% sample (all contests) ● 19 more precincts to capture left over contests ● Random selection: – Draw film canisters with precinct #s from bowl – Selection process took about 1 hour

Conducting the 1% Manual Audit in Santa Cruz County (2006) ● Audits took place at County Elections Dept. ● If discrepancies are found, they must be resolved ● Total effort required: – Staff of up to 15 working for 5 days – Up to 14 counters paid $15/hour – 1 or 2 supervisors paid $25 to $33/hour ● Different counting method used for – Optical scan paper ballots – DRE VVPATs ● Audits of VVPATs much more labor intensive

Method for 1% manual audit of optical scan paper ballots ● Precincts to audit selected at random ● First pass counts total # of ballots for precinct ● Compare to post-election ballot accounting form – Completed by poll workers when polls close – Accounts for all ballots (voted, spoiled, unused, etc.) – Reconciles ballots cast to voters signing poll book ● Separate pass through ballots for each contest ● Manual tally of the ballots compared to: – Summary tape printed by PCOS scanner – Initial central tabulation for precinct ● Two 4-person audit boards + floating supervisor

Method for 1% manual audit of DRE VVPAT printouts ● Precincts to audit selected at random ● Compare to post-election ballot accounting form ● All contests to be audited counted in single pass ● Manual tally compared to: – Summary tape printed by DRE – Initial central tabulation for precinct ● One 2-person audit board + floating supervisor ● VVPATs printed on a continuous roll of paper ● 2 paint rollers on stands used to wind VVPAT

Paint-roller apparatus used to wind through VVPAT rolls ● 2 paint rollers mounted to stands ● Adjust spacing to match VVPAT length

Parameters of 10% Manual Audit in Santa Cruz County (2006) ● Adopted by Board of Supervisors January 2006 ● First 10% audit of DREs in November 2006 ● 170 in-person voting precincts / 117 polling places ● 124 DREs deployed to those polling places ● 13 units selected for audit (10% of DREs) ● Randomly selected 13 precincts (14 DREs / 18 pcts) – Some VVPATs have votes from multiple precincts – Included “early votes” cast by voters from those pcts. – Drew film canisters with precinct #s from bowl ● 573 VVPATs audited; avg. of 32/precinct (41/DRE)

Conducting the 10% Manual Audit in Santa Cruz County ● Used same paint rollers as for 1% manual audit ● All contests audited in a single pass ● “PostIt” notes applied to VVPAT for checkpoints ● Two 4-person audit boards + floating supervisor ● Effort required: staff of 8+ working for 3 days ● Audits took place at County Elections Dept. ● 1 mangled VVPAT from a jammed printer ● 2 VVPATs with no selections recorded – 1 discovered during audit, 1 during a recount

Continuous-roll VVPAT printouts more difficult to manually audit ● Takes time to wind from one VVPAT to the next ● Winding-time motivates single pass counting ● Mix of precincts on same VVPAT roll for DREs at some multi-precinct polling places ● Need to identify and skip over: – Voided VVPATs – Provisional VVPATs – VVPATs from precincts not selected for audit

Some attempts to make VVPATs easier to audit manually ● Some jurisdictions cut VVPAT rolls into pieces – Eliminates winding-time penalty – Can set aside voided and provisional VVPATs ● BUT: – Cutting VVPAT rolls takes time and risks damage – Thin, lightweight paper pieces are hard to handle – Complicates long-term storage ● Audits easier if precincts don't share same DRE ● Use paper ballots, not DRE, for provisionals (but makes out-of-precinct ballots harder to tabulate)

Difficulties in predicting audit costs from existing audit data ● Existing cost data is aggregated and anecdotal ● Lack fine-grained data for costs of each step ● Audit costs vary with type of voting system, e.g., DRE+VVPAT, opscan, Ink-A-Vote, digital paper, etc. ● Number of ballots to be audited is uncertain: – Depends on random selection of precincts – Huge variation in ballots cast per precinct (0 to 553) ● Costs may not scale linearly as sample increases ● Training costs if temporary staff needed ● Speed and accuracy decline as fatigue sets in

Research needed to measure costs of audit components ● Ability to predict audit costs urgently needed by – Legislators writing election audit legislation – Elections officials developing audit procedures – Jurisdictions choosing new voting technologies ● Elections officials are best suited to collect the data ● Need to reach consensus on proper metrics ● Standard reporting format needs to be defined ● Some other entity needed to collate / analyze data ● EAC, states, or foundations should sponsor research