1. Voting System Planning and Testing R. Michael Alvarez Caltech/MIT VTP Voting Systems Testing Summit 2005

2. Alvarez, Voting System Testing Summit 2005 Voting systems The entire process, end-to-end (registration, balloting, tabulation..) –Yes, precinct-voting devices are critical, but so are VR, absentee/early voting, tabulation –Planning and testing should involve the entire process, especially potential points of failure One important concern now are statewide voter registration systems.

3. Alvarez, Voting System Testing Summit 2005 Security Planning End-to-end security review, threat assessment Protect physical facility, use procedural controls (chain of custody) Software and hardware protection Many eyes involved in each step Openness and transparency

4. Alvarez, Voting System Testing Summit 2005 Example: L&A, Travis County

5. Alvarez, Voting System Testing Summit 2005 Contingency Planning Assume bad things will happen! Moynihan (2005), “Leveraging Collaborative Networks in Infrequent Emergency Situations”: 1. Pre-plan, but expect to plan more once emergency occurs 2. Identify necessary resources and match them with organizational competence 3. Create trust where you can, find alternatives when you can’t 4. Take advantage of technology to improve coordination and efficacy 5. Establish, formalize and communicate basic procedures

6. Alvarez, Voting System Testing Summit 2005 Testing In the US, we generally procure first, experiment later. Testing often only around procurement. We need to develop protocols for testing practices: –Pilot test voting systems, starting small, building to real experiments; using experimental methods, with treatment and control groups (and use real people!) –Field test usability, ballot designs, accessibility –Auditing, monitoring, data analysis from field –Security testing, including “tiger team” and parallel monitoring programs –Think of testing as a continuing and on-going process (before, during and after procurement) –Disseminate testing results, feedback into certification

7. Alvarez, Voting System Testing Summit 2005 Example: Buenos Aires

8. Alvarez, Voting System Testing Summit 2005 Conclusions and Questions Need to develop security plans Need to develop contingency plans Need testing protocols for voting systems –Standards or guidelines for plans and testing protocols? –Best practices? –Who should help develop plans and protocols (EAC, NIST, NAS, State/local officials)? –Resources?