1. © Copyright 2009 TEM Consulting, LP - All Rights Reserved Presentation To Travis County, TX - May 27, 2009Rev 1 – 05/22/09 - HSB US Voting System Conformity Assessment System Prepared for Travis County Texas May 27, 2009

2. © Copyright 2009 TEM Consulting, LP - All Rights Reserved Presentation To Travis County, TX - May 27, 2009Rev 1 – 05/22/09 - HSB Conformity Assessment Conformity assessment is a system designed to assure that products and services meet requirements. Certification is a common component in a conformity assessment system. Certification of a product is a means of providing assurance that it complies with specified standards and other normative documents.

3. © Copyright 2009 TEM Consulting, LP - All Rights Reserved Presentation To Travis County, TX - May 27, 2009Rev 1 – 05/22/09 - HSB System Certification Several types of certification systems exist: Some comprise type testing only Other systems include initial testing and field surveillance Still others include initial testing of a product and assessment of its suppliers’ quality systems, followed by routine audits that take into account the factory quality system and the testing of samples from the factory and the open market.

4. © Copyright 2009 TEM Consulting, LP - All Rights Reserved Presentation To Travis County, TX - May 27, 2009Rev 1 – 05/22/09 - HSB Voting System Certification In the US, Voting System Certification includes: National testing and certification by the EAC State testing and certification Local acceptance testing (recommended) Assessment of its suppliers’ quality systems field surveillance by both EAC and state officials

5. © Copyright 2009 TEM Consulting, LP - All Rights Reserved Presentation To Travis County, TX - May 27, 2009Rev 1 – 05/22/09 - HSB International Conformity Assessment Conformity assessment is an international concern to assure the safety of food, pharmaceuticals and other critical items. It is also a necessity to support international trade. To meet these needs a number of treaties and agreements have been created. These agreements are implemented by processes, created and supervised by organizations such as: International Standards Organization (ISO) International Laboratory Assessment Cooperation (ILAC)

6. © Copyright 2009 TEM Consulting, LP - All Rights Reserved Presentation To Travis County, TX - May 27, 2009Rev 1 – 05/22/09 - HSB Purpose of Presentation This presentation discusses the elements common to conformity assessment systems and how these are used in the US Election System for certification and decertification of voting equipment.

7. © Copyright 2009 TEM Consulting, LP - All Rights Reserved Presentation To Travis County, TX - May 27, 2009Rev 1 – 05/22/09 - HSB Components of a Certification System Initial Type Testing Assessment of the Supplier’s Quality System Field Information & Feedback User Involvement

8. © Copyright 2009 TEM Consulting, LP - All Rights Reserved Presentation To Travis County, TX - May 27, 2009Rev 1 – 05/22/09 - HSB Key Participants EAC Voting System Test Laboratories Texas Secretary of State, Elections Division Travis County Office of Elections Vendors Voters

9. © Copyright 2009 TEM Consulting, LP - All Rights Reserved Presentation To Travis County, TX - May 27, 2009Rev 1 – 05/22/09 - HSB National ProgramState & Local Programs The primary concern of the EAC program is that the system design, as represented in the system submitted for testing, meets the requirements The primary concern of the state and local election official is that: Meets the specific needs of a jurisdiction the units delivered meet and continue to meet the requirements over their useful life.

10. © Copyright 2009 TEM Consulting, LP - All Rights Reserved Presentation To Travis County, TX - May 27, 2009Rev 1 – 05/22/09 - HSB Key Issues for Certification System What is a minimal acceptable system? Are the testing lab/testers/lab assessors qualified? Will the vendor deliver units within manufacturing tolerances to those tested? How will the election officials know if non-compliant units are delivered and what corrective actions can it take? Will election officials and poll workers use the systems as intended?

11. © Copyright 2009 TEM Consulting, LP - All Rights Reserved Presentation To Travis County, TX - May 27, 2009Rev 1 – 05/22/09 - HSB

12. © Copyright 2009 TEM Consulting, LP - All Rights Reserved Presentation To Travis County, TX - May 27, 2009Rev 1 – 05/22/09 - HSB Processes Processes are necessary to implement the components of a conformity assessment system for voting systems and to provide reasonable assurance that the key concerns of such a system are carefully addressed.

13. © Copyright 2009 TEM Consulting, LP - All Rights Reserved Presentation To Travis County, TX - May 27, 2009Rev 1 – 05/22/09 - HSB Processes Technical Standard Establish normative standard 2002 VSS & EAC VVSG Refine and Keep standard current Laboratory Accreditation NIST NVLAP Accredit to ISO 17025 to assure: Technical competence Management and quality process control EAC Accredit as Voting System Test Laboratory

14. © Copyright 2009 TEM Consulting, LP - All Rights Reserved Presentation To Travis County, TX - May 27, 2009Rev 1 – 05/22/09 - HSB Processes Technical Reviewers Review and give recommendations to EAC TR’s will have specialized qualifications for various topics EAC will perform routine performance evaluations Product Evaluation Test plan Testing and test report Witness testing will be part of the process Test report review and system certification

15. © Copyright 2009 TEM Consulting, LP - All Rights Reserved Presentation To Travis County, TX - May 27, 2009Rev 1 – 05/22/09 - HSB Processes Vendor Registration Review the vendor configuration control system Review the vendor quality control system

16. © Copyright 2009 TEM Consulting, LP - All Rights Reserved Presentation To Travis County, TX - May 27, 2009Rev 1 – 05/22/09 - HSB EAC / NIST TGDC Technical Guidelines EAC / NIST TGDC Technical Guidelines Candidate System Accredited Test Laboratory Accredited Test Laboratory EAC Technical Review Team Technical Review Team Certified System System Description System Description NIST Software Ref. Lib. NIST Software Ref. Lib. Vendor Quality & Configuration Control Processes Vendor Quality & Configuration Control Processes State & Local Evaluation Field Deployment and Use Receiving Inspection

17. © Copyright 2009 TEM Consulting, LP - All Rights Reserved Presentation To Travis County, TX - May 27, 2009Rev 1 – 05/22/09 - HSB EAC / NIST TGDC Technical Guidelines EAC / NIST TGDC Technical Guidelines Accredited Test Laboratory Accredited Test Laboratory EAC Technical Review Team Technical Review Team NIST NVLAP & Software Ref. Lib. NIST NVLAP & Software Ref. Lib. Vendors State & Local Election Officials Citizens Local Receiving Inspectors Local Receiving Inspectors

18. © Copyright 2009 TEM Consulting, LP - All Rights Reserved Presentation To Travis County, TX - May 27, 2009Rev 1 – 05/22/09 - HSB

19. © Copyright 2009 TEM Consulting, LP - All Rights Reserved Presentation To Travis County, TX - May 27, 2009Rev 1 – 05/22/09 - HSB Conclusion A well constructed certification system provides satisfactory answers to central issues: What is a minimal acceptable system? Are the testing lab/testers/lab assessors qualified? Will the vendor deliver units within manufacturing tolerances to those tested? Will the election officials know if non-compliant units are delivered and what actions can they take? Will election officials and poll workers use the systems as intended?