12/9-10/2009 TGDC Meeting Usability and Accessibility Progress and Challenges Sharon Laskowski, PhD National Institute of Standards and Technology

12/9-10/2009 TGDC Meeting Progress has been substantial Requirements answered ~80% of critical usability and accessibility issues Most of VVSG 2.0 Chapter 3 was included in VVSG 1.1 VVSG 1.1 includes updated requirements for color and contrast Style guide for documentation usability completed, plain language benefits confirmed Design vs. Performance requirements We have made progress with test methods that measure usability performance User testing with pass/fail criteria (“benchmarks”) Universal usability continues to be an overarching goal Electronic machines need to be designed for the wide range of voters There are still a number of challenges that need more research (the 20%) Page 2 12/9-10/2009 TGDC Meeting

Performance-based test protocols are completed and will be validated Usability benchmarks Stable for the original demographics too much variability when demographics expanded Validating in 3 locations to finalize protocol and obtain values for “calibration machine” Accessibility throughout the voting session Poll worker documentation usability Accessible Voting Station usability testing needs more research Wide variability of benchmark values Difficult to obtain large sets of test participants Issues with wheelchair and dexterity testing Challenge is to establish pass/fail criteria Page 3 12/9-10/2009 TGDC Meeting

Wheelchair access requirements VVSG 2.0 requirements are harmonized with ADA Accessibility Guidelines for Buildings and Facilities (ADAAG) Designed for non-motorized wheelchairs Should be updated to account for motorized wheelchair anthropometry based on new Access Board research Page 4 12/9-10/2009 TGDC Meeting

Dexterity Dexterity has been inadequately addressed in existing system designs and hence, in the VVSG, for both navigation and for paper ballots One size does not fit all (huge individual variability) Possible solution is universal plug to support alternate switches or devices Research for more universal solutions covering broader sets of voters is needed Think out of the box, e.g., control by voice/sound command EAC Accessibility Grants Program has potential to provide solutions Page 5 12/9-10/2009 TGDC Meeting

Audio ballot usability Current audio ballots are difficult to navigate due to a high cognitive load Research is needed to create requirements to improve usability Studies exist in other domains having audio interfaces, such as automated phone interfaces VBP is a special subset as it has no visual interface Page 6 12/9-10/2009 TGDC Meeting

Cognitive disabilities Thus far our approach to cognitive disabilities is based on the concept of universal usability Clear instructions in plain language Clean design and navigation to lower cognitive load for everyone But, there is more that could be investigated We started with an analysis in “Implications of graphics on usability and accessibility for the voter”, VOTE-ID 2009 Proceedings, Lecture Notes in Computer Science 5767 Springer 2009 : In general, graphics may not help and may harm voter performance Further research is needed using carefully designed experiments EAC Accessibility Grants Program has potential to provide solutions Page 7 12/9-10/2009 TGDC Meeting

Usability for poll workers and other audiences We have focused on poll worker tasks during voting, but have not examined the human factors challenges in other parts of the process Audits, early voting, etc. Further research in these areas is needed VVSG, VVSG test methods, and other explanatory documents We have many lessons learned, public comments, and empirical evidence about our audiences and the use of these documents As we finalize these documents, we are considering how to further improve their usability for these audiences Page 8 12/9-10/2009 TGDC Meeting

Usability and accessibility testing by the VSTLs Our research shows that Testing needs to be performed by usability and accessibility experts Testing with human participants requires an appropriate facility We are in the process of developing a set of qualifications for the VSTL (Voting System Test Lab) personnel and lab facilities for NVLAP ISSUE: Current accreditation appears to require VSTLs to have permanent staff and onsite capability for human factors conformance testing Page 9 12/9-10/2009 TGDC Meeting

VVSG 2.0 and Beyond We have a mature set of requirements Test methods exist and require validation BUT, addressing the challenges in the “20%” gap is critical to achieving fully accessible, usable systems Page 10 12/9-10/2009 TGDC Meeting