1. PROTEA: A Process for procuring accessible software Dr Richard Walker E-Learning Development Team Manager & VLE Service Group Leader University of York Professor Helen Petrie Professor of Human Computer Interaction University of York

2. ABOUT THE UNIVERSITY OF YORK Ranked #7 in The Times Higher Education 100 Under 50 world ranked universities (2013) Home to more than 15,000 students and 3,000 staff Blackboard users since 2005 Currently on Blackboard Learn v9.1 Service Pack 12 (licensing community, content and learning management system and mobile learning building block)

3. ABOUT THE HCI RESEARCH GROUP AT YORK One of the oldest research groups studying human- computer interaction in the UK, if not the world Founded in 1984 – centred in Computer Science, but with members in Archaeology, Electronics, Health Sciences, History, Management, Psychology, Sociology and Theatre Film TV Key values – putting humans at the centre of human- computer interaction Particular interest in people with disabilities, older people, e-learning

4. BACKGROUND – FROM THE E-LEARNING DEVELOPMENT TEAM Have a strong commitment to accessibility and usability Institutional staff and student VLE surveys have highlighted usability issues, for example: In designing modules Uploading content EARL reading list application Tools felt to be “clunky”

5. BACKGROUND – FROM THE E-LEARNING DEVELOPMENT TEAM Have worked with the HCI Group on two occasions: On upgrade from v8 of Blackboard Academic Suite to the next generation release 9.1 Service Pack 5 (in 2011) To review version 9.1 Service Pack 12 – as an additional action to the VLE retender (see below) We report our findings to the University’s E-Accessibility Forum and use the information to inform our training and support to system users

6. BACKGROUND – FROM THE HCI RESEARCH GROUP We have done many evaluations of accessibility and usability of websites and web-based systems, in particular: A formal investigation of websites for the Disability Rights Commission of Great Britain We evaluated 100 websites with 50 people with disabilities, 10 people per website Asked them to conduct typical tasks on each websites We found that if we evaluated with blind, partially sighted and dyslexic participants we found 80% of accessibility problems

7. OUR CHALLENGE We wanted to support the university in providing more accessible software, for both students and staff One of the difficulties is in procuring new software – how do we know it will be accessible, most software does not come with any details of accessibility If it does, it is usually a statement of conformance to guidelines, perhaps Web Content Accessibility Guidelines (WCAG) – is this sufficient?

8. OUR CHALLENGE Two problems: WCAG guidelines are not relevant to all software and quite hard to apply to very complex systems Guidelines such as WCAG do not tell you definitely that students/staff with disabilities will be able to use the software, they are not precise enough, you need testing with users

9. OUR SOLUTION: PROTEA: PROTOCOL FOR TESTING E-ACCESSIBILITY So we created a protocol for user testing of software: Can be applied to any kind of software, to be used by students or staff Can be conducted by our group, other groups in the university or by the software provider (who would provide evidence that it has been followed)

10. OUR SOLUTION: PROTEA: PROTOCOL FOR TESTING E-ACCESSIBILITY Need to evaluate the software with 3 blind people, 3 partially sighted people and 3 people with dyslexia, as close as possible to the real users of the software being evaluated The blind and partially sighted people should use a mix of different assistive technologies to access the software Blind participants – JAWS, WindowEyes, VoiceOver, NVDA Partially sighted participants – SuperNOVA, ZoomText Dyslexia participants – may change screen configuration

11. OUR SOLUTION: PROTEA: PROTOCOL FOR TESTING E-ACCESSIBILITY Need to pick a series of 3 - 5 “typical tasks” for the participants to do: “typical task” means something quite “meaty” Send an email to your tutor asking for a meeting Upload an essay Find the reference list for your archaeology course next term Start with easy tasks, work up to harder ones, perhaps weave into a story

12. OUR SOLUTION: PROTEA: PROTOCOL FOR TESTING E-ACCESSIBILITY A facilitator sits with the participant as they go through the tasks, to assist if needed If possible, sessions should be recorded for later analysis and discussion Participant talks through what they are doing (“think aloud” or “verbal protocol”)

13. OUR SOLUTION: PROTEA: PROTOCOL FOR TESTING E-ACCESSIBILITY If the participant finds a problem, they pause and rate it on a scale 1 (= cosmetic, small annoyance) to 4 (= major, I’m really stuck) If the participant gets really stuck (e.g. the screen reader cannot interpret the page), the facilitator helps move the participant on BUT participant should not be “lead” through the task – not told where to click or the particular names of button, links etc

14. OUR SOLUTION: PROTEA: PROTOCOL FOR TESTING E-ACCESSIBILITY Occasionally might have some of the development team sit in BUT only if the participant is comfortable with that and will not feel intimidated Gather all the problems encountered, analyse for which groups of users/assistive technologies are being affected

15. OUR RESULTS We applied PROTEA as part of the recent procurement process for the VLE at York We did find some unexpected accessibility problems that we fed back to Blackboard We also highlighted some issues where we need to educate our staff members who develop content to ensure accessibility

16. OUR RESULTS Issue 1: Assistive Technology unable to give access to functionality Problem: screen reader users could not interact fully with the Content Editor Consequences: could not send emails, contribute to fora, answer quiz questions

17. OUR RESULTS Issue 2: Navigation within software Problem: Moving between frames was difficult and inconsistent for screen reader users Consequences: users could not navigate to where they wanted to be, became “trapped” in frames

18. OUR RESULTS Issue 3: Auditory information needs different handling to visual information Problem: In character limited text boxes, screen readers announced the character count after every keypress

19. OUTCOMES AND RECOMMENDATIONS Blackboard have been very responsive in investigating these issues They are the kinds of problems which will not necessarily be picked up by testing to guidelines Need really realistic evaluation with users and a range of assistive technologies

20. OUTCOMES AND RECOMMENDATIONS Other useful outcomes of using this kind of evaluation: VLE support staff become much more aware of problems that students with disabilities may face Can create better support materials, training for students/staff who are creating content

21. OUTCOMES AND RECOMMENDATIONS We have used the PROTEA method not only with Blackboard, but with several systems being procured by the university and created in-house Both systems for students and for staff We have also worked with the UK government (on DirectGov) and a major bank

22. THANK YOU! If you would like more information, we will be setting up a website in the next weeks: www.yorkhci.org/protea We are happy to provide general advice for free and of course can undertake evaluations Helen.Petrie@york.ac.uk