ACCESSIBILITY TESTING, EVALUATING AND REPORTING AHG 2013 Hadi Rangin University of Illinois EASI Track

Overview Background on accessibility evaluation and reporting Evaluation methodology Common Accessibility Testing Criteria Essential Accessibility Criteria (Show-stoppers) Reporting evaluation results Identify audience Closing remarks Contacts and FAQ

Background Increasing attention to accessibility by administrators Checking for accessibility after adoption Some checks for accessibility in the RFP process Inconsistency in accessibility evaluation and reporting

Statement of the Problem No normalized evaluation and reporting Depends very much on who has performed & furnished report Tests miss essential features and include testing for non-critical elements Reports include raw data from automated accessibility checkers without explanation Reports are not targeted for their audience Lack of impact of the issues, prioritization of issue and possible solutions Administrators need reliable data to make strategic decisions

The Goal Define guidelines for accessibility evaluation & reporting Define a methodology for accessibility testing & evaluation Communicate how to write effective reports Engage vendors in constructive collaboration

Evaluation Methodology Create a list of functional tasks for the application. Define Accessibility/Usability Requirements. Divide functional tasks into a logical series of steps Start with Testing & Evaluation based on the functional tasks and criteria identified Watch for Show-stoppers, recommended, and other potential technical criteria. Record your observations and findings

Create A List of Functional Tasks An application is functionally accessible if the user can: Accomplish applicable tasks successfully, independently and reliably in a reasonable time. Functional tasks are unique to each application Important: work with responsible entities to learn how the application is going to be used in your setting Prioritize functional tasks Functional tasks require manual testing Be ready to spend a lot of time on it

Functional tasks example: Webmail Logging in and out E-mail summary Finding desired message(s) Reading an e-mail Tagging or moving an e-mail to desired folder Sending an e-mail Misc. tasks

Define Accessibility/Usability Requirements (1) Keep in mind at all times: Consider the differing needs of users with disabilities: Physical disabilities, low-vision, color-blind, blind, deaf, cognitive impairments Goal is functional accessibility; answer the following questions: How usable is the application for diverse groups of users? What are the impacts of accessibility issues in completing the task?

Define Accessibility/Usability Requirements(2) 3. Consider the following from the perspectives of users with differing needs: Perceivable – Information and user interface components must be presentable in ways that the user can understand. Operable – User interface components and navigation must be operable. Understandable - Information about and the operation of the user interface must be understandable. Robust – Content must be robust enough that it can be interpreted reliably by a wide variety of user agents, including assistive technologies.

Divide functional tasks into steps(1) No magic recipe that would work for all applications. Key factor is that all the relevant functionality must be available to all AT users. Example: Reading an Email Can user identify that he is in the right folder (Inbox)?

Divide functional tasks into steps(2) Can user navigate to the desired message(s)? The user needs to know: Message status (read, unread, replied to, flagged) Subject Date Sender Recipients Presence of attachments Can user open the message and read the message body? Can user access and open the attachments?

Before you begin Specify: Computer system characteristics Browser information Assistive technology used Any third-party software information Accessibility testing tools Study all manuals, configuration/optimization documentation Study specific configuration information for users with disabilities Engage the IT professionals in charge of the applications Have a copy of the Show-stoppers & common technical requirements Have a second person to observe/take notes

Begin with Testing and Evaluation Testing and evaluation require good knowledge of accessibility and familiarity with APPLICATION development. As you go through each step for each functional task, check for technical accessibility of that step You can use automated testing tools to determine selected technical accessibility issues: FAE, WAVE, Accessibility Evaluator for Firefox, OAA Accessibility Inspector (in development), etc. Never use an AT like Jaws to determine the technical accessibility issues. Take good notes & make screenshots whenever possible Mark any functional tasks that violate Show-stopper criteria Stop if you see too many Show-stopper violations Use the Common Testing Categories & Criteria as a guide

Common Accessibility Testing Categories Structure, Navigation, Orientation Styling Content Forms Help and Documentation Personalization and Customization

Page Structure, navigation, orientation The most important elements of accessibility. Logical structure is critical for effective & efficient interaction with an application. User must always know where he/she is. User must be able to locate and move to the desired components User must be able to interact with the page with keyboard Visual layout and order must match the keyboard order

Styling Use of table for layout Text zooming and window resizing High-contrast mode Stylistic images

Content Link and link text Data table Grouping relevant items Informational graphics Use of color Audio/video Embedded objects Challenge-response mechanism

Forms Primary method for user to interact with application User needs to concentrate on the data being entered Not being challenged or distracted by the complexity of the interface and navigation Entering data with certainty Notification of warning, error, or successful form submission Being able to identify easily places where errors have occurred and navigate to them All form controls must be accessible

Help and Documentation New, complicated, and challenging applications Understanding the layout and general features of the application Some tasks require complicated instructions Quick guide in the application Presence of supported accessibility features Quick accessibility guide

Personalization and Customization Users have different needs and ways of viewing and interacting with application Adaptation of application to user's individual needs instead of user adapting to the application Many global settings and personalization/customization features can improve or limit accessibility/usability

Essential technical accessibility criteria (Show-stoppers) (1) All critical functionality of a given application must be operable via the keyboard without relying on keyboard shortcuts This includes all menus, other navigation, and dynamic objects (such as image slideshows) A clear visual indicator of the current item in focus must be displayed (similarly to how mouse hover works in a webpage) Form fields and buttons must have valid programmatic labels that are available to assistive technologies Color alone must not be used to convey application state and other information

Essential technical accessibility criteria (Show-stoppers) (2) The application must have a logical and programmatically discernible structure that follows the visual structure of the application. The reading order and tab order for an application must be logical for the program. The user interface must remain usable in high-contrast color modes. Informational images, charts, and graphs must have descriptive alternative text. Charts and graphs must also have additional detailed descriptions of their content.

Essential technical accessibility criteria (Show-stoppers) (3) Flash objects and other embedded applications must accept keyboard focus and never trap it. All contents and possible controls in those objects must be also accessible and operable with keyboard. Audio content must have transcripts, video with audio must have captions and audio description (where appropriate), and any associated media player controls must be operable by keyboard. CAPTCHAs (generically referred to as a challenge-response mechanism) must be accessible.

Reporting Evaluation Results An effective evaluation report concisely does three things: It communicates whether or not the application is accessible. It communicates the impact of any issues found, in regards to the specific application. It prioritizes the results for remediation, from most impactful to least.

Effective Report Structure Like a Funnel: Moving from a broad conceptual overview to detailed, conformance-level information. Executive Summary Evaluation Methods Include a list of identified functional tasks that were tested Prioritized Issues With applicable conformance criteria from accessibility standards and guidelines Recommendations for remediation (as appropriate) Include links to applicable resources for best practices and applicable techniques

Executive Summary Good for administrators, purchasers, and account representatives. An executive summary should include: The name of the application being evaluated Include major and minor version number if appropriate. A statement of impact Can the application be considered accessible? Who will be affected by any issues found, and what is the severity of the impact? A statement of how well the application conforms to legal accessibility requirements Helps readers determine legal risk of deployment (especially administrators)

Executive Summary (continued) An executive summary should include: A brief outline of major functional issues Ex: users cannot log in successfully or cannot read email without difficulty A remediation statement How difficult would it be to correct the issues found, and who would need to fix them? Needed by administrators and purchasers to estimate the cost involved to perform remediation and the possible effect on project timelines Suggestions of viable alternative applications (for internal reports) If there are no alternative solutions, administrators and purchasers would likely want to know this.

Evaluation Methods Establishes the validity of the evaluation The functional task list lets others know what areas of the application were evaluated. Assists others in reproducing the results. Include: Information about what tools were used to evaluate the application. Include version or model numbers where possible. Identify web browsers that were used, including version numbers. Identify any mobile devices used for testing, including the major and minor revision of the OS on the device.

Prioritized Issues Helps create a concrete checklist for remediators Group issues by functional task Prioritize the tasks from most critical to least critical For each task: Identify any problems found Include the applicable conformance criteria from the accessibility standards Prioritize the conformance criteria, from most impactful to least impactful

Prioritized Issues (continued) Many conformance issues arise multiple times and are application-wide. Applies to most Show-stopper issues. Ex: Application menus are not keyboard accessible Call out application-wide issues in their own section. Group by issue, from most impactful to least. Include applicable conformance criteria.

Recommendations for Remediation Helps developers with limited or no knowledge about accessibility. Helps to create a positive partnership with vendors. Provides best-practice recommendations for remediation of issues. Links to resources: Help developers understand accessibility issues See working examples

Identify the Audience Internal (your organization) Vendors Administrators and purchasers Technical support and documentation staff Development teams Vendors Top managers / CEO Sales representatives and account managers Project managers and development teams General Public Online journals promoting accessibility awareness End-users Disability advocacy groups Articles and FAQs for accessibility professionals

Reports for Upper-Level Administrators and Purchasers Usually non-technical Give them the Executive Summary Let them know more detailed report is available if desired.

Reports for Project Managers, Account Representatives, & the General Public Need a general overview & functional understanding of the issues Give them: Executive Summary Methodology Issues by Task

Reports for Development Teams Need to remediate any issues found Give developers the full report Let them know that you are willing to help

Closing Remarks Define unique functional tasks. For an effective evaluation report: Define unique functional tasks. Test & evaluate the application based on the functional tasks. Identify your audience. Target the report accordingly.

Hadi Rangin, hadi@illinois.edu Contact Information Hadi Rangin, hadi@illinois.edu Questions?