Bring Accessibility into the Development Lifecycle with CI Testing Alistair Garrison Director of Accessibility Research

Formerly Known As SSB BART Group Who is Level Access? Formerly Known As SSB BART Group New Corporate Identity – Name and Branding New Website – Coming Spring 2017!! No Change in our Innovative Technology or Excellent Service CSUN 2017 Assistive Technology Conference

What is Continuous Integration? Continuous Integration (CI) is a development practice; It requires developers to check-in their code to a shared repository – ideally several times a day. Each check-in forces an automatic re-build; then re-test of the software (from the integrated code) So integration issues can be detected immediately. CSUN 2017 Assistive Technology Conference

CSUN 2017 Assistive Technology Conference CI process flow… Developer develops Checks-in changes CI fetches changes CI builds CI tests CI notifies – Build Success / Failure CSUN 2017 Assistive Technology Conference

So, let’s see a perfect-world example… The ACME Corporation wants a web page for selling its products; CSUN 2017 Assistive Technology Conference

So, let’s see a perfect-world example… To interested customers (who previously had to use a mail order catalog); CSUN 2017 Assistive Technology Conference

The Product Owner, a Designer and a Business Analyst sit down to talk… The Business Analyst (BA) asks questions; The Product Owner states what they want; The Designer draws ideas; CSUN 2017 Assistive Technology Conference

CSUN 2017 Assistive Technology Conference After a short while… Concepts / ideas start to solidify… They want: A web page; Pictures of their products; With show / hide buttons above each picture; to reveal more product information. CSUN 2017 Assistive Technology Conference

CSUN 2017 Assistive Technology Conference So… Designer creates a wire- frame of page; BA creates user stories for the show / hide function (component-level); BA creates user stories for completing a task on the page (integration-level); CSUN 2017 Assistive Technology Conference

For clarity - a user story is… A description in the everyday language of the end-user that captures what a user does, or needs to do, in order to complete a task. CSUN 2017 Assistive Technology Conference

User stories: special format… In our example, the ACME Corp uses the Gherkin format when creating user stories. Given (the situation); When (I do something or something happens); And (optionally I do something else); Then (this is what I expect has happened)… CSUN 2017 Assistive Technology Conference

Component-level acceptance test Example, from ACME… Component-level acceptance test Scenario 1: Show / Hide Vanilla Widget – Show state. Show X CSUN 2017 Assistive Technology Conference

Component-level acceptance test Example, from ACME… Component-level acceptance test Given the Show / Hide Widget button is focused; And the Show / Hide Widget button has the value “Show X”; Show X CSUN 2017 Assistive Technology Conference

Component-level acceptance test Example, from ACME… Component-level acceptance test When the Show / Hide Widget button is activated; Show X CSUN 2017 Assistive Technology Conference

Component-level acceptance test Example, from ACME… Component-level acceptance test Then the Show / Hide Widget button has the value “Hide X”; And the Show / Hide Widget panel is visible; And the Show / Hide Widget panel contains the text “X”. Hide X X CSUN 2017 Assistive Technology Conference

Integration-level acceptance test… Example, from ACME… Integration-level acceptance test… Scenario 1: Show product details Basically, the vanilla widget is used in context. Show product details CSUN 2017 Assistive Technology Conference

Integration-level acceptance test… Example, from ACME… Integration-level acceptance test… Given the Show / Hide button is focused; And the Show / Hide button has the value “Show product details”; Show product details CSUN 2017 Assistive Technology Conference

Integration-level acceptance test… Example, from ACME… Integration-level acceptance test… When the Show / Hide button is activated; Show product details CSUN 2017 Assistive Technology Conference

Integration-level acceptance test… Example, from ACME… Integration-level acceptance test… Then the Show / Hide button has the value “Hide product details”; And the Show / Hide panel is visible; And the Show / Hide panel contains the text “Super magnet”. Hide product details Super magnet CSUN 2017 Assistive Technology Conference

FYI, in terms of development methods ACME Corporation uses… BDD - Behavior-driven development; BDD specifies that software should be built / tested in terms of its desired behaviors. Where "desired behaviors" are generally defined through user stories in a special format e.g. Gherkin. CSUN 2017 Assistive Technology Conference

ACME uses Cucumber for BDD testing… With Selenium Webdriver + Assertion Libraries e.g. Chrome Webdriver / Chai; But, it could have chosen loads of other frameworks… CSUN 2017 Assistive Technology Conference

So, turning user stories into Acceptance Test for Cucumber… The user stories formatted in Gherkin are associated with code functions e.g. step definitions, page objects, etc… so they become Acceptance Tests. CSUN 2017 Assistive Technology Conference

CSUN 2017 Assistive Technology Conference So, we now have… Designs; Acceptance Tests (Gherkin); BDD Component-level acceptance tests; BDD Integration-level acceptance tests; CSUN 2017 Assistive Technology Conference

So where do they fit into ACME’s CI process? Pre-Check-In CSUN 2017 Assistive Technology Conference

In the pre-check-in process… Continuous Integration Update from Version Control Build (local) Inputs Designs Component-Level Acceptance Tests Test: Unit tests + Acceptance Tests Code Test Check-In CSUN 2017 Assistive Technology Conference

CSUN 2017 Assistive Technology Conference And… Post-Check-In CSUN 2017 Assistive Technology Conference

In the post check-in test phase… Continuous Integration CI runs all unit tests (fast); CI runs a smoke test of acceptance tests (slow); tests that cover the highest traffic tasks / journeys; happy paths; CI runs all acceptance tests (nightly). Unit Tests Smoke Test Test type Nightly Number of tests CSUN 2017 Assistive Technology Conference

So, how do we bring Accessibility testing into this process?… CSUN 2017 Assistive Technology Conference

Accessibility testing… Accessibility testing is effectively an extension of User Acceptance Testing (UAT). Ensuring: Users – including defined groups of disabled & older users; Using – defined assistive technologies, browsers, and operating systems; Can – effectively and efficiently undertake necessary journeys within a product (e.g., web site or web application); To – complete all core goals. CSUN 2017 Assistive Technology Conference

Moving forward, it is not… In the context of Continuous Integration Simply putting the url of a page into an on-line tool; and pressing test – in order to check: The source code; or The initial DOM state of the page (post-load). As, a user’s journey through a page can result in many DOM stages; and each state needs all its components to be accessible in order for a user to have an accessible journey; CSUN 2017 Assistive Technology Conference

In today’s development pipelines it is… Accessibility Testing The integration of stand-alone Accessibility Testing Libraries with QA Frameworks for BDD, Unit Testing, etc… To allow each DOM state configuration attained through-out a user story (component level / integration level) to be tested for accessibility. CSUN 2017 Assistive Technology Conference

Where Accessibility Testing libs are… Stand-alone libraries of accessibility checks – often coded in JavaScript (e.g. Level Access’s Access Engine). Which can be integrated with loads of testing frameworks. Called through APIs which allow all tests to be run, single tests to be run, etc. CSUN 2017 Assistive Technology Conference

Methods of injecting a lib… Injecting lib into web page Adding the JavaScript through a <script tag… Achieved dynamically through JavaScript in (Cucumber) step definitions; Through a manual testing toolbar; CSUN 2017 Assistive Technology Conference

Methods of injecting a lib… Inserting lib into mobile build For Mobile it can be better to “instrument the code” – in effect, include the testing lib in the non-release code. So it’s baked-in from the start. It’s just a little easier if you’re testing with frameworks like Appium – driving real phones. CSUN 2017 Assistive Technology Conference

So, let’s see a perfect-world example again… The ACME Corporation wants a web page for selling its products; CSUN 2017 Assistive Technology Conference

So, let’s see a perfect-world example again… To interested customers (who previously had to use a mail order catalog); CSUN 2017 Assistive Technology Conference

The Product Owner, a Designer and a Business Analyst sit down to talk… The Business Analyst (BA) asks questions; The Product Owner states what they want; The Designer draws ideas; CSUN 2017 Assistive Technology Conference

CSUN 2017 Assistive Technology Conference After a short while… Concepts / ideas start to solidify… They want: A web page; Pictures of their products; With show / hide buttons above each picture; to reveal more product information. All to be accessible! CSUN 2017 Assistive Technology Conference

CSUN 2017 Assistive Technology Conference Why… Amongst other reasons… One of their very good customers has a lot of temporary disabilities. CSUN 2017 Assistive Technology Conference

CSUN 2017 Assistive Technology Conference So, they… Determine the accessibility requirements their aiming to meet; Select an Accessibility Testing Library that integrates with their BDD testing tools; Define relevant design pattern of behaviors (in terms of accessibility) for their page / widgets. CSUN 2017 Assistive Technology Conference

CSUN 2017 Assistive Technology Conference The… Designs are altered to ensure accessibility is considered; Component-Level Acceptance tests are extended to include calls to the Accessibility Checking Library; and additional behaviours (in terms of accessibility). CSUN 2017 Assistive Technology Conference

CSUN 2017 Assistive Technology Conference Component-Level Accessibility acceptance test using Accessibility Checking Library (BDD format) Scenario 1: Accessibility Test Point - Show / Hide Widget – Show state. Given the Show / Hide Widget button is focused; And the Show / Hide Widget button has the value “Show X”; When I run Accessibility Tests Then no Accessibility Tests will fail. Show X CSUN 2017 Assistive Technology Conference

CSUN 2017 Assistive Technology Conference Component-Level Accessibility acceptance test using Accessibility Checking Library (BDD format) Scenario 2: Accessibility Test Point - Show / Hide Widget – Hide State. Given the Show / Hide Widget button is focused; And the Show / Hide Widget button has the value “Hide X”; And the Show / Hide Widget panel is visible; And the Show / Hide Widget panel contains the text “X”. When I run Accessibility Tests Then no Accessibility Tests will fail. Hide X X CSUN 2017 Assistive Technology Conference

CSUN 2017 Assistive Technology Conference Component-Level Accessibility acceptance test based on behaviors that support accessibility Scenario 3: Accessibility Behavior - Show / Hide Widget - Show. Given the Show / Hide Widget button is focused; And the Show / Hide Widget button has the value “Show X”; When the Show / Hide Widget button is activated; Then the Show / Hide Widget button has the value “Hide X”. (satisfying 4.1.2 as status update is communicated). And [optional] the Show / Hide Widget button has an aria-expanded attribute set to the value “true”. Hide X Show X X CSUN 2017 Assistive Technology Conference

CSUN 2017 Assistive Technology Conference And… Integration-Level Acceptance tests are extended to cover additional behaviours (in terms of accessibility). CSUN 2017 Assistive Technology Conference

CSUN 2017 Assistive Technology Conference Example Integration-level A11y acceptance test, based on behaviors that support accessibility Scenario 1: Keyboard - Read more about product. Given that the ACME product page is open. And the tab key has been used to focus the “Show product information” button When the enter button is pressed. Then the “Show product information” button becomes “Hide product information”. And an area becomes visible that shows “product information” for the “super magnet” product. CSUN 2017 Assistive Technology Conference

CSUN 2017 Assistive Technology Conference So, they now have… Designs (that take into account selected accessibility requirements); Component-level acceptance tests Component-level accessibility acceptance tests – that include behaviours needed for accessibility support; Integration-level acceptance tests Integration-level accessibility acceptance tests – that include behaviours needed for accessibility support; CSUN 2017 Assistive Technology Conference

What does this impact in CI? Continuous Integration Unit tests – no; Functional tests: Component-level acceptance tests - yes; Integration-level acceptance tests - yes; CSUN 2017 Assistive Technology Conference

In terms of the number of tests… Continuous Integration Acceptance Tests for A11y Integration Tests for A11y Component Tests for A11y Acceptance Tests for A11y Integration Tests Component Tests for A11y Component Tests Component Tests Pre Check-In Post Check-In Unit Tests Unit Tests CSUN 2017 Assistive Technology Conference

But, what does this bit of extra effort give you!!! Far higher testing coverage relating to accessibility; Far higher confidence that all users will be able to complete their journeys in an unhindered way. = Happier users! CSUN 2017 Assistive Technology Conference

Questions?