Story-Based Test Automation Using Free Tools Intelligent Assurance and Testing 15-Apr-17 Story-Based Test Automation Using Free Tools Paul Gerrard paul@gerrardconsulting.com @paul_gerrard gerrardconsulting.com

Intelligent Definition and Assurance Overview Behaviour-Driven Development and similar approaches are still developer-focused Don’t engage testers as they should Don’t support higher complexity requirements Don’t scale to a larger requirements bases In this session, we’ll cover: Acceptance-driven development with free tools How Story Platform for QA (a sub-set of Business Story Manager) and Robot Framework can be used to implement this approach. Intelligent Definition and Assurance

Intelligent Definition and Assurance Background We won’t go into much detail of Behaviour-Driven Development http://en.wikipedia.org/wiki/Behavior-driven_development for an intro Requirements-test hierarchies: Go to http://businessstorymanager.com and watch the videos. Intelligent Definition and Assurance

Behaviour-Driven Development using current tools Test Fixtures/ Test Step Defs. + Story (structured) Generated Test Code BDD Tool Test Outcomes Test Tool executes tests System Under Test Write the Code + Refactor the Code Intelligent Definition and Assurance

Intelligent Definition and Assurance BDD and tool support A big benefit to developers is that BDD makes the Test-Driven Development approach easier Behaviour-Driven Development: A collaborative approach to specification Uses Domain–Specific Language in specifications Implements Ubiquitous Language Currently, tools only support the automated testing part of the BDD regime. Intelligent Definition and Assurance

Story Platform QA (BSM subset) Robot Framework Requirements Dictionary Keyword Definitions + Story (structured) Test Cases Test Export Test Outcomes Test Status Red Amber Green Test Library e.g. Webdriver executes tests System Under Test Write the Code + Refactor the Code Intelligent Definition and Assurance

Business Story Manager and Test Frameworks STORY: Feature Header Business Scenario Given “Search page” When “I find a book” And “I add to basket” Then “basket has my book” Just enough to clarify understanding Test Scenario Pre-conditions “Search page” Steps “enter search text” “click search button” “scroll down to find book” “select book” “view more information to verify” “select add to basket” Post Conditions/outcomes “Basket has selected book” Requirement Generated Story Based Test Cases Manually maintained Keyword Definitions 1 per Feature 1 per application Technical Code Per Environment Test Config. Example: Robot Framework Test Log Drivers Selenium (Web) HTTP (web) SSH DB Library Etc. System(s) Under Test

A walkthrough of the process using http://sp.qa How Does it Work? A walkthrough of the process using http://sp.qa

The detail for the requirements selected on the left is shown here.

Intelligent Definition and Assurance The feature identifies a capability of the system described in the requirement Intelligent Definition and Assurance

Intelligent Definition and Assurance The scenario provides an illustration of a feature in use. Examples provide broader coverage of the feature. Intelligent Definition and Assurance

NB: The colour coding highlights terms defined in the dictionary, and data items in scenarios. (The dictionary is available in the full product). This test scenario has been edited to provide more detail. These are the automated test keywords.

The Lucky Numbers Application

Intelligent Definition and Assurance Lucky numbers features: Registration Submit ticket View my Game Plays System Status Change my profile Reset Password etc. The numbers are submitted and validated and then the user is asked to confirm the ‘play’. Intelligent Definition and Assurance

Intelligent Definition and Assurance

Exporting Test Scenarios

We’ve shown just one test scenario here for illustration We’ve shown just one test scenario here for illustration. Normally, all test scenarios are exported.

Intelligent Definition and Assurance Dropbox asks for confirmation that you want to post the test case files to your account. Intelligent Definition and Assurance

BSM lists all the files that have been posted to your Dropbox account. Intelligent Definition and Assurance

Generated files and resource files The files generated by BSM have been synchronised with your Dropbox account These resource files correspond to your generated test case files. You maintain your keyword definitions here. Intelligent Definition and Assurance

Robot script and resource file The Robot Script files with test cases are generated by Business Story Manager The tester maintains the resource file with the keyword definitions. Intelligent Definition and Assurance

Tests running on a Linux VM that shares Dropbox Terminal window driving the tests System Under Test

…and the same test on Windows 7 Intelligent Definition and Assurance

Intelligent Definition and Assurance

Results upload service Use a batch file or shell script to run the tests, then upload the results to BSM/SP.QA loadRobotOutput.py Python uploads the output file Intelligent Definition and Assurance

Intelligent Definition and Assurance

Business Story Manager Robot Framework Requirements Dictionary Keyword Definitions + Story (structured) Test Cases Test Export Test Outcomes Test Status Red Amber Green Test Library e.g. Webdriver executes tests System Under Test Write the Code + Refactor the Code Intelligent Definition and Assurance

Behaviour-Driven Tooling The essential architecture of BDD tools is that A tool parses stories and generates test code The test code can be used for TDD The developer (almost always) writes the step definitions/fixture code and runs the tests The stories, test code and steps are all source-controlled (in the developers’ library) What we have done with SP.QA and Robot is allow non-techies to own this process and align it with stakeholder needs properly. Intelligent Definition and Assurance

Intelligent Definition and Assurance Summary SPQA and Robot Framework combined: Support requirements of any complexity Implement keyword-driven test automation Manage requirements, stories and tests to any scale Acceptance-driven approach: Traceable from requirements to tests and back Automated story acceptance and regression testing Build automated end-to-end acceptance tests with reusable keywords. Intelligent Definition and Assurance

http://sp.qa Story Platform for QA SP.QA provides a subset of Business Story Manager functionality and is FREE-TO-USE.

Visit SP.QA to access the free Story Platform for QA web service.

Go to the Image Gallery to see the requirements-test process in pictures

SP.QA Scope What you get FREE Service Limits Requirements module Business Stories and Scenarios Test Scenarios Robot Framework Interface FREE Service Limits Limited to 100 business stories 5 users, 1 application, 1 project

Intelligent Definition and Assurance BSM Scope What you get Stakeholder and Risk Management modules Lean Task Management module Dictionary and Library modules Role-Based Access Control All SP.QA Scope Limitations None Commercials £30 per user per month Intelligent Definition and Assurance

Story-Based Test Automation Using Free Tools Intelligent Assurance and Testing 15-Apr-17 Story-Based Test Automation Using Free Tools Paul Gerrard paul@gerrardconsulting.com @paul_gerrard gerrardconsulting.com