1. Story-Based Test Automation Using Free Tools
Intelligent Assurance and Testing
15-Apr-17
Story-Based Test Automation Using Free Tools
Paul Gerrard
@paul_gerrard
gerrardconsulting.com

2. Intelligent Definition and Assurance
Overview
Behaviour-Driven Development and similar approaches are still developer-focused
Don’t engage testers as they should
Don’t support higher complexity requirements
Don’t scale to a larger requirements bases
In this session, we’ll cover:
Acceptance-driven development with free tools
How Story Platform for QA (a sub-set of Business Story Manager) and Robot Framework can be used to implement this approach.
Intelligent Definition and Assurance

3. Intelligent Definition and Assurance
Background
We won’t go into much detail of Behaviour-Driven Development
for an intro
Requirements-test hierarchies:
Go to
and watch the videos.
Intelligent Definition and Assurance

4. Behaviour-Driven Development using current tools
Test Fixtures/
Test Step Defs. +
Story (structured)
Generated Test Code
BDD Tool
Test Outcomes
Test Tool
executes tests
System Under Test
Write the Code + Refactor the Code
Intelligent Definition and Assurance

5. Intelligent Definition and Assurance
BDD and tool support
A big benefit to developers is that BDD makes the Test-Driven Development approach easier
Behaviour-Driven Development:
A collaborative approach to specification
Uses Domain–Specific Language in specifications
Implements Ubiquitous Language
Currently, tools only support the automated testing part of the BDD regime.
Intelligent Definition and Assurance

6. Story Platform QA (BSM subset)
Robot Framework
Requirements
Dictionary
Keyword Definitions +
Story (structured)
Test Cases
Test Export
Test Outcomes
Test Status
Red
Amber
Green
Test Library
e.g. Webdriver
executes tests
System Under Test
Write the Code
+ Refactor the Code
Intelligent Definition and Assurance

7. Business Story Manager and Test Frameworks
STORY: Feature Header
Business Scenario
Given “Search page”
When “I find a book”
And “I add to basket”
Then “basket has my book”
Just enough to clarify understanding
Test Scenario
Pre-conditions “Search page”
Steps “enter search text”
“click search button”
“scroll down to find book”
“select book”
“view more information to verify”
“select add to basket”
Post Conditions/outcomes “Basket has selected book”
Requirement
Generated
Story Based
Test Cases
Manually maintained
Keyword Definitions
1 per Feature
1 per application
Technical Code
Per Environment
Test
Config.
Example: Robot Framework
Test
Log
Drivers
Selenium (Web)
HTTP (web)
SSH
DB Library
Etc.
System(s) Under Test

8. A walkthrough of the process using http://sp.qa
How Does it Work?
A walkthrough of the process using

9. The detail for the requirements selected on the left is shown here.

10. Intelligent Definition and Assurance
The feature identifies a capability of the system described in the requirement
Intelligent Definition and Assurance

11. Intelligent Definition and Assurance
The scenario provides an illustration of a feature in use. Examples provide broader coverage of the feature.
Intelligent Definition and Assurance

12. NB: The colour coding highlights terms defined in the dictionary, and data items in scenarios. (The dictionary is available in the full product).
This test scenario has been edited to provide more detail. These are the automated test keywords.

13. The Lucky Numbers Application

14. Intelligent Definition and Assurance
Lucky numbers features:
Registration
Submit ticket
View my Game Plays
System Status
Change my profile
Reset Password
etc.
The numbers are submitted and validated and then the user is asked to confirm the ‘play’.
Intelligent Definition and Assurance

15. Intelligent Definition and Assurance

16. Exporting Test Scenarios

17. We’ve shown just one test scenario here for illustration
We’ve shown just one test scenario here for illustration. Normally, all test scenarios are exported.

18. Intelligent Definition and Assurance
Dropbox asks for confirmation that you want to post the test case files to your account.
Intelligent Definition and Assurance

19. BSM lists all the files that have been posted to your Dropbox account.
Intelligent Definition and Assurance

20. Generated files and resource files
The files generated by BSM have been synchronised with your Dropbox account
These resource files correspond to your generated test case files. You maintain your keyword definitions here.
Intelligent Definition and Assurance

21. Robot script and resource file
The Robot Script files with test cases are generated by Business Story Manager
The tester maintains the resource file with the keyword definitions.
Intelligent Definition and Assurance

22. Tests running on a Linux VM that shares Dropbox
Terminal window driving the tests
System Under Test

23. …and the same test on Windows 7
Intelligent Definition and Assurance

24. Intelligent Definition and Assurance

25. Results upload service
Use a batch file or shell script to run the tests, then upload the results to BSM/SP.QA
loadRobotOutput.py Python uploads the output file
Intelligent Definition and Assurance

26. Intelligent Definition and Assurance

27. Business Story Manager
Robot Framework
Requirements
Dictionary
Keyword Definitions +
Story (structured)
Test Cases
Test Export
Test Outcomes
Test Status
Red
Amber
Green
Test Library
e.g. Webdriver
executes tests
System Under Test
Write the Code
+ Refactor the Code
Intelligent Definition and Assurance

28. Behaviour-Driven Tooling
The essential architecture of BDD tools is that
A tool parses stories and generates test code
The test code can be used for TDD
The developer (almost always) writes the step definitions/fixture code and runs the tests
The stories, test code and steps are all source-controlled (in the developers’ library)
What we have done with SP.QA and Robot is allow non-techies to own this process and align it with stakeholder needs properly.
Intelligent Definition and Assurance

29. Intelligent Definition and Assurance
Summary
SPQA and Robot Framework combined:
Support requirements of any complexity
Implement keyword-driven test automation
Manage requirements, stories and tests to any scale
Acceptance-driven approach:
Traceable from requirements to tests and back
Automated story acceptance and regression testing
Build automated end-to-end acceptance tests with reusable keywords.
Intelligent Definition and Assurance

30. http://sp.qa Story Platform for QA
SP.QA provides a subset of Business Story Manager functionality and is FREE-TO-USE.

31. Visit SP.QA to access the free Story Platform for QA web service.

32. Go to the Image Gallery to see the requirements-test process in pictures

33. SP.QA Scope What you get FREE Service Limits Requirements module
Business Stories and Scenarios
Test Scenarios
Robot Framework Interface
FREE Service Limits
Limited to 100 business stories
5 users, 1 application, 1 project

34. Intelligent Definition and Assurance
BSM Scope
What you get
Stakeholder and Risk Management modules
Lean Task Management module
Dictionary and Library modules
Role-Based Access Control
All SP.QA Scope
Limitations
None
Commercials
£30 per user per month
Intelligent Definition and Assurance

35. Story-Based Test Automation Using Free Tools
Intelligent Assurance and Testing
15-Apr-17
Story-Based Test Automation Using Free Tools
Paul Gerrard
@paul_gerrard
gerrardconsulting.com