1. Move from Scripted Manual Testing to Scenario-Based Testing
Andrea Fox
Sr. Test Analyst
October 4, 2017

2. Remembering the co- founder and CEO of Cerner. A life lost too soon.
Dedication of talk
Remembering the co- founder and CEO of Cerner. A life lost too soon.
“The future is not guaranteed, but I like our chances of being the ones who can positively impact the delivery of care.” ~Neal Patterson
I wanted to dedicate my talk to Neal Patterson the founder and CEO of Cerner. Neal was an inspiration to many of his associates, but it was his public speaking style that made me believe I too could be a speaker. He was very real and a lot of the times would get lost in thought etc. but it was his realness and his passion that made it worth hearing him.

3. Summary Identifying the issue Behavior driven tests
How we arrived at today’s scenarios
What are scenario tests?
Tips for writing
Test Evidence
Pros and Cons of scenario testing
Q&A
Who here writes/runs tests manually? Are you using scripts?

4. The issue
The problem – the manual, scripted tests were not enough to find defects
Only covered a single path
Corrective action was to add another test that scripted the failure scenario
Spent too much time writing, running, and maintaining (sometimes)
Test repository was out of control

5. BDT – Behavior Driven Testing
Had us look at the system from the outside to outline behavior
Started to make us think about what is being tested

6. BDT - Example Given the account has a balance of $2000
When the customer requests a withdrawal of $100
Then the account is debited by $100, and the account balance updates to $1900

7. Why BDT didn’t necessarily work
Written in given, when, then statements
Writing scenarios in the description area as well as the steps
Spent more time trying to write them then run them and hard to understand
Good start – not quite what we needed
Behavior driven testing started us on the right track. We were starting to think more about what needs to be tested, but got caught up in the design aspect. We had a hard time writing the tests in the given, when, then statements and the engineers had a hard time wrapping their head around it while testing.

8. The Transition
Originally presented to the group the idea of using exploratory testing
Formed the idea of writing steps as scenarios
When talking about exploratory I had talked about how we didn’t need the direction the steps were providing but leaving it open for the tester to interpret giving them more freedom. The director didn’t like the idea just having requirements so mentioned scenarios.

9. What is scenario testing?
Definition: 1)an outline of the plot of a dramatic work, giving particulars as to the scenes, characters, situations etc. 2) an imagined or projected sequence of events, especially any of several detailed plans or possibilities
In testing it’s the goal or action the user wants to accomplish
Scenario testing is the happy medium between scripted testing and exploratory testing.

10. Best practice tips for writing scenario tests
The scenarios tell a story. Can be simple to complex.
The test isn’t written to just pass, but to make the tester want to look at all of the angles
The scenarios are credible. Can this actually happen to a client? If so it should be included
Evaluate whether the test scenarios passed or failed. Are they written in a way that makes the results obvious?
Put the time into planning – mind maps, outlines, lists, etc.
Kaner, Cem. (2003) An Introduction to Scenario Testing.

11. Example of Mind Map

12. Test Evidence Just as important as the scenarios!
Evidence should show the story that was tested
Make sure to include any thing you deem relevant to proving the scenario passed or failed: screenshots, explanations of what was tested, pass/fail status, etc.

13. Test evidence example

14. Pros and Cons Took less time to write Gain solution knowledge faster
Allows for multiple paths to be taken during testing
Places focus on what is being tested
All members of the team (engineers, architect, solution designer, and tester) take part in scenario planning sessions
If a scenario fails testing doesn’t have to necessarily stop
Can be hard to run without solution knowledge
Spend more time planning
Not always the best choice for early testing
Can expose more design flaws than actual code problems (not necessarily bad)

15. Any questions?

16. Thank you for attending!
Andrea Fox
2800 Rockcreek Pkwy
Kansas City, MO 64117