1. Exploratory Test Automation
Jim Trentadue
Software Quality Consulting Director
Original Software

2. Agenda 1 2 3 4 5 6 What is ET What drives ET
Good automation practice with ET 4
Enabling Test Automation for ET
There are two ways this slide could be operated:
You may hyperlink the various Agenda Items to the slides that encompass their respective content (do not forget to add a ‘back’ button linking back to this slide)
Alternatively you may copy & paste the agenda slide besides the respective content slides so that when you press next, it shows the agenda slide once more, but this time, the upcoming topic will be highlighted.
NOTE: This slide is suitable for 5 menu items max, in case you need to add more menu items, then first group all the textboxes (including the numbered ones) together and resize them to make em smaller. The text in these text boxes has been set to “shrink to fit” settings, so now if you reduce the size of the textboxes the font size will change automatically, thereby making it easier to add more menu items without any problems. 5
Case Study 6
Session Recap

3. What is ET?

4. What is Exploratory Testing?
Only for the context of automation and this presentation
Original
Modified
Exploratory Testing is simultaneous learning, test design and test execution
- “Exploratory Testing Explained”, James Bach 2003
Simultaneously designing and executing tests to learn about the system using your insights from the last experiment to inform the next
- “Explore It!”, Elisabeth Hendrickson ↔ ↔
Engineer
Evaluate

5. Structured vs. Unstructured?
What does ET look like?
Ad hoc or Test Case?
Chartered or Uncharted?
Bounded or Unbounded?
Define the scope of Exploratory Testing

6. Exploratory Tester types
Deciding skills and experience of doing ET
Experienced
vs.
New Testers?
Testing skill level
High
Low
App. SME
vs.
New to app.?
System skill level
High
Low
Developer
vs.
Testers?
IT Role
High
Low

7. What drives ET?

8. ET mission Drivers for Exploratory Tests Concern Destruct Observe
Risk: Module, App., etc.
Examine App. Behavior
Cause-Effect
Break Software!
Concern
Destruct
Observe
Curiosity

9. Mood for ET
Using ‘Six Thinking Hats’ as a reference

10. Bolstering by Automation
ET Release Planning
Highlighting the need for automation
Data
Navigation
Timing
Test Case 1
Facts:
▪ Executed 12 times in release
▪ Module has been modified in each sprint (10)
▪ TC1 passes, but related complex tests are failing
Bolstering by Automation
▪ Frequent & random data injection
▪ Navigation paved more paths
▪ Sporadic timeouts on forms / pages are captured precisely

11. Good Automation Practice with ET

12. Decision to automate When should an exploratory test be automated?
Dina Hart, “A Casual Approach to Testing” on ministryoftesting.com

13. Exploratory Automation structure
Where to begin with automating these tests?
“TOURING TESTS”
“CHARTER YOUR EXPLORATIONS”
Simple Template
Explore (target), with (resources) to discover (information)
Good Charters
Direction without over specifying test actions
Generating Charters
Good time to draft charters is during requirements discussions
Nightmare Headline Game
Inspiring from our worst fears
Planning Charters
Chartering is more fluid than typical test planning
Practice Sessions
Draft charters for exploring a software package
Business District
Historical District
Entertainment District
Tourist District
Hotel District
Seedy District
‘Exploratory Software Testing’ by James Whitaker
“Explore It!” by Elisabeth Hendrickson

14. ET Automated Test Types
Three types that can be covered with automation
Planned, but random
Data
Field Data Injection, Data Reusability, Mapping Data Patterns
Objects identified, but sporadic
Navigation
Back and Forth, Object State Transition, Object Touches
Events known, but timings unknown
Timings
Varying Queries, Page Events (both fast and slow), Additional Instances

15. Enabling Test Automation for ET

16. Data Mapping Bloodline of your application Reuse Injection
Analyzing data output; map to existing or new automated test cases
Mapping
Leveraging this for subsequent transactions
Reuse
Random Data injected in page / form fields, both good and bad
Injection

17. Navigation Touch Touring the world…in your system Transition Seesaw
Random touch on an object to see what’s available and what isn’t
Touch
Determine an object's state in expected or unexpected transition
Transition
Back and forth actions to see if object / data pattern emerge
Seesaw

18. Timings Again Events Queries How responsive is your design
Open another instance of the app on same PC or same server
Again
By executing these, does the performance fluctuate on other events?
Events
Run large queries, small, and something in between
Queries

19. Case Study

20. CS - Exploratory Test Automation
Introduction to Model-Based Automation
Trouble Manager Functions
Create ticket to track problem
Run electrical test on phone line
Diagnose the problem
Fix the problem
Notify customer the issue is fixed
Close the ticket
Background
Testing each critical scenario in a Trouble Manager system for a major telecommunications company seemed impossible & harder yet was when certain conditions would appear
Solution
Implement a Model-Based Automation approach to each major function
Exploratory Test Automation in ‘Experiences of Test Automation’ by Graham & Fewster

21. CS – Defining Workflows
Functions that flow through the system
Basic Workflow
Improved Workflow
CloseTicket
Start
Diagnosed
Closed
RepairTicket
CreateTicket
DiagnoseTicket
CloseTicket
Created
Created
Repaired
AnalyzeTicket
AnalyzeTicket
Original Path
CreateTicket
AnalyzeTicket
Diagnose Ticket
RepairTicket
CloseTicket
Second Path
CreateTicket
AnalyzeTicket
Diagnose Ticket
RepairTicket
CloseTicket
Third Path
CreateTicket
AnalyzeTicket
Diagnose Ticket
CloseTicket
Fourth Path
CreateTicket
AnalyzeTicket
Diagnose Ticket
RepairTicket
DiagnoseTicket
CloseTicket
AnalyzeTicket
Exploratory Test Automation in ‘Experiences of Test Automation’ by Graham & Fewster

22. CS – Key Points Summarizing key points in this case study
Partial knowledge of a test result if automated, can enable for more tests to be run and checked automatically
Contain work within the module for what can possibly be done
Draw different logical ‘models’ of what can happen in the application
Automate the tour of the modelled paths through your system

23. Session Recap

24. Presentation Recap Key takeaways for Automating Exploratory Tests
Define Exploratory Testing only for the context of your automation
Understand the drivers for ET in your testing organization
Review your structure and types for Automated ET to decide appropriately
Create your automated ET ‘tour’ in three areas: Data, Navigation & Timings
Reference this Case Study or others for practical examples

25. Thank you for attending this presentation!
Jim Trentadue
Software Quality Consulting Director
Original Software