1. www.origsoft.comwww.origsoft.com | © Original Software 1 Jim Trentadue Software Quality Consulting Director Original Software jtrentadue@origsoft.com Transform Manual Testing processes to incorporate Test Automation

2. www.origsoft.comwww.origsoft.com | © Original Software 2 Agenda 22 Automation Models and Frameworks 33 Incorporating Automation into Standard Testing Processes 44 55 Case Study Deep-dive into Testing Artifacts Test activities / deliverables by SDLC 11 Presentation Recap 66

3. www.origsoft.comwww.origsoft.com | © Original Software 3 SDLC Test Activities / Deliverables

4. www.origsoft.comwww.origsoft.com | © Original Software 4 Test Activities within an Agile SDLC Project Initiation The End Game (System Test) Release Planning Release to Prod/Support Each Iteration 1 … X Get an understanding of the project Participate in sizing stories; Create test plans Participate in sprint planning, estimating tasks Write and execute story tests Pair-test with other testers, developers Business validation (customers) Automate new functional test cases; Run automated regression test cases Run project load tests Demo to the stakeholders Release mgmt tests mock deploy on staging; Smoke test on staging Perform load test (if needed) Complete regression test; Participate in releases readiness Business testers perform UAT Participate in release to production Participate in retrospectives Agile Testing – Crispin & Gregory Typical Agile process of activities to deliverables

5. www.origsoft.comwww.origsoft.com | © Original Software 5 Test activities within a Waterfall SDLC Project Initiation Test Strategy Analysis Design Develop Testing Deploy Test Scenarios Test Cases Test Scripts Test Results Test Summary Standard V-Model set of testing activities

6. www.origsoft.comwww.origsoft.com | © Original Software 6 Test Automation working models and frameworks in the industry

7. www.origsoft.comwww.origsoft.com | © Original Software 7 Record \ Playback Structured Testing (invokes more conditions) Data Driven Keyword Driven Model / Object Based Actions Based Hybrid: combines 2 or more of the previous frameworks Test Automation Frameworks Below is the evolution of automation frameworks

8. www.origsoft.comwww.origsoft.com | © Original Software 8 0-10% UI Centric Automation 20-40% Service or Middle Tier Automation 50-60% Unit level Automation +80%, UI Centric Automation 0-10% Service or Middle Tier Automation 0-10% Unit level Automation Velocity Partners – The Agile Test Automation pyramid, Mike Cohn Key points with Test Automation & Agile Agile automation  Smaller units of work built uniquely vs. end-to-end  Constant review of automation backlog  Automation of forms not complete

9. www.origsoft.comwww.origsoft.com | © Original Software 9 Automated Software Testing: Introduction, Management and Performance – Dustin, Rashka, Paul D. System Design & Development C. Small Tool Pilot/Prototype A. System Life-Cycle ProcessF. Production and Maintenance B. Business Analysis and Requirements E. Integration and Test Mapping ATLM against a Waterfall SDLC How an Automated Test Lifecycle Management maps to SDLC

10. www.origsoft.comwww.origsoft.com | © Original Software 10 Fusing Automation into the Process

11. www.origsoft.comwww.origsoft.com | © Original Software 11 Agile Testing – Crispin & Gregory Test Automation – Agile fusion Automated & Manual Manual AutomatedTools Technology-Facing Supporting the Team Business-Facing Q3 Q1Q4 Q2 Critique Product Unit Tests Component Tests Performance & Load Testing Security Testing “ility” Testing Functional Tests Examples Story Tests Prototypes Simulations Exploratory Testing Scenarios Usability Testing UAT Alpha / Beta

12. www.origsoft.comwww.origsoft.com | © Original Software 12 Test Automation – Agile (ish) fusion No one admits to doing pure Waterfall anymore Define TA scope in Release Test Plan Automated & Manual Scenarios Prepare error-handling (new and regression) Record carefully Replay CM Test Automation activities: Project Initiation Test Strategy Analysis Design Develop Testing Deploy Test Scenarios Test Cases Test Scripts Test Results Test Summary

13. www.origsoft.comwww.origsoft.com | © Original Software 13 Deep-dive into Testing Artifacts

14. www.origsoft.comwww.origsoft.com | © Original Software 14 Test Strategy / Test Plan Not listing automation for accomplishing the testing on schedule Omitting variables: Resources, Data, Objects, Licenses, Environments Omitting automation How the tool selection was done and how testers will use this To include automation in every iteration / build To obtain buy-in for automation

15. www.origsoft.comwww.origsoft.com | © Original Software 15 Test Stories / Test Scenarios Planning at a wrong level for automation Ambiguous areas: Usability, Reliability, Maintainability, etc. Topics not automatable To define the ‘what’ along with ‘how’ Peer reviews to ensure accuracy in automation To parse candidates

16. www.origsoft.comwww.origsoft.com | © Original Software 16 Test Scripts / Test Cases Planning automation until late in schedule Delaying automation creation & execution at start of schedule/sprint Automation as a finality Frequent automated test runs from the onset End-to-end regression test runs for changes Better Tests; Shift-Right

17. www.origsoft.comwww.origsoft.com | © Original Software 17 Retrospectives / Test Summary The notion that TA is like a manual test case Not scheduling a maintenance window Not doing tweaks TA metric reviews per iteration / sprint To automate completed manual tests To improve automation

18. www.origsoft.comwww.origsoft.com | © Original Software 18 Case Study

19. www.origsoft.comwww.origsoft.com | © Original Software 19 Background & Solution Titled: Automation Through the Back Door (Manual Test Support) Background: To improve the rate of automation, changes to the TA framework were required to support manual testing Technical Solution: Develop framework called command-driven testing What is Command-Driven Testing? Uses simple command keywords (SELECT, BUTTON) Interpreter scripts are the same for all product Using advantages from Data-Driven testing, navigation was in a DRIVER-File; data in a DATA-File Two files built together form a command script Sequentially, the script-runner reads the commands in the DRIVER-File. DATA-Codes are substituted with data from the DATA-File With Command-Driven testing: Testers don’t need to learn tool scripting Separation in navigation & data make command scripts flexible & reusable DRIVER-Files can be ported to different apps & can migrate to another tool The test tool is needed only to prepare the templates and to run the tests DRIVER-File GOTO, Customer Window INPUT, FirstName, INPUT, Name INPUT, Address INPUT, Bank Account SELECT, Button, OK CLOSE DATA-File, “John”, “Doe”, “1 Main Dr.” Script-Runner Loop thru statements GOTO Interpret & execute INPUT Interpret & execute SELECT Script-Runner Test Tool Test Execution Experiences of Test Automation – Graham & Fewster

20. www.origsoft.comwww.origsoft.com | © Original Software 20 Step 1 Record the TC with the capture functionality of automation solution Step 2 Translate generated script to command-script Step 3 Develop TC’s in template and build test suites Process for Implementation Experiences of Test Automation – Graham & Fewster Prerequisites: 1)Can’t start without knowledge of application under test and test cases to be automated 2)Need working engine to interpret commands for all test cases 3)All GUI elements used in execution must be registered by the capture / replay tool for object name normalization Prereqs 1.Test Cases 2.Command Library 3.Registration of GUI Controls STEP 1 Automator →Capture→Tool Script STEP 2 Tool Script ↓ Machine ↓ Command Script STEP 3 Command Script →Tester→Suite Automated Test Suite ↓ Test Engine ↓ Test Results

21. www.origsoft.comwww.origsoft.com | © Original Software 21 Case Study - Key Points Command –Driven framework was adopted for manual testing as well Helped limit testers working on same template concurrently Defect reporting was much easier with rework to recreate the issue Constant reviews to assess ongoing needs for automated and manual testing Implementation supported partial automated test execution; for test prep tasks Manual tests now focused on customer-specific conditions instead

22. www.origsoft.comwww.origsoft.com | © Original Software 22 Presentation Recap

23. www.origsoft.comwww.origsoft.com | © Original Software 23 Recap of the Presentation Outline your process by phase and deliverable; insert automation often Research industry frameworks and apply best practices Fuse automation topics into each testing phase with corresponding deliverable Delve into your testing process to plan and avoid topics around automation Present a real-life case study to management to build your case

24. www.origsoft.comwww.origsoft.com | © Original Software 24 Thank you for attending this presentation! Jim Trentadue Software Quality Consulting Director Original Software jtrentadue@origsoft.com