Property of Progressive Insurance & Casualty Company 1 Agile Test Automation Created and Modified By: Nishant Awasthi January 13, 2009

Property of Progressive Insurance & Casualty Company2 Agile Test Automation Treat testware as software Application of agile development principles to the test automation project.  Requirements modeling.  Automation architecture modeling.  Acceptance testing of automated scripts.  Time Boxed iterations with frequent delivery of testware.

Property of Progressive Insurance & Casualty Company3 Agile Automation Process… contd.. Continuous integration of testware  Running tests with the latest development build Continuous integrated testing  Execute tests daily in the Development and QA environment  Provide continuous feedback Lock-step with the development and the QA team for iterative releases of testware

Property of Progressive Insurance & Casualty Company4

5

6

7

8 Automation Ceremonies NameFrequencyAgenda Automation Planning Meeting (APM) Every other week Following Development Iteration planning meeting. 1)Review automation progress 2)Reprioritize any existing test cases as needed 3)Review test cases prioritized for the current iteration a.Overview of functionality b.Determine automatability c.Determine dependencies (script modification, new development etc.) d.Change Automation Status as needed e.Assign automation developer to analyze the test plan and create a task in Automation Tracker. Automation Release meeting (ARM) Every other week Following Development iteration releases. 1)Determine velocity and burn down. 2)Determine current workload. 3)Synchronize scripts for release. Automation AcceptanceAd Hoc, as and when required. 1)Synch up with QA analyst and get feedback on the behavior and functionality of the script. 2)Mark the script for elevation in the daily run of automated regression Testware.

Property of Progressive Insurance & Casualty Company9

10 Automation Prioritization AttributeDescription Business Risk of Not Automating TestIf one script will be testing a higher-risk feature, it should be assigned a higher priority. Number of Planned ExecutionsIf a script will be executed more times, it should be assigned a higher priority. Efficiency (Automated vs. Manual)If one script will eliminate more manual testing it should have a higher priority. Volatility of Features Being TestedIf one script is testing a more volatile feature, it should be assigned a lower priority, since it will require more revisions due to the volatility. Dependency of Other TestsIf a high priority task depends on the completion of the candidate task, then the candidate task should also be assigned a high priority. Reuse PotentialScripts which can be reused should have higher priority than one-off scripts. Open DefectsScripts which test an open defect should be prioritized according to the severity/priority of the defect. AutomatibilityIf a test cannot be automated it should receive a low priority (at best). If it will take significant effort to automate, especially in relation to the benefit to the automation, it should also receive a low priority. UsabilityIf a script tests usability as opposed to proper functioning of the application, it should receive a lower priority.

Property of Progressive Insurance & Casualty Company11

Property of Progressive Insurance & Casualty Company12 Model based test script design Model : description of a system’s behavior  Simpler than the actual system  State modeling is a common practice in embedded systems world Finite State Model : consists of a set of states, a set of input events and the relations between them. Generate Sequences of Test Actions from the FSM

Property of Progressive Insurance & Casualty Company13 Model based test script …contd… Application behavior model  Exploratory modeling  Walkthroughs  Paired exploration  Capturing states of the application  When is this action possible?  What is the outcome of this action? Automated test action sequences

Property of Progressive Insurance & Casualty Company14 Automation Steps 1) Test Plan Analysis Analyze and refactor the manual test plan. Group similar test conditions. 2) Script design and development Modularize your script based on the different states or state transitions your application Keep a single flow either positive or negative in each action that you record. Return back to the initial state in your script. Record the flow you are automating. Try to add all the objects on a particular page from the active screen. Use smart identification or vary the object recognition properties if required. Playback your script Repeat these steps for the negative flow 3) Parameterization Use the data table of QTP for positive data parameterization. Repeat the parameterization for negative data. 4) Checkpoints Use custom and built in checkpoints 5) Script Elevation Get your script acceptance tested and reviewed Elevate it to Test Director

Property of Progressive Insurance & Casualty Company15

Property of Progressive Insurance & Casualty Company16

Property of Progressive Insurance & Casualty Company17

Property of Progressive Insurance & Casualty Company18 Roles and Responsibilities QA AnalystAutomation DeveloperApplication Developer 1)Mark automation status pending after the test plan is created. 2)Prioritize the test plan with respect to need for automation 3)Review and walkthrough the prioritized test plan to the automation developer. 4)Notify the automation developer working on the test plan if there are any changes. 5)Provide feedback to the automation developer after the automation status is “Ready for Release” 1)Pick up test plans that says “Creation complete” prioritized and assigned. 2)After picking up the test plan change Automation Status to In Process and Update Automation Comments field with name/time stamp. 3)Estimate and define task(s) in Automation tracker (task planning system) 4)Create automation script. 5)Synch up with QA analyst and get feedback on the behavior and functionality of the script after developing the script. 6)Mark the test plan as ready for release after getting it reviewed. 7)Release the test script into the daily run of “Automated Regression Testware” and mark automation status as completed. 8)Facilitate automation ceremonies. 1)Keep automation team in loop while making UI changes. 2)Help automation team with controls and technical details of the app as and when required.