True ROI of Automation? Allscripts India LLP Sarbajit Sahani, Expert Software Engineer Raushan Raj, Associate Software Engineer Allscripts India LLP

Abstract Measurement of ROI on automation is a keen-witted situation. Deriving ROI can be obtained during various stages of SDLC such as development, client testing and maintenance. It is also important to consider the size and nature of AUT. Most importantly ROI should be re-visited periodically. If we can calculate it by using a mathematical formula then it will be more acceptable.

Method One approach to calculating the ROI is to use a framework called ‘Value Based Framework’, which calculates the ROI of automation testing using a mathematical formula which is based on a number of factors.

ROI Calculation By Using Value Based Framework Factors Considered: Application Details Product Size – Number of Screens or Functional Points. Product Maturity – In Development Stage or a live product. Test Objective Presentation Layer – UI Automation by using a tool. Middle Tier – Server Layer Automation directly dealing with Class Files. Database – Service Layer or Database testing by considering SPs and GPs. Automation Cost Development – Cost of developing the Automation Test Cases. Execution – Cost of running the automated test cases. Maintenance – Cost of Maintaining the automated test cases. Environment – Cost of Maintaining Environments. Automation Life Span Test Life Span – For how many runs the test will add value. Functionality Life Span – For how many days testing the functionality will add value. Frequency of Regression

Value Based Framework ROI = (M * R) – ((C * F) + R * Mc) M - Manual Testing Cost R - Number of Runs C - Cost of Automation F - Factors need to consider for Automation Mc - Automation Maintenance cost

Value Based Framework M – Manual Testing Cost Cost Involved for manual testing which includes test case writing, execution etc. Cost which can be avoided for the test cases which is planned for automation

Value Based Framework R – Number of Runs Number of runs considered to calculate ROI for Automation

Value Based Framework C – Cost of Automation UI Automation – 0% to 100% Server Layer Automation – 0% to 100% Hybrid Automation – 0% to 100% % of Value addition with respect to Automation Development Cost

Value Based Framework F – Factors need to consider for Automation Stage Of Automation In Sprint Automation – 0% to 100% After GA Automation – 0% to 100% % of Value addition with respect to Automation Development Cost Type of Automation Pre-Requisite Creation – 0% to 100% Single flow for multiple Iteration – 0% to 100% Localization Testing – 0% to 100% % of Value addition with respect to Automation Development Cost

Value Based Framework F – Factors need to consider for Automation Business Complexity High Complexity – 0% to 100% Medium Complexity – 0% to 100% Low Complexity – 0% to 100% % of Value addition with respect to Automation Development Cost Customer Complexity High Complexity – 0% to 100% Medium Complexity – 0% to 100% Low Complexity – 0% to 100% % of Value addition with respect to Automation Development Cost

Mc – Automation Maintenance cost Automation Tool Environment Automation Training Result Analysis Script Maintenance

Tool to generate ROI of Automation

Tool to generate ROI of Automation

References & Appendix https://www.infoq.com/news/2009/04/testing-roi

Sarbajit Sahani Raushan Raj Author Biography: Lead automation engineer at Allscripts instrumental for strategizing, designing and implementing effective test automation for multiple Products. Over 12 years’ industry experience in feasibility study, framework design and automation testing. Connect me at LinkedIn: https://www.linkedin.com/in/sarbajit-sahani-b8b37887 Co-Author Biography: Raushan Raj New entrant to automation world with less than two years of industry experience but unbound energy and passion for automation makes me question everything and try every tool in town. Connect me at LinkedIn: https://www.linkedin.com/in/raushan-raj-a4b7b536/

Thank You!!!