1. True ROI of Automation? Allscripts India LLP
Sarbajit Sahani, Expert Software Engineer
Raushan Raj, Associate Software Engineer
Allscripts India LLP

2. Abstract
Measurement of ROI on automation is a keen-witted situation. Deriving ROI can be obtained during various stages of SDLC such as development, client testing and maintenance. It is also important to consider the size and nature of AUT.
Most importantly ROI should be re-visited periodically. If we can calculate it by using a mathematical formula then it will be more acceptable.

3. Method
One approach to calculating the ROI is to use a framework called ‘Value Based Framework’, which calculates the ROI of automation testing using a mathematical formula which is based on a number of factors.

4. ROI Calculation By Using Value Based Framework
Factors Considered:
Application Details
Product Size – Number of Screens or Functional Points.
Product Maturity – In Development Stage or a live product.
Test Objective
Presentation Layer – UI Automation by using a tool.
Middle Tier – Server Layer Automation directly dealing with Class Files.
Database – Service Layer or Database testing by considering SPs and GPs.
Automation Cost
Development – Cost of developing the Automation Test Cases.
Execution – Cost of running the automated test cases.
Maintenance – Cost of Maintaining the automated test cases.
Environment – Cost of Maintaining Environments.
Automation Life Span
Test Life Span – For how many runs the test will add value.
Functionality Life Span – For how many days testing the functionality will add value.
Frequency of Regression

5. Value Based Framework ROI = (M * R) – ((C * F) + R * Mc)
M - Manual Testing Cost
R - Number of Runs
C - Cost of Automation
F - Factors need to consider for Automation
Mc - Automation Maintenance cost

6. Value Based Framework M – Manual Testing Cost
Cost Involved for manual testing which includes test case writing, execution etc. Cost which can be avoided for the test cases which is planned for automation

7. Value Based Framework R – Number of Runs
Number of runs considered to calculate ROI for Automation

8. Value Based Framework C – Cost of Automation
UI Automation – 0% to 100%
Server Layer Automation – 0% to 100%
Hybrid Automation – 0% to 100%
% of Value addition with respect to Automation Development Cost

9. Value Based Framework F – Factors need to consider for Automation
Stage Of Automation
In Sprint Automation – 0% to 100%
After GA Automation – 0% to 100%
% of Value addition with respect to Automation Development Cost
Type of Automation
Pre-Requisite Creation – 0% to 100%
Single flow for multiple Iteration – 0% to 100%
Localization Testing – 0% to 100%
% of Value addition with respect to Automation Development Cost

10. Value Based Framework F – Factors need to consider for Automation
Business Complexity
High Complexity – 0% to 100%
Medium Complexity – 0% to 100%
Low Complexity – 0% to 100%
% of Value addition with respect to Automation Development Cost
Customer Complexity
High Complexity – 0% to 100%
Medium Complexity – 0% to 100%
Low Complexity – 0% to 100%
% of Value addition with respect to Automation Development Cost

11. Mc – Automation Maintenance cost
Automation Tool
Environment
Automation Training
Result Analysis
Script Maintenance

12. Tool to generate ROI of Automation

13. Tool to generate ROI of Automation

14. References & Appendix

15. Sarbajit Sahani Raushan Raj Author Biography:
Lead automation engineer at Allscripts instrumental for strategizing, designing and implementing effective test automation for multiple Products. Over 12 years’ industry experience in feasibility study, framework design and automation testing.
Connect me at LinkedIn:
Co-Author Biography:
Raushan Raj
New entrant to automation world with less than two years of industry experience but unbound energy and passion for automation makes me question everything and try every tool in town.
Connect me at LinkedIn:

16. Thank You!!!