1. Analytics driven testing
Anusha Mukkavilli – Associate Consultant
Nibhanupudi Suguna- Associate Consultant
Capgemini India Pvt. Ltd
Logo of your organization

2. Abstract
Digitalization is fundamentally changing the business climate. To be successful, organizations must develop and adopt agile business models that respond faster to changing market and economic conditions. To support this business shift, the software industry re-invented itself to allow for fast-paced software systems evolution. This evolution has been made possible by many innovative technologies, architectures and tools (such as Cloud infrastructures, Open platforms, Building Blocks Method, etc.) combined with new practices and methodologies
In this context, development cycles are increasingly shorter while testing resources often remain at best unchanged. As consequence, testing became a big challenge mainly for large and scale software systems. IT professionals seek for industrial ways to maintain the quality of their systems as whole while they continuously and independently evolve.

3. Analytics driven testing provides practical ways to industrialize a software testing approach based on software metrics which consists of gathering data from various metrics as inputs and then building a business intelligent system that shows the test gaps as outputs.
The Goal of analytics driven approach is to
Determine a quantitative measure of test coverage, which is imperative to measure quality.
Find untested areas of the software application.
Identify redundant test cases that do not increase test coverage.

4. Analytics driven approach in QA
Gives insights on the customer sentiments that help in customer-centric QA
Provides insights to start, stop, and prioritize testing
Increases testing efficiency and predictability
Reduces overall cost due to early defect detection
Accelerates time to market
Going beyond the traditional QA methodologies and taking an analytics-based approach has become key factor in the next generation of QA. This helps in predicting the future failure looking at the past data and taking the proactive measures for future.

5. Introduction
For many software testing firms to accelerate growth in competitive world it is imperative for them to transform their business models.
This is not possible until and unless the companies reinvent themselves to allow for fast-paced software systems evolution.
This evolution has been made possible by many innovative technologies, architectures and tools combined with new practices and methodologies which drastically shorten their time-to-market

6. What is Analytics? Analytics is defined as the process of measuring, recording, tracking, and analyzing data to study real-time patterns and usage of an application or a workflow.
Some of the key players in this field are Google Analytics, Yahoo Analytics and many others.
Analytics data gives you better performance insights and also helps you understand, prioritize and in decision making on what needs testing and optimizing first on your application/website

7. Challenges that complicate testing process
Different expectation in terms of KPI reports, Test results, Audit reports, Test management reports, other metrics, etc. 
Unable to produce desired and on-demand analytical reports
Is the project going in the right track?
Who is the right tester for this assignment?
Where exactly I am incurring more cost?
Will I be able to meet the deadline?
What measures should I take to accomplish the project with in the set deadlines?

8. World Quality Report

9. Advantages of Analytics driven approach in Software Testing
It helps in identifying right tester for particular task
It helps you dis­cover, mon­i­tor and set tar­gets for your key suc­cess met­rics
It helps in identifying issues impacting various areas of project
It helps in proactively identifying the risks and mitigating the risks at the earliest stage
It helps in identifying where the delay is and what is the issue
It helps in Improve Planning, Quality and Delivery
It Predictive Analytics helps in making right decisions at right time.

10. Gartner’s Hype Cycle for emerging trends in testing 2013

11. Analytics process 1. Analytics process components:
A communications platform.
A feedback system.
Training.
A text analytics engine.
2. Real time versus right time – information and insights must be provided at the right time and not necessarily continuously.
3. Insights – the process of interpretations must be standardized so that different managers come to the same conclusion after analyzing the data.

12. Way to develop and design a dashboard
Analytics toolbox
Descriptive analytics – creating simple counts, distributions, visualizations describing your data.
Predictive analytics – predicting organizational and process behaviour, e.g., can you predict at the beginning of a release how it will end?
Prescriptive analytics – prescribe corrective actions and suggest mitigations after identifying aberrations, risks, etc.
Way to develop and design a dashboard
Follow an accepted methodology
Specific process/goal in order to be useful.
Do not mix strategic metrics with operational metrics
Provide drill down capabilities to better focus on and target your issues.

13. Way to develop ‘meaningful’ metrics
Goals need to be defined based on organizationally adopted
quality frameworks (TMMI, ISO, IEEE, etc.).
Questions need to be defined so that their answers meet the goals.
Defined metrics are a natural outcome of the questions.
On which processes should organizations focus when considering the analytics initiative?
The three major quality processes that directly impact the analytics initiative are:
1. Requirements management
2. Test automation
3. Defect management

14. How can we get organizational and management support for the QA analytics initiative?
The organization defines a framework that drives the QA improvement function. Management understands that their success is directly proportional to the quality of the delivered applications. Management always needs to balance between committing resources to deliver quality applications and lowering QA costs. Budgets are justified by continuously demonstrating the quality of application delivery through efficient quality processes.

15. Business benefits-Real world example
Tricentis Tosca a small Austrian startup raised $165M to deliver continuous testing at the speed of DevOps
Tricentis being named a Leader in Gartner’s Magic Quadrant for Software Test Automation for the second year in a row. According to the Forrester report, “Tricentis Tosca provides top test automation and optimization design capabilities, test assets reuse, and combined automation.”

16. Conclusion 
There are many heuristic approaches which are emerging to push the boundaries beyond pure analytics leading to continuous and touch less automation at the speed of thought. For example, MBT (model based testing) and Test Coverage, PBT (Theorem Proving Based Testing), AI, Machine Learning, Deep learning etc.
The proposed approach proves to be not only effective in saving manual effort, time, and energy but also in helps in reducing the testing costs and deriving better ROI early in the testing life cycle. Ultimately, it improves efficiency and effectiveness of the testing operations.

17. References & Appendix  
211.html 

18. Author Biography
Anusha Mukkavilli
IT professional with 4+ years of work experience, currently working for Progressive Insurance account
Nibhanupudi Suguna

19. Logo of your organization
Thank You!!!
Logo of your organization