1. AI and Software Testing
Vikram Raghuwanshi
Senior Consultant
Capgemini Technology Services

2. ABSTRACT
This presentation gives an Introduction to Artificial Intelligence; and how AI touches our lives on day to day basis. It talks about the usage of AI in QA. It explains that how to deploy AI Systems for QA, and what benefits can be expected out of this. This will help Test Managers to decide if using AI techniques for software testing; will be helpful to their organization

3. What is AI
When Machines and devices are capable of taking their own independent decision, we say that these machines and devices exhibit Artificial Intelligence.
Figure: An AI Robot

4. Some applications of AI
Virtual Assistants
Driverless cars
Driverless Trains
Space Exploration
Security
Robotic Surgeons
Advanced algorithms used in Stock markets
Cyborg technology
Figure: Google Driverless car

5. How AI based systems work
AI based systems employ heuristic algorithms to generate analytical models based on historical Test data. It also generates Charts and Mind Maps. Based on these models, AI system takes decision in day to day Test activities. For example, it analysis defects already logged, prepares a model, and then selects tests more likely to find defects in subsequent releases of the Application Under Test.

6. Heuristic algorithms used in AI
The term heuristic is used for algorithms which find solutions among all possible ones, but they do not guarantee that the best will be found
Some heuristic algorithms used in AI systems are:
Fuzzy systems
Neural networks
Evolutionary computation

7. Models Used by AI Systems
Some models generally used by AI Systems are:
Naive Bayes
ARIMA
Decision Tree
Is it Sunny?
Yes No
Wear white clothes
Is it raining
Yes No
Wear Rain Coat
Wear normal clothes
Figure: Decision Tree

8. Prerequisites on using AI based system
Following are the Prerequisites:
Data Cleansing & Standardization
Test plan
Test Cases
Defects
Requirements Traceability Matrix
Any other related artifacts
Relevant access to data source
AI Engine needs access rights to various systems. For example, for ALM it needs query editor access, for TFS it needs developer access

9. Integrating AI Based System
Quite a Few vendors in open market offer AI solutions for Testing. Chose a System and do a POC. The system will have an API to communicate with your existing applications. Ask your development team / Vendor to do the integration.
It will involve integration of various tools, as depicted in the next slide

10. Integrating AI Based System
AI Based Tool
CI Tools
Build Tools
Database Systems
Test Automation Tools
Reporting Tools
Test Management Tools
Dashboard

11. Integrating AI Based System
AI tool will need a lot of historical data, so that it may generate Analytics Based on that Data
AI Tool
Test cases
Defects
Downtime Logs
Code Quality Report
RTM

12. Applications of AI in Software Testing
Test Plan Support
Chat Bots
An AI system can be built nowadays, to support software testing. It can have the features given alongside. It can have Chat Bots, which will chat with customers and users, collect feedback to help improve the software, and will also collect information about problems faced by the end users. The AI system will than log defects based on chat inputs
Test Suite Analysis
Defect analysis
RTM analysis
Exhaustive Reporting
Identification of Test Cases
Preserving Of test Artifacts
Continuous Integration
Machine Learning

13. Analytics Generated with AI Tool
AI tool gets various data feeds; it then gives critical predictive analysis driven at real time to the testing teams across project’s lifecycle assisting them to keep adjusting to various moving parameters influencing outcomes of the project. Analytics generated:
Assets analytics: Captures various metrics around analytics and provide recommendations on improving the leverage from individual assets. Analyses assets usage trend, execution duration, attached criticality through requirements, defects mapping etc.
Staff analytics: Assists understanding the distribution of capabilities, specific skills among the complete team (including client’s own staff and vendor resources), track real time progress being made on workforce transformation initiatives, predict resource needs for ongoing and upcoming programs upfront.

14. Analytics Generated with AI Tool
Customer analytics: Customer feedbacks from multiple social channels providing critical feedback for applications designs, indicating usage patterns for specific apps allowing testing teams to plan better around digital testing coverage.
Delivery analytics: Presents various analytics for testing delivery and trends to all possible levels and help testing teams to make decisive calls on testing plans in real time
Rightsizing solutions: Combination of various analytical utilities guiding testing organization to size their testing appropriately real time
Predictive analytics: Provides predictive views for all key dimensions associated with success of any project.

15. Benefits of Using AI in QA
Environment provisioning
Selecting Tests
Execution & Reporting
Early completion of Software Testing Life Cycle
Better coverage – Test suggested by the system shall ensure optimal code coverage
Will help in getting Quality software with less time and effort
Will help in creation of Reusable prediction models
Unattended execution – In case of failures during Test Execution, AI system shall execute recovery routines and achieve unaided execution
Will help in integrating Source Control Systems, Build Servers, environment subsystems, Test management software and Cognitive Systems.

16. Example of an actual AI enabled System for QA
Capgemini’s Smart QA is an analytics-driven platform that helps banks, financial institutions, and insurers build an end-to-end ecosystem, which includes testing assets, a test environment, test data and performance data
It has An analytics engine, which includes cognitive capabilities powered by smart techniques and insights which test architects can use to prioritize what needs to be tested, optimize testing efforts, and identify areas of improvement. Smart QA also knows how much testing is required and which assets should be used.

17. Smart QA ‘s Sample Dashboard
Testing summary view is given below. Defect status with respect to Project, Status, Severity, and Priority is tracked in various sections of this dashboard.

18. Smart QA ‘s Sample Dashboard
Defect & Execution Metrics view is given below. It represents the metrics related to Testing efficiency.

19. Conclusion and Recommendation
AI is next level of automation, lot of case studies already exist on the benefits of automation and how it saved time and efforts for enterprises, mostly in Regression suite execution. Case studies related to AI testing are not easily available in public domain yet. Vendors offering AI solutions, for example IBM, can definitely help in this regard.
We should appreciate the benefits AI is giving us and we should try to contribute in whatever way we can. Given the benefits, we should move towards using AI into Software Testing Life Cycle.

20. References
General search on AI on Google
en.wikipedia.org

21. Author Biography
Vikram is having 12+ years of rich IT experience. He has worked on support, S/W Development, S/W testing and Automated testing. He has keen interest in current & emerging technologies. His educational qualifications include BSc. ( PCM ) & MCA