MBML_Efficient Testing Methodology for Machine Learning Sastry Bvssrs Senior IT Quality Analyst Nitin Upreti Project Manager Optum

..then the Strategies… First some concepts… ...then Conclusion. ABSTRACT Machine Learning is the industry buzz word since past two decades. As the data is spread across in a non homogeneous manner We need to perform effective Testing to ensure the correctness of data and the functionality of the machine which is performing the analytics. Few of the Applications functionality deal with understanding and responding to the properties of data sets and their interactions and validating those applications will be a tedious task for Testing Teams. This presentation outlines how we can perform Test Efficiency & derive QA strategy by defining Efficient Testing Methodology for Machine Learning. ..then the Strategies… First some concepts… Model Based Machine Learning Business Challenges What is Machine Learning? How Data is Spread? Testing Framework for Machine Learning (conceptually, Technically, Team & Style) ...then Conclusion. Benefits to business Metrics

CHALLENGE In a recent IT Economic survey*, just 62 percent of functional Testing is covered in traditional Model Performance Testing & security Testing are a barrier in Traditional Testing Traditional Testing has limitation in terms of complex testing with interactions Defects are identified at later stages which incurs in huge cost to fix the issue * - Source: Informa

WHAT IS Machine Learning? Machine learning is the subfield of computer science that, according to Arthur Samuel, gives "computers the ability to learn without being explicitly programmed.“ Source: Wikipedia

Traditional Testing

How Data is Spread?

How Machine Learning is Treated?

Deriving Models in Machine Learning

Model Representation of Requirements/Modules Testing Framework Model Representation of Requirements/Modules Perform Ranking of various paths to Perform Test Execution Perform Context Driven Testing Automation of Modules Perform Perspective Analytics using R- Language

Test Coverage Index Factors Contributing to Test Coverage Index are: Test Scenarios Generated Requirement Traceability Matrix Mapping across Requirements Depth of the Model Positive & Negative Scenarios Test Optimization

Test Maturity Index Factors Contributing to Test Maturity Index are: Skill Set of the Team Trainings Attended Certifications Test Planning Test Design Test Execution Defect Management

Automation Maturity Index Factors Contributing to Automation Maturity Index are: Automated Regression Bed Usage of Automation Tools Performance Measures Automation Penetration

BENEFITS TO BUSINESS Efficient Test Case Optimization $$ Structured Testing Approach Efficient Automation Approach Automatic Test Case Generation to the exhaustive Level Less Expensive Rework Reduction in Production Defects due to early defect Detection Efficient Analytics as a whole Effective Test Scenario Mining using Model Paths $$ $ $$$ Model Based Testing provide more options for Exploratory Testing Speed Up to 50% increase in speed to market through streamlined software delivery Test Coverage Increased Test Coverage and reduced manual Effort due to Model Based Testing Reduced defects in later test phases by up to 30% and overall higher quality Operational state is more stable and secure and changes are systematically auditable DELIVERY RISK RESILENCY

Author Biography   B V S S R S Sastry works as a Senior IT Quality Analyst at Optum Global Solutions, Hyderabad and have 5+ years of experience in Integrated Test Environment Services, ITIL & CMMI Level 5 assessment Process Improvement in Insurance domain. He holds a Master’s Degree in Technology from JNTU Hyderabad. He is a certified ISTQB professional, ITIL V3 certified. He has published 17 International white papers till date. Nitin Upreti works as a Project Manager at Optum Global Solutions, Gurgaon. He has a vast experience of 14 + years in the IT industry with expertise ranging from White Box testing, Mainframes, Web technologies and transforming Legacy applications into Big data based solutions. He is a certified professional from ISTQB and the Academy of Health care Management, USA. He Holds a Post Graduate Diploma in Business Management along with a Bachelor degree in Computer Applications.

Thank You!!!