ADT (Analytics Driven Testing) Quality & Agility through Data Driven Decisions Vinod Sundararaju Antony, Director – Projects Jancy Emima Alphonse, Senior Manager – Projects Ankur Joshi, Manager – Business Development Cognizant Technology Solutions

Abstract The Need Multitude Products across various Channels/Devices Organisations focus on bringing right competitive advantage Direct testing effort onto the right focus areas The Solution Analytics Driven Testing (ADT) focuses on “Right” (through internal analytics) “Relevant” (through external analytics) To bring in benefits of Optimised Coverage Superior Quality Enhanced UX & Brand Image

Analytics Driven Testing - Approach Data & Analytics Analytics driven testing enabled by actionable insights Outcomes “Right” Product - Internal Analytics Optimised Coverage Code Coverage Defect Details Defect analytics Code Coverage Analytics Superior Quality With Reduced Cost Testers “Relevant” Product –External Analytics Enhanced User Experience & Brand Image Web & Mobile Analytics Social Sentiment listening / Analytics

Pass % of Prioritized Tests Defect analytics Inputs to Defect Analytics Identify Defect Prone Areas Align QA based on insights Master business processes Analyze defect patterns Prioritisation of Functional, Regression Testing Historic Defects data with business process mapping Static Testing of focus areas Requirements & Impacted Business Process Identify Hotspots Defect Density Pass % of Prioritized Tests

Code Coverage Analytics Baseline Tests Manual/Automated Execution of impacted tests Tester Map code to tests Impacted test cases Check in code Deploy to QA Env. Identify impacted test footprints Developer Code Coverage %

Web & Mobile Analytics Identify type of analytics Collect & analyse metrics Align QA based on insights Page view/ visit stats Browser stats Clicks, impressions New vs return ratio Visit/ session duration Prioritise testing of business processes / pages that have higher traffic & conversion rate Server log file analysis Page tagging Focused Compatibility Tests Packet/ Network sniffing Derive Load volume patterns for Performance Testing % defect leakage on compatibility issues App store scores

Social Sentiment listening / Analytics Social Data Sources Social Listening & Analytics Improved Product UX Crawl app stores, social media sites Study social feedback Derive sentiment score Competitor Analysis Feedback is segregated into the preset attributes Findings form requirements for product Improved Test Coverage Analyze data using reporting tools Arrive at insights/additional validation scenarios Net Promoter Score (NPS) % of Customer feedback addressed

Tying Analytics to Business Processes Day in a life/Real time Scenarios Business process models Business Process Execution & Reporting Analytics based insights Business Processes Business Components & Scenarios Analyze key business processes through day in a life/real time scenarios Arrive at insights Identify process areas, create business requirement vs. business process traceability matrix Create business process models Identify key business components, rules and activities and test suites Integrate E2E business scenarios with testing plan/strategy Execute the E2E business scenarios Business Process Reporting

Manage ‘Just-Enough’ data for actionable insights Decide which Data to use Handle Analytics Leverage Insights Set Analytics Objective Understand customers better Improve products / services Improve fraud detection Better management of governance, risk & compliance Improve internal efficiencies Internal – Customer Transactions, internal supply chain, defects External - Website Usage, Social Media, Mobile Apps, open/ 3rd -party data etc. Finalise technology, tools, & data sets Set guidelines for stakeholder differentiated communication Analyze data Bring actionable insights into Product Design Align Analytics insights to business processes Use insights to add tests & improve product testing

References & Appendix Becoming an analytics driven organisation to create value - http://www.ey.com/Publication/vwLUAssets/EY-global-becoming-an-analytics-driven-organisation/$FILE/ey-global-becoming-an-analytics-driven-organisation.pdf Press Release : http://www.gartner.com/newsroom/id/3198917 7 Common Reasons Users are Abandoning your App - http://www.webanalyticsworld.net/2016/08/why-users-are-abandoning-your-mobile-app.html Making data analytics work: Three key challenges : http://www.mckinsey.com/business-functions/business-technology/our-insights/making-data-analytics-work

Author Biography Vinod Sundararaju Antony - Vinod has 14 years of experience with Software Testing and Quality Assurance and has played roles such as Global Test Delivery Manager, Quality Consultant, and Transformation Consultant. Jancy Emima Alphonse - Jancy has 12 years of experience with Software Testing and Quality Assurance and has played roles such as Test Program Manager, Quality Consultant, and Transformation Consultant. Ankur Joshi – Ankur has 7 years of experience in Software Quality Assurance and has played roles such as Business Development Manager and SDET.

Thank You!!!