Continuous Automated Chatbot Testing Kiran Kumar Ponugoti - Sr Test Architect V. Karthik – Sr. Solution Architect

Quality, Service, Performance and Customer Satisfaction Abstract Artificially intelligent computer programs that can “chat” back in response to a human, called chatbots, are virtual agents or assistants. They replicate the effectiveness of the best agents, to enable digital self-service. Chatbot Landscape Optimize Quality, Service, Performance and Customer Satisfaction Support Customer’s Needs

Chatbot Testing with Continuous Integration Deployment tool Test data generator

Natural Language Processing Chatbot Workflow & Testing scope UI Testing Compatibility Functional Accessibility Mobile Unit Testing API Testing Security Performance Data transformation Data base #1 User input a Phrase at Chat Client #6 Chatbot packages data into proper response for display at chat client #4 Intent is used to call upon proper service, using entity information to find proper data #2 Chatbot sends the phrase to a machine learning NLP engine #5 Data is returned to Chatbot #3 NLP engine extracts users intent and entities from given phrase and sends back to chatbot Chatbot Chat Client Natural Language Processing Data Services

Security & Performance Other Areas Intelligence Quotient Chatbot Testing - Testable components Conversational Flow Ensure a Problem-Solving Approach Check Context Awareness Add Alternative, Yet Valid, Response to the Same Text Include Pleasant Responses to Erroneous User Inputs Include Help Features User Experience The Onboarding Process Using Generic Templates Balancing Out Text Length Ensure Correct Link Navigation Including Feedback Options Security & Performance Masking Sensitive Data Testing for Outages Penetration Testing Response Time / Performance Test Simulate Backend Load Other Areas Multiple Channel Compatibility Web Services / Microservices Orchestrated Testing Mobile, Web & App Server, Middleware & Backend 3rd Party Systems / API’s Intelligence Quotient Sharpen Your Bot’s Memory Enhance Bot’s Ability to Validate User’s Intent

Challenges & Solutions: Automation Solution Test Data Generation and management Virtualize / Mocking of test data where ever possible Multi layer Access points, authentication areas and data Security Framework will Encrypt and decrypt secured test data flow Test Usability, Compatibility, Accessibility, Platform dependency and Linguistic challenges Framework handlers address browsers, devices, platforms, OS, Different ways of inputs, Accessibility & data validation Integration with Customer facing application like, Mobile App, Website, Call Centre, Support Portal and perform End 2 End testing Framework handlers address testing like Integration, End 2 End, Mobile, Browser / OS / Platform Compatibility, Response Validation by DB Testing & NLP validations API Integration & Add On Services testing like, Integrate with existing Backend services and validate Endpoints. Speech to Text, Text to Speech, Alchemy Language, Tone Analyzer conversation validations API Automation solution will Validate all possible endpoints request and response Input / Output schemas. Different modes of input Data conversion and processing validation scenarios are validated. Continuous integrations of automated testing and build management Automation solution will integrate to Deployment tool to support Continuous integration and continuous development approach with seamless testing Variance in test scenarios, test data and Platform independent Framework Platform independent Framework to Handle Data variance scenarios with same code base

Benefits Better confidence on your Chatbot to address customer queries Faster Releases of different Chatbots for specific purposes Reduced cost of testing by simplifying the complete testing process Reduced Risk of deploying Chatbots in production Better Business by enhanced customer experience

Author Biography Total 17 Years of IT experience. Currently working as Senior Test Architect, with responsibility of Tech Support, Delivery Technical Support, Design UI Automation Frameworks. Total 15+ years of experience in Software Engineering GUI Automation, Performance, Accessibility Testing and Software Engineering Tools. Qualification : B.Tech (cse), MBA (IT)

Thank You!!!