1. Continuous Automated Chatbot Testing
Kiran Kumar Ponugoti - Sr Test Architect
V. Karthik – Sr. Solution Architect

2. Quality, Service, Performance and Customer Satisfaction
Abstract
Artificially intelligent computer programs that can “chat” back in response to a human, called chatbots, are virtual agents or assistants. They replicate the effectiveness of the best agents, to enable digital self-service.
Chatbot Landscape
Optimize
Quality, Service, Performance and Customer Satisfaction
Support
Customer’s Needs

3. Chatbot Testing with Continuous Integration
Deployment tool
Test data generator

4. Natural Language Processing
Chatbot Workflow & Testing scope
UI Testing
Compatibility
Functional
Accessibility
Mobile
Unit Testing
API Testing
Security
Performance
Data transformation
Data base
#1 User input a Phrase at Chat Client
#6 Chatbot packages data into proper response for display at chat client
#4 Intent is used to call upon proper service, using entity information to find proper data
#2 Chatbot sends the phrase to a machine learning NLP engine
#5 Data is returned to Chatbot
#3 NLP engine extracts users intent and entities from given phrase and sends back to chatbot
Chatbot
Chat Client
Natural Language Processing
Data Services

5. Security & Performance Other Areas Intelligence Quotient
Chatbot Testing - Testable components
Conversational Flow
Ensure a Problem-Solving Approach
Check Context Awareness
Add Alternative, Yet Valid, Response to the Same Text
Include Pleasant Responses to Erroneous User Inputs
Include Help Features
User Experience
The Onboarding Process
Using Generic Templates
Balancing Out Text Length
Ensure Correct Link Navigation
Including Feedback Options
Security & Performance
Masking Sensitive Data
Testing for Outages
Penetration Testing
Response Time / Performance Test
Simulate Backend Load
Other Areas
Multiple Channel Compatibility
Web Services / Microservices
Orchestrated Testing
Mobile, Web & App
Server, Middleware & Backend
3rd Party Systems / API’s
Intelligence Quotient
Sharpen Your Bot’s Memory
Enhance Bot’s Ability to Validate User’s Intent

6. Challenges & Solutions:
Automation Solution
Test Data Generation and management
Virtualize / Mocking of test data where ever possible
Multi layer Access points, authentication areas and data Security
Framework will Encrypt and decrypt secured test data flow
Test Usability, Compatibility, Accessibility, Platform dependency and Linguistic challenges
Framework handlers address browsers, devices, platforms, OS, Different ways of inputs, Accessibility & data validation
Integration with Customer facing application like, Mobile App, Website, Call Centre, Support Portal and perform End 2 End testing
Framework handlers address testing like Integration, End 2 End, Mobile, Browser / OS / Platform Compatibility, Response Validation by DB Testing & NLP validations
API Integration & Add On Services testing like,
Integrate with existing Backend services and validate Endpoints.
Speech to Text, Text to Speech, Alchemy Language, Tone Analyzer conversation validations
API Automation solution will Validate all possible endpoints request and response Input / Output schemas. Different modes of input Data conversion and processing validation scenarios are validated.
Continuous integrations of automated testing and build management
Automation solution will integrate to Deployment tool to support Continuous integration and continuous development approach with seamless testing
Variance in test scenarios, test data and Platform independent Framework
Platform independent Framework to Handle Data variance scenarios with same code base

7. Benefits
Better confidence on your Chatbot to address customer queries
Faster Releases of different Chatbots for specific purposes
Reduced cost of testing by simplifying the complete testing process
Reduced Risk of deploying Chatbots in production
Better Business by enhanced customer experience

8. Author Biography
Total 17 Years of IT experience.
Currently working as Senior Test Architect, with responsibility of Tech Support, Delivery Technical Support, Design UI Automation Frameworks.
Total 15+ years of experience in Software Engineering GUI Automation, Performance, Accessibility Testing and Software Engineering Tools.
Qualification : B.Tech (cse), MBA (IT)

9. Thank You!!!