1. 10 Point Success Strategy in Crowd Testing
Sree Ram Prasad, Manager
Nilesh Kumar, Sr. Consultant
Deloitte Consulting

2. Abstract Agenda Crowd Testing – Brief Refresher
With Increasing digitization in every aspect of life, breakneck pace of development, ever shortening shelf-life of mobile apps, and more focus on user experience, it is more pertinent than even to look at Crowd Testing as our best ally in Quality Control.
To set the context, we’ll briefly touch upon Crowd testing in general, benefits of crowd testing and types of testing which can be targeted through Crowd testing, and some of the major challenges.
We intend to presents a 10 point agenda to overcome all the major challenges, and introduce innovative techniques to reap definite success from Crowd Testing projects.
Agenda
Crowd Testing – Brief Refresher
Benefits of Crowd Testing
5 Drivers for Crowd Testing
Targeted Testing Types
Major Challenges
Success Strategy
Caution with Crowd Testing
References & Appendix
Author Biography

3. Crowd Testing – Brief Refresher
Collective Cognizance
Real World Testing
Diversity
Focus on consumer experience
Localization
Cost Effective
Easy to scale up or down
Shorter time to market

4. 5 Drivers for Crowd Testing

5. Diversified Tester Base Collective Cognizance
Benefits of Crowd Testing
Diversified Tester Base
Extensive Coverage
Cost Effective
Faster Turnover
Collective Cognizance
Feedback

6. Targeted Testing Types
Localization Testing
Performance Testing
Beta Testing
Security Testing
Web/Mobile
Testing
Usability and Functional Testing

7. Major Challenges in Crowd Testing

8. 10 Point Success Strategy
Train the Tester
Recorded Demos
Mock Screens in Design Document
Micro-learning 1
Focus on Coverage
Crowd Re-orientation
Real Time Coverage Report
Crowd Feedback to Drive 2

9. 10 Point Success Strategy cont.…
Communicate to Succeed
Chat-bots to Answer Generic Queries
Busting Language Barriers
Multiple Channels to Disseminate Information
Maintaining Confidentiality & Building Trust
Non-Disclosure Agreements
Meetups
Four Colour Ethics Meter 3 4

10. 10 Point Success Strategy cont.…
Motivated Testers
Interactive and Engaging Test Management
Reward Bonus Points for Active Users
Crowd Contests 5
Address Technological Challenges
Version Control
Test Management
Confidentiality and security
Feature Toggle, App vs APIs, Web Hosted App 6

11. 10 Point Success Strategy cont.…
Mute False alarms
Crowd Awareness Surveys
Crowd Evaluation Matrices
Correct Test Data Sample Space
Defect Logs and Diagnostic report
Mandatory Execution Evidence
Deriving value from Bugs
Defect Guidelines
Pro-active Exception Handling
Bug Revalidation
Value Proposition of Bug
Analytics Based Defect Assessment 7 8

12. 10 Point Success Strategy cont.…
Intelligent Crowd Sampling
Qualifier Round for Crowd
Analytics Based Crowd Sampling
Crowd Skills Survey
Device Verification 9
Cohesive Team
Inclusion
Brand Loyalty
Long Term Relations
Crowd Satisfaction Survey 10

13. Caution with Crowd Testing
Crowd Testing is not an ultimate solution to all the testing needs.
Engagement in Crowd Testing should be exercised with utmost caution, or be rather avoided when
Dealing with highly sensitive and confidential data
Possibility of security compromise and intellectual property theft
Possibility of Exposure of critical and core business logic
Unrealistic expectations from Crowd Testing
Automation has better ROI than Crowd Testing

14. References & Appendix

15. Author Biography Sree Ram Prasad Bhatiprolu Nilesh Kumar
Sree Ram Prasad Bhatiprolu
Manager, Deloitte USI
Sree Ram Prasad Bhatiprolu is Manager with Deloitte Consulting USI, Hyderabad, India. Sree Ram is certified scrum master with 14 years of IT experience, worked across LSHC (Life sciences and Health care), TMT (Technology, Media & Telecommunications) and FSI (finance, securities and insurance) industries. Sree Ram holds Masters degree in software engineering from BITS Pilani. He has about 14 years of experience in leading, managing and delivering testing engagements. He has extensive experience in Automation, Integration Testing, Functional Testing and Performance and Testing.
Sree Ram Prasad Bhatiprolu
Nilesh Kumar
Sr. Consultant, Deloitte USI
Nilesh Kumar is a Sr. Consultant with Deloitte Consulting USI, Hyderabad, India. He has over 7 years of experience in Software Quality and Assurance. Nilesh has worked on various automation testing tools like QTP, Selenium, RFT and various test management tools. He is ISTQB and PAHM certified professional.
Nilesh Kumar

16. Thank You!!!