1. Addressing Test coverage in Continuous Testing
Barkha Tyagi – Test Architect
Sameer Chawla – Sr. Test Architect 1

2. Release times are reducing but “Test Coverage” is increasing
Abstract
Continuous Testing is on the rise and increasingly significantly every year
Waterfall
Agile
DevOps
Months / Years
Usability
Customer Experience
Time per Release
Weeks / Months
Multiple Devices
Security
Accessibility
Multiple Browsers
Performance
Application Requirements
Functional / Integration testing
Days / Weeks
Expansion in Test Coverage
Release times are reducing but “Test Coverage” is increasing

3. Analysis – Increasing test coverage in Continuous Testing (CT)
Technical users
Increased need of frequent changes
‘Mandate security and performance
Talent to test
Lack of skilled testers.
Inter personal and analyzing skills.
Rising digital platforms
Multiple browsers
Multiple devices are in market
How and when to start test
Need to maintain speed with balance
Squeezed timeline
More mind and more ideas.
If idea not floated on time it would be “stale” soon.

4. Production Environment
Snapshot: Handling test coverage challenges in CT
Product in Production
Crowd Source testing
Automation Focused
Regression Suite
Enhanced Talent
Automation
Mutation testing
Start early – Plan idle periods of team members
Continuous Integration
Continuous Testing
Dev Environment
Unit Testing
Test Environment
Integration Test / Functional Test / Performance Test / Multi device Test / Compatibility Test / Security Test / Regression Test
Production Environment
Deploy in Production / Feedback
Staging Environment
Acceptance Test
Product Backlog
Sprint Backlog
Rigorous practices with brainstorming sessions – to handle multiple environment issues

5. Automation is the integral part of Continuous Delivery
Techniques and Best Practices: Integrating automated testing in all CT phases to achieve delivery with quality on time
Build
Build creation process can be automated using different tools.
Automation is the integral part of Continuous Delivery
Develop
In this phase Unit and other required test cases automation will help
Integration
Jenkins, Marven Sonar etc. tools are available to automate integration process.
Functional, Security, Compatibility ,performance etc. test cases can be automated using for faster execution
Deploy
Build package creation and deployment in production can be done through tools only.

6. Pass Mutant 1 Fail Source Code Mutant 2
Techniques and Best Practices: Mutation Testing
Creating Mutants - Change your source Code by adding fake error in it.
Errors can be added by changing ‘Value, Definition or Statement”
Source Code
Mutant 1
Pass
Fail
Mutant 2
Run Test
Result: High quality test suite which may help in stable and reliable automation.

7. Techniques and Best Practices: Crowd Sourced Testing
Not a hired test team but the crowd is testing the application
Software is tested on diverse and realistic platforms.
Specific groups can be selected through the crowd to test software.
Can be useful for compatibility, browser, device and usability testing

8. Adopt new testing techniques “Focused” regression Suite
Lessons Learnt: Overcoming test coverage challenges in CT ..1/2
Automation
Adopt new testing techniques
“Focused” regression Suite
Automated test buckets for UI and API test cases help in saving time and covering more.
Mutation Testing
Crowd Source Testing
The “Focused” Test coverage, ‘Prioritization of test scenarios’ on the basis of revenue generation
User experience is a valuable approach that straightforwardly cops up with continuity of design, develop and test in CD.

9. Start early and work together Enhance talent Rigorous practices
Lessons Learnt: Overcoming test coverage challenges in CT ..2/2
Start early and work together
All team members should start work together.
Plan Idle periods of team members due to any dependency.
Effectively utilize time by starting test planning, including designing and testing
Automate test cases at the right time parallel to dev/unit coding phase.
Enhance talent
Work on manual power to bring manual testers close to the technical skills that is a key area to enhance test quality, coverage and reduce time to test.
Rigorous practices
Understand environment issues and set up rigorous practices with brainstorming sessions.

10. References & Appendix
Mutation testing information available on "
Two-Week Sprint: referanced from my own current project cycle and from "
Reference - Automation “Automagic"...Gartner’s Approach to IT Automation in the Data Center.

11. Author Biography
Barkha Tyagi – Test Architect
10+ years experience in IT Industry in managing the end to end testing of multiple projects operated in Agile/Scrum.
Working as Test Architect in NIIT Technologies
Masters in computers.
Having an \'A\' Level Certification from Software Technology Group Delhi.
Sameer Chawla – Sr. Test Architect
HP Certified automation test professional with 11+ years of rich industry experience
Presently working in the role of a Senior Test Architect with responsibilities of:
Test Automation, PoC Creation
Test Team Management & Technical Training
MCA from M.D.U.