1. De-mystifying Big Data Testing using new generation tools / technology
Manish Singhal – Senior Test Manager
Mohit Saini – Test Data analyst
Sopra Steria

2. Abstract
Digital Blast has enabled vast population of this world into digital usage of technology. This include social media, online marketing, online shopping, online payment, customer preferences.
All this gets converted into data which is used by companies to do real time analysis and offer faster services to the clients.
As per International Institute for Analytics (IIA), worldwide revenues for big data and business analytics will grow from $130.1 billion in 2016 to more than $203 billion in 2020, at a compound annual growth rate (CAGR) of 11.7%.
There is a growing needs to ensure that all aspects of testing ( Data integration, Data scalability , Speed of data transformation / processing ) is covered in Big Data testing with key focus on reducing testing turn around time.
This white paper will focus on:
Challenges on big data testing
Generation of Test data for Big data projects
Usage of tools (Test Data Maker, Agile Requirement Designer, QuerySurge Blue Prism)

3. 1 2 3 4 5 | Current Market Scenario Key Challenges in Big Data Testing
Agenda 1 |
Current Market Scenario 2
Key Challenges in Big Data Testing 3
Tools 4
Usage of Tools for automation 5
Summary

4. Current Market Scenario

5. Big Data Testing Challenges
Ensuring the test coverage for all the possible negative and positive data combinations as per the complex business rules / requirements
Identifying / Covering the edge conditions
Testing coverage with in the test schedule
Ensuring data integrity is maintained
Data consistency
Coping with the changing business requirements / Agile ways of working

6. Tools to De-mystify Big Data testing
Test Data maker assists in creating, maintaining and provisioning of the test data needed to rigorously test evolving applications. It uniquely combines elements of data sub-setting, masking and synthetic, on-demand data generation to enable testing teams to meet the agile needs of the organization
Agile Requirements Designer is an end-to-end test automation and test case design tool. The optimal set of manual or automated tests can be derived automatically from requirements modeled as unambiguous flowcharts and are linked to the right data and expected results. These tests are updated automatically when the requirements change, allowing organizations to deliver quality software which reflects changing user needs.

7. Tools to De-mystify Big Data testing contd.
Test Data maker assists in creating, maintaining and provisioning of the test data needed to rigorously test evolving applications. It uniquely combines elements of data sub-setting, masking and synthetic, on-demand data generation to enable testing teams to meet the agile needs of the organization
Agile Requirements Designer is an end-to-end test automation and test case design tool. The optimal set of manual or automated tests can be derived automatically from requirements modeled as unambiguous flowcharts and are linked to the right data and expected results. These tests are updated automatically when the requirements change, allowing organizations to deliver quality software which reflects changing user needs.

8. Agile Requirements Designer / Test Data Maker
Example : Sample online food ordering side:
Requirement: Able to book food online for my present location for a choice of restaurant
Sample Test Data Model

9. QuerySurge

10. Blue Prism
Challenge is to run the complete process where we have to:
Take file from the source control system
Place it in desired servers / folders
Wait for file to be processed
Wait for the output files to be generated
Ready the source and result files for query Surge
Initiate QuerySurge
Run comparison scripts
Verify the results

11. Summary
With efficient usage of tools, we have made testing of a big data very simpler.
Model based testing using Agile Requirements Designer, we are generating test cases covering ensuring complete coverage
Using Test Data Maker, ensuring that test data is generated for all the combinations, ensuring that requirement changes does not involve a huge rework as only models needs to be changed
Ensuring that test execution does not take huge effort by using QuerySurge
Automate repeatable tasks (which are prone to human error) by doing end to end process automation using Blue Prism.

12. References & Appendix
NewVantage Partners Big Data Executive Survey
International Institute for Analytics (IIA), worldwide revenues for big data and business analytics

13. Author Biography
Manish Singhal has total IT experience of over 20 years with Test Management, Software quality assurance & control, Black Box Testing, Client interaction, managing SDLC and software development process with leading software consulting companies and clients. Manish is current working as Senior Test Manager in SopraSteria and is handling key account on Big Data Testing space for a leading financial institution in UK.
Manish is specialist in Formulating Testing Process & Methodologies with comprehensive hands on experience in designing Test proposals, Creation of estimates, Test strategy formulations, Test management and Test execution with strong people management capabilities.
Manish has extensive experience in managing large cross-functional, geographically dispersed IT Testing and Quality Assurance projects on time and within budget. Manish also is proficient in setting up Quality procedures & processes of Software Development Life Cycle Management along with responsibility for driving and sustaining CMMi Level 5 assessment for Testing Service line.

14. Author Biography
Mohit Saini With over 5.5 years of experience in IT industry, worked as part of the Quality Assurance teams across various Projects where clients affiliated to Finance Domain such as Debt Management, Share market etc. Proficient in SQL queries, Functional, Database/ETL with zeal to be more technical oriented. Recently acquired command on CA Grid tools such as Agile Requirements Designer as part of the data team. Always supportive for the team and have good soft skills.

15. Thank You!!!