De-mystifying Big Data Testing using new generation tools / technology Manish Singhal – Senior Test Manager Mohit Saini – Test Data analyst Sopra Steria

Abstract Digital Blast has enabled vast population of this world into digital usage of technology. This include social media, online marketing, online shopping, online payment, customer preferences. All this gets converted into data which is used by companies to do real time analysis and offer faster services to the clients. As per International Institute for Analytics (IIA), worldwide revenues for big data and business analytics will grow from $130.1 billion in 2016 to more than $203 billion in 2020, at a compound annual growth rate (CAGR) of 11.7%. There is a growing needs to ensure that all aspects of testing ( Data integration, Data scalability , Speed of data transformation / processing ) is covered in Big Data testing with key focus on reducing testing turn around time. This white paper will focus on: Challenges on big data testing Generation of Test data for Big data projects Usage of tools (Test Data Maker, Agile Requirement Designer, QuerySurge Blue Prism)

1 2 3 4 5 | Current Market Scenario Key Challenges in Big Data Testing Agenda 1 | Current Market Scenario 2 Key Challenges in Big Data Testing 3 Tools 4 Usage of Tools for automation 5 Summary

Current Market Scenario

Big Data Testing Challenges Ensuring the test coverage for all the possible negative and positive data combinations as per the complex business rules / requirements Identifying / Covering the edge conditions Testing coverage with in the test schedule Ensuring data integrity is maintained Data consistency Coping with the changing business requirements / Agile ways of working

Tools to De-mystify Big Data testing Test Data maker assists in creating, maintaining and provisioning of the test data needed to rigorously test evolving applications. It uniquely combines elements of data sub-setting, masking and synthetic, on-demand data generation to enable testing teams to meet the agile needs of the organization Agile Requirements Designer is an end-to-end test automation and test case design tool. The optimal set of manual or automated tests can be derived automatically from requirements modeled as unambiguous flowcharts and are linked to the right data and expected results. These tests are updated automatically when the requirements change, allowing organizations to deliver quality software which reflects changing user needs.

Tools to De-mystify Big Data testing contd. Test Data maker assists in creating, maintaining and provisioning of the test data needed to rigorously test evolving applications. It uniquely combines elements of data sub-setting, masking and synthetic, on-demand data generation to enable testing teams to meet the agile needs of the organization Agile Requirements Designer is an end-to-end test automation and test case design tool. The optimal set of manual or automated tests can be derived automatically from requirements modeled as unambiguous flowcharts and are linked to the right data and expected results. These tests are updated automatically when the requirements change, allowing organizations to deliver quality software which reflects changing user needs.

Agile Requirements Designer / Test Data Maker Example : Sample online food ordering side: Requirement: Able to book food online for my present location for a choice of restaurant Sample Test Data Model

QuerySurge

Blue Prism Challenge is to run the complete process where we have to: Take file from the source control system Place it in desired servers / folders Wait for file to be processed Wait for the output files to be generated Ready the source and result files for query Surge Initiate QuerySurge Run comparison scripts Verify the results  

Summary With efficient usage of tools, we have made testing of a big data very simpler. Model based testing using Agile Requirements Designer, we are generating test cases covering ensuring complete coverage Using Test Data Maker, ensuring that test data is generated for all the combinations, ensuring that requirement changes does not involve a huge rework as only models needs to be changed Ensuring that test execution does not take huge effort by using QuerySurge Automate repeatable tasks (which are prone to human error) by doing end to end process automation using Blue Prism.

References & Appendix NewVantage Partners Big Data Executive Survey 2017 http://newvantage.com/wp-content/uploads/2017/01/Big-Data-Executive-Survey-2017-Executive-Summary.pdf International Institute for Analytics (IIA), worldwide revenues for big data and business analytics

Author Biography Manish Singhal has total IT experience of over 20 years with Test Management, Software quality assurance & control, Black Box Testing, Client interaction, managing SDLC and software development process with leading software consulting companies and clients. Manish is current working as Senior Test Manager in SopraSteria and is handling key account on Big Data Testing space for a leading financial institution in UK. Manish is specialist in Formulating Testing Process & Methodologies with comprehensive hands on experience in designing Test proposals, Creation of estimates, Test strategy formulations, Test management and Test execution with strong people management capabilities. Manish has extensive experience in managing large cross-functional, geographically dispersed IT Testing and Quality Assurance projects on time and within budget. Manish also is proficient in setting up Quality procedures & processes of Software Development Life Cycle Management along with responsibility for driving and sustaining CMMi Level 5 assessment for Testing Service line.

Author Biography Mohit Saini With over 5.5 years of experience in IT industry, worked as part of the Quality Assurance teams across various Projects where clients affiliated to Finance Domain such as Debt Management, Share market etc. Proficient in SQL queries, Functional, Database/ETL with zeal to be more technical oriented. Recently acquired command on CA Grid tools such as Agile Requirements Designer as part of the data team. Always supportive for the team and have good soft skills.

Thank You!!!