1. QA Validation in Big Data
Chandrakanth Gande
Sr. Consultant
Capgemini

2. B G Abstract Need for Big Data
Characteristics and features of Big Data
Big Data uses in various Industries
Big Data and its approach in the testing practices
Abstract

3. Need for BIG Data TRADITIONAL DATABASE IS HIGHLY EXPENSIVE
STORAGE OF LARGE VOLUME OF DATA
SEMI STRUCTURED AND UNSTRUCTURED DATA
Need for
BIG Data
LATENCY ISSUES

4. Variety Volume Velocity Variability Characteristics of Big Data
Handles large volumes of data
Structured, Semi Structured & Unstructured data
Variety
Volume
Velocity
Variability
Speed at which data flows in from sources
Manages data effectively in inconsistencies

5. Big Data uses in Various industries
It captures voices of the flight crew, recordings of microphones and earphones, and the performance information of the aircraft.
Black Box Data
Social Media Data
Stock Exchange Data
Power Grid Data
Transport data includes model, capacity, distance and availability of a vehicle.
Transport Data
Social media such as Face book and Twitter hold information and the views posted by millions of people across the globe.
The stock exchange data holds information about the ‘buy’ and ‘sell’ decisions made on a share of different companies made by the customers
The power grid data holds information consumed by a particular node with respect to a base station

6. Big Data Life Cycle
Data sourcing
Extraction & Structuring
Data Modelling
Data Analysis & Interpretation
Data Sourcing: Data is obtained from different sources and in different forms.
Extraction & Structuring: Data is obtained from different formats and will be making into a standard usable format.
Data Modelling: standard format is placed into rows and columns in database.
Data Analysis & Interpretation: Data is analysed using statistical methods. Decisions are made based on the processed data.

7. Validation steps in Big Data
To verify the right data is extracted and loaded into the correct HDFS location.
To verify the Business Logic is validated every node and multiple nodes.
Output files are generated and ready to move in EDW or any other system.
Data Staging
Map Reducing
Output
Validation

8. Configuration of Reports
Validation steps in Big Data
To verify the mapping of fields in the report. Validation of Layout, “drill-in” on the report with various data sets, Scheduling reports & accuracy, Printing reports,etc
Data is available for all the privileged users based on the security level or hierarchy of the business
Data in the reports are accessible based on the configuration assigned to users.
Dashboard Testing
Data Security & Access
Configuration of Reports

9. Validation steps in Big Data
End to End testing of reports at various levels is required to ensure the quality of the reports is not impacted.
To Obtain and understand the actual performance under load of Big Data applications, such as response time, maximum online user data capacity size, and maximum processing time. .
End to End Testing
Performance Testing

10. References & Appendix
Bigdata-madesimple.com
Author Biography
Chandrakanth Gande is Sr. Consultant in Banking and Capital Markets Vertical. He has 9 years of experience in IT, and Banking. He has involved in the QA activities for Banking projects.

11. Logo of your organization! !
Logo of your organization