Journey of Quality Analysts towards Data Analytics   Journey of Quality Analysts towards Data Analytics Vijai Krishnamoorthy, Expert Quality Engineer Jaladi, Venkata Naga Bhaskar Rao, Expert Quality Engineer Jandhyala, Vardhan, Sr Software Engineer

Abstract Data Analytics is a process of analysing collection of data and remodelling it to arrive a desired conclusion which helps to come up with predictive Analysis. Analysing data is a basic need of any business, which plays a vital role in delivering an efficient product and to perform outstanding service supporting client by coming with predictive analysis. Our intent is to explain the role that should be played by a functional tester in data analytics and areas where tester can bring a huge difference in data analytics.

Challenges in Big Data Testing Are we Utilizing Functional testers with Database/domain knowledge for Big data testing? Are we Increasing the cost to company, by not Utilizing  in-house Functional testing Experts? Are Corporates/Testers ready to transform Functional testers as Big data testers ?  Do big data testers ready to work as Functional Testers as well as Big data testers?

Role of Functional Tester Coverage / Completeness of Data Accuracy and correctness of data Freshness Richness

Big Data Processing Batch Real Time Interactive

Verification Conformity Duplication Consistency Accuracy Validity Data Completeness

3V 3Vs of Big Data Testing Challenges Less Structured Formats (website links, emails, Twitter responses, pictures/images, written text on various platforms) which make its analysis more difficult. 3V Velocity Variety Volume

Stages of Functional Validation of 3 Vs Data Extraction Testing Reports & Visualization Testing Data Quality Analysis

Functional Tester to Big Data Tester Data Flow validation Data Acquisition based on business use cases and Validating data movement across different layers. Testing the Data Aggregation and Data filtering mechanisms. End-to-End Data validation and transformation logic based on business rules Data Integrity Validate for Data completeness with referential integrity checks, Data constraints and duplication checks along with error conditions. Testing boundaries to identify schema limitations for each layer.

Functional Tester to Big Data Tester Data Ingestion Layer Ability to connect with different data models and replay the data through messaging systems, Monitor Data loss. Fault Tolerance, Continuous availability and connection to different data streams Data Processing Layer Business Rules validation & Validating Map-Reduce process. Data Integrity, Data Transformation, Data Aggregation and Consolidation Data Processing Performance & Exception Handling.

Functional Tester to Big Data Tester Data Storage Layer Read & Write Timeouts, Continuous Availability, Load balancing and Query Performance Analysis Reports Testing Validation for Measures & Dimensions, Real-time Custom Reporting, Drill up / down mechanisms, Business reports & Charts.

Looking at Future According to IDC, revenues for big data business analytics will grow from $130.1 billion in 2016 to more than $203 billion in 2020 As per a study, by 2020, global big data market is expected to grow at a CAGR of about 37 percent and this will be as a result of the increase in penetration of big data in diverse sectors The importance of Big Data can’t be overstated and organizations who tap into it will surely reap its benefit in the nearest and distant future

Key Takeaways Every business should look towards Data Analytics, and it will be best if we can convert our functional testers who are well versed with domain and product to a Big data tester Functional testers are good in structured data validations, by upscaling and adapting new techniques of processing unstructured data, they could perform Big Data testing more efficiently Big Data Analytics helps in faster and better decision making, by predicting from the data, who else is better than a functional tester to validate the data given

Author Biography Vijai Krishnamoorthy an Expert Quality Engineer Working with Allscripts from past 8 years. Expertise in Healthcare domain Email: Vijai.Krishnamoorthy@allscripts.com Bhaskar Rao Jaladi an Expert Quality Engineer Working with Allscripts from past 8 years. Submitted 2 IDFs related Health care in USPTO. Email: venkatanagabhaskarrao.jaladi@allscripts.com Vardhan Jandhyala Senior Software Engineer Working with Allscripts from past 6 years. Expertise in Automation Testing Email: Vardhan.Jandhyala@gmail.com

Thank You!!!