1. How Cutting Edge Big Data and Analytics Lets J. D
How Cutting Edge Big Data and Analytics Lets J.D. Power Compete in the 21st Century
Jonathan Miller, Ramki Ramaswamy
VP-CTO VP Application Development

2. Agenda Our process of transformation Into the weeds
What has it done for us

3. Transformation timeline
3 Years Ago
Strictly dealt with survey data only
Data was collected using paper and telephone surveys
Expensive and needed OCR and manual techniques
Needed a lot of clean up
Depended on various analytic engines like SAS, SPSS and even Matlab
2 Years Ago
Migrated to a modern eSurvey methodology
Rules built into the survey
Cleaner data with minimal post processing
Reduced production costs
Largely depended on survey data
Standardized manual analysis using SAS
1 Year Ago
A modern noSql data management platform was designed
Automated the high risk and high value paths
Created an architecture to support multiple types of data including survey data
Ability to load and analyze data in near real time
Today
Full feature data management platform
Custom workflows to alert on data during load
Automated outcome oriented performance improvement suggestions
Show multiple data set information side by side to allow for correlation
Empower the user to analyze the data in near real time

4. How did we get there
Understanding the exact requirements based on our current outputs and customer needs
Identify and segregate of our customer and their organization topology
Understanding the data attributes and what they mean for each businesses
Abundant POC across different industries
Reduce cost and move to a more generation aligned strategy
Increased speed to market on analysis and actionable results
Build vs Buy analysis
Using Agile methodology

5. Understanding our Data
First step to understand the business behind our data attributes
Creating a high level topology of the data like Customer Demographics, Transaction details, Product information etc.
Realizing that we are in Big Data territory
Additive nature of data
Correlation needs between data
Structuring of “Unstructured data”

6. Our Data Warehouse Security
Batch data processing like Scoring, sample weight calculation etc.
Data Files
Pre-Processing like validation and micro batching
Data Warehouse
Apache SOLR cloud based data mart with sharding
Data Streams
Live data mart loading using Lambdas and streams
Event Data
Data From API
AWS Cloud Infrastructure

7. A demo of our MDM

8. Our Platform

9. A demo of VoX