1 Intel Information Technology Labs Big Data: Into the Deep John David Miller, Principal Engineer Intel Information Technology Labs

2 Intel Information Technology Labs Intel’s Interest in Big Data Hardware Cloud / Datacenter infrastructure, sensors Software Hadoop & HBase optimization tools Internal Decision-making, sense-making, understanding, efficiency

3 Intel Information Technology Labs A Few Intel Big Data Use Cases Big Database: like traditional BI, but bigger Marketing: usage data  platform optimization, demographics –“Who’s using what apps, when, and how?” Manufacturing: tool & chip instrumentation  increase yield –“How can we spot trends before they become problems?” HVAC: environmental sensor data  better control, cost saving –“How can we better manage our facilities?” Deep Analytics: heterogeneous data, complex, open-ended Cybersecurity  threat mitigation –“Who’s trying to break in? Who’s trying to steal info from the inside?” Strategic Planning: market awareness  contingency planning –“How will the flooding in Thailand affect us?” HR: employee understanding  better, happier workforce –“Are we hiring the right people to be successful at Intel?” Marketing: web/social media analytics  consumption prediction –“How many PCs will we sell next year?”

4 Intel Information Technology Labs Deep Analytics: Digging Deeper 1 st order: answers directly codified in the data –Database reports, dashboards, e.g., traditional BI 2 nd order: can infer the answer from existing data –Recommendation systems, textual search, predictive analytics 3 rd order: answers from transforming data into higher structure –Entity / semantic graphs –{people, places, things} + facts, relationships 4 th order: using derived structure to synthesize / simulate –System dynamics modeling  possible future outcomes –Simulates from generated model, not historical, statistical data

5 Intel Information Technology Labs Towards Deep Analytics Mature Nascent Visualization, Context-Awareness, Personalization Machine-learning, Knowledge representations, Data/Text/Media Analytics Hardware, OS, Cloud / Datacenter management Wisdom-of-Crowds, Collective Intelligence How many widgets will we sell next year? How will (random world event) affect us? Are we hiring the right people? What features do our customers want? Deep Analytics addresses complex, open-ended, high-value questions Technology Stack Infra- structure Machine intelligence interface

6 Intel Information Technology Labs Information Ecology workflow

7 Intel Information Technology Labs Big Data Technology Hadoop isn’t the end-all, but a beginning “Birth of a new operating system” Low cost  shift to “Keep everything” – and figure it out later Still need a mix or hybrid of Hadoop + so-called MPP databases..but transferring data back & forth is expensive Hadoop 2: beyond MapReduce MPI, Graphlab, Spark, …