BIG DATA – WHAT’S THE BIG DEAL The call would start soon, please be on mute. Thanks for your time and patience.

WHO AM I  Debarchan, from Calcutta with an Indian heart and a global mind. .NET programmer who fell in love with Open Source, specifically Apache Hadoop  Author.  Community enthusiast.  Cricket and music lover.  One who gets really scared and bored if tomorrow is exactly like today.  Known is a drop, the unknown is an ocean.

WHAT IS BIG DATA?

How do I optimize my fleet based on weather and traffic patterns? SOCIAL & WEB ANALYTICS LIVE DATA FEEDS ADVANCED ANALYTICS What’s the social sentiment for my brand or products How do I better predict future outcomes? A NEW SET OF QUESTIONS

COMMON BIG DATA CUSTOMER SCENARIOS GAIN COMPETITIVE ADVANTAGE BY MOVING FIRST AND FAST IN YOUR INDUSTRY Web app optimization Smart meter monitoring Equipment monitoring Advertising analysis Life sciences research Fraud detection Healthcare outcomes Weather forecasting Natural resource exploration Social network analysis Churn analysis Traffic flow optimization IT infrastructure optimization Legal discovery

THE BIG DATA LIFECYCLE

MANAGE ANY DATA, ANY SIZE, ANYWHERE

Extremely large volume of unstructured web logs Ad hoc analysis of logs to prototype patterns Hadoop data cluster feeds large 24TB cube Business users analyze cube data E.g. STRUCTURED & UNSTRUCTURED DATA

THE BIG DATA LIFECYCLE

ENRICH BY CONNECTING TO THE WORLDS DATA Discove r Combine Refine

POWER OF COMBINING THE WORLDS DATA Value

E.g. VALUE OF EXTERNAL DATA

THE BIG DATA LIFECYCLE

INSIGHTS ON ANY DATA, ALL USERS, WHEREVER THEY ARE

INSIGHTS FOR ALL USERS THROUGH FAMILIAR TOOLS PB TB GB

16 Application written in java for Big Data Processing Uses the “Map-Reduce” Processing Paradigm Characteristics: How is it different from traditional SQL Server? 1. Optimized for distributed storage and computing of data 2. Highly-scalable (scale out model) 3. Commodity HW-based 4. Open Source  Very low cost for acquisition and storage Hadoop Data Analytics Dataflow

17 Distributed Storage (HDFS) Distributed Processing (Map Reduce)

18 Welcome to the Zoo! Need to Know* StreamInsight Good to Know*

19 Feed us back Support Team’s blog: Facebook Page: Facebook Group: Read more: Next Session: Apache Hadoop – A deep dive

© 2012 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.