Turning back time … … to 1998

my first data warehouse: Microsoft Ignite 2016 9/22/2018 2:05 PM my first data warehouse: © 2016 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

(working with 1m records) SLOW (working with 1m records)

Fast forward >> … to 2016

System Center Marketing 9/22/2018 SQL Server 2016 Industry leader in Mission Critical OLTP built-in Most secure database built-in Highest performing data warehouse built-in End-to-end mobile BI on any device built-in In-database Advanced Analytics built-in Industry leader 6 years in a row least vulnerable SQL Server Oracle MySQL SAP HANA #1 performance TPC-H Oracle is #5 #2 SQL Server #1 #3 A fraction of the cost Microsoft Tableau Oracle $120 $480 $2,230 Self-service BI per user R + in-memory at massive scale In-memory across all workloads Consistent experience from on-premises to cloud The above graphics were published by Gartner, Inc. as part of a larger research document and should be evaluated in the context of the entire document. The Gartner document is available upon request from Microsoft. Gartner does not endorse any vendor, product or service depicted in its research publications, and does not advise technology users to select only those vendors with the highest ratings or other designation. Gartner research publications consist of the opinions of Gartner's research organization and should not be construed as statements of fact. Gartner disclaims all warranties, expressed or implied, with respect to this research, including any warranties of merchantability or fitness for a particular purpose. National Institute of Standards and Technology Comprehensive Vulnerability Database update 10/2015 TPC-H 10TB non-clustered results as of 04/06/15, 5/04/15, 4/15/14 and 11/25/13, respectively. http://www.tpc.org/tpch/results/tpch_perf_results.asp?resulttype=noncluster © 2012 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Big Data Platform Session Code Regan Murphy Sr. Tech Evangelist Microsoft New Zealand @nzregs regan.murphy@microsoft.com

Microsoft Ignite 2016 9/22/2018 2:05 PM Big Data definition Big data is high-volume, high-velocity and/or high-variety information assets that demand cost-effective, innovative forms of information processing that enable enhanced insight, decision making, and process automation. – Gartner, Big Data Definition* * Gartner, Big Data (Stamford, CT.: Gartner, 2016), URL: http://www.gartner.com/it-glossary/big-data/ © 2016 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Here are some of the ways big data is being used Offering customers exactly what they want, when they want it Personalized offers Product recommendations New product introduction Getting the right products to the right places Inventory management Supply chain optimization Marketing mix optimization Fixing problems proactively before they start Predictive maintenance Operational efficiency Customer service improvement Improving visibility and making accurate predictions Exploring new business opportunities Remote monitoring Demand forecasting Risk and compliance management Cross-sell and upsell Product-as-a-service New data-driven services

Telco Billing Apps Web Mobile

Telco Billing Microsoft Ignite 2016 9/22/2018 2:05 PM Apps Web Mobile © 2016 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Telco Data Pipeline Microsoft Ignite 2016 9/22/2018 2:05 PM Transactional Database Data Factory SQL Server Power BI HDInsight (Hadoop and Spark) SQL Data Warehouse Blob Storage Apps Web Mobile Bots JSON Stream Analytics Power BI Event Hubs Historical Reporting Near Real Time Insight © 2016 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Transform data into intelligent action Information Management Big Data Stores Machine Learning and Analytics Intelligence People Data Sources Machine Learning Cognitive Services Data Factory Data Lake Store SQL Data Warehouse Data Lake Analytics Bot Framework Apps Web Mobile Bots Data Catalog Apps HDInsight (Hadoop and Spark) Event Hubs Cortana Sensors and devices Dashboards & Visualizations Stream Analytics Automated Systems Power BI Data Intelligence Action

Event Hubs Cloud-scale telemetry ingestion from websites, apps, and devices Log millions of events per second in near real time Connect devices using flexible authorization and throttling Use time-based event buffering Get a managed service with elastic scale Reach a broad set of platforms using native client libraries Pluggable adapters for other cloud services

Stream Analytics Real-time stream processing in the cloud Perform real-time analytics for your Internet of Things solutions Stream millions of events per second Get mission-critical reliability and performance with predictable results Create real-time dashboards and alerts over data from devices and applications Correlate across multiple streams of data Use familiar SQL-based language for rapid development

Demo: Event Hubs, Stream Analytics, Power BI

Telco Data Pipeline Microsoft Ignite 2016 9/22/2018 2:05 PM Transactional Database Data Factory SQL Server Power BI HDInsight (Hadoop and Spark) SQL Data Warehouse Blob Storage Apps Web Mobile Bots JSON Stream Analytics Power BI Event Hubs Historical Reporting Near Real Time Insight © 2016 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Azure HDInsight A Cloud Spark and Hadoop service for the Enterprise Reliable with an industry leading SLA Enterprise-grade security and monitoring Productive platform for developers and scientists Cost effective cloud scale Integration with leading ISV applications Easy for administrators to manage 63% lower TCO than deploying your own Hadoop on-premises* *IDC study “The Business Value and TCO Advantage of Apache Hadoop in the Cloud with Microsoft Azure HDInsight”

Demo: HD Insight

Azure Data Lake Store A No limits Data Lake that powers Big Data Analytics Petabyte size files and Trillions of objects Scalable throughput for massively parallel analytics HDFS for the cloud Always encrypted, role-based security & auditing Enterprise-grade support

Azure Data Lake Analytics A No limits Analytics Job Service to power intelligent action Start in seconds, scale instantly, pay per job Develop massively parallel programs with simplicity Debug and optimize your big data programs with ease Virtualize your analytics Enterprise-grade security, auditing and support

Azure Data Factory Compose and orchestrate data services at scale Create, schedule, orchestrate, and manage data pipelines Visualise data lineage Connect to on-premises and cloud data sources Monitor data pipeline health Automate cloud resource management

Azure SQL Data Warehouse A relational data warehouse as a service, fully managed by Microsoft Elastic scale and performance Massively parallel processing Scale to petabytes of data Instant-on compute scales in seconds Query relational/non-relational Use simple billing compute and storage Pay for what you need, when you need it with dynamic pause Bring DW to the cloud without rewriting

Demo: Data Factory

Telco Data Pipeline Microsoft Ignite 2016 9/22/2018 2:05 PM Transactional Database Data Factory SQL Server Power BI HDInsight (Hadoop and Spark) SQL Data Warehouse Blob Storage Apps Web Mobile Bots JSON Stream Analytics Power BI Event Hubs Historical Reporting Near Real Time Insight © 2016 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

9/22/2018 2:05 PM © 2014 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.