1. Agenda for the Day 01 – Introduction to APS & Demo
02 – APS Hardware, Software & Architecture
03 – Distribution Theory & Design
----- Meal Break
04 – APS Data Loading
05 - HDI Region Software, Tools & Polybase
06 – BI Integration & End-to-End Demo

2. Microsoft Analytics Platform System 01 – Introduction To APS
Dan Kogan | Microsoft ​Sr. Product Marketing Manager
Jesse Fountain| Microsoft ​WW TSP Lead
Sanjay Soni | Microsoft ​Sr. Product Marketing Manager
September 20, 2018

3. AGENDA APS Introduction APS High-Level Architecture
Key Features / Differentiators
APS Integrated Demonstration

4. The traditional data warehouse
Microsoft Analytics Platform System
9/20/2018
The traditional data warehouse
BI and analytics
Dashboards
Reporting
Data warehouse
ETL
Data sources
OLTP
ERP
CRM
LOB
© 2014 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

5. The traditional data warehouse
Microsoft Analytics Platform System
9/20/2018
The traditional data warehouse
BI and analytics
Dashboards
Reporting
Real-time
data 2
Data warehouse
ETL
Increasing
data volumes 1
New data
sources and types 3
Cloud-born
data 4
Data sources
OLTP
ERP
CRM
LOB
Non-relational data
Devices
Web
Sensors
Social
© 2014 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

6. Microsoft Analytics Platform System
9/20/2018
Introducing the Microsoft Analytics Platform System The turnkey modern data warehouse appliance
Enterprise-ready Big Data
Next-generation performance at scale
Engineered for optimal value
Relational and non-relational data in a single appliance
Enterprise-ready Hadoop
Integrated querying across Hadoop and PDW using T-SQL
Direct integration with Microsoft BI tools such as Microsoft Excel
Near real-time performance with In-Memory Columnstore
Ability to scale out to accommodate growing data
Removal of data warehouse bottlenecks with MPP SQL Server
Concurrency that fuels rapid adoption
Industry’s lowest data warehouse appliance price per terabyte
Value through a single appliance solution
Value with flexible hardware options using commodity hardware
© 2014 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.

7. Financial Services
In a US$1.2 million win, a Canadian bank is transforming operations and staying competitive by choosing Microsoft APS and self-service BI tools for all employees
Challenge
To stay competitive, a recently rebranded Canadian bank wanted a better analytics and business intelligence (BI) solution
Faced rapid data growth from 5 TB to 25 TB over the next three years
Wanted to give 1,000 employees more self-service BI features and access to data anytime, anywhere from mobile devices
Needed better insight into customer behavior, but the existing Major RDBMS data warehouse was costly and difficult to maintain.
Lacked the analytics capabilities and scalability needed to support business growth and transformation
Hoped to control costs—maintaining BI tools from multiple vendors had become cumbersome and expensive
Solution
Chose the Microsoft Analytics Platform System and is standardizing its BI environment with technologies such as Microsoft SQL Server Analysis Services and SQL Server Reporting Services
Provides real-time, self-service BI access to all employees, driving better business decisions companywide
Improves insight to identify new revenue opportunities as well as the reasons existing customers leave the bank
Gains more scalable, flexible solution with potential for cloud and big-data integration

8. Why an Appliance?

9. Microsoft Analytics Platform System
9/20/2018
Blazing-fast performance MPP and In-Memory Columnstore for next-generation performance 9
Up to 100x faster queries
Updateable clustered columnstore vs. table with customary indexing
Up to 15x more compression
Columnstore index representation C1 C2 C3 C4 C5 C6
Parallel query execution
Store data in columnar format for massive compression
Load data into or out of memory for next-generation performance with up to 60% improvement in data loading speed
Updateable and clustered for real-time trickle loading
Query
Results
© 2014 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.

10. Microsoft Analytics Platform System
9/20/2018
Software and hardware re-designed together Built-in software features lead to efficient hardware
Windows Server 2012 built-in software drives efficient hardware
Virtualization for streamlining hardware footprint
High-end storage features built into software
Storage
Spaces
Windows virtualized storage
Storage pool
Reduced costs
Built-in best practices
Physical
storage
© 2014 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

11. Microsoft Analytics Platform System
9/20/2018
Scaling out your data to petabytes Scale-out technologies in the Analytics Platform System
Scale out
Multiple nodes with dedicated CPU, memory, and storage
Ability to incrementally add hardware for near-linear scale to multiple petabytes
Ability to handle query complexity and concurrency at scale
No “forklift” of prior warehouse to increase capacity
Ability to scale out HDInsight and PDW
PDW / HDInsight
PDW / HDInsight
PDW / HDInsight
PDW / HDInsight
PDW / HDInsight
PDW / HDInsight
PDW
0 terabytes
6 petabytes 11
© 2014 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

12. Microsoft Analytics Platform System
9/20/2018
Connecting islands of data with PolyBase Bringing Hadoop point solutions and the data warehouse together for users and IT
Select…
Result set
Provides a single T-SQL query model for PDW and Hadoop with rich features of T- SQL, including joins without ETL
Uses the power of MPP to enhance query execution performance
Supports Windows Azure HDInsight to enable new hybrid cloud scenarios
Provides the ability to query non-Microsoft Hadoop distributions, such as Hortonworks and Cloudera
SQL Server
Parallel Data
Warehouse
Microsoft Azure
HDInsight
PolyBase
Hortonworks for Windows and Linux
Cloudera
Microsoft HDInsight
© 2014 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.

13. Recent POC Results Public Transport Company (vs. Major RDBMS)
200 user concurrency test, 7 years worth of data
ETL Batch window test
Major RDBMS
APS
2 hours
12 minutes
4 hours
3 minutes
Advertising Company (vs. Hadoop)
120 node Hadoop cluster vs. 10 node APS
Hadoop queries executed through Hive
Hadoop
APS
7 minutes
1 minute
5 minutes
1 second
Internal Testing (vs. SQL Server Standard)
SQL Server on SMP (32-core, 256GB, 152 HDDs)
SQL Server on SMP (80-core, 1TB, 192 HDDs)
APS-HP Single rack (128-core, 2TB, 288 HDDs)
48 min
34 min
< 1 min

14. Retail
A global retailer of consumer electronics that reported US$42 billion in annual sales in 2013 is improving operational efficiency companywide by using APS to power real-time reporting for 80,000 stores.
Challenge
A leading global retailer of consumer electronics wanted to help its thousands of outlets operate more efficiently, but faced a daily struggle to deliver timely performance reports to 80,000 stores
Needed better access to operational data, including point-of-sale purchases, orders, and transport updates for approximately 125,000 retail employees and 1,200 corporate managers
Wanted analytics and BI tools that would be easy to use for employees in multiple roles working in diverse locations, including store floors worldwide
Expanding its existing MPP Data Warehouse architecture to meet the SLAs would be too costly
Solution
The retailer replaced its MPP Data Warehouse and BI Toolset with the Microsoft Analytics Platform System in an end-to-end solution that includes Microsoft BI tools and integrates with an existing Hadoop cluster
Improves operational performance with real-time, on-demand insight into multiple business processes and data sources, easily handling 40,000 reports per hour
Gains unprecedented insight into at least five years of performance information
Scales easily and affordably to support new workloads and additional functionality
Cuts annual power costs by millions of dollars by consolidating with APS and using built-in technology like clustered columnstore index

15. APS High-Level Architecture

16. APS Logical Architecture
Control Server
A.K.A. ‘The Brains’’
1. Optimizer creates parallel query plan
2. Each compute server runs a portion of the query in parallel
3. Data is combined and returned to user
User Query
Optimizer
Metadata
Statistics
Data Movement Services …
Compute Server
Compute Server A.K.A. ‘The Brawn’
Compute Server
DMS
Balanced Storage
DMS
Balanced Storage
DMS
Balanced Storage

17. Automatic MapReduce pushdown
Microsoft Analytics Platform System
9/20/2018
Automatic MapReduce pushdown
Source systems
Hadoop / Data Lake
(Cloudera, Hortonworks, HDInsight)
Analytics / Ad-hoc / Visualization
SQL Server
Data Marts
Reporting Services
Analysis Services
Microsoft
HDInsight
SQL Server Parallel Data Warehouse
PolyBase
MapReduce
T-SQL
APS
Day / Hour / Minute Refresh
© 2014 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

18. APS High-Availability No Single Point-Of-Failure
Infiniband 2
Ethernet 2
Control Host
Compute Host 1
Compute 1 VM
Compute 1 VM
FAB AD
VMM
CTL
Infiniband 1
Infiniband 1
Ethernet 1
Ethernet 1
Compute Host 2
Failover Host
Compute 2 VM

19. the Analytics Platform System Appliance
9/20/2018
Concurrency that fuels rapid adoption Great performance with mixed workloads
Analytics Platform System
ETL/ELT with SSIS, DQS, MDS
ERP
CRM
LOB
APPS
SQL Server SMP
Intra-Day
CRTAS
Link Table
PDW
HDInsight
PolyBase
ETL/ELT with DWLoader
Near real-time
Reporting and cubes
Columnstore
Real-Time
ROLAP / MOLAP
DirectQuery
Hadoop / Big Data
Polybase
SNAC
BI Tools
Ad hoc queries
Fast ad hoc

20. Key Features & Differentiators

21. Microsoft Analytics Platform System
9/20/2018
How PolyBase works
Direct and parallelized HDFS access
Enhancing the Data Movement Service (DMS) of APS to allow direct communication between HDFS data nodes and PDW compute nodes
Non-relational data
Social apps
Sensor and RFID
Mobile apps
Web apps
Hadoop
Relational data
Traditional schema-based data warehouse applications
PDW
Regular T-SQL
Results
External table
External data
source
External file
format
Enhanced PDW query engine
HDFS bridge
© 2014 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

22. Microsoft Analytics Platform System
9/20/2018
Big Data insights for anyone New insights with familiar tools through native Microsoft BI integration
Takes advantage of high adoption of Excel, Power View, PowerPivot, and SQL Server Analysis Services
Minimizes IT intervention for discovering data with tools such as Microsoft Excel
Everyone else using
Microsoft BI tools
Offers Hadoop tools like MapReduce, Hive, and Pig for data scientists
Enables DBA and power users to join relational and Hadoop data with T-SQL
Power users
Data scientist
© 2014 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

23. End-to-end Solution Azure Market Place Alerts, Notifications
Big Data Sources (Raw, Unstructured)
SQL Server StreamInsight
Data & Compute Intensive App
Summarize & Load via Polybase
SQL Server FTDW Data Marts
Business Insights
Sensors
Hortonworks or Cloudera Hadoop
SQL Server Reporting Services
Devices
HDInsight on Windows Azure
HDInsight on Windows Server
SQL Server Anlalytical Platform Server
Interactive Reports
Integrate/Enrich
Bots
SQL Server Analysis Server
Azure Market Place
ETL with SSIS, DQS, MDS
Performance Scorecards
Crawlers
ERP
CRM
LOB
APPS
Source Systems

24. + ~ 3-4 months ~ 3 weeks APS Reduces TIMELINE SQL Server SQL Server
Infrastructure
5-8 weeks
Installation
2-4 weeks
System Design
2-3 weeks
D/W Requirements
Data Warehouse H/W Design
Hadoop Requirements
Hadoop H/W Design
Infrastructure Acquisition
DW H/W Installation
Hadoop H/W Installation
DW S/W Installation
Hadoop S/W Installation
Networking
Database Design
Physical Data Design
Index Design
Configuration Settings
Performance Design
1 week
SQL Server
~ 3 weeks
D/W Requirements
Hadoop Requirements
Analytics Platform System

25. Microsoft’s big picture approach
DATA TYPES
DATA SOURCES
Apps, Biz process, ERP, CRM
INFRASTRUCTURE DM
SQL Server
ETL
TOOLS
Known, known
Excel
Power Pivot
Power View
Data Explorer
USERS
Business
Structured
MM/DD/YYYY
Known, known
Known, known
Known, known
Analytical Platform Server
Polybase
Semi-structured
web logs, RFID, “the internet of things”
Known, unknown
Machines and other devices
Pig, R, map reduce, algorithms, machine learning
Known, unknown
“Data Scientists” “Quants”
HDInsight
Known, unknown
Known, unknown
Un-structured
text, video, audio
Unknown, unknown
Collaboration and social
, blogs, documents
Indexing engine algorithms
Unknown, unknown
Everyone
Unknown, unknown
Unknown, unknown

26. Transportation and Logistics
With APS, a major city’s transit agency is improving the travel experience for millions with near-real-time insight into data collected from billions of trips across multiple types of public transportation.
Challenge
To improve customer experience, a government agency needed a better analytics solution to gain insight into billions of annual journeys across multiple types of public transit
Agency responsible for a transport network needed to improve travel for millions
Wanted to collect data from a recently implemented contactless card system, but lacked a sufficiently scalable platform
Needed to look at data from multiple trips, including 1 billion subway journeys across 270 stations, and 2 billion bus journeys on 8,500 buses
Subway journeys expected to increase 30% by 2020, and bus journeys are growing at the fastest rate since 1946
Solution
Implemented the Microsoft Analytics Platform System
Provides more granular analytics and data on congestion and travel disruption to help improve customer experience
Gains 10x faster performance while querying 42x more data, and makes information accessible to 100x more users
Automates refunds—identifies incorrect billing and immediately refunds the money to the customer accounts, avoiding the inconvenience and cost of customers’ contacting a representative to manually correct the error
Collects more big data—integrates sensor information from a rental bicycle system called Barclays Cycle Hire. Integrates social media such as Twitter to better understand customers’ reactions. Other possible sources of data include WiFi, CCTV, and law-enforcement agencies.

27. Demo

28. Microsoft Cybercrime Center APS/BI Demo
9/20/2018
Microsoft Cybercrime Center APS/BI Demo
Sanjay Soni
Sr. Technical Product Marketing Manager
(for Power BI + End to end BA demos)
Microsoft
© 2012 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

29. The Microsoft Cybercrime Center

30. The impact of cybercrime
Online Child Exploitation
The NCMEC has reviewed more than 90 million images and videos of child pornography.***
Financial Fraud
53% of the world’s securities exchanges were targeted in 2012
Cybercrime costs consumers $113 billion a year*
1 in 5 small and medium enterprises are targeted by cyber criminals**
Every second, 12 people are victims of cybercrime – nearly 400 million every year*
50% of online adults have been victims in the past year
* 2013 Norton Report
** National Cyber Security Alliance
*** National Center for Missing and Exploited Children

31. A new era in the fight against cybercrime
Microsoft
Digital Crimes Unit
Microsoft Consulting Services
Trustworthy Computing
Government Affairs
Public Sector
Cybercrime Center
Security, Reliability, Privacy
Solutions, Initiatives, Innovations
Policy, Legislation
Risk Assessment, Cybersecurity Services
Industry Partners
Criminal Law Enforcement
Government
Proactive Disruption
We work with our customers and partners to proactively target online criminals
MALICIOUS
SOFTWARE CRIMES
IP CRIMES
CHILD EXPLOITATION
Viruses
Trojans
Worms
Botnets
Counterfeiting
End-User Piracy
Child Abuse Images
Sex Trafficking

32. Botnet Analytics – Architecture
Certs
ISPs
Others
Partners / Subscribers
Internet with
billions of devices
Internet with
billions of devices
Azure
Machine Learning
[Sinkhole]
SQL Server
Azure HDInsight
Microsoft
Analytics Platform System
(APS)
Visualizations
& insights
Excel & Power BI
Processing 200M+ transactions per day and growing, the Microsoft Cybercrime Center APS & BI are powered by:

33. Use of Analytics Platform System/ Parallel Data Warehouse
Cybercrime analytics team uses PDW
Microsoft’s data scientists support cybercrime operations by using PDW to run advanced algorithms
We can aggregate 500 million rows of data a day into tables that bring even more speed to our ability to respond to rapidly evolving threats.
566,773,255 calls a day into the sinkhole from over 25 million distinct IP address representing many more than that number of computers, because some IP addresses are corporate consolidation points.
Massive performance improvements:
Our data is now responding at the speed of thought.
Cybercrime analytics team uses PDW
Microsoft’s data scientists support cybercrime operations by using PDW to run advanced algorithms
We can aggregate 500 million rows of data a day into tables that bring even more speed to our ability to respond to rapidly evolving threats.
566,773,255 calls a day into the sinkhole from over 25 million distinct IP address representing many more than that number of computers, because some IP addresses are corporate consolidation points.
Massive performance improvements:
Our data is now responding at the speed of thought.
Query
Before PDW
After using PDW 1
2 days, 8 minutes
<3 minutes 2
4 hours
2 seconds

34. Microsoft Analytics Platform System
9/20/2018
© 2014 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
© 2014 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.