Overview of Azure Data Lake Store

Fundamentals Reliable Unlimited Storage Optimized for Analytics Automatically replicates your data Three copies within a single region Highly available Unlimited Storage Unlimited account sizes Individual file sizes from gigabytes to petabytes No limits to scale Optimized for Analytics Built for running large analytics systems that require massive throughput Optimized for parallel computation over petabytes of data Automatically optimizes for any throughput

Secure your Data Access control Auditing Encryption POSIX-compliant Access Control Lists (ACLs) on Files and Folders * Integrated with Azure Active Directory Auditing Audit logs for all operations Audit logs that can be analyzed with ADL U- SQL Scripts Encryption Transparent server-side encryption * Azure-managed (Azure Key Vault) and customer- managed keys* * Features arriving by GA

HDFS for the Cloud Built from the ground up as a Hadoop file system Other HDI Cluster Types Tools running in HDI Hadoop Distros Microsoft R Services (Revolution R) Works Today Hadoop Works Today Sqoop Works Today Hortonworks* By GA Apache Hadoop Version 2.8 and above Storm Works Today Distcp Works Today Cloudera* By GA HBase Works Today Spark By GA * Features arriving by GA

ADL Store Azure Blob Storage Scenarios Billing WebHDFS Authentication Optimized for Analytics General purpose bulk storage Billing Pay for amount stored and for I/O operations WebHDFS Implements WebHDFS No WebHDFS Authentication Azure Active Directory Access Keys Authorization POSIX-style ACLs Access Keys Data Encryption Transparent Server-side Encryption* Client-Side Encryption * Features arriving by GA

Ingress and Egress Services ADL SDKs Tools ADL REST endpoints SMSG Readiness 11/8/2018 Ingress and Egress Services Azure Data Factory ADL Copy Service Azure Import/Export Service Azure Stream Analytics* ADL SDKs .NET SDK Node.Js SDK Java SDK * Python SDK * Apache Sqoop(TM) - efficiently transferring bulk data between Hadoop and structured datastores such as relational databases. DistCp Version 2 (distributed copy) - for large inter/intra-cluster copying using MapReduce Tools Apache Sqoop™ DistCp Azure Portal Azure PowerShell Azure X-Platform CLI ADL REST endpoints Curl Any HTTP REST Client * Features arriving by GA © 2015 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Integration with Azure Data Factory SMSG Readiness 11/8/2018 Integration with Azure Data Factory Sources Sinks Azure Blob Azure Table Azure Blob Azure SQL Database Azure SQL Data Warehouse Azure Table Azure DocumentDB Azure Data Lake Store Azure SQL Database Note that all the source/sink options from SQL Server onwards can be hosted on-premises or on Azure IaaS. SQL Server File system Azure SQL Data Warehouse Oracle database MySQL database Azure DocumentDB DB2 database Teradata database Azure Data Lake Store Sybase database PostgreSQL database SQL Server © 2015 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

http://aka.ms/AzureDataLake