Microsoft Connect 2016 11/24/2018 10:05 AM © 2016 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

U-SQL: a new developer language to process big data using C#/SQL Michael Rys Principal Program Manager @MikeDoesBigData

Azure Data Lake Analytics Store and analyze data of any kind and size Develop faster, debug and optimize smarter Interactively explore patterns in your data No learning curve Managed and supported Dynamically scales to match your business priorities Enterprise-grade security Built on YARN, designed for the cloud YARN U-SQL Azure Data Lake Analytics HDInsight Hive R Server HDFS Azure Data Lake Store

 Declarativity and Extensibility are equally native to the language! Get benefits of both: Declarativity: Does scale and performance for you Extensibility: Let’s you add your business logic Makes it easy for you by unifying: Unstructured and structured data processing Declarative SQL and custom imperative Code Queries over data in ADLA and SQL DBs in Azure Increase productivity and agility from Day 1 and at Day 100 for YOU! Why U-SQL? Traditional SQL for Big Data: + Declarative – Hard to extend Traditional Programming Languages for Big Data: + Extensible – Requires a lot of code/knowledge to scale and perform

The origins of U-SQL SCOPE – Microsoft’s internal Big Data language SQL and C# integration model Optimization and Scaling model Runs 100’000s of jobs daily Hive Complex data types (Maps, Arrays) Data format alignment for text files T-SQL/ANSI SQL Many of the SQL capabilities (windowing functions, meta data model etc.) The origins of U-SQL U-SQL SCOPE Hive T-SQL/ ANSI SQL

Query data where it lives Easily query data in multiple Azure data stores without moving it to a single store Benefits Avoid moving large amounts of data across the network between stores Single view of data irrespective of physical location Minimize data proliferation issues caused by maintaining multiple copies Single query language for all data Each data store maintains its own sovereignty Design choices based on the need Push SQL expressions to remote SQL sources Projections Filters Joins Azure Data Lake Storage Azure Storage Blobs Query Write Query Write Azure SQL in VMs Azure Data Lake Analytics Query U-SQL Query Query Query Azure SQL DB Azure SQL Data Warehouse

Microsoft Connect 2016 11/24/2018 10:05 AM Show me U-SQL https://github.com/Azure/usql/tree/master/Examples/TweetAnalysis https://github.com/Azure/usql/tree/master/Examples/ImageApp © 2016 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

This is why U-SQL! Unifies natively SQL’s declarativity and C#’s extensibility Unifies querying structured and unstructured data Unifies querying Data Lake and SQL Server (in Azure) data Increase productivity and agility from Day 1 forward for YOU! Sign up for an Azure Data Lake account at http://www.azure.com/datalake and give us your feedback via http://aka.ms/adlfeedback!

Additional Resources Blogs, presentations and community pages: http://aka.ms/AzureDataLake http://usql.io (U-SQL Github) http://blogs.msdn.microsoft.com/mrys/ http://blogs.msdn.microsoft.com/azuredatalake/ http://www.slideshare.net/MichaelRys Documentation, articles, and videos: http://aka.ms/usql_reference https://azure.microsoft.com/en-us/documentation/services/data-lake-analytics/ https://msdn.microsoft.com/en-us/magazine/mt614251 https://channel9.msdn.com/Search?term=U-SQL#ch9Search https://www.youtube.com/results?search_query=U-SQL ADL forums and feedback http://aka.ms/adlfeedback https://social.msdn.microsoft.com/Forums/azure/en-US/home?forum=AzureDataLake http://stackoverflow.com/questions/tagged/u-sql Re-visit Connect on Channel 9. Continue your education at Microsoft Virtual Academy online.

11/24/2018 10:05 AM © 2016 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.