1. Microsoft Connect 2016 11/24/2018 10:05 AM
© 2016 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

2. U-SQL: a new developer language to process big data using C#/SQL
Michael Rys
Principal Program Manager
@MikeDoesBigData

3. Azure Data Lake Analytics
Store and analyze data of any kind and size
Develop faster, debug and optimize smarter
Interactively explore patterns in your data
No learning curve
Managed and supported
Dynamically scales to match your business priorities
Enterprise-grade security
Built on YARN, designed for the cloud
YARN
U-SQL
Azure Data Lake Analytics
HDInsight
Hive
R Server
HDFS
Azure Data Lake Store

4.  Declarativity and Extensibility are equally native to the language!
Get benefits of both:
Declarativity: Does scale and performance for you
Extensibility: Let’s you add your business logic
Makes it easy for you by unifying:
Unstructured and structured data processing
Declarative SQL and custom imperative Code
Queries over data in ADLA and SQL DBs in Azure
Increase productivity and agility from Day 1 and at Day 100 for YOU!
Why U-SQL?
Traditional SQL for Big Data:
+ Declarative
– Hard to extend
Traditional Programming Languages for Big Data:
+ Extensible
– Requires a lot of code/knowledge to scale and perform

5. The origins of U-SQL SCOPE – Microsoft’s internal Big Data language
SQL and C# integration model
Optimization and Scaling model
Runs 100’000s of jobs daily
Hive
Complex data types (Maps, Arrays)
Data format alignment for text files
T-SQL/ANSI SQL
Many of the SQL capabilities (windowing functions, meta data model etc.)
The origins of U-SQL
U-SQL
SCOPE
Hive
T-SQL/ ANSI SQL

6. Query data where it lives Easily query data in multiple Azure data stores without moving it to a single store
Benefits
Avoid moving large amounts of data across the network between stores
Single view of data irrespective of physical location
Minimize data proliferation issues caused by maintaining multiple copies
Single query language for all data
Each data store maintains its own sovereignty
Design choices based on the need
Push SQL expressions to remote SQL sources
Projections
Filters
Joins
Azure
Data Lake Storage
Azure
Storage Blobs
Query
Write
Query
Write
Azure SQL in VMs
Azure Data
Lake Analytics
Query
U-SQL
Query
Query
Query
Azure SQL DB
Azure SQL Data Warehouse

7. Microsoft Connect 2016
11/24/ :05 AM
Show me U-SQL
© 2016 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

8. This is why U-SQL!
Unifies natively SQL’s declarativity and C#’s extensibility
Unifies querying structured and unstructured data
Unifies querying Data Lake and SQL Server (in Azure) data
Increase productivity and agility from Day 1 forward for YOU!
Sign up for an Azure Data Lake account at and give us your feedback via

9. Additional Resources Blogs, presentations and community pages:
(U-SQL Github)
Documentation, articles, and videos:
ADL forums and feedback
Re-visit Connect on Channel 9.
Continue your education at Microsoft Virtual Academy online.

10. 11/24/ :05 AM
© 2016 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.