1. U-SQL Object Model

2. Meta Data Object Model Legend Procedures Table Types ADLA Catalog
C# Extractors
C# Reducers
Database
C# Processors
C# Fns
C# UDTs
C# UDAgg
C# Applier
[1,n]
C# Combiners
C# Outputters
Creden-tials
Data Source
Schema
C# Assemblies
[0,n]
Ext. tables
tables
views
TVFs
Procedures
Table Types
Legend
Statistics
Clustered Index
Abstract objects
User objects
MD Name
C# Name
partitions
Contains
Refers to
Implemented
and named by

3. U-SQL Catalog Naming Discovery Sharing Naming Securing Discovery
Default database and schema context: master.dbo
Quote identifiers with []: [my table]
Stores data in ADL Storage /catalog folder
Discovery
Visual Studio Server Explorer
Azure Data Lake Analytics Portal
SDKs and Azure PowerShell commands
Sharing
Within an Azure Data Lake Analytics account
Securing
Secured with AAD principals at catalog level (inherited from ADL Storage)
Naming
Discovery
Sharing
Securing

4. DEMO
Create shareable data and code

5. Views and TVFs Views for simple cases
CREATE VIEW V AS EXTRACT…
CREATE VIEW V AS SELECT …
Cannot contain user-defined objects (such as UDFs or UDOs)
Will be inlined
Table-Valued Functions (TVFs)
CREATE FUNCTION F string = "default") [TABLE ( … )]
AS BEGIN = … END;
Provides parameterization
One or more results
Can contain multiple statements
Can contain user-code (needs assembly reference)
Will always be inlined
Infers schema or checks against specified return schema
Views for simple cases
TVFs for parameterization and most cases

6. Procedures Allows encapsulation of non-DDL scripts
CREATE PROCEDURE P string = "default“) AS
BEGIN
…; TO …;
INSERT INTO T …;
END;
Provides parameterization
No result but writes into file or table
Can contain multiple statements
Can contain user code (needs assembly reference)
Will always be inlined
Cannot contain DDL (no CREATE, DROP)
Allows encapsulation of non-DDL scripts

7. Table types Enables you to name a table schema
Provides reuse for function/procedure definitions
CREATE TYPE T AS TABLE(c1 string, c2 int );
CREATE FUNCTION F T) T AS BEGIN = … END;

8. Tables CREATE TABLE CREATE TABLE AS SELECT CREATE TABLE T (col1 int
, col2 string
, col3 SQL.MAP<string,string>
, INDEX idx CLUSTERED (col1 ASC)
PARTITIONED BY HASH (driver_id) );
Structured Data
Built-in Data types only (no UDTs)
Clustered index (must be specified): row-oriented
Fine-grained partitioning (must be specified):
HASH, DIRECT HASH, RANGE, ROUND ROBIN
CREATE TABLE T (INDEX idx CLUSTERED …) AS SELECT …;
CREATE TABLE T (INDEX idx CLUSTERED …) AS EXTRACT…;
CREATE TABLE T (INDEX idx CLUSTERED …) AS myTVF(DEFAULT);
Infer the schema from the query
Still requires index and partitioning
CREATE TABLE
CREATE TABLE AS SELECT