2. Go Large with SQL Azure Federations Do you want shards with that?
SESSION CODE: COS212
Chris Hewitt
Quant Analyst
AusAlgo
Go Large with SQL Azure Federations Do you want shards with that?
(c) 2011 Microsoft. All rights reserved.

3. Agenda
What
Why
How
Lessons

4. What are Federations in SQL Azure?
Federations are SQL Azure objects that support scaling-out to multiple databases providing:
Unlimited Scalability
Dynamic Elasticity
Simplified Multi-Tenancy

5. Scale CPU/DB size 11 Few Nodes Many Nodes
Small Capacity ‘Infinite’ Capacity
Scale Applications horizontally to many nodes.
Practically never run out of capacity:
Scale beyond scale-up and single database limitations (xxxGB limit db size limit, computational capacity of a single node)
Scale to massive computational capacity
Utilize unlimited storage
Fully exploit pay-as-you-go…
Expand and Contract without downtime to amplify elasticity
Build apps that grow with its workload over time.
Handle bursting workloads, peaks and valleys…

6. Tenancy Models Cloud Tenancy Models Multiple tenants per database
Multiple databases per tenant
Tenant_id =55
Tenant_id =55
Tenant_id =57
Single tenant per database
Tenant_id =55
Tenant_id =56
Cloud Tenancy Models
Single-database-per-tenant does not work for long tail and large tenants
Utilize multiple-tenants-per-database and multiple-databases-per-tenant as well for full flexibility

7. Typical Usage Scenarios
12/8/2018 6:08 PM
Typical Usage Scenarios
Web Scale
Multi Tenant
Workload Spikes
“NoSQL”
Web Scale DB Solutions
Scale requirements that need to break through the scale-up ceilings and scale to workloads to handle populations on the web
Multi Tenant SaaS ISVs
ISVs coming from Single Tenant-per-database model that are expanding to flexible tenancy model with multiple-tenants-per-database or multiple-databases-per-tenant.
Workloads with Spikes
Applications that has periods of spikes in their workloads with variance in capacity and scale requirements
NoSQL Applications (more on this next…)
(c) 2011 Microsoft. All rights reserved.
© 2007 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

8. NoSQL Gene in SQL Azure Federations
Scale-out for massive parallelism (Sharding)
Loosened Consistency Mode (somewhat ACIDic)
Lightweight local storage (tempdb)
Support for Unstructured and Semi Structured Data
NoSQL: Typically a different compromise on the CAP Theorem.
The CAP theorem, also known as Brewer's theorem, states that it is impossible for a distributed computer system to simultaneously provide all three of the following guarantees:
Consistency (all nodes see the same data at the same time)
Availability (a guarantee that every request receives a response about whether it was successful or failed)
Partition tolerance (the system continues to operate despite arbitrary message loss)
According to the theorem, a distributed system can satisfy any two of these guarantees at the same time, but not all three.[3]
SQL Azure Federations:
Scale-out for massive parallelism
First class sharding support in the platform
Loosened Consistency Mode
Eventual Consistency across members – members have independent schemas
Local consistency within members – members are databases with full ACID properties
Lightweight local storage as well as permanent consistent storage
With each federation member, you also get access to the local tempdb
Support for Unstructured and Semi Structured Data
XML, Blob types and Key-Value Pairs

9. Federations Federations CustomerFederationeration
SalesDB
CustomerFederationeration
CustomerFederationeration
CustomerFederation
Federation Root
Federation Members
Federations:
Federation are contained within a user database just like other objects. Above, SalesDB represent a user database with federations. There can be many federations to represent varying scale-out needs within a single database.
Federation Members:
Federation use regular SQL Azure databases to achieve scale-out. Databases contain parts of federations data are called federation members and are there to provide the computational capacity for parts of the federations workload.
Federation members cover the full range of data for a federation and are managed by the federation dynamically as data is repartitioned.

10. Atomic Units Federation Distribution Key Atomic Units
Federations
SalesDB
member: Range [1000, 2000)
CustomerFederationeration
CustomerFederationeration
CustomerFederation
AU PK=5
AU PK=25
AU PK=35
AU PK=5
AU PK=25
AU PK=35
AU PK=1005
AU PK=1025
AU PK=1035
Federation Root
Federation Members
Atomic Units
Federation Distribution Key
The key used for data distribution in the federations. In the federation definition, the distribution key represented by a label for the distribution key, a data type to specify the valid data domain for the distribution such as uniqueidentifier or bigint, and distribution type to specify the style for distributing the data such as range.
In SQL Azure Federations v1, the supported datatypes for federation key are: INT, BIGINT, UNIQUEIDENTIFIER, and VARBINARY(900).
Atomic Unit
Represent a single instance of a federation key. An AU contains all rows in all federated tables with the same federation key value. These collection of rows are guaranteed to stay together and is never SPLIT further into multiple members. With a federation distribution key such as tenant_id, atomic unit refers to all data for tenant_id=55 in federated tables.
Federation Root
The database that houses federation object and is the central repository for information about distribution of scaled-out data.

11. Another View A Federation Root Can contain many Federations
SalesDB
CustomerFederation
Can contain many Federations
Which may have many Federation Members
member: Range [1000, 2000)
Which contain Atomic Units
AU PK=5
AU PK=25
AU PK=35
AU PK=5
AU PK=25
AU PK=35
AU PK=1005
AU PK=1025
AU PK=1035

12. Types of Tables Global Tables Federated Tables Reference Tables
Federations
SalesDB
Federated Tables
Reference Tables
CustomerFederationeration
CustomerFederationeration
CustomerFederation
Federation root
Federation members
Federated Tables
Refer to tables that contain data that is distributed by the federation. Federated tables are created in federation members and contain a federation distribution key. Federated tables contain data that is part of the scaled out data.
Reference Tables
Refer to tables that contain reference information to optimize lookup queries in federations. Reference tables are created in federation members. Reference tables typically contain small lookup information useful for query processing such as zipcodes that is cloned to each federation member.
Central Table
Refer to tables that are created in the federation root for typically low traffic objects such as metadata.

13. Repartitioning Operations
ALTER FEDERATION CustomerFederation SPLIT AT (tenant_id=7500)
SalesDB
Orders_federation
CustomerFederationeration
CustomerFederation
Range Distribution [min,1000, 2000, 3000 … 5000, 10000, Max]
Repartitioning Operations:
Federation allow online repartitioning of data through T-SQL commands such as SPLIT.
By repartitioning CustomerFederation with SPLIT, administrators can move data to new federation members without downtime and expand computational capacity of the system.

14. Current and Future Repartitioning
-- In SQL Azure Federations V1 ALTER FEDERATION TickFederation SPLIT AT(iid = 100) GO -- (future) ALTER FEDERATION TickFederation SPLIT AT(10,20,30,…90) ALTER FEDERATION TickFederation MERGE AT(10,20,30,…90) ALTER FEDERATION TickFederation MERGE AT(iid = 100)

15. Data Dependent Routing
-- Connect to Root Database
-- then
USE FEDERATION CustomerFederation(customer_id = 5075) …
SalesDB
Orders_federation
CustomerFederationeration
CustomerFederation
Range Distribution [min,1000, 2000, 3000 … 5000, 10000, Max]
Data Dependent Routing:
Applications always connect to the root and issue the USE FEDERATION routing statement to connect to atomic units (for example customer_id = 5075).
As data is repartitioned, atomic units move around but address to the atomic unit guarantees routing to the correct federation member.

16. Connecting to Federations FILTERING = ON
Connection to Root - SqlClient, ODBC
“Server=tcp:servername.db.windows.net;Db=salesdb;User ID=uname;Password=pword;Encrypt=True”
-- THEN USE FEDERATION TO CONNECT TO THE MEMBER CONTAINING THE KEY.
USE FEDERATION CustomerFederation(cid = 55) WITH RESET, FILTERING=ON GO
SalesDB
member: Range [100,200)
CustomerFederation
CustomerFederation
Uszipcodes
(reference)
FILTERING=ON
Connect to the Atomic Unit
Reference Data stays fully visible.
No Changes allowed to the global state of the federation member:
No DDL, No DML on Reference Data
Good for…
Most of the workload of the application: safe programming model
Leakage Errors: Engine level predicate injection prevents data leakage
orders
(federated)
orderdetails
(federated)
Customer_id=155

17. Connecting to Federations FILTERING = OFF
Connection to Root - SqlClient, ODBC
“Server=tcp:servername.db.windows.net;Db=salesdb;User ID=uname;Password=pword;Encrypt=True”
-- THEN USE FEDERATION TO CONNECT TO THE MEMBER CONTAINING THE KEY.
USE FEDERATION CustomerFederation(cid = 155) WITH RESET, FILTERING=OFF GO
SalesDB
member: Range [100,199)
CustomerFederation
CustomerFederation
Uszipcodes
(reference)
FILTERING=OFF
Connects to the full member
Unrestricted Access to the member: Same as connecting to the database name
DDL, DML and Access to All Atomic Units within the members allowed
Good for…
Management Tasks: Schema Deployment
Fan-out Querying – queries spanning multiple atomic units
Customer_id=
orders
(federated)
orderdetails
(federated)

18. demo Federations 12/8/2018 6:08 PM
© 2007 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

19. … Demo Scenario Market DB Tick Federation Historical Events 100s? ? ? ?
100s
Results DB
Simulated Orders

20. IDENTITY No IDENTITY Columns in Federated Databases
Use UniqueIdentifier instead
Reference Data?

21. De-Normalised Federation Key has to be in each federated table
A unique or clustered index on a federated table must contain the federated column
(c) 2011 Microsoft. All rights reserved.

22. No Distributed Transactions
(c) 2011 Microsoft. All rights reserved.

23. SQL Azure Federations V1 (vs DIY)
Federations enable building data tiers with
unlimited scalability,
best price-performance with dynamic and online elasticity,
Simplifies multi-tenancy.
Federations Improve Developer Experience
Deliver Superior Management Experience
Federations Improve Developer Experience
Robust Connection Management
Federation Directory for connection routing
Filtering connections for safe programming model
Deliver Superior Management Experience
Online data redistribution through operations such as SPLIT.

24. Federations – Future Enhancements
Fan-out Queries
Schema Versioning and Management
Developer Experience on Premise
Fan-out Queries: Ability to query multiple members with a single query
Schema Versioning and Management: Fine grain control over schema rollout and upgrades.
Developer Experience on Premise: Full end to end development experience on premise for folks developing with on premise technologies.

25. Federation Technology Preview Nominations
Now Open! Federations Technology Preview Program Nominations
Information on How to Nominate your Application
Click on the Survey Link
Fill-out the Survey Questions
Wait for communication from the preview team!

26. Question & Answer Session
(c) 2011 Microsoft. All rights reserved.

27. Win a Touch Mouse! 10 prizes drawn every day!*
Complete your session evaluation on Schedule Builder for your chance to win a Windows 7 Touch Mouse
Exclusive for Windows 7
Eleven multi touch gestures
Contoured shape for superior comfort
10 prizes drawn every day!*
*Please see Registration for full terms and conditions

28. SIMPLY PARTICIPATING IN TECH∙ED AUSTRALIA
GET REWARDED FOR
SIMPLY PARTICIPATING IN TECH∙ED AUSTRALIA
DOWNLOAD QR CODE READER FOR YOUR MOBILE DEVICE
Windows Phone 7 we suggest Quick Reader
Symbian we suggest the built in reader or mobile Tag
IOS we suggest QR Reader
Blackberry we suggest Mobile Tag
Android phones we suggest Barcode Scanner
GO TO THE TECHQUEST WEBSITE & CREATE A PROFILE
EXAMPLE ONLY

29. (c) 2011 Microsoft. All rights reserved.
12/8/2018 6:08 PM
© 2010 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
(c) 2011 Microsoft. All rights reserved.
© 2007 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

30. Resources www.msteched.com/Australia Sessions On-Demand & Community
Microsoft Certification & Training Resources
technet.microsoft.com/en-au
Resources for IT Professionals
Resources for Developers
(c) 2011 Microsoft. All rights reserved.