1. Design considerations for storing data in the Cloud
Eric Nelson
Microsoft UK

2. Session Objectives SQL Azure vs/and Windows Azure Storage
Drill into Windows Azure Storage
Next session: Keith Burns drills into SQL Azure

3. New online community http://ukazure.ning.com/

4. Windows Azure Platform

5. Windows Azure Platform
Windows Azure – Your code and data
SQL Azure Database – Your relational data
AppFabric* – Your messages
Code
Data
Relational
Data
Firewall friendly messaging between cloud and on-premise

6. Azure Application = WebRole + Worker Role + Storage
Your Service
Web Site
(ASPX, ASMX, WCF)
Web Site
(ASPX, ASMX, WCF)
Web Role
(ASPX, WCF)
Queues LB
Internet
Worker Service
Worker Role LB
Blobs
Tables
SQL Azure

7. Windows Azure and SQL Azure
Azure Storage
SQL Azure
Vision
Access
Relational? (today)
Relational? (tomorrow)
Analogy

8. Windows Azure and SQL Azure
Azure Storage
SQL Azure
Vision
Highly scalable, highly available store in the Cloud
Access
Uses WCF Data Services - REST
Relational? (today) No
Relational? (tomorrow)
Analogy

9. Windows Azure and SQL Azure
Azure Storage
SQL Azure
Vision
Highly scalable, highly available store in the Cloud
Scalable, highly available relational store in the Cloud
Access
Uses WCF Data Services - REST
SqlClient + TSQL
(Use Azure Web Role + WCF Data Services if REST is required)
Relational? (today) No
Yes – but with some limitations
Relational? (tomorrow)
Yes – with less and less limitations
Analogy

10. Windows Azure and SQL Azure
Azure Storage
SQL Azure
Vision
Highly scalable, highly available store in the Cloud
Scalable, highly available relational store in the Cloud
Access
Uses WCF Data Services - REST
SqlClient + TSQL
(Use Azure Web Role + WCF Data Services if REST is required)
Relational? (today) No
Yes – but with some limitations
Relational? (tomorrow)
Yes – with less and less limitations
Analogy
File System
RDBMS – as it is 

11. Cost impacting Architecture

12. Windows Azure and SQL Azure
Azure Storage
SQL Azure
Maximum Amount of data in a single “database”
Price per GB per month
Price to store 1KB per month

13. Windows Azure and SQL Azure
Azure Storage
SQL Azure
Maximum Amount of data in a single “database”
100TB
10GB (50GB this summer)
Price per GB per month
Price to store 1KB per month

14. Windows Azure and SQL Azure
Azure Storage
SQL Azure
Maximum Amount of data in a single “database”
100TB
10GB (50GB this summer)
Price per GB per month
£0.09
£6.06
Price to store 1KB per month

15. Windows Azure and SQL Azure
Azure Storage
SQL Azure
Maximum Amount of data in a single “database”
100TB
10GB (50GB this summer)
Price per GB per month
£0.09
£6.06 *
Price to store 1KB per month
£0.00

16. Azure Application = WebRole + Worker Role + Storage
Your Service 2
Web Site
(ASPX, ASMX, WCF)
Web Site
(ASPX, ASMX, WCF)
Web Role
(ASPX, WCF)
Queues LB
Internet
Worker Service
Worker Role LB
Blobs
Tables 1
SQL Azure

17. SQL Azure

18. + SQL Azure http://www.mygreatsqlazureidea.com SQL Azure Database
“Next big thing”

19. SQL Azure Application Architectures
SQL Azure access from within Datacenter
SQL Azure access from outside
Application/ Browser
Application / Tools
SOAP/REST
HTTP/S
WCF Data Svcs/REST - EF
HTTP/S
Web Role
Windows Azure
T-SQL (TDS)
T-SQL (TDS)
SQL Azure
SQL Azure
Code Near
Code Far

20. Demo
The sys.database_usage view lists the number, type, and duration of databases on the server and the sys.bandwidth_usage view describes the bandwidth used with each database.
The sys.database_usage view contains the following fields:
Field Definition time
The date when the usage events occurred.
sku
The type of database, either Business for the Business Edition or Web for the Web Edition.
quantity
The maximum number of databases of an SKU type that existed during that day. When a database is created, the quantity is increased immediately. When a database is dropped, the quantity is reduced the next day.
The sys.bandwidth_usage view contains the following fields:
The hour when the bandwidth was consumed. The rows in this view are on a per-hour basis. For example, :00: means that the bandwidth was consumed on September 19, 2009 between 2:00 A.M. and 3:00 A.M.
database_name
The name of the database that used bandwidth.
direction
The type of bandwidth that was used.
Ingress is data moving into the SQL Azure database.
Egress is data moving out of the SQL Azure database.
class
The class of bandwidth that was used.
Internal is data from within the Windows Azure platform.
External is data from outside the Windows Azure platform.
time_period
The time period when the usage occurred is either Peak or OffPeak. The Peak time is based on the region in which the server was created. For example, if a SQL Azure server was created in the "US_Northwest" region, the Peak time is defined as being between 10:00 A.M. and 6:00 P.M. PST.
The amount of bandwidth, in kilobytes (KBs), that was used.

21. Windows Azure Storage “NoSQL”

22. Azure Storage Storage Queues Blobs Account Tables
Blobs
Account
Tables

23. Queues
Queues
Messages
“Body1”
“Body2”
ThumbnailJobsQ
“Body3”

24. Blobs Containers Blobs Each Blob is a list of blocks Block operation
Photo1.png
Photo2.png
Easy to use REST Put/Get/Delete interface
Can read from any Offset, Length of Blob
Conditional Put and Get Blob
Max Blob size
50 GB using PutBlock and PutBlockList
64 MB using PutBlob
Blocks provide continuation for blob upload
Put Blob/BlockList == Replace Blob for CTP
Can replace an existing blob with new blob/blocks
Each Blob is a list of blocks
A Block can be up to 4MB each
Each block can be variable size
Each block has a 64 byte ID, scoped by blob name
Block operation
PutBlock
Puts an uncommitted block defined by the block ID for the blob
Block list operations
PutBlockList
Provide the list of blocks to comprise the readable version of the blob
Can use uncommitted blocks only
GetBlockList
Returns the list of committed blocks (not the block data)
Block ID and size of block are returned for each block
Pictures
Photo3.png

25. Content Delivery Network
“Near” London
“Near” Sydney
CDN Node
Photo1.png
Containers
Pictures
Photo1.png
Blobs
Photo3.png
Photo2.png
Default time 72 hours
Cache-control header – in put blob set header or set properties
“Near” London
“Near” Sydney
Photo1.png
Windows Azure Blob service URL:
Photo1.png
Windows Azure CDN URL:
Photo1.png

26. Tables - EAV MyAzureTable1 Entity: Shiny red Car
PK, RK, TS, att1, att2, att8
Entity: Blue Bird
PK, RK, TS, att1, att4
Entity: 2008 Tax Return
PK, RK, TS, att1, att2, att8, att9, att23, att46, att40

27. Windows Azure Tables Massively Scalable Tables Highly Available
Billions of entities (rows) and TBs of data
Automatically scales to thousands of servers as traffic grows
Highly Available
Can always access your data
Durable
Data is replicated at least 3 times

28. No Referential Integrity
Tables - Capabilities
What tables don’t do
What tables can do
Not relational 
Cheap 
No Referential Integrity 
Very Scalable 
If these are important to you, use SQL Azure
No Joins 
Flexible 
Limited Queries 
Durable 
No Group by 
Store LOTS of stuff  
No Aggregations 
No Transactions 
No Transactions 

29. Demo
The Query Tables operation supports the query options defined by the ADO.NET Data Services Framework. For more information, see ADO.NET Data Services Query Options (1st link below). So here are some of the things that could help you get going:
You need not URI Encode your query as it will be done by application itself.
Please escape single quotes with two single quotes. For example: if your partition key is a’a then use a’’a.
Query table operation in Azure storage does not support SELECT clause just yet. So if you want to query for a particular condition just specify that condition. For example if you want to query your table for certain PartitionKey values, just type (PartitionKey eq 'YourPartitionKey').
Some examples of queries:
(PartitionKey eq '1')
(RowKey eq 'test')
(Name eq 'Amit')
(PartitionKey eq '1' and RowKey gt '5'), (PartitionKey eq ‘1’ and RowKey lt ‘10’ or Name eq ‘Amit’)

30. Summary
Important: You have SQL Azure and Windows Azure Storage to take advantage of
SQL Azure has a well known API and behavior
Windows Azure Storage has a new API and behavior
UK Site
UK Community /
My blog