Art by Andrew Fryer

Dan Cohen Microsoft Consulting Services Microsoft Israel Team blog: My blog:

Agenda Developer tools Windows Azure Compute Compute Storage Storage SQL Azure Windows Azure platform AppFabric Windows Azure Interoperability Monitoring and Debugging Microsoft Codename “Dallas”

Operating System App/Web Server Frameworks Your Application OS Services Virtualized Instance Hardware Windows Azure Platform.NET based Application Platform as a Service (PaaS) Platform as a Service (PaaS) Windows Azure OS IIS / WAS.NET Framework Lots of stuff… Custom Hyper-V Hardware

hardware software facilities power/cooling IT labor suppor t network securi ty maintenanc e managemen t tools disaster recovery backup Acquisition cost is 10% of IT Spend Operating cost is 90% of IT Spend Source: IDC

Windows Azure Platform

Windows Azure Compute Demo: Create an account / project

Windows Azure, On One Slide The “cloud” is 1000s of geo-distributed servers & networking Windows Azure is analogous to An operating system for the cloud The “operating system by analogy” provides Application hosting Automated service management Upload your code and “service model”; press START Durable storage at massive scale

Windows Azure, In One Picture Desktop VS Tools WA SDK Storage Cluster MSFT Datacenters Business Portal Developer Portal Compute Cluster … … Service Management Service REST … Cloud VM Runtime API VM Setup User Code

Azure Service Architectures A service consists of An isolation boundary A set of component roles each with endpoints Numbered, identical instances of each role

Azure Service Architectures A role instance is a set of: CodeConfiguration local data deployed in a dedicated VM Web Role: The role is hosted on IIS Worker Role: The role is an executable create your own web server host a database host Java VM, Ruby etc.) Worker Role Web Role Web Role

Azure Service Architectures A service is reachable externally via a single VIP for load-balanced traffic to multiple endpoints Endpoints are reachable internally over TCP, HTTP Runtime API exposes & updates IP/Port values LB

Windows Azure Compute Demo: “Hello Cloud” with Visual Studio 2008/ 2010

Sample: Notification Of New Instances

Sample: Inter-role communication

2 VM Roles (currently) available: Web role External synchronous HTTP/S communication Worker role External HTTP/S / TCP communication Mental model: “it’s like a Windows Service” Windows Azure Operating System Automatic /manual upgrades See: Storage Local storage is volatile (!) 4 standard sizes Small -> Extra Large See:

100% of the time, you will be in the 10% where this operation takes more than 115 seconds.

BlobContainerAccount cohowinery images PIC01.JPG PIC02.JPG videos VID1.AVI /

CloudStorageAccount account = CloudStorageAccount.FromConfigurationSetting("CloudStorageAccount"); CloudBlobClient blobClient = new CloudBlobClient( account.BlobEndpoint, account.Credentials); // Create Container CloudBlobContainer cloudContainer = blobClient.GetContainerReference(containerName); bool hasCreated = cloudContainer.CreateIfNotExist(); // Access Blob in the Container CloudBlob cloudBlob = cloudContainer.GetBlobReference(blobName); //BlobRequestOptions has retry policy, timeout etc. BlobRequestOptions options = new BlobRequestOptions(); //Upload the local file to Blob service cloudBlob.UploadFile(uploadFileName, options); //Download to local file name cloudBlob.DownloadToFile(downloadFileName, options);

10 GB Movie Windows Azure Storage Block Id 1 Block Id 2 Block Id 3 Block Id N blobName = “blob.wmv”; PutBlock(blobName, blockId1, block1Bits); PutBlock(blobName, blockId2, block2Bits); ………… PutBlock(blobName, blockIdN, blockNBits); PutBlockList(blobName, blockId1, blockId2…,blockIdN); blobName = “blob.wmv”; PutBlock(blobName, blockId1, block1Bits); PutBlock(blobName, blockId2, block2Bits); ………… PutBlock(blobName, blockIdN, blockNBits); PutBlockList(blobName, blockId1, blockId2…,blockIdN); blob.wmv

0 10 GB 10 GB Address Space

Windows Azure Blob Service DemoBlob Application Lease Drive X:

//Create Local Storage resource and initialize the local cache for drives CloudDrive.InitializeCache(localCacheDir, cacheSizeInMB); CloudStorageAccount account = CloudStorageAccount.FromConfigurationSetting("CloudStorageAccount"); //Create a cloud drive (PageBlob) CloudDrive drive = account.CreateCloudDrive(pageBlobUri); drive.Create(1000 /* sizeInMB */); //Mount the network attached drive on the local file system string pathOnLocalFS = drive.Mount(cacheSizeInMB, DriveMountOptions.None); //Use NTFS APIs to Read/Write files to drive … //Snapshot drive while mounted to create backups Uri snapshotUri = drive.Snapshot(); //Unmount the drive drive.Unmount();

EntityTableAccount cohowinery customers Name =… = … Name =… = … winephotos Photo ID =… Date =… Photo ID =… Date =…

PartitionKey (Category) RowKey (Title) TimestampReleaseDate Action Fast & Furious…2009 Action The Bourne Ultimatum…2007 … ……… Animation Open Season 2…2009 Animation The Ant Bully…2006 PartitionKey (Category) RowKey (Title) TimestampReleaseDate Comedy Office Space…1999 … ……… SciFi X-Men Origins: Wolverine…2009 … ……… War Defiance…2008 PartitionKey (Category) RowKey (Title) TimestampReleaseDate Action Fast & Furious…2009 Action The Bourne Ultimatum…2007 … ……… Animation Open Season 2…2009 Animation The Ant Bully…2006 Comedy Office Space…1999 … ……… SciFi X-Men Origins: Wolverine…2009 … ……… War Defiance…2008 Server B Table = Movies [Comedy - MaxKey) Server B Table = Movies [Comedy - MaxKey) Server A Table = Movies [MinKey - Comedy) Server A Table = Movies [MinKey - Comedy) Server A Table = Movies Server A Table = Movies

[DataServiceKey("PartitionKey", "RowKey")] public class Movie { /// Movie Category is the partition key public string PartitionKey { get; set; } /// Movie Title is the row key public string RowKey { get; set; } public DateTime Timestamp { get; set; } public int ReleaseYear { get; set; } public double Rating { get; set; } public string Language { get; set; } public bool Favorite { get; set; } }

CloudStorageAccount account = CloudStorageAccount.FromConfigurationSetting("CloudStorageAccount"); CloudTableClient tableClient = new CloudTableClient( account.TableEndpoint, account.Credentials); // Create Movie Table string tableName = “Movies“; tableClient.CreateTableIfNotExist(tableName); TableServiceContext context = tableClient.GetDataServiceContext(); // Add movie context.AddObject(tableName, new Movie("Action", “White Water Rapids Survival")); context.SaveChangesWithRetries(); // Query movie var q = (from movie in context.CreateQuery (tableName) where movie.PartitionKey == "Action" && movie.Rating > 4.0 select movie).AsTableServiceQuery (); foreach (Movie movieToUpdate in q) { movieToUpdate.Favorite = true; context.UpdateObject(movieToUpdate); } context.SaveChangesWithRetries( ); SaveChangesOptions.Batch

MessageQueueAccount order processing customer ID order ID cohowinery

Azure Queue Input Queue (Work Items)

Azure Queue Input Queue (Work Items)

CloudStorageAccount account = CloudStorageAccount.FromConfigurationSetting("CloudStorageAccount"); CloudQueueClient queueClient = new CloudQueueClient( account.QueueEndpoint, account.Credentials); //Create Queue CloudQueue queue = queueClient.GetQueueReference(queueName); queue.CreateIfNotExist(); //Add Message CloudQueueMessage message = new CloudQueueMessage(“some content"); queue.AddMessage(message); //Get Message message = queue.GetMessage(TimeSpan.FromMinutes(3) /*Invisibility timeout*/); // Process Message within the Invisibility Timeout //Delete Message queue.DeleteMessage(message);

Windows Azure Storage Explorers (April 2010) Windows Azure Storage ExplorerBlock BlobPage BlobTablesQueuesFree Azure Blob ClientXY Azure Blob Compressor Azure Blob Compressor Enables compressing blobs for upload and download XY Azure Blob ExplorerXY Azure Storage ExplorerXXXY Azure Storage Simple ViewerX XXY Cerebrata Cloud Storage StudioXXXXY/N Cloud Berry ExplorerXXY Clumsy Leaf Azure Explorer Clumsy Leaf Azure Explorer Visual studio plug-in XXXXY Factonomy Azure UtilityXY Gladinet Cloud DesktopXN MyAzureStorage.com MyAzureStorage.com A portal to access blobs, tables and queues XXXXY Space BlockXY Windows Azure Management ToolXXXXY

Cerebrata Cloud Storage Studio

Azure MMC Snap-in

Cerebrata – Azure Diagnostics Manager

Business Analytics Reporting

Windows Azure Compute Demo: Create a SQL Azure database

SQL Azure Network Topology Application Internet Azure Cloud Internet Azure Cloud LB TDS (tcp) Applications use standard SQL client libraries: ODBC, ADO.Net, PHP, … Load balancer forwards ‘sticky’ sessions to TDS protocol tier Security Boundary SQL Gateway Scalability and Availability: Fabric, Failover, Replication, and Load balancing Gateway: TDS protocol gateway, enforces AUTHN/AUTHZ policy; proxy to backend

SQL Azure Network Topology Application Internet Azure Cloud Internet Azure Cloud LB TDS (tcp) Applications use standard SQL client libraries: ODBC, ADO.Net, PHP, … Load balancer forwards ‘sticky’ sessions to TDS protocol tier Security Boundary SQL Gateway Scalability and Availability: Fabric, Failover, Replication, and Load balancing Gateway: TDS protocol gateway, enforces AUTHN/AUTHZ policy; proxy to backend

Application Topologies SQL Azure access from within MS Datacenter (Azure compute) SQL Azure Access from outside MS Datacenter (On-premises) SQL Azure Access from within and outside MS Datacenter (On-premises & Azure Compute) Application/ Browser SOAP/RESTHTTP/S Astoria/REST - EDM HTTP/S App Code (ASP.NET) App Code (ASP.NET) App Code (ASP.NET) App Code (ASP.NET) T-SQL (TDS) SQL Azure WindowsAzure Code Near App code/ Tools T-SQL (TDS) SQL Azure WindowsAzure Code Far Hybrid SQL Azure Data Sync Data Sync WindowsAzure SQL Azure SQL Server App code/ Tools App Code (ASP.NET) App Code (ASP.NET) App Code (ASP.NET) App Code (ASP.NET) T-SQL (TDS)

SQL Azure Cluster Node 55 DB3 Node 14 Node 19 Node 21 Node 99 Node 2 Node 76 Shards: Managing Lots of Data App Server Master DB DB1 DB2 Node 33 DB3 DB4 DB5 DB6

SQL Azure Cluster Node 14 Node 19 Node 21 Node 99 Node 2 Node 76 Copies: Managing Lots of Reads App Server Master DB DB C1 DB C2 Node 33 DB C3 DB C4 DB C5 DB C6 LB

What are your application’s requirements? Storage and Transactional throughput Storage Requirements LowHigh Transactional Requirements Low High Single Database No Partitioning Partitioned Data Partitioning Based on Application Requirements (Storage) Partitioned Data Partitioning based on Application Requirements (IOPS) Partitioned Data Partitioning based on Application Requirements (IOPS, Storage or both)

SQL Azure Tips Developing on a local SQL Express instance has some advantages Easy to get started, you already know how to do it! Full fidelity with the designer and debugging tools Reduces latency when using local Azure development tools Reduces bandwidth and databases costs for development Some caveats Remember to alter your VS build settings to switch the connection string when deploying Use tools (like SQLAzureMW) to keep you within the supported SQL Azure features SQLAzureMW Always test in SQL Azure before deploying to production

SQL Azure Migration Wizard

SQL Azure Tips SQL Azure connection strings follow normal SQL syntax Applications connect directly to a database “Initial Catalog = ” in connection string No support for context switching (no USE ) Some commands must be in their own batch Create/Alter/Drop Database & Create/Alter/Drop Login, & Create/Alter USER with FOR/FROM LOGIN Encryption security Set Encrypt = True, only SSL connections are supported TrustServerCertificate = False, avoid Man-In-The-Middle-Attack! Format of username for authentication: ADO.Net: Data Source=server.database.windows.net; User Setup your firewall rules first!

SQL Azure Tips SQL Connections: Retry on failure Connections can drop for variety of reasons Idleness Transient (network) errors Intentional throttling First step: reconnect immediately Handles idleness- and transient-disconnects Gateway handles connection retry for app Connections attempted for ~30s before failure What to do on connection failure? Wait (10 seconds), then retry Change your workload if throttled

SQL Connection Retry Pattern while (true) { using (SqlConnection connection = new SqlConnection(connStr)) using (SqlConnection connection = new SqlConnection(connStr)) { try try { connection.Open(); connection.Open(); using (SqlCommand cmd = connection.CreateCommand()) using (SqlCommand cmd = connection.CreateCommand()) { cmd.CommandText cmd.CommandText cmd.CommandType = CommandType.StoredProcedure; cmd.CommandType = CommandType.StoredProcedure; cmd.Parameters.Add(new BCPJobId)); cmd.Parameters.Add(new BCPJobId)); cmd.ExecuteNonQuery(); cmd.ExecuteNonQuery(); } } catch (Exception exc) catch (Exception exc) { // deal with error // deal with error } } // more stuff // more stuff // ….. // …..}

Data Access APIs Supported APIs Connection String ADO.Net Encrypt=True and to User ID ODBC Encrypt=yes and to Uid SupportedUnsupported ADO.Net.Net 3.5 SP1 and 4.0 ODBC - SNAC 10 Entity Framework.Net 3.5 SP1 and 4.0 SQL Server 2008 Driver for PHP v1.1 OleDB

Data Access APIs cont. Protocols Authentication Mode SupportedUnsupported TCP/IP over port 1433Dynamic ports Named pipes Shared memory SupportedUnsupported SQL AuthWindows Auth

SQL Azure Labs Data Sync

Windows Azure Platform

Service Remoting Control Web services through the Internet using Service Bus Sender solicits information from listeners Service Bus Service Bus Access Control

Eventing Notify remote parties of events Sender transmits information to listeners Listeners may handle events in different ways Service Bus Service Bus Access Control

Tunneling Traverse network borders without opening firewalls Use an HTTP channel to mimic needed protocol Service Bus Service Bus Access Control

Azure AppFabric scenario #1 Scenario: How do we connect Org. A to Org. B ? Employees of Org. B wish to be informed of events from org. A. Organization A Internal Application Organization B Internal Application

Azure AppFabric scenario Scenario: How do we connect Org. A to Org. B ? Option #1 – On-premise integration BizTalk + SQL + Windows + firewall licenses ? Hardware, electricity, hosting ? Maintenance, personnel, support services ? Organization A Internal Application Organization B Internal Application Firewall BizTalk + SQL Server + Windows Servers BizTalk + SQL Server + Windows Servers

Azure motivation scenario Scenario: How do we connect Org. A to Org. B ? Option #2 – Windows Azure AppFabric integration Cost: 3.99$ per connection / month (volume discount) Data transfers: $0.10 in / $0.15 out per GB Organization A Internal Application Organization B Internal Application Service Bus Service Bus Access Control

Question: Is BizTalk Server Dead ? In a word: No. In two words: Hell, no!

Business Partner CRM HR E-Commerce ERP Business Partner Integration Server (EAI/B2B) Enterprise Service Bus (ESB) Windows Azure Platform Cloud-enabled SOA platform AppFabric Service Bus

Device On-Premise Cloud Enabling hybrid applications with AppFabric Request processing and scheduling Website Front End Website Front End Real Estate agent laptop/ device Real Estate agent laptop/ device CRM System

Device On-Premise Cloud Enabling hybrid applications with AppFabric Request processing and scheduling Website Front End Website Front End Real Estate agent laptop/ device Real Estate agent laptop/ device CRM System

Cloud On-Premise Enabling hybrid applications with AppFabric Request processing and scheduling Website Front End Website Front End Service Bus Service Bus Access Control

Cloud On-Premise Enabling hybrid applications with AppFabric Request processing and scheduling Website Front End Website Front End Service Bus Service Bus Access Control

Cloud On-Premise Enabling hybrid applications with AppFabric Request processing and scheduling Website Front End Website Front End Service Bus Service Bus Access Control Rules OUT claims IN claims

Cloud On-Premise Enabling hybrid applications with AppFabric Website Front End Website Front End Service Bus Service Bus Access Control Request processing and scheduling two-way one-way multicast multiplex pub/sub buffered Communication Patterns

Demo: Simple Publish/Subscribe & Multicast Enabling hybrid applications with AppFabric

On-Premise Site B On-Premise Site B On-Premise Site A On-Premise Site A Service Bus Service Bus Access Control

Demo: Distributed Cloud based app with Windows Azure & AppFabric Enabling hybrid applications with AppFabric

Windows Azure LB Web Role Worker Role Azure Queue

Windows Azure Interoperability

Hosting non-MS applications Mental Model: Windows OS Non-admin user USB drive with “runme.bat” Can your app run that way? Use Worker Role Configure Endpoints as necessary Call Process.Start(…) Continue monitoring from the Worker Role

Windows Azure Storage 4 Java Windows Azure Storage Blobs, tables, queues, drives Highly scalable REST interface Java storage library (windowsazure4j.org) windowsazure4j.org Built by Soyatec

Resources Building Java Applications with Windows Azure Developing PHP & MySQL Applications with Windows Azure Ruby on Rails on Windows Azure: Sample App: Downloads:

What is PaaS ? Cloud infrastructure, built bottom up VM Image Deployment; Network Programming; VM Allocation Global Datacenters Replicated Data Store Service Model Processing; Service Health Model Storage Abstractions Blobs, Tables, Queues Programming Model Developer Tools Developer platform, designed top down Load Balanced Partitions Service Management

Thank you!