Cloud Computing Systems Lin Gu Hong Kong University of Science and Technology Sept. 26, 2011 Windows Azure—Microsoft Cloud Computing Technologies

Cloud Systems Infrastructure as a Service (IaaS): basic compute and storage resources – E.g., Amazon AWS/EC2, VMWare vCloud Platform as a Service (PaaS): cloud application infrastructure – E.g. Google App Engine, Salesforce.com, Windows Azure Software as a Service (SaaS): cloud applications – E.g. Google Docs, Microsoft Office Web Companions, Office 365

How does cloud differ from: Leased computers Fabric computing/unified computing Cluster Computing Grid Computing

When the Sloan Digital Sky Survey started work in 2000, its telescope in New Mexico collected more data in its first few weeks than had been amassed in the entire history of astronomy. Now, a decade later, its archive contains 140TB of information.

The Large Synoptic Survey Telescope, which is perhaps world’s largest digital camera (3200M pixels) due to come on stream in Chile in 2016, will acquire that quantity of data every five days.

Windows Azure – Why not just Windows? Since 1970’s, microprocessors revolutionized the IT industry Commoditization of computing and communication Miniaturization of computing capability – Preemptive multitasking, multiple-issue out-of-order processors, virtualization migrate to PC-class computers What brings you here cannot take you there: the capacity we need today exceeds even the largest single supercomputer ever built – Beyond scientific computing: larger data and massive concurrency, interactive applications, business transactions, …

… Fabric Compute Storage Application Fabric Cloud Based Computation with Windows Azure VM Web RoleWorker Role Agent main() { … } Load Balancer HTTP IIS ASP.NET, WCF, etc.

High-Level Architecture Hypervisor Guest PartitionHost PartitionGuest Partition Hardware Virtualization Stack (VSP) Drivers Host OS Server Core Applications Virtualization Stack (VSC) Guest OS Server Enterprise Virtualization Stack (VSC) Guest OS Server Enterprise NIC Disk1 VMBUS Disk2 CPU

Windows Azure Compute Service Virtual Machines The VMs are provided by a cloud-optimized hypervisor Applications see a 64-bit Windows Server 2008 interface – Development fabric: a desktop replica of Windows Azure

Windows Azure Deployment Step 1: Allocate nodes – Across fault domains, across update domains Step 2: Place OS and role images on nodes Step 3: Configure settings Step 4: Configure load- balancers, Start Roles Step 5: Maintain desired number of roles – Failed roles automatically restarted – Node failure results in new nodes automatically allocated Fault-tolerant high-performance distributed computing “fabric” Load-balancers

HV-enabled Server Core base VHD Image-Based Deployment Host Partition Host partition differencing VHD Guest Partition Guest partition differencing VHD Server Enterprise base VHD Guest partition differencing VHD Application VHD Server Core base VHD Server Enterprise base VHD Maintenance OS App1 Package App3 Package App2 Package Guest Partition

… Fabric Compute Storage Application BlobsQueues HTTP Windows Azure Storage Service Types of Storage Tables

Table... Table Windows Azure Storage Tables Entity... Entity Property Name Type Value Storage Accounts

Windows Azure Storage Tables: Limitation Access via REST – You can’t use ordinary ADO.NET No SQL – A new approach for developers to learn – No aggregates – You can’t easily move relational data to or from it No schema – Entities and properties are not exactly rows (tuples) and columns (attributes)

Windows Azure Storage Tables: Strengths Massively scalable Perspective: – Applied to the right problem, Windows Azure Tables are a beautiful thing But they’re not the optimal solution for many other applications – Amazon, Google, and others provide similar abstractions

Queue Using Queues An application model Web Role ASP.NET, WCF, etc. Worker Role main() { … } 1) Receive work 2) Put work in queue 3) Get work from queue 4) Do work To scale, add more of either

Service Bus Exposing internal applications on the Internet isn’t a comfortable solution – Firewalls get in the way, may be slow, expensive Service Bus: provides a cloud-based intermediary between clients and internal applications – Direct or queued – It also provides a service registry that clients can use to find the services they need

Access Control Service Bus Registry Endpoints Organization Y Organization X Application Service Bus 2) Discover endpoints 1) Register endpoints 3) Access application

SQL Azure.NET Services Windows Azure Applications Others Windows Mobile Windows Mobile Windows Vista/XP Windows Vista/XP Windows Server Windows Server SQL Azure Database Others (Future) SQL Azure (a.k.a. SQL Services)

SQL Azure SQL Azure Database – Formerly known as SQL Data Services (SDS) and SQL Server Data Services (SSDS) – “Huron” Data Hub Built on the Microsoft Sync Framework, which is “a comprehensive synchronization platform that enables collaboration and offline access for applications, services, and devices with support for any data type, any data store, any transfer protocol, and any network topology.” Limitations in functionality and scalability – In the future: reporting, analysis, extract/transform/load (ETL) services, …

Platform Services Software Services Application Services Infrastructure Services The Microsoft Cloud Categories of Services

User Private Cloud Public Cloud Services Application Patterns Table Storage Service Table Storage Service Blob Storage Service Blob Storage Service Queue Service Queue Service ASP.NET (Web Role) ASP.NET (Web Role) ASP.NET (Web Role) ASP.NET (Web Role) ASP.NET (Web Role) ASP.NET (Web Role) ASP.NET (Web Role) ASP.NET (Web Role) ASP.NET (Web Role) ASP.NET (Web Role) ASP.NET (Web Role) ASP.NET (Web Role) ASP.NET (Web Role) ASP.NET (Web Role) ASP.NET (Web Role) ASP.NET (Web Role) ASP.NET (Web Role) ASP.NET (Web Role) Web Svc (Web Role) Web Svc (Web Role) ASP.NET (Web Role) ASP.NET (Web Role) ASP.NET (Web Role) ASP.NET (Web Role) ASP.NET (Web Role) ASP.NET (Web Role) ASP.NET (Web Role) ASP.NET (Web Role) Jobs (Worker Role) Jobs (Worker Role) Silverlight Application Silverlight Application Web Browser Mobile Browser Mobile Browser WPF Application WPF Application Service Bus Access Control Service Workflow Service Workflow Service User Data User Data Application Data Application Data Reference Data Web Application Enterprise Data Enterprise Web Svc Enterprise Application Data Service Data Service Storage Service Storage Service Identity Service Identity Service Application Service Application Service Enterprise Identity

User Private Cloud Public Services Application Patterns Table Storage Service Table Storage Service Blob Storage Service Blob Storage Service Queue Service Queue Service ASP.NET (Web Role) ASP.NET (Web Role) ASP.NET (Web Role) ASP.NET (Web Role) ASP.NET (Web Role) ASP.NET (Web Role) ASP.NET (Web Role) ASP.NET (Web Role) ASP.NET (Web Role) ASP.NET (Web Role) ASP.NET (Web Role) ASP.NET (Web Role) ASP.NET (Web Role) ASP.NET (Web Role) ASP.NET (Web Role) ASP.NET (Web Role) ASP.NET (Web Role) ASP.NET (Web Role) Web Svc (Web Role) Web Svc (Web Role) ASP.NET (Web Role) ASP.NET (Web Role) ASP.NET (Web Role) ASP.NET (Web Role) ASP.NET (Web Role) ASP.NET (Web Role) ASP.NET (Web Role) ASP.NET (Web Role) Jobs (Worker Role) Jobs (Worker Role) Silverlight Application Silverlight Application Web Browser Mobile Browser Mobile Browser WPF Application WPF Application Service Bus Access Control Service Workflow Service Workflow Service User Data User Data Application Data Reference Data Grid / Parallel Computing Application Enterprise Data Enterprise Web Svc Enterprise Application Data Service Data Service Storage Service Storage Service Identity Service Identity Service Application Service Application Service Enterprise Identity