Lecture 2. CAP and Challenges

Slides:



Advertisements
Similar presentations
Summary of Cloud Computing (CC) from the paper Abovce the Clouds: A Berkeley View of Cloud Computing (Feb. 2009)
Advertisements

Creating HIPAA-Compliant Medical Data Applications with Amazon Web Services Presented by, Tulika Srivastava Purdue University.
Hello i am so and so, title/role and a little background on myself (i.e. former microsoft employee or anything interesting) set context for what going.
UC Berkeley Above the Clouds A Berkeley View of Cloud Computing 1 UC Berkeley RAD Lab.
INTRODUCTION TO CLOUD COMPUTING CS 595 LECTURE 6 2/13/2015.
The Microsoft Cloud Azure Platform This presentation incorporates some content from Microsoft.
By: Kathleen Walters CLOUD COMPUTING Definition Cloud computing allows multiple computers to connect to one main network. Instead of installing different.
What is Cloud Computing? o Cloud computing:- is a style of computing in which dynamically scalable and often virtualized resources are provided as a service.
The Cloud: Demystified Neil Cattermull Frontier Technology.
Cloud Computing: Overview 1. This lecture What is cloud computing? What are its essential characteristics? Why cloud computing? Classification/service.
Cloud Computing Stuart Dillon-Roberts. “In the simplest terms, cloud computing means storing & accessing data & programs over the Internet instead of.
Cloud computing Tahani aljehani.
An Introduction to Cloud Computing. The challenge Add new services for your users quickly and cost effectively.
Cloud Computing Systems Lin Gu Hong Kong University of Science and Technology Sept. 21, 2011 Windows Azure—Overview.
Plan Introduction What is Cloud Computing?
Cloud Computing and its Implementation ELYSIUM TECHNOLOGIES PRIVATE LIMITED Madurai | Trichy | Coimbatore | Kollam | Cochin | Tirunelveli T. Udhaya Shankar,
A Brief Overview by Aditya Dutt March 18 th ’ Aditya Inc.
Cloud Computing: Overview
Cloud Computing for the Enterprise November 18th, This work is licensed under a Creative Commons.
Cloud Computing Saneel Bidaye uni-slb2181. What is Cloud Computing? Cloud Computing refers to both the applications delivered as services over the Internet.
CEG7380 Cloud Computing Lecture 1
Cloud Computing. What is Cloud Computing? Cloud computing is a model for enabling convenient, on-demand network access to a shared pool of configurable.
An emerging computing paradigm where data and services reside in massively scalable data centers and can be ubiquitously accessed from any connected devices.
Introduction to Cloud Computing
Presented by: Sanketh Beerabbi University of Central Florida COP Cloud Computing.
Introduction. Readings r Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn. 3 m Note: All figures from this book.
What is the cloud ? IT as a service Cloud allows access to services without user technical knowledge or control of supporting infrastructure Best described.
Cloud Computing is a Nebulous Subject Or how I learned to love VDF on Amazon.
3/12/2013Computer Engg, IIT(BHU)1 CLOUD COMPUTING-1.
Web Technologies Lecture 13 Introduction to cloud computing.
Chapter 3 Pre-Incident Preparation Spring Incident Response & Computer Forensics.
Cloud Architecture. SPI Model Cloud Computing Classification Model – SPI Cloud Computing Classification Model – SPI - SaaS: (Software as a Service) -
1 TCS Confidential. 2 Objective : In this session we will be able to learn:  What is Cloud Computing?  Characteristics  Cloud Flavors  Cloud Deployment.
Windows Azure Overview for IT Pros Anton Boyko. Intro to Cloud Computing Intro to Windows Azure Cloud Services Web Sites Virtual Machines Workload Options.
 Cloud Computing technology basics Platform Evolution Advantages  Microsoft Windows Azure technology basics Windows Azure – A Lap around the platform.
© 2012 Eucalyptus Systems, Inc. Cloud Computing Introduction Eucalyptus Education Services 2.
INTRODUCTION TO CLOUD COMPUTING. CLOUD  The expression cloud is commonly used in science to describe a large agglomeration of objects that visually appear.
Above the Clouds: A Berkeley View of Cloud Computing Annajiat Alim Rasel, P Shimul Bala, P Raquibul Bari, P Annajiat Alim.
CS 6027 Advanced Networking FINAL PROJECT ​. Cloud Computing KRANTHI ​ CHENNUPATI PRANEETHA VARIGONDA ​ SANGEETHA LAXMAN ​ VARUN ​ DENDUKURI.
Agenda  What is Cloud Computing?  Milestone of Cloud Computing  Common Attributes of Cloud Computing  Cloud Service Layers  Cloud Implementation.
Clouding with Microsoft Azure
Prof. Jong-Moon Chung’s Lecture Notes at Yonsei University
Lecture 6: Cloud Computing
Unit 3 Virtualization.
Introduction to Cloud Technology
Chapter 6: Securing the Cloud
Lecture 2. Ecosystem, CAP, and Challenges
Understanding The Cloud
Avenues International Inc.
Dr.S.Sridhar, Director, RVCT, RVCE, Bangalore
Cloud Security– an overview Keke Chen
VIRTUALIZATION & CLOUD COMPUTING
An Introduction to Cloud Computing
Hybrid Cloud Architecture for Software-as-a-Service Provider to Achieve Higher Privacy and Decrease Securiity Concerns about Cloud Computing P. Reinhold.
Amazon Web Services Submitted By- Section - B Group - 4
Dr.S.Sridhar, Director, RVCT, RVCE, Bangalore
Andrew McCombs March 10th, 2011
Physical Architecture Layer Design
CEG7380 Cloud Computing Lecture 1
Introduction to Enterprise Systems
Above the Clouds A Berkeley View of Cloud Computing
Outline Virtualization Cloud Computing Microsoft Azure Platform
Cloud Computing and its Implementation
Internet and Web Simple client-server model
Cloud Computing: Concepts
NIST Cloud Computing Reference Architecture
Views of Cloud Computing
April 17, 2009 eGov Workshop: Cloud Computing and Interoperability Daniel Burton Senior Vice President Global Public Policy Salesforce.com.
Cloud Computing for Wireless Networks
Presentation transcript:

Lecture 2. CAP and Challenges COSC6376 Cloud Computing Lecture 2. CAP and Challenges Instructor: Weidong Shi (Larry), PhD Computer Science Department University of Houston

Outline Ecosystem CAP Challenges

Summary Assignment Paper can be downloaded from the class website Due next Tuesday in class

NIST: Interactions between Actors in Cloud Computing Cloud Consumer Cloud Provider Cloud Broker Cloud Auditor Cloud Carrier The communication path between a cloud provider & a cloud consumer The communication paths for a cloud auditor to collect auditing information The communication paths for a cloud broker to provide service to a cloud consumer

Conceptual Reference Diagram Cloud Carrier Cloud Consumer Cloud Auditor Broker Security Audit Privacy Impact Audit Performance Cloud Service Management Service Layer Business Support Service Arbitrage Aggregation Service Intermediation Provisioning/ Configuration Portability/ Interoperability Physical Resource Layer IaaS SaaS PaaS Resource Abstraction and Control Layer Hardware Facility

Resource Abstraction All problems in computer science can be solved by another level of indirection (abstraction) - David John Wheeler

Six Layers of Cloud Services Salesforce.com, Webex, … App Engine, Microsoft Azure Amzon AWS, Racksapce, IBM Ensembles Savvis, Intermap, Digital Realty Trust AT & T VMWare, IBM, Xen

Spectrum of Clouds Instruction Set VM (Amazon EC2, 3Tera) Bytecode VM (Microsoft Azure) Framework VM Google AppEngine, Force.com Lower-level, Less management Higher-level, More management EC2 Azure AppEngine Force.com

Amazon EC2 Like physical hardware, users can control nearly the entirely software stack, from the kernel upwards. A few API calls to request and configure the virtualized hardware. No limit on the kinds of applications that can be hosted. Low level of virtualization-raw CPU cycles, IP connectivity-allow developers to code whatever they want. Hard to offer scalability and failover.

Google AppEngine and Force.com Does one thing well: running web apps App Engine handles HTTP(S) requests, nothing else Think RPC: request in, processing, response out Works well for the web and AJAX; also for other services Request-reply based. Not suitable for general- purpose. Severely rationed in how much CPU time they can use in a request. Automatic scaling and high-availability.

Microsoft’s Azure Written using the .NET libraries, and compiled to the language independent managed environment. General -purpose computing. Users get a choice of language, but can not control the operating system or runtime. Libraries provide automatic network configuration and failover/scalability but need users' cooperation also.

Spectrum Azure General-purpose Can not control OS A degree of scalability Google appengine/force.com Highly scalable Yet not general-purpose Amazon EC2 General-purpose Hard to offer scalability

Major Cloud Providers and Service Offerings

Public, Private, and Hybrid Clouds

Hybrid Clouds Using multiple clouds for different applications to match needs Moving an application to meet requirements at specific stages in its lifecycle, from early development through unit test, scale testing, pre-production and ultimately full production scenarios Moving workloads closer to end users across geographic locations, including user groups within the enterprise, partners and external customers Meeting peak demands efficiently in the cloud while the low steady-state is handled internally Maintaining confidential data on better protected clouds while allowing distributed computation on more computationally efficient ones.

Cloud Interoperability Standards Open Cloud Computing Interface – Infrastructure EC2 API Simple Storage Service (S3) API Windows Azure Storage Service REST APIs Windows Azure Service Management REST APIs Deltacloud API Rackspace Cloud Servers API Rackspace Cloud Files API Cloud Data Management Interface vCloud API GlobusOnline REST API

CAP

The CAP Theorem Three properties of a system: consistency, availability and partitions Availability Consistency Partition tolerance 19

The CAP Theorem Once a writer has written, all readers will see that write Availability Consistency Partition tolerance

Consistency Model A consistency model determines rules for visibility and apparent order of updates. For example: Row X is replicated on nodes M and N Client A writes row X to node N Some period of time t elapses. Client B reads row X from node M Does client B see the write from client A? Consistency is a continuum with tradeoffs For NoSQL, the answer would be: maybe CAP Theorem states: Strict Consistency can't be achieved at the same time as availability and partition-tolerance.

Consistency Case 1 Case 2 Upload a picture to facebook Send a message to your friend to check out the picture Will your friend see it? Case 2 Post a comment C1 on your friend’s page at time t Post another comment C2 10 seconds later Will your friend see two comments with C1 first, followed by C2

Eventual Consistency When no updates occur for a long period of time, eventually all updates will propagate through the system and all the nodes will be consistent For a given accepted update and a given node, eventually either the update reaches the node or the node is removed from service

GPS Powered Distributed Database Spanner allows server nodes to coordinate without a whole lot of communication.’ Google Spanner, the Largest Single Database on Earth

The CAP Theorem Every request received by a non-failing node in the system must result in a response (must terminate) System is available during software and hardware upgrades and node failures Availability Consistency Partition tolerance

Availability Traditionally, thought of as the server/process available five 9’s (99.999 %). However, for large node system, at almost any point in time there’s a good chance that a node is either down or there is a network disruption among the nodes. Want a system that is resilient in the face of network disruption

The CAP Theorem A system can continue to operate in the presence of a network partitions. Availability Consistency Partition tolerance

The CAP Theorem You can have at most two of these three properties for any shared-data system To scale out, you have to partition. That leaves either consistency or availability to choose from In almost all cases, you would choose availability over consistency C A P Availability Partition-resilience Claim: every distributed system is on one side of the triangle.

Challenges

Adoption Challenges Challenge Opportunity Availability Multiple providers & DCs Data lock-in Standardization Data Confidentiality, Auditability, and privacy Encryption, VLANs, Firewalls; Geographical Data Storage; Privacy preserving data outsourcing

Challenges and Opportunities Availability of Service Service Duration Data S3 outage: authentication service overload leading to unavailability 2hours 2/15/08 S3 outage: Single bit error leading to gossip protocol blowup. 6-8hours 7/20/08 AppEngine partial outage: programming error 5 hours 6/17/08 Gmail. 1.5hours 08/11/08

Adoption Challenges Challenge Opportunity Availability Multiple providers & DCs Data lock-in Standardization Data Confidentiality, Auditability, and privacy Encryption, VLANs, Firewalls; Geographical Data Storage; Privacy preserving data outsourcing

Senior Execs Move Forward with Cloud Investments

Legal framework of Cloud Computing Legal compliance issues Service levels and performance Cross-border issues Data protection, rights and usage Privacy and security Termination and transition

Compliance of Cloud Computing Auditing requirements Many contracts impose auditing possibilities that include physical inspection how can these auditing requirements be complied with when geographically decentralized cloud services are used? Compliance IaS Data retention obligations Tax related storage requirements Labor law related book keeping requirements SaaS Electronic invoicing legislation Ecommerce legislation Electronic signature legislation

HIPPA Compliance? What is HIPPA? What is Regulated? Health Insurance Portability and Accountability Act of 1996 – a Federal Law What is Regulated? Accountability: Protects health data integrity, confidentiality and availability Reduces Fraud and Abuse Establishes Standards for Protection of Health Information Privacy (Operational, Consumer Control, Administration) Security (Administrative, Physical, Technical, Network) Definition of Privacy Privacy is the right of an individual to keep his/her individual health information from being disclosed

Cross Border Data Transfer/Storage EU Only use cloud provider with data center within EU e.g. Amazon EC2: choice of location (US East, US West or Ireland) Australia Financial services companies must first notify Australian Prudential Regulatory Authority (APRA) of data offshore transfer Make sure that agreement is concluded with the cloud provider

Cross Border Data Transfer/Storage Applicable Law & Competent court If outside own country, any litigation can become prohibitively expensive . . Data stored in the U.S. is subject to U.S. law, for example: US Patriot Act – US government’s authority extends to compel disclosure of records held by cloud providers

Challenges of Datasets over Multiple Clouds Interesting datasets might be available in different clouds Different cloud providers Private or public clouds Services mashing up datasets Inevitably crossing clouds Federated cloud architectures

Growth Challenges Challenge Opportunity Data transfer bottlenecks FedEx-ing disks, Data Backup/Archival Performance unpredictability Improved VM support, flash memory, scheduling VMs Scalable storage Invent scalable store Bugs in large distributed systems Invent Debugger that relies on Distributed VMs Scaling quickly Invent Auto-Scaler that relies on ML; Snapshots

Challenges and Opportunities ∙Data Transfer bottlenecks Obstacles: large data transferring is expensive. e.g. Ship 10 TB from UC Berkeley to Amazon Bandwidth: 20 M/s Time: 45 days Money: $1000 Opportunities: Ship disks. Make it attractive to keep data in cloud. Reduce the cost of WAN bandwidth.

Growth Challenges Challenge Opportunity Data transfer bottlenecks FedEx-ing disks, Data Backup/Archival Performance unpredictability Improved VM support, flash memory, scheduling VMs Scalable storage Invent scalable store Bugs in large distributed systems Invent Debugger that relies on Distributed VMs Scaling quickly Invent Auto-Scaler that relies on ML; Snapshots

Real-time Bidding (Ads)

Algorithms on Big data Working on “Big Data” Data mining Machine learning Visualization Traditionally assume data is in flat files or relational databases Distributed data organization puts new challenges Redesign algorithms Redesign frameworks

Policy and Business Challenges Opportunity Reputation Fate Sharing Offer reputation-guarding services like those for email Software Licensing Pay-for-use licenses; Bulk use sales

Come to the Dark Side Spam as a service Crimeware as a service Password cracking cloud DoS attack as a service How likely is the risk buy services using stolen credit card numbers create ec2 instances using stolen keys attack authentication (SOAP, XML. XML wrapping attacks) hijack cloud infrastructure

Botnet as a Service

C & C Activities 2013 GLOBAL THREAT INTELLIGENCE REPORT (GTIR)”.

Underground E-shop selling access to malware-infected hosts

Botnet prices (Trend Micro) DDoS attacks Spamming (e-mail, social networks) Covert channel for information exchange PsyOPS in social networks Bitcoins