An Overview of Cloud Computing Raghu Ramakrishnan Chief Scientist, Audience and Cloud Computing Research Fellow, Yahoo! Research Reflects many discussions with: Eric Baldeschwieler, Jay Kistler, Chuck Neerdaels, Shelton Shugar, and Raymie Stata and joint work with the Sherpa team, in particular: Brian Cooper, Utkarsh Srivastava, Adam Silberstein and Nick Puz in Y! Research Chuck Neerdaels, P.P. Suryanarayanan and many others in CCDI 1

CCDI—Research Collaboration Yahoo! Research Raghu Ramakrishnan Brian Cooper Utkarsh Srivastava Adam Silberstein Nick Puz Rodrigo Fonseca CCDI Chuck Neerdaels P.P.S. Narayan Kevin Athey Toby Negrin Plus Dev/QA teams

SCENARIOS Pie-in-the-sky Need some slides up front to layout what our definition of a cloud entails (since it’s a rapidly emerging buzzword that has no intrinsic definition. ) SCENARIOS 3

Living in the Clouds We want to start a new website, FredsList.com Our site will provide listings of items for sale, jobs, etc. As time goes on, we’ll add more features And illustrate how more cloud capabilities (and corresponding infrastructure components) are used as needed List of capabilities/components is illustrative, not exhaustive Our cloud provides a “dataset” abstraction FredsList doesn’t worry about the underlying components

Simple Web Service API’s Step 1: Listings FredsList wants to store listings as (key, category, description) FredsList.com application DECLARE DATASET Listings AS ( ID String PRIMARY KEY, Category String, Description Text ) 1234323, transportation, For sale: one bicycle, barely used 5523442, childcare, Nanny available in San Jose 215534, wanted, Looking for issue 1 of Superman comic book Simple Web Service API’s Database Sherpa

FredsList.com application Step 2: Search FredsList’s customers quickly ask for keyword search FredsList.com application ALTER Listings SET Description SEARCHABLE “bicycle” “dvd’s” “nanny” Simple Web Service API’s Database Search Sherpa Vespa Messaging YMB

FredsList.com application Step 3: Photos FredsList decides to add photos to listings FredsList.com application ALTER Listings ADD Photo BLOB Simple Web Service API’s Database Storage Search Foreign key photo → listing MObStor Sherpa Vespa Messaging YMB

FredsList.com application Step 4: Data Analysis FredsList wants to analyze its listings to get statistics about category, do geocoding, etc. FredsList.com application ALTER Listings MAKE ANALYZABLE Hadoop program to generate fancy pages for listings Pig query to analyze categories Hadoop program to geocode data Simple Web Service API’s Compute Database Storage Search Grid Foreign key photo → listing MObStor Sherpa Vespa Batch export Messaging YMB

Step 5: Performance FredsList wants to reduce its data access latency FredsList.com application ALTER Listings MAKE CACHEABLE Simple Web Service API’s Compute Database Storage Search Caching Grid Foreign key photo → listing MObStor Sherpa Vespa memcached Batch export Messaging YMB

EYES TO THE SKIES Motherhood-and-Apple-Pie Need some slides up front to layout what our definition of a cloud entails (since it’s a rapidly emerging buzzword that has no intrinsic definition. ) EYES TO THE SKIES 10

Why Clouds? On-demand infrastructure to create a fundamental shift in the OE curve. Let’s us: Do things we can’t do Reduce time to market Build more robustly, more efficiently, more globally, more completely, for a given budget Cloud services should do heavy lifting of heavy-lifting of scaling & high-availability Today, this is done at the app-level, which is not productive

Requirements for Cloud Services Multitenant. A cloud service must support multiple, organizationally distant customers. Elasticity. Tenants should be able to negotiate and receive resources/QoS on-demand. Resource Sharing. Ideally, spare cloud resources should be transparently applied when a tenant’s negotiated QoS is insufficient, e.g., due to spikes. Horizontal scaling. It should be possible to add cloud capacity in small increments; this should be transparent to the tenants of the service. Metering. A cloud service must support accounting that reasonably ascribes operational and capital expenditures to each of the tenants of the service. Security. A cloud service should be secure in that tenants are not made vulnerable because of loopholes in the cloud. Availability. A cloud service should be highly available. Operability. A cloud service should be easy to operate, with few operators. Operating costs should scale linearly or better with the capacity of the service.

Types of Cloud Services Two kinds of cloud services: Horizontal Cloud Services Functionality enabling tenants to build applications or new services on top of the cloud Functional Cloud Services Functionality that is useful in and of itself to tenants. E.g., various SaaS instances, such as Saleforce.com; Google Analytics and Yahoo!’s IndexTools; Yahoo! properties aimed at end-users and small businesses, e.g., flickr, Groups, Mail, News, Shopping Could be build on top of horizontal cloud services or from scratch Yahoo! has been offering these for a long while (e.g., Mail for SMB, Groups, Flickr, BOSS, Ad exchanges)

Horizontal Cloud Services Horizontal cloud services are foundations on which tenants build applications or new services. They should be: Semantics-free. Must be "generic infrastructure,” and not tied to specific app-logic. May provide the ability to inject application logic through well-defined APIs Broadly applicable. Must be broadly applicable (i.e., it can't be intended for just one or two properties). Fault-tolerant over commodity hardware. Must be built using inexpensive commodity hardware, and should mask component failures. While each cloud service provides value, the power of the cloud paradigm will depend on a collection of well-chosen, loosely coupled services that collectively make it easy to quickly develop and operate innovative web applications.

What’s in the Horizontal Cloud? Simple Web Service API’s Horizontal Cloud Services Provisioning & Virtualization e.g., EC2 Batch Storage & Processing e.g., Hadoop & Pig Operational Storage e.g., S3, MObStor, Sherpa Edge Content Services e.g., YCS, YCPI Other Services Messaging, Workflow, virtual DBs & Webserving ID & Account Management Metering, Billing, Accounting Shared Infrastructure Security Monitoring & QoS Common Approaches to QA, Production Engineering, Performance Engineering, Datacenter Management, and Optimization 15

Yahoo! CCDI Thrust Areas Fast Provisioning and Machine Virtualization: On demand, deliver a set of hosts imaged with desired software and configured against standard services Multiple hosts may be multiplexed onto the same physical machine. Batch Storage and Processing: Scalable data storage optimized for batch processing, together with computational capabilities Operational Storage: Persistent storage that supports low-latency updates and flexible retrieval Edge Content Services: Support for dealing with network topology, communication protocols, caching, and BCP Rest of today’s talk

Hadoop: Batch Storage/Analysis Why is batch processing important? Whether it’s response-prediction for advertising machine-learned relevance for Search, or content optimization for audience, data-intensive computing is increasingly central to everything Yahoo! does Hadoop is central to addressing this need Hadoop is a case-study in our cloud vision Processes enormous amounts of data Provides horizontal scaling and fault-tolerance for our users Allows those users to focus on their app logic [Workflow] High-level query layer (Pig) Map-Reduce HDFS OK

SHERPA To Help You Scale Your Mountains of Data A project in Y!R focused on a long-range problem, origins in earlier work at Wisconsin. Basis for the Goldrush hack, which won the recent Local hack competition, and could contribute to creation/refinement of Y! Local content and Next Gen Search.

The Yahoo! Storage Problem Small records – 100KB or less Structured records - tens, hundreds or thousands of fields Extreme data scale - Tens of TB Extreme request scale - Tens of thousands of requests/sec Low latency globally - 20+ datacenters worldwide High Availability - outages cost $millions Variable usage patterns - as applications and users change 19

The next generation global-scale record store The Sherpa Solution The next generation global-scale record store Record-orientation: Routing, data storage optimized for low-latency record access Scale out: Add machines to scale throughput (while keeping latency low) Asynchrony: Pub-sub replication to far-flung datacenters to mask propagation delay Consistency model: Reduce complexity of asynchrony for the application programmer Cloud deployment model: Hosted, managed service to reduce app time-to-market and enable on demand scale and elasticity 20

What is Sherpa? CREATE TABLE Parts ( ID VARCHAR, StockNumber INT, Status VARCHAR … ) E 75656 C A 42342 E B 42521 W C 66354 W D 12352 E F 15677 E A 42342 E B 42521 W C 66354 W D 12352 E E 75656 C F 15677 E Structured, flexible schema E 75656 C A 42342 E B 42521 W C 66354 W D 12352 E F 15677 E Geographic replication Parallel database Hosted, managed infrastructure 21

What Will Sherpa Become? A 42342 E B 42521 W C 66354 W D 12352 E F 15677 E E 75656 C A 42342 E B 42521 W C 66354 W D 12352 E F 15677 E Indexes and views CREATE TABLE Parts ( ID VARCHAR, StockNumber INT, Status VARCHAR … ) E 75656 C A 42342 E B 42521 W C 66354 W D 12352 E F 15677 E Geographic replication Parallel database Structured, flexible schema Hosted, managed infrastructure

Sherpa Design Goals Scalability Geographic replication Thousands of machines Easy to add capacity Restrict query language to avoid costly queries Geographic replication Asynchronous replication around the globe Low-latency local access High availability and fault tolerance Automatically recover from failures Serve reads and writes despite failures Consistency Per-record guarantees Timeline model Option to relax if needed Multiple access paths Hash table, ordered table Primary, secondary access Hosted service Applications plug and play Share operational cost 23 23

Technology Elements Applications PNUTS API Tabular API PNUTS Query planning and execution Index maintenance Distributed infrastructure for tabular data Data partitioning Update consistency Replication YCA: Authorization YDOT FS Ordered tables YDHT FS Hash tables YMB Pub/sub messaging Zookeeper Consistency service 24 24

Data Manipulation Per-record operations Multi-record operations Get Set Delete Multi-record operations Multiget Scan Getrange Web service (RESTful) API 25

Tablets—Hash Table 0x0000 0x2AF3 0x911F 0xFFFF Name Description Price Grape Grapes are good to eat $12 Lime Limes are green $9 Apple Apple is wisdom $1 Strawberry Strawberry shortcake $900 0x2AF3 Orange Arrgh! Don’t get scurvy! $2 Avocado But at what price? $3 Lemon How much did you pay for this lemon? $1 Tomato Is this a vegetable? $14 0x911F Banana The perfect fruit $2 Kiwi New Zealand $8 0xFFFF 26

Tablets—Ordered Table Name Description Price A Apple Apple is wisdom $1 Avocado But at what price? $3 Banana The perfect fruit $2 Grape Grapes are good to eat $12 H Kiwi New Zealand $8 Lemon How much did you pay for this lemon? $1 Lime Limes are green $9 Orange Arrgh! Don’t get scurvy! $2 Q Strawberry Strawberry shortcake $900 Tomato Is this a vegetable? $14 Z 27

Flexible Schema Posted date Listing id Item Price 6/1/07 424252 Couch $570 763245 Bike $86 6/3/07 211242 Car $1123 6/5/07 421133 Lamp $15 Color Red Condition Good Fair

Detailed Architecture Remote regions Local region Clients REST API Routers YMB Tablet controller Storage units 29

Tablet Splitting and Balancing Each storage unit has many tablets (horizontal partitions of the table) Storage unit may become a hotspot Storage unit Tablet Overfull tablets split Tablets may grow over time Shed load by moving tablets to other servers 30

QUERY PROCESSING 31

Accessing Data SU SU SU Get key k Get key k Record for key k 1 4 2 3 32

Bulk Read SU SU SU {k1, k2, … kn} Get k1 Get k2 Get k3 1 2 33 Scatter/ gather server SU SU SU 33

Range Queries in YDOT Clustered, ordered retrieval of records Storage unit 1 Canteloupe Storage unit 3 Lime Storage unit 2 Strawberry Router Apple Avocado Banana Blueberry Storage unit 1 Canteloupe Storage unit 3 Lime Storage unit 2 Strawberry Grapefruit…Lime? Grapefruit…Pear? Lime…Pear? Canteloupe Grape Kiwi Lemon Lime Mango Orange Storage unit 1 Storage unit 2 Storage unit 3 Strawberry Tomato Watermelon Apple Avocado Banana Blueberry Strawberry Tomato Watermelon Lime Mango Orange Canteloupe Grape Kiwi Lemon

Updates SU SU SU Sequence # for key k Write key k Write key k 1 8 Sequence # for key k Write key k Routers Message brokers 2 Write key k 3 Write key k 7 4 Sequence # for key k 5 SUCCESS SU SU SU 6 Write key k 35

ASYNCHRONOUS REPLICATION AND CONSISTENCY 36

Asynchronous Replication 37

Consistency Model Goal: make it easier for applications to reason about updates and cope with asynchrony What happens to a record with primary key “Brian”? Record inserted Update Update Update Update Update Update Update Delete v. 1 v. 2 v. 3 v. 4 v. 5 v. 6 v. 7 v. 8 Time Time Generation 1 38

Consistency Model Read Stale version Stale version Current version Time Generation 1 39

Consistency Model Read up-to-date Stale version Stale version Current version v. 1 v. 2 v. 3 v. 4 v. 5 v. 6 v. 7 v. 8 Time Generation 1 40

Consistency Model Read ≥ v.6 Stale version Stale version Current version v. 1 v. 2 v. 3 v. 4 v. 5 v. 6 v. 7 v. 8 Time Generation 1 41

Consistency Model Write Stale version Stale version Current version Time Generation 1 42

Consistency Model Write if = v.7 Stale version Stale version ERROR Stale version Stale version Current version v. 1 v. 2 v. 3 v. 4 v. 5 v. 6 v. 7 v. 8 Time Generation 1 43

Consistency Model Mechanism: per record mastership Write if = v.7 ERROR Stale version Stale version Current version v. 1 v. 2 v. 3 v. 4 v. 5 v. 6 v. 7 v. 8 Time Generation 1 44

Consistency Techniques Per-record mastering Each record is assigned a “master region” May differ between records Updates to the record forwarded to the master region Ensures consistent ordering of updates Tablet-level mastering Each tablet is assigned a “master region” Inserts and deletes of records forwarded to the master region Master region decides tablet splits These details are hidden from the application Except for the latency impact!

Mastering Tablet master 46 A 42342 E B 42521 W C 66354 W D 12352 E E 75656 C F 15677 E A 42342 E B 42521 W Tablet master C 66354 W D 12352 E E 75656 C F 15677 E A 42342 E B 42521 W C 66354 W D 12352 E E 75656 C F 15677 E 46

Bulk Insert/Update/Replace Client feeds records to bulk manager Bulk loader transfers records to SU’s in batches Bypass routers and message brokers Efficient import into storage unit Client Bulk manager Source Data

Bulk Load in YDOT YDOT bulk inserts can cause performance hotspots Solution: preallocate tablets

Index Maintenance How to have lots of interesting indexes, without killing performance? Solution: Asynchrony! Indexes updated asynchronously when base table updated Planned functionality

SHERPA IN CONTEXT 50

MObStor Yahoo!’s next-generation globally replicated, virtualized media object storage service Better provisioning, easy migration, replication, better BCP, and performance New features (Evergreen URLs, CDN integration, REST API, …) The object metadata problem addressed using Sherpa, though MObStor is focused on blob storage. 51 51

Storage & Delivery Stack

The World Has Changed Web applications need: Scalability! Preferably elastic Geographic distribution High availability Reliable storage Web applications can do without: Complicated queries Strong transactions

Structured record storage Web Data Management CRUD Point lookups and short scans Index organized table and random I/Os $ per latency Scan oriented workloads Focus on sequential disk I/O $ per cpu cycle Structured record storage (PNUTS) Large data analysis (Hadoop) Object retrieval and streaming Scalable file storage $ per GB Blob storage (SAN/NAS)

Retrieval from single table of objects/records Types of Record Stores Query expressiveness S3 PNUTS Oracle Simple Feature rich Object retrieval Retrieval from single table of objects/records SQL

Types of Record Stores Consistency model S3 PNUTS Oracle Best effort Strong guarantees Eventual consistency Timeline consistency ACID Program centric consistency Object-centric consistency

Types of Record Stores Elasticity (ability to add resources on demand) PNUTS S3 Oracle Not scalable Elastic Limited (via data distribution) VLSD (Very Large Scale Distribution /Replication)

Data Stores Comparison User-partitioned SQL stores Microsoft Azure SDS Amazon SimpleDB Multi-tenant application databases Salesforce.com Oracle on Demand Mutable object stores Amazon S3 Versus PNUTS More expressive queries Users must control partitioning Limited elasticity Highly optimized for complex workloads Limited flexibility to evolving applications Inherit limitations of underlying data management system Object storage versus record management

Application Design Space Get a few things Sherpa MObStor YMDB MySQL Oracle Filer BigTable Scan everything Everest Hadoop Records Files 59

Alternatives Matrix Consistency model Global low Structured SQL/ACID latency Structured access Consistency model SQL/ACID Operability Availability Updates Elastic Sherpa Y! UDB MySQL Oracle HDFS BigTable Dynamo Cassandra 60

Further Reading Efficient Bulk Insertion into a Distributed Ordered Table (SIGMOD 2008) Adam Silberstein, Brian Cooper, Utkarsh Srivastava, Erik Vee, Ramana Yerneni, Raghu Ramakrishnan PNUTS: Yahoo!'s Hosted Data Serving Platform (VLDB 2008) Brian Cooper, Raghu Ramakrishnan, Utkarsh Srivastava, Adam Silberstein, Phil Bohannon, Hans-Arno Jacobsen, Nick Puz, Daniel Weaver, Ramana Yerneni

Opening Up Yahoo! Search Phase 1 Phase 2 SearchMonkey Messaging: • SearchMonkey is another proof point in Yahoo!’s big bets strategy o Openness – SearchMonkey will create access to the structured data that the web is based on and build a better experience for Yahoo! Search users o Partner of choice - Yahoo! Search is opening up its platform to any size site owner or developer to play in the search ecosystem with self-service tools o Insights – site owners will have better insight to search traffic drivers; better able to create targeted results for better engagement • Yahoo! Search is a leading starting point on the Web for users and the first major search engine to open its search page to site owners and developers and allow them to create visibly differentiated search results. • By opening up the search platform, Yahoo! Search improves the way search engines display results to the benefit of users, site owner and developers. Giving site owners and developers control over the appearance of Yahoo! Search results. BOSS takes Yahoo!’s open strategy to the next level by providing Yahoo! Search infrastructure and technology to developers and companies to help them build their own search experiences. 62

Search Results of the Future yelp.com Gawker babycenter New York Times epicurious LinkedIn answers.com webmd

BOSS Offerings API CUSTOM ACADEMIC BOSS offers two options for companies and developers and has partnered with top technology universities to drive search experimentation, innovation and research into next generation search. API A self-service, web services model for developers and start-ups to quickly build and deploy new search experiences. CUSTOM Working with 3rd parties to build a more relevant, brand/site specific web search experience. This option is jointly built by Yahoo! and select partners. ACADEMIC Working with the following universities to allow for wide-scale research in the search field: University of Illinois Urbana Champaign Carnegie Mellon University Stanford University Purdue University • MIT Indian Institute of Technology Bombay University of Massachusetts (Slide courtesy Prabhakar Raghavan) 64

Partner Examples About Medium Founded in 2006 in Boulder, Colorado, Me.dium is a social browsing software company offering a browser extension that allows people to surf with friends for the first time. By revealing this new Social Exploration Environment TM (SEE), Me.dium graphically connects users with their friends and others enabling users to interact online, similarly to how one interacts with people in the real world. Me.dium’s Social Search Alpha is the first crowd-powered search engine.   Social Search enables people to find relevant information based on the activity of crowds right now.  See an entirely new layer of information over and above what traditional search provides.  You get the hottest, most active content related to your query - as determined by the current activity of the crowds. Me.dium combined the BOSS API with it’s insight into the real time surfing activity of the crowds to build a unique "Crowd-Powered" social search engine prototype. 65

QUESTIONS? 66