Experiences With RAMCloud Applications John Ousterhout, Jonathan Ellithorpe, Bob Brown.

Slides:



Advertisements
Similar presentations
MinCopysets: Derandomizing Replication in Cloud Storage
Advertisements

Fast Crash Recovery in RAMCloud
What happens when you try to build a low latency NIC? Mario Flajslik.
University of Chicago Department of Energy The Parallel and Grid I/O Perspective MPI, MPI-IO, NetCDF, and HDF5 are in common use Multi TB datasets also.
Daniel Schall, Volker Höfner, Prof. Dr. Theo Härder TU Kaiserslautern.
Analysis of RAMCloud Crash Probabilities Asaf Cidon Stanford University.
John Ousterhout Stanford University RAMCloud Overview and Update SEDCL Retreat June, 2014.
RAMCloud: Scalable High-Performance Storage Entirely in DRAM John Ousterhout Stanford University (with Nandu Jayakumar, Diego Ongaro, Mendel Rosenblum,
RAMCloud Scalable High-Performance Storage Entirely in DRAM John Ousterhout, Parag Agrawal, David Erickson, Christos Kozyrakis, Jacob Leverich, David Mazières,
RAMCloud 1.0 John Ousterhout Stanford University (with Arjun Gopalan, Ashish Gupta, Ankita Kejriwal, Collin Lee, Behnam Montazeri, Diego Ongaro, Seo Jin.
Log-Structured Memory for DRAM-Based Storage Stephen Rumble, Ankita Kejriwal, and John Ousterhout Stanford University.
SEDCL: Stanford Experimental Data Center Laboratory.
CS 140 Lecture Notes: Technology and Operating SystemsSlide 1 Technology Changes Mid-1980’s2009Change CPU speed15 MHz2 GHz133x Memory size8 MB4 GB500x.
Fixing the Embarrassing Slowness of OpenDHT on PlanetLab Sean Rhea, Byung-Gon Chun, John Kubiatowicz, and Scott Shenker UC Berkeley (and now MIT) December.
Secondary Storage CSCI 444/544 Operating Systems Fall 2008.
CS 142 Lecture Notes: Large-Scale Web ApplicationsSlide 1 RAMCloud Overview ● Storage for datacenters ● commodity servers ● GB DRAM/server.
CERN openlab Open Day 10 June 2015 KL Yong Sergio Ruocco Data Center Technologies Division Speeding-up Large-Scale Storage with Non-Volatile Memory.
Metrics for RAMCloud Recovery John Ousterhout Stanford University.
1 © Copyright 2009 EMC Corporation. All rights reserved. Agenda Storing More Efficiently  Storage Consolidation  Tiered Storage  Storing More Intelligently.
RAMCloud Design Review Recovery Ryan Stutsman April 1,
It’s Time for Low Latency Steve Rumble, Diego Ongaro, Ryan Stutsman, Mendel Rosenblum, John Ousterhout Stanford University 報告者 : 厲秉忠
What We Have Learned From RAMCloud John Ousterhout Stanford University (with Asaf Cidon, Ankita Kejriwal, Diego Ongaro, Mendel Rosenblum, Stephen Rumble,
RAMCloud Overview John Ousterhout Stanford University.
RAMCloud: a Low-Latency Datacenter Storage System John Ousterhout Stanford University
RAMCloud: Concept and Challenges John Ousterhout Stanford University.
RAMCloud: A Low-Latency Datacenter Storage System Ankita Kejriwal Stanford University (Joint work with Diego Ongaro, Ryan Stutsman, Steve Rumble, Mendel.
What is SharePoint? Module 1. Module Overview  Defining SharePoint  Understanding How SharePoint is Used  Interacting with SharePoint.
Cool ideas from RAMCloud Diego Ongaro Stanford University Joint work with Asaf Cidon, Ankita Kejriwal, John Ousterhout, Mendel Rosenblum, Stephen Rumble,
X-Stream: Edge-Centric Graph Processing using Streaming Partitions
John Ousterhout Stanford University RAMCloud Overview and Update SEDCL Forum January, 2015.
Data Structures & Algorithms and The Internet: A different way of thinking.
The Magic of the Cloud: Supercomputers for Everyone, Everywhere Prof. Eric A. Brewer UC Berkeley.
Low-Latency Datacenters John Ousterhout Platform Lab Retreat May 29, 2015.
How computer’s are linked together.
Does everyone in our school have equal opportunities for technology use? What about the kids who can't read yet? What about someone who has trouble hearing?
WINDOWS XP PROFESSIONAL AUTOMATING THE WINDOWS XP INSTALLATION Bilal Munir Mughal Chapter-2 1.
Durability and Crash Recovery for Distributed In-Memory Storage Ryan Stutsman, Asaf Cidon, Ankita Kejriwal, Ali Mashtizadeh, Aravind Narayanan, Diego Ongaro,
RAMCloud: Low-latency DRAM-based storage Jonathan Ellithorpe, Arjun Gopalan, Ashish Gupta, Ankita Kejriwal, Collin Lee, Behnam Montazeri, Diego Ongaro,
RAMCloud: Scalable High-Performance Storage Entirely in DRAM John Ousterhout Stanford University (with Christos Kozyrakis, David Mazières, Aravind Narayanan,
© 2009 Lenovo Lenovo Confidential Lenovo Confidential Lenovo Confidential Lenovo Confidential Lenovo Confidential Lenovo Rapid Drive 2.0 TSC, lenovo.
Log-structured Memory for DRAM-based Storage Stephen Rumble, John Ousterhout Center for Future Architectures Research Storage3.2: Architectures.
CS 140 Lecture Notes: Technology and Operating Systems Slide 1 Technology Changes Mid-1980’s2012Change CPU speed15 MHz2.5 GHz167x Memory size8 MB4 GB500x.
Interoperability and Image Analysis KC Stegbauer.
Fast Crash Recovery in RAMCloud. Motivation The role of DRAM has been increasing – Facebook used 150TB of DRAM For 200TB of disk storage However, there.
11.1Database System Concepts. 11.2Database System Concepts Now Something Different 1st part of the course: Application Oriented 2nd part of the course:
RAMCloud Overview and Status John Ousterhout Stanford University.
Near Real-Time Verification At The Forecast Systems Laboratory: An Operational Perspective Michael P. Kay (CIRES/FSL/NOAA) Jennifer L. Mahoney (FSL/NOAA)
David P. Anderson Space Sciences Laboratory University of California – Berkeley Public and Grid Computing.
)1()1( Presenter: Noam Presman Advanced Topics in Storage Systems – Semester B 2013 Authors: A.Cidon, R.Stutsman, S.Rumble, S.Katti,
Spring 2003CS 4611 Replication Outline Failure Models Mirroring Quorums.
Best Available Technologies: External Storage Overview of Opportunities and Impacts November 18, 2015.
CSci8211: Distributed Systems: RAMCloud 1 Distributed Shared Memory/Storage Case Study: RAMCloud Developed by Stanford Platform Lab  Key Idea: Scalable.
John Ousterhout Stanford University RAMCloud Overview and Update SEDCL Retreat June, 2013.
Windows Terminal Services for Remote PVSS Access Peter Chochula ALICE DCS Workshop 21 June 2004 Colmar.
Studies of LHCb Trigger Readout Network Design Karol Hennessy University College Dublin Karol Hennessy University College Dublin.
Prospects of Software Industry in Bangladesh Natasha Ahmed – Maliha Tasnim – Anamika Almas Storage Devise Natasha Ahmed.
RAMCloud and the Low-Latency Datacenter John Ousterhout Stanford Platform Laboratory.
Administering Change Documenting a Running Network Upgrading a Network Moving a Network 1.
29/04/2008ALICE-FAIR Computing Meeting1 Resulting Figures of Performance Tests on I/O Intensive ALICE Analysis Jobs.
Red, Green, Black & Blue: Western Digital Hard Drives Explained | ☎ (+1) | ☎ (+1)
Log-Structured Memory for DRAM-Based Storage Stephen Rumble and John Ousterhout Stanford University.
Deployment of a Virtualized Server Grid
Disaster Planning and Recovery
Large-scale file systems and Map-Reduce
Dedicated Hosting Servers In The US With Faster Bandwidth Connectivity Fast bandwidth connectivity for dedicated hosting servers is vital for any enterprise.
Grid Canada Testbed using HEP applications
RAMCloud Architecture
RAMCloud Architecture
CS 140 Lecture Notes: Technology and Operating Systems
Lecture 15 Reading: Bacon 7.6, 7.7
Presentation transcript:

Experiences With RAMCloud Applications John Ousterhout, Jonathan Ellithorpe, Bob Brown

● January 2014: RAMCloud 1.0 (first practical version) ● Application experiments so far:  Stanford: natural language processing, graph algorithms  Open Networking Laboratory: ONOS (operating system for software defined networks)  CERN: high energy physics (visiting scientist, summer 2014)  Huawei: real-time device management ● Challenges  Low-latency networking not yet commonplace  RAMCloud not cost-effective at small scale  RAMCloud is too slow (!!) January 30, 2015RAMCloud Application ExperiencesSlide 2 Overview

January 30, 2015RAMCloud Application ExperiencesSlide 3 Scale and Recovery Crashed Master Recovery Masters ~500 MB/sec/server Fast crash recovery: partition lost data Cluster Size Server Capacity Cluster Capacity Recovery Time 101 servers50 GB5 TB1 sec 201 servers100 GB20 TB1 sec 6 servers100 GB600 GB40 sec 6 servers2.5 GB15 GB1 sec 11 servers5 GB55 GB1 sec Small clusters can’t have both fast recovery and large capacity/server

● Choice #2 is 100x faster than RAMCloud  And, can store data in application-specific fashion  But, data must partition  What about persistence? January 30, 2015RAMCloud Application ExperiencesSlide 4 Fast But Not Fastest … … RAMCloud Servers Choice #1: 5-10 µs remote access Choice #2: ns local access Clients

● Technology transfer is a numbers game:  Odds of success with any one group or project are low  Must try many experiments to find the right fit ● Our goals:  Learn something from every test case  Keep improving RAMCloud ● Application issues suggest new research opportunities January 30, 2015RAMCloud Application ExperiencesSlide 5 Conclusion