Pelican: A building block for exascale cold data storage

Slides:



Advertisements
Similar presentations
2  Industry trends and challenges  Windows Server 2012: Modern workstyle, enabled  Access from virtually anywhere, any device  Full Windows experience.
Advertisements

Large Scale Computing Systems
Energy Efficiency through Burstiness Athanasios E. Papathanasiou and Michael L. Scott University of Rochester, Computer Science Department Rochester, NY.
4/11/2017.
Pelican: A Building Block for Exascale Cold Data Storage
Clouds are like cotton candy, and irons … make shirts flat? Barney Boisvert
CS Spring 2011 CS 414 – Multimedia Systems Design Lecture 27 – Media Server (Part 3) Klara Nahrstedt Spring 2011.
ANALYZING STORAGE SYSTEM WORKLOADS Paul G. Sikalinda, Pieter S. Kritzinger {psikalin, DNA Research Group Computer Science Department.
Towards Energy Efficient Hadoop Wednesday, June 10, 2009 Santa Clara Marriott Yanpei Chen, Laura Keys, Randy Katz RAD Lab, UC Berkeley.
What will my performance be? Resource Advisor for DB admins Dushyanth Narayanan, Paul Barham Microsoft Research, Cambridge Eno Thereska, Anastassia Ailamaki.
Cassandra Database Project Alireza Haghdoost, Jake Moroshek Computer Science and Engineering University of Minnesota-Twin Cities Nov. 17, 2011 News Presentation:
Reconfigurable Network Topologies at Rack Scale
UC Berkeley Improving MapReduce Performance in Heterogeneous Environments Matei Zaharia, Andy Konwinski, Anthony Joseph, Randy Katz, Ion Stoica University.
Towards Energy Efficient MapReduce Yanpei Chen, Laura Keys, Randy H. Katz University of California, Berkeley LoCal Retreat June 2009.
Energy Efficient Prefetching – from models to Implementation 6/19/ Adam Manzanares and Xiao Qin Department of Computer Science and Software Engineering.
Slide 1 ISTORE: System Support for Introspective Storage Appliances Aaron Brown, David Oppenheimer, and David Patterson Computer Science Division University.
Online Auctions in IaaS Clouds: Welfare and Profit Maximization with Server Costs Xiaoxi Zhang 1, Zhiyi Huang 1, Chuan Wu 1, Zongpeng Li 2, Francis C.M.
Everest: scaling down peak loads through I/O off-loading D. Narayanan, A. Donnelly, E. Thereska, S. Elnikety, A. Rowstron Microsoft Research Cambridge,
Cloud Computing Economics Ville Volanen
Ji-Yong Shin Cornell University In collaboration with Mahesh Balakrishnan (MSR SVC), Tudor Marian (Google), and Hakim Weatherspoon (Cornell) Gecko: Contention-Oblivious.
New Challenges in Cloud Datacenter Monitoring and Management
CoolAir Temperature- and Variation-Aware Management for Free-Cooled Datacenters Íñigo Goiri, Thu D. Nguyen, and Ricardo Bianchini 1.
SilverLining. Stuff we're covering Hardware infrastructure and scaling Cloud platform as a service The SilverLining Project.
Cloud Data Center/Storage Power Efficiency Solutions Junyao Zhang 1.
Middleware Enabled Data Sharing on Cloud Storage Services Jianzong Wang Peter Varman Changsheng Xie 1 Rice University Rice University HUST Presentation.
Dynamic Resource Allocation Using Virtual Machines for Cloud Computing Environment.
Unifying Primary Cache, Scratch, and Register File Memories in a Throughput Processor Mark Gebhart 1,2 Stephen W. Keckler 1,2 Brucek Khailany 2 Ronny Krashinsky.
Storage Management in Virtualized Cloud Environments Sankaran Sivathanu, Ling Liu, Mei Yiduo and Xing Pu Student Workshop on Frontiers of Cloud Computing,
1EMC CONFIDENTIAL—INTERNAL USE ONLY Why EMC for SQL Performance Optimization.
School of EECS, Peking University Microsoft Research Asia UStore: A Low Cost Cold and Archival Data Storage System for Data Centers Quanlu Zhang †, Yafei.
Webscale Computing Mike Culver Amazon Web Services.
Building Green Cloud Services at Low Cost Josep Ll. Berral, Íñigo Goiri, Thu D. Nguyen, Ricard Gavaldà, Jordi Torres, Ricardo Bianchini.
Challenges towards Elastic Power Management in Internet Data Center.
Enabling Dynamic Data and Indirect Mutual Trust for Cloud Computing Storage Systems.
Ji-Yong Shin Cornell University In collaboration with Mahesh Balakrishnan (MSR SVC), Tudor Marian (Google), Lakshmi Ganesh (UT Austin), and Hakim Weatherspoon.
CEPH: A SCALABLE, HIGH-PERFORMANCE DISTRIBUTED FILE SYSTEM S. A. Weil, S. A. Brandt, E. L. Miller D. D. E. Long, C. Maltzahn U. C. Santa Cruz OSDI 2006.
Dilip N Simha, Maohua Lu, Tzi-cher chiueh Park Chanhyun ASPLOS’12 March 3-7, 2012 London, England, UK.
Systems and Networking Challenges in Cloud Computing: Toward Software-Defined Clouds Aditya Akella TA: Aaron Gember Fall
Fast File System 2/17/2006. Introduction Paper talked about changes to old BSD 4.2 File System (FS) Motivation - Applications require greater throughput.
Symbiotic Routing in Future Data Centers Hussam Abu-Libdeh Paolo Costa Antony Rowstron Greg O’Shea Austin Donnelly MICROSOFT RESEARCH Presented By Deng.
Storing and Serving Multimedia. What is a Media Server? A scalable storage manager Allocates multimedia data optimally among disk resources Performs memory.
Copyright © 2010, Performance and Power Management for Cloud Infrastructures Hien Nguyen Van; Tran, F.D.; Menaud, J.-M. Cloud Computing (CLOUD),
Accounting for Load Variation in Energy-Efficient Data Centers
CSE791 COURSE PRESENTATION QIUWEN CHEN Workload-aware Storage.
Presented by: Xianghan Pei
Zeta: Scheduling Interactive Services with Partial Execution Yuxiong He, Sameh Elnikety, James Larus, Chenyu Yan Microsoft Research and Microsoft Bing.
Scriber- Vibha Goyal Date:- March 03, 2016 Course:- CS 525University of Illinois at Urbana Champaign CalvinFS: Consistent WAN Replication and Scalable.
Scientific days, June 16 th & 17 th, 2014 This work has been partially supported by the LabEx PERSYVAL-Lab (ANR-11-LABX ) funded by the French program.
R2C2: A Network Stack for Rack-scale Computers Paolo Costa, Hitesh Ballani, Kaveh Razavi, Ian Kash Microsoft Research Cambridge EECS 582 – W161.
SizeCap: Efficiently Handling Power Surges for Fuel Cell Powered Data Centers Yang Li, Di Wang, Saugata Ghose, Jie Liu, Sriram Govindan, Sean James, Eric.
XFabric: a Reconfigurable In-Rack Network for Rack-Scale Computers Sergey Legtchenko, Nicholas Chen, Daniel Cletheroe, Antony Rowstron, Hugh Williams,
Log-Structured Memory for DRAM-Based Storage Stephen Rumble and John Ousterhout Stanford University.
Network Requirements for Resource Disaggregation
Why does your datacenter need HyperConverged Infrastructure?
Flamingo: Enabling Evolvable HDD-based Near-Line Storage
Measurement-based Design
FlashGraph: Processing Billion-Node Graphs on an Array of Commodity SSDs Scribed by Vinh Ha.
PA an Coordinated Memory Caching for Parallel Jobs
Understanding Real World Data Corruptions in Cloud Systems
Gregory Kesden, CSE-291 (Cloud Computing) Fall 2016
Research on Disks and Disk Scheduling
Decibel: Isolation and Sharing in Disaggregated Rack-Scale Storage
Be Fast, Cheap and in Control
NSF cloud Chameleon: Phase 2 Networking
Rethinking Cost and Performance of Database Systems
SQL Server on Amazon Web Services
SQL Server on Amazon Web Services
Design Tradeoffs for SSD Performance
Presentation transcript:

Pelican: A building block for exascale cold data storage Shobana Balarishnan, Richard Black, Austin Donnelly, Paul England, Adam Glass, Dave Harper, Sergey Legtchenko, Aaron Ogus, Eric Peterson, Antony Rowstron Microsoft Research, Microsoft Adapted from their OSDI and SDC slides. EECS 582 – W16

Background: cold data in the cloud EECS 582 – W16

Background: cold data in the cloud EECS 582 – W16

Right-provisioning EECS 582 – W16

Pelican: rack-scale appliance for cold data EECS 582 – W16

Prototype EECS 582 – W16

Impact of right provisioning on resources x12 x6 EECS 582 – W16 x16

Data placement: maximizing request concurrency EECS 582 – W16

Data placement: maximizing request concurrency EECS 582 – W16

Data placement: maximizing request concurrency EECS 582 – W16

IO scheduling: “spin up is the new seek” EECS 582 – W16

Challenges of right-provisioning EECS 582 – W16

Evaluation FP: full provisioning EECS 582 – W16

Throughput EECS 582 – W16

Time to first byte EECS 582 – W16

Power consumption EECS 582 – W16

Conclusion Pros and Cons Reduce capital cost and operating cost Meet requirements from cold data workload Sensitive to hardware changes Manually handle many constraints EECS 582 – W16

x4 x2 Related Works Amazon Glacier Facebook cold storage datacenter Two billion photos Maybe right-provisioning idea Blue-ray optical disk x4 x2 EECS 582 – W16

Q&A Thank You! EECS 582 – W16