WheelFS Jeremy Stribling, Frans Kaashoek, Jinyang Li, Robert Morris

Slides:

Advertisements

Similar presentations

Summary Cache: A Scalable Wide-Area Web Cache Sharing Protocol Li Fan, Pei Cao and Jussara Almeida University of Wisconsin-Madison Andrei Broder Compaq/DEC.

Advertisements

Searching over Many Sites in Jeremy Stribling Joint work with: Jinyang Li, M. Frans Kaashoek, Robert Morris MIT Computer Science and Artificial Intelligence.

Building Distributed, Wide-Area Applications with WheelFS Jeremy Stribling, Emil Sit, Frans Kaashoek, Jinyang Li, and Robert Morris MIT CSAIL and NYU.

Recovery Techniques in Mobile Databases Prepared by Ammar Hamamra.

Don’t Give Up on Distributed File Systems Jeremy Stribling, Emil Sit, Frans Kaashoek, Jinyang Li, and Robert Morris MIT CSAIL and NYU.

Simplifying Wide-Area Application Development with WheelFS Jeremy Stribling In collaboration with Jinyang Li, Frans Kaashoek, Robert Morris MIT CSAIL &

Serverless Network File Systems. Network File Systems Allow sharing among independent file systems in a transparent manner Mounting a remote directory.

11-May-15CSE 542: Operating Systems1 File system trace papers The Zebra striped network file system. Hartman, J. H. and Ousterhout, J. K. SOSP '93. (ACM.

Flexible, Wide-Area Storage for Distributed Systems Using Semantic Cues Jeremy Stribling Thesis Defense, August 6, 2009 Including material previously published.

OverCite: A Distributed, Cooperative CiteSeer Jeremy Stribling, Jinyang Li, Isaac G. Councill, M. Frans Kaashoek, Robert Morris MIT Computer Science and.

Overlay Networks + Internet routing has exhibited scalability - Internet routing is inefficient -Difficult to add intelligence to Internet Solution: Overlay.

Consistency in NFS and AFS. Network File System (NFS) Uses client caching to reduce network load Built on top of RPC Server cache: X Client A cache: XClient.

1 Principles of Reliable Distributed Systems Tutorial 12: Frangipani Spring 2009 Alex Shraer.

Efficient replica maintenance for distributed storage systems Byung-Gon Chun, Frank Dabek, Andreas Haeberlen, Emil Sit, Hakim Weatherspoon, M. Frans Kaashoek,

Cooperative Computing for Data Intensive Science Douglas Thain University of Notre Dame NSF Bridges to Engineering 2020 Conference 12 March 2008.

CS 672 Paper Presentation Presented By Saif Iqbal “CarNet: A Scalable Ad Hoc Wireless Network System” Robert Morris, John Jannotti, Frans Kaashoek, Jinyang.

(ITI310) By Eng. BASSEM ALSAID SESSIONS 8: Network Load Balancing (NLB)

Distributed File Systems Sarah Diesburg Operating Systems CS 3430.

Cloud Distributed Computing Environment Content of this lecture is primarily from the book “Hadoop, The Definite Guide 2/e)

Wide-area cooperative storage with CFS Frank Dabek, M. Frans Kaashoek, David Karger, Robert Morris, Ion Stoica.

Distributed systems [Fall 2014] G Lec 1: Course Introduction.

Microsoft FrontPage 2003 Illustrated Complete Finalizing a Web Site.

IMDGs An essential part of your architecture. About me

Distributed File Systems Overview  A file system is an abstract data type – an abstraction of a storage device.  A distributed file system is available.

Jalisa Eady Definitions Mr. Gabbard Pd

WheelFS Jeremy Stribling, Frans Kaashoek, Jinyang Li, Robert Morris MIT CSAIL and New York University.

Offline Web Apps - an offline web application is a list of URLs for HTML, CSS, JavaScript, images, or any other kind of resource. Most browsers have offline.

1 Secure Peer-to-Peer File Sharing Frans Kaashoek, David Karger, Robert Morris, Ion Stoica, Hari Balakrishnan MIT Laboratory.

Caching Consistency and Concurrency Control Contact: Dingshan He

Graduate school experience in a systems group Jinyang Li assistant professor New York University.

OLE Slide No. 1 Object Linking and Embedding H OLE H definition H add other information to documents H copy.

Distributed systems [Fall 2015] G Lec 1: Course Introduction.

Module 4: Managing Access to Resources. Overview Overview of Managing Access to Resources Managing Access to Shared Folders Managing Access to Files and.

WheelFS. Agenda Introduction Background Feature System Architecture Design and Details Conclusion.

Bristol Bay Weekly Report + Mixed Salmon Update. If you do not work with Bristol Bay data, feel free to take a break now.

Cloud Distributed Computing Environment Hadoop. Hadoop is an open-source software system that provides a distributed computing environment on cloud (data.

1 Secure Peer-to-Peer File Sharing Frans Kaashoek, David Karger, Robert Morris, Ion Stoica, Hari Balakrishnan MIT Laboratory.

Information Systems in Organizations 2.2 Systems Architecture: Devices, Network, Data and Apps.

Libraries in the digital age Collection & preservation for generational access part two The LOCKSS Program.

DISTRIBUTED FILE SYSTEM- ENHANCEMENT AND FURTHER DEVELOPMENT BY:- PALLAWI(10BIT0033)

Introduction to Operating Systems

Data Management with Google File System Pramod Bhatotia wp. mpi-sws

Distributed File Systems

Module 4: Managing Access to Resources

Steve Ko Computer Sciences and Engineering University at Buffalo

Steve Ko Computer Sciences and Engineering University at Buffalo

Concurrent Version Control

ECE 671 – Lecture 16 Content Distribution Networks

Introduction to Operating Systems

EECS 498 Introduction to Distributed Systems Fall 2017

Jinyang Li’s Research Distributed Systems Wireless Networks

Distributed P2P File System

Distributed systems [Fall 2016] G Jinyang Li

ColdFusion Performance Troubleshooting and Tuning

CSE 486/586 Distributed Systems Consistency --- 1

CSE 451: Operating Systems Winter Module 22 Distributed File Systems

The Memory B. Ramamurthy C B. Ramamurthy.

Distributed File Systems

Distributed File Systems

Module P3 Practical: Building a webapp in nodejs and

Locality Optimizations in Tapestry Sahara/OceanStore Winter Retreat

CSE 451: Operating Systems Spring Module 21 Distributed File Systems

Distributed File Systems

CSE 451: Operating Systems Winter Module 22 Distributed File Systems

Distributed File Systems

Consistent Hashing and Distributed Hash Table

Distributed File Systems

Caching 50.5* + Apache Kafka

If my file system only has lots of big video files what block size do I want? Large (correct) Small.

Presentation transcript:

WheelFS Jeremy Stribling, Frans Kaashoek, Jinyang Li, Robert Morris MIT CSAIL and New York University

I Want to Publish an SOSP Paper But I don’t want to do a lot of work Idea: New cooperative Web cache!

Exploit a Network File System Apache Caching Proxy Apache Caching Proxy Apache Caching Proxy Apache Caching Proxy foo.html Network File Server (/mnt/awesome-cdn/cache)

It Is Too Good to Be True Tries to mask failures with long timeouts Strict consistency prevents use of cached data under failures My naïve SOSP strategy might not work

So What Are My Options? Give up dream and just not graduate Submit random paper [Rooter, WMSCI ’05] Build an FS that would work for real apps Apache can use old copies of data And can always fall back to the origin Why not tell the FS?

WheelFS: Storage for Wide-area Apps File system API Semantic cues give apps control of behavior under failures  Make Apache distributed with 1 line change

Cooperative Web Cache on WheelFS Apache Caching Proxy Apache Caching Proxy Apache Caching Proxy Apache Caching Proxy WheelFS WheelFS WheelFS WheelFS /wfs/cache/ .maxtime=500 /.bestversion /.lax /$URL Use newest version you can find quickly Allow multiple versions of a directory to be updated Time out operations after T milliseconds

I’m Running Out of Time, So: A compact but powerful set of cues Useful to other applications, including: PlanetLab measurement studies Parallel Grid computations Distribute make See you at the next big conference!