Content Distribution Network

Slides:



Advertisements
Similar presentations
Authentication Applications. will consider authentication functions will consider authentication functions developed to support application-level authentication.
Advertisements

What is OceanStore? - 10^10 users with files each - Goals: Durability, Availability, Enc. & Auth, High performance - Worldwide infrastructure to.
Lakshmi Narayana Gupta Kollepara 10/26/2009 CSC-8320.
Pond: the OceanStore Prototype CS 6464 Cornell University Presented by Yeounoh Chung.
The Zebra Striped Network Filesystem. Approach Increase throughput, reliability by striping file data across multiple servers Data from each client is.
Chapter 14 From Cryptography and Network Security Fourth Edition written by William Stallings, and Lecture slides by Lawrie Brown, the Australian Defence.
U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science Emery Berger University of Massachusetts Amherst Operating Systems CMPSCI 377 Lecture.
OceanStore: An Infrastructure for Global-Scale Persistent Storage John Kubiatowicz, David Bindel, Yan Chen, Steven Czerwinski, Patrick Eaton, Dennis Geels,
OCT1 Principles From Chapter One of “Distributed Systems Concepts and Design”
Group Management, Permissions, and Revocation in OceanStore Barbara Engelhardt George Porter Naveen Sastry UC Berkeley January 2002.
Andrew Schroeder Networking Aspects of Cloud Computing.
OceanStore: Data Security in an Insecure world John Kubiatowicz.
OceanStore: An Architecture for Global-Scale Persistent Storage Professor John Kubiatowicz, University of California at Berkeley
Concurrency Control & Caching Consistency Issues and Survey Dingshan He November 18, 2002.
OceanStore An Architecture for Global-Scale Persistent Storage Motivation Feature Application Specific Components - Secure Naming - Update - Access Control-
University of Pennsylvania 11/21/00CSE 3801 Distributed File Systems CSE 380 Lecture Note 14 Insup Lee.
OceanStore: An Architecture for Global - Scale Persistent Storage John Kubiatowicz, David Bindel, Yan Chen, Steven Czerwinski, Patric Eaton, Dennis Geels,
Distributed Computer Security 8.2 Discretionary Access Control Models - Sai Phalgun Tatavarthy.
Sanjay Ghemawat, Howard Gobioff, and Shun-Tak Leung Google∗
Windows ® Powered NAS. Agenda Windows Powered NAS Windows Powered NAS Key Technologies in Windows Powered NAS Key Technologies in Windows Powered NAS.
Presented by: Alvaro Llanos E.  Motivation and Overview  Frangipani Architecture overview  Similar DFS  PETAL: Distributed virtual disks ◦ Overview.
CSE 486/586, Spring 2012 CSE 486/586 Distributed Systems Distributed File Systems Steve Ko Computer Sciences and Engineering University at Buffalo.
Cloud Computing.
CSC 456 Operating Systems Seminar Presentation (11/13/2012) Leon Weingard, Liang Xin The Google File System.
Jan 17, 2001CSCI {4,6}900: Ubiquitous Computing1 Announcements I will be out of town Monday and Tuesday to present at Multimedia Computing and Networking.
Failure Resilience in the Peer-to-Peer-System OceanStore Speaker: Corinna Richter.
Pond: the OceanStore Prototype Sean Rhea, Patric Eaton, Dennis Gells, Hakim Weatherspoon, Ben Zhao, and John Kubiatowicz University of California, Berkeley.
The Data Grid: Towards an Architecture for the Distributed Management and Analysis of Large Scientific Dataset Caitlin Minteer & Kelly Clynes.
OceanStore: An Infrastructure for Global-Scale Persistent Storage John Kubiatowicz, David Bindel, Yan Chen, Steven Czerwinski, Patrick Eaton, Dennis Geels,
By: Ashish Gohel 8 th sem ISE.. Why Cloud Computing ? Cloud Computing platforms provides easy access to a company’s high-performance computing and storage.
Distributed File Systems Overview  A file system is an abstract data type – an abstraction of a storage device.  A distributed file system is available.
Distributed File System By Manshu Zhang. Outline Basic Concepts Current project Hadoop Distributed File System Future work Reference.
Naming CSCI 4780/6780. Names & Naming System Names have unique importance –Resource sharing –Identifying entities –Location reference Name can be resolved.
Introduction to DFS. Distributed File Systems A file system whose clients, servers and storage devices are dispersed among the machines of a distributed.
Introduction to Database Systems1. 2 Basic Definitions Mini-world Some part of the real world about which data is stored in a database. Data Known facts.
OceanStore: An Architecture for Global- Scale Persistent Storage.
Fast Crash Recovery in RAMCloud. Motivation The role of DRAM has been increasing – Facebook used 150TB of DRAM For 200TB of disk storage However, there.
Security fundamentals Topic 5 Using a Public Key Infrastructure.
Distributed File Systems Architecture – 11.1 Processes – 11.2 Communication – 11.3 Naming – 11.4.
Secure Location-Independent Autonomic Storage Architectures GR/S44501/01 February January 2007 Graham Kirby, Alan Dearle, Ron Morrison & Stuart.
Cloud Archive By: Kimberly Nolan. What it is?  The goal of a cloud archiving service is to provide a data storage (ex. Google drive and SkyDrive) as.
Dsitributed File Systems
Distributed File System. Outline Basic Concepts Current project Hadoop Distributed File System Future work Reference.
OceanStore : An Architecture for Global-Scale Persistent Storage Jaewoo Kim, Youngho Yi, Minsik Cho.
CMSC 818J: Privacy enhancing technologies Lecture 2.
Key management issues in PGP
Course: Cluster, grid and cloud computing systems Course author: Prof
Option 2: The Oceanic Data Utility: Global-Scale Persistent Storage
Amazon Storage- S3 and Glacier
Cryptography and Network Security
OceanStore: An Architecture for Global-Scale Persistent Storage
Authentication Applications
Scaling for the Future Katherine Yelick U.C. Berkeley, EECS
Plethora: Infrastructure and System Design
Providing Secure Storage on the Internet
How to Protect Big Data in a Containerized Environment
Mobile Agents.
7.1. CONSISTENCY AND REPLICATION INTRODUCTION
OceanStore: Data Security in an Insecure world
Today: Coda, xFS Case Study: Coda File System
OceanStore: An Architecture for Global-Scale Persistent Storage
AWS Cloud Computing Masaki.
Outline Announcements Lab2 Distributed File Systems 1/17/2019 COP5611.
CSE 542: Operating Systems
CSE 542: Operating Systems
Today: Distributed File Systems
Outline Review of Quiz #1 Distributed File Systems 4/20/2019 COP5611.
Mid term grades Mean = 48.59, Median = 48.5, Min = 40, Max = 56.
Review Stateless (NFS) vs Statefull (AFS)
Outline for today Oceanstore: An architecture for Global-Scale Persistent Storage – University of California, Berkeley. ASPLOS 2000 Feasibility of a Serverless.
Presentation transcript:

Content Distribution Network Outline for today Oceanstore: An architecture for Global-Scale Persistent Storage – University of California, Berkeley. ASPLOS 2000 Chord Content Distribution Network 23-May-19 4/598N: Computer Networks

Content Distribution Networks (slides courtesy Girish Borkar: Udel) original content Replica congested Replica Not congested Client 23-May-19 4/598N: Computer Networks

Applications operate on objects in persistent store E.g. files (traditional operating systems), persistent objects (in a object based system) Applications operate on objects in persistent store Powerpoint operates on a persistent .ppt file, mutating its contents Palm calendar operates on my calendar which is replicated in myYahoo, Palm Desktop and the Pilot itself Storage is cheap but maintenance is not ~ 4 $/GB 23-May-19 4/598N: Computer Networks

Global Persistent Store Persistent store is fundamental for future ubiquitous computing because it allows "devices" to operate transparently, consistently and reliably on data. Transparent: Permits behavior to be independent of the device themselves Consistently: Allows users to safely access the same information from many different devices simultaneously. Reliably: Devices can be rebooted or replaced without losing vital configuration information 23-May-19 4/598N: Computer Networks

Persistent store on a wide-scale 10 billion users, 10,000 files per user = 100 trillion files!! Information: should be separated from location. To achieve uniform and highly-available access to information, servers must be geographically distributed, but exploit caching close to clients for performance must be secure must be durable must be consistent 23-May-19 4/598N: Computer Networks

Oceanstore system model: Data Utility CaliforniaStore IndianaStore USAStore SanJoseStore Ameritech End User with roaming access 23-May-19 4/598N: Computer Networks

Oceanstore system model: Data Utility CaliforniaStore IndianaStore USAStore SanJoseStore Ameritech End User with roaming access 23-May-19 4/598N: Computer Networks

Untrusted infrastructure (utility model – telephone) Oceanstore Goals Untrusted infrastructure (utility model – telephone) Only clients can be trusted Servers can crash, or leak information to third parties Most of the servers are working correctly most of the time Class of trusted servers that can carry out protocols on the clients behalf (financially liable for integrity of data) Nomadic Data Access Data can be cached anywhere, anytime (promiscuous caching) Continuous introspective monitoring to locate data close to the user 23-May-19 4/598N: Computer Networks

Oceanstore Persistent Object Named by a globally unique id (GUID) Such GUIDs are hard to use. If you are expecting 10 trillion files, your GUID will have to be a long (say 128 bit) ID rather than a simple name passwd vs 12agfs237dfdfhj459uxzozfk459ldfnhgga self-certifying names secureHash(/id=surendar,ou=uga,key=<SecureKey>/etc/passwd) -> uniqueId Map uniqueId->GUID Users would use symbolic links for easy usage /etc/passwd -> uniqueId 23-May-19 4/598N: Computer Networks

SecureHash Pros: Cons: The self-certifying name specifies my access rights Cons: If I lose the key, the data is lost Key management issues Keys can be upgraded Keys can be revoked How do we share data? 23-May-19 4/598N: Computer Networks

All read-shared-users share an encryption key Access Control All read-shared-users share an encryption key Revocation: Data should be deleted from all replicas Data should be re-encrypted New keys should be distributed Clients can still access old data till it is deleted in all replicas All writes are signed Validity checked by Access Control Lists (ACLs) If A says trust B, B says trust C, C says trust D, what can you infer about A ? D 23-May-19 4/598N: Computer Networks

Oceanstore Persistent Object Objects are replicated on multiple servers. Replicated objects are not tied to particular servers i.e. floating replicas Replicas located by a probabilistic algorithm first before using a deterministic algorithm Data can be active or archival. Archival data is read-only and spread over multiple servers – deep archival storage 23-May-19 4/598N: Computer Networks

Application level conflict resolution Updates Objects are modified through updates (data is never overwritten) i.e. versioning system Application level conflict resolution Updates consist of a predicate and value pair. If a predicate evaluates to true, the corresponding value is applied. <room 453 free?>, <reserve room> <room 527 free?>, <reserve room> <else> <go to Jittery Joes> This is similar to Bayou which we will explore later in the semester 23-May-19 4/598N: Computer Networks

Oceanstore uses introspection to monitor system behavior Use this information for cluster recognition Use this information for replica management 23-May-19 4/598N: Computer Networks