Outline for today Feasibility of a Serverless Distributed File System deployed on an Existing set of Desktop PCs – Microsoft research. ACM SIGMETRICS 2000.

Slides:



Advertisements
Similar presentations
TRANSACTION PROCESSING SYSTEM ROHIT KHOKHER. TRANSACTION RECOVERY TRANSACTION RECOVERY TRANSACTION STATES SERIALIZABILITY CONFLICT SERIALIZABILITY VIEW.
Advertisements

CMPT 401 Summer 2007 Dr. Alexandra Fedorova Lecture X: Transactions.
Location and Tracking Spring 2004: Location Recognition Larry Rudolph Location of what? Services applications, resources, sensors, actuators where.
1 Countermeasures against Consistency Anomalies in Databases with Relaxed ACID Properties. By Lars Frank Copenhagen Business School.
CMPT Dr. Alexandra Fedorova Lecture X: Transactions.
2003/4/21CSE 6362 Intelligent Environments Spring The Anatomy of a Context- Aware Application Computer Science and Engineering University of Texas.
1 ACID Properties of Transactions Chapter Transactions Many enterprises use databases to store information about their state –e.g., Balances of.
5.1 © 2004 Pearson Education, Inc. Exam Managing and Maintaining a Microsoft® Windows® Server 2003 Environment Lesson 5: Working with File Systems.
70-290: MCSE Guide to Managing a Microsoft Windows Server 2003 Environment Chapter 7: Advanced File System Management.
Transaction. A transaction is an event which occurs on the database. Generally a transaction reads a value from the database or writes a value to the.
Distributed Databases
Academic Year 2014 Spring. MODULE CC3005NI: Advanced Database Systems “DATABASE RECOVERY” (PART – 1) Academic Year 2014 Spring.
Managing Multi-User Databases AIMS 3710 R. Nakatsu.
Jan 17, 2001CSCI {4,6}900: Ubiquitous Computing1 Announcements I will be out of town Monday and Tuesday to present at Multimedia Computing and Networking.
Software.
Jan 24, 2001CSCI {4,6}900: Ubiquitous Computing1 Announcements Homework #2 is due in 2 weeks. If your Homework 1 is not working (and you believe that it.
Sentient Computing Presenter : Alhaf malik.K Syed Ammal Engineering College, Ramanathapuram.
Introduction to Database Systems1. 2 Basic Definitions Mini-world Some part of the real world about which data is stored in a database. Data Known facts.
Database Environment Chapter 2. Data Independence Sometimes the way data are physically organized depends on the requirements of the application. Result:
Eduardo Gutarra Velez. Outline Distributed Filesystems Motivation Google Filesystem Architecture The Metadata Consistency Model File Mutation.
Feb 1, 2001CSCI {4,6}900: Ubiquitous Computing1 Eager Replication and mobile nodes Read on disconnected clients may give stale data Eager replication prohibits.
Chapter 10 Recovery System. ACID Properties  Atomicity. Either all operations of the transaction are properly reflected in the database or none are.
Jan 17, 2001CSCI {4,6}900: Ubiquitous Computing1 Announcements Did you hear about the Microsoft site crash yesterday? Compiling in Solaris (gemini) gcc.
Configuring, Managing and Maintaining Windows Server® 2008 Servers Course 6419A.
CSE 486/586 CSE 486/586 Distributed Systems Consistency Steve Ko Computer Sciences and Engineering University at Buffalo.
Dsitributed File Systems
Storage Systems CSE 598d, Spring 2007 Lecture 13: File Systems March 8, 2007.
Overview Issues in Mobile Databases – Data management – Transaction management Mobile Databases and Information Retrieval.
70-290: MCSE Guide to Managing a Microsoft Windows Server 2003 Environment, Enhanced Chapter 7: Advanced File System Management.
Lesson 9: SOFTWARE ICT Fundamentals 2nd Semester SY
Dr. Ir. Yeffry Handoko Putra
The Anatomy of a Context-Aware Application
Introduction To DBMS.
Managing Multi-User Databases
Databases We are particularly interested in relational databases
Database Applications (15-415) DBMS Internals- Part XIII Lecture 22, November 15, 2016 Mohammad Hammoud.
CSE 486/586 Distributed Systems Consistency --- 1
Introduction to Operating System (OS)
Chapter 3: Windows7 Part 3.
Outline Announcements Fault Tolerance.
Today: Coda, xFS Case Study: Coda File System
Database Applications (15-415) DBMS Internals- Part XIII Lecture 25, April 15, 2018 Mohammad Hammoud.
CSE 486/586 Distributed Systems Consistency --- 1
EEC 688/788 Secure and Dependable Computing
Database Systems Instructor Name: Lecture-3.
Transaction Processing Concepts
EEC 688/788 Secure and Dependable Computing
Outline Announcements Lab2 Distributed File Systems 1/17/2019 COP5611.
Database Security Transactions
Lecture 21: Replication Control
Chapter 2: Operating-System Structures
Introduction to Operating Systems
CSE 542: Operating Systems
Lecture 20: Intro to Transactions & Logging II
Transaction management
Database Recovery 1 Purpose of Database Recovery
CPSC-608 Database Systems
Outline Review of Quiz #1 Distributed File Systems 4/20/2019 COP5611.
File System Management
Federated, Available, and Reliable Storage for an Incompletely Trusted Environment Atul Adya, William J. Bolosky, Miguel Castro, Gerald Cermak, Ronnie.
EEC 688/788 Secure and Dependable Computing
Database Applications (15-415) DBMS Internals- Part XIII Lecture 24, April 14, 2016 Mohammad Hammoud.
Introducing NTFS Reliability Security Long file names Efficiency
Content Distribution Network
Chapter 2: Operating-System Structures
Lecture 21: Replication Control
Abstractions for Fault Tolerance
Concurrency Control.
Transaction Communication
Outline for today Oceanstore: An architecture for Global-Scale Persistent Storage – University of California, Berkeley. ASPLOS 2000 Feasibility of a Serverless.
Presentation transcript:

Outline for today Feasibility of a Serverless Distributed File System deployed on an Existing set of Desktop PCs – Microsoft research. ACM SIGMETRICS 2000 14-May-19 Ubiquitous Computing

MSR Serverless Distributed File System They’ve actually implemented this system within Microsoft and hence have real results Assumption 1: not-fully-trusted environment Assumption 2: Disk space is not that free Each disk is partitioned into three areas: Scratch area – for local computations Global storage area Local cache for global storage 14-May-19 Ubiquitous Computing

Efficiency consideration Compress data in storage Coalesce distinct files that have identical contents Probably an artifact of Windows environment that stores files in specific locations e.g. c:\windows\system\ File are replicated Machines that are topologically close Machines that are lightly loaded Non-cache reads and writes to prevent buffer cache pollution 14-May-19 Ubiquitous Computing

Files in a directory are replicated together Replica management Files in a directory are replicated together When new machines join, its data is replicated to other machines Replicas of other files are moved into the new machine When machine leaves, the data in that machine is replicated in other machines from other replicas 14-May-19 Ubiquitous Computing

File updates are digitally signed Security File updates are digitally signed File contents are encrypted before replication Convergent encryption to coalesce encrypted file Encryption: Hash(file contents) -> uniqueHash Encrypt(unencrypterfile, uniqueHash)->encryptedfile User1: encrypt(UserKey1, uniqueHash) -> Key1 User2: encrypt(UserKey2, uniqueHash) -> Key2 Decryption User1: decrypt(UserKey1, Key1) -> uniqueHash Decrypt(encryptedfile, uniqueHash) -> unencryptedfile 14-May-19 Ubiquitous Computing

Users specify the session guarantees required for each session Application API Related read, write operations to objects form a session (defined by the application developer) Users specify the session guarantees required for each session Applications can register call back functions for exceptions 14-May-19 Ubiquitous Computing

Transactions (Database technology) A transaction is a program unit that must be executed atomically; either the entire unit is executed or none at all. The transaction either completes in its entirety, or it does not (or at least, nothing appears to have happened). A transaction can generally be thought of as a sequence of reads and writes, which is either committed or aborted. A committed transaction is one that has been completed entirely and successfully, whereas an aborted transaction is one that has not. If a transaction is aborted, then the state of the system must be rolled-back to the state it had before the aborted transaction began. 14-May-19 Ubiquitous Computing

ACID semantics Atomicity – each transaction is atomic, every operation succeeds or none at all Consistency – maintaining correct invariants across the data before and after the transaction Isolation - either has the value before the atomic action or after it, but never intermediate Durability – persistent on stable storage (backups, transaction logging, checkpoints) 14-May-19 Ubiquitous Computing

Relax the ACID constraints Relaxed semantics Relax the ACID constraints We could relax consistency for better performance (ala Bayou) where you are willing to tolerate inconsistent data for better performance. For example, you are willing to work with partial calendar update and are willing to work with partial information rather than wait for confirmed data. More on this later on in the course. 14-May-19 Ubiquitous Computing

Naming and Location Management The Anatomy of a Context-Aware Application – Andy Harter, Andy Hopper, Pete Steggles, Andy Ward and Paul Webster. AT&T Labs, Cambridge, UK 14-May-19 Ubiquitous Computing

History of AT&T Cambridge Lab It used to be Olivetti Research Lab ORL developed active badges, VNC etc.. VNC allows you to teleport across machines by moving the display to another terminal near you 14-May-19 Ubiquitous Computing

Motivation for context-aware application Users application should be available where-ever the user goes, in a suitably adapted form Context aware application is one which adapts its behavior to a changing environment E.g. Follow-Me applications Context aware applications need to know the location of users and equipment, and the capabilities of the equipment and networking infrastructure 14-May-19 Ubiquitous Computing

Components of Context-Aware Applications A fine-grained location system, which is used to locate and identify objects A detailed data model, which describes the essential real-world entities that are involved in mobile applications A persistent distributed object system, which presents that data model in a form accessible to applications 14-May-19 Ubiquitous Computing

Components of Context Aware Applications Resource monitors, which run on networked equipment and communicate status information in a centralized repository A spatial monitoring, which enables event-based location-aware applications Important lesson: It is very important to present location information in a form suitable to the application 14-May-19 Ubiquitous Computing

Out-door: Global Positioning Satellite (GPS) Location system Out-door: Global Positioning Satellite (GPS) Developed by US military, available for civilian use Locates using triangulation with multiple satellites Accurate up to a few meters. (Military version more accurate) In-door: Radio-based, Infra-red, Ultra-sound For indoor systems, you have to worry about interference from multipath sources, obstructions and devices (such as fluorescent bulb ballasts) Ultrasound is more immune to these interferences 14-May-19 Ubiquitous Computing

Radio transceiver, ultrasonic transducer and control logic Bat Unit Radio transceiver, ultrasonic transducer and control logic Each bat has a GUID Use the radio, ultrasonic transducer and the speed of sound in air (estimated from ambient temperature) to estimate location Use multiple receivers to get 3D location using multilateration Reflections of ultrasonic waves – statistical outlier elimination 14-May-19 Ubiquitous Computing

Bats 14-May-19 Ubiquitous Computing

Sentient Computing 14-May-19 Ubiquitous Computing

Discussion 14-May-19 Ubiquitous Computing