Wingu A Synchronization Layer For Safe Concurrent Access to Cloud Storage Craig Chasseur and Allie Terrell December 20, 2010

Why We Love Clouds ● Availability ● Stays up even when whole datacenters go down ● Accessible from anywhere with an Internet connection ● Scalability ● Support many concurrent operations on many items of data ● Support many users sharing

Why We Love Clouds ● Availability ● Stays up even when whole datacenters go down ● Accessible from anywhere with an Internet connection ● Scalability ● Support many concurrent operations on many items of data ● Support many users sharing

Overview ● Background ● Motivation and Goals ● Implementation ● Wrap Up

Overview ● Background ● Motivation and Goals ● Implementation ● Wrap Up

Amazon S3 Bucket obj Region Bucket obj

Consistency in S3 ● Most regions ● Eventual consistency for Object data ● Eventual consistency for Bucket contents ● Northern California ● Read-after-write consistency for Object data ● Eventual consistency for Bucket contents ● Neither model provides synchronization of shared access ● Last writer wins, updates may be lost

Overview ● Background ● Motivation and Goals ● Implementation ● Wrap Up

An Experiment To Measure Latency ● Using PlanetLab, we measured the time it took for a new Object in S3 ● To become readable ● To appear in the Bucket's list of contents ● For each PlanetLab site, we record the earliest and latest possible time that the Object could have become accessible at the site

Bucket List Time To Availability (s)

Design Goals ● Strong consistency guarantees ● A successful commit means changes are globally available ● There is a single canonical directory structure and a single canonical version for every object ● Prevent write conflicts ● Locking – don't assume that updates can commute ● Notify users of potential and actual changes to checked out files and directories

Overview ● Background ● Motivation and Goals ● Implementation ● Wrap Up

Architecture Amazon S3 Client Master Client

S3's Role /foo/bar/file.txt Bucket /tmp.txtconfig

S3's Role /foo/bar/file.txt Bucket /tmp.txtconfig Version ID (S3): P6L2hOMowwhpT1mtliXCMjoh0Wfh1FbF Hash (wingu): IlljY7PeQLBvmB+4XYIxLowO1RE= Written By (wingu): craig on wintermute.myisp.net

S3's Role /foo/bar/tmp.txt Bucket /tmp.txtconfig Version: Version: Version: 32323

Client's Role /file1.txt : 'r' /dir : 'r' /dir/file2.bin : 'w'... Map of Paths to LocksMount Point /mnt/s3

Master's Role Directory Tree user1 user2 user3... List of Connected Clients

Master's Role: Checkout For Read Directory TreeList of Connected Clients r rr r user1 user2 user3...

Master's Role: Lock For Write Directory TreeList of Connected Clients w rr r user1 user2 user3...

Master's Role: Lock For Write Directory TreeList of Connected Clients w rr r user1 user2 user3... readerswriter queued writers

Master's Role Directory TreeList of Connected Clients user1 user2 user3...

Master's Role Directory TreeList of Connected Clients user1 user2 user3... cached queuedlocked

Updates ClientMasterS3Readers Ask for write lock Acquire write lock Write Get version ID Update notification Success Visible? Hashes match? Success Notify all readers

Error Handling ● Many errors can be detected by the client using only local knowledge ● Errors or failures detected on the master don't alter the canonical state – the system always remains consistent and synchronized ● Errors detected on the master are reported to the originating client

Design Goals ● Strong consistency guarantees ● A successful commit means changes are globally available ● There is a single canonical directory structure and a single canonical version for every object ● Prevent write conflicts ● Locking – don't assume that updates can commute ● Notify users of potential and actual changes to checked out files and directories

Overview ● Background ● Motivation and Goals ● Implementation ● Wrap Up

Summary ● Amazon S3 consistency is weak ● Wingu provides a consistency management layer ● A single master safely synchronizes all access to a shared S3 Bucket

Conclusions ● Strong consistency can be built on top of weakly consistent systems ● A single master adds little overhead ● Traffic and processing for synchronization is a small fraction of that for actual storage ● Explicit Locking Has Pros and Cons ● Pro: no conflicting writes for any data type ● Con: reduced concurrency if writes can commute

Questions? Presentation made with Open Office Impress

Use Case For Shared Cloud Storage ● Users may check out all or part of a shared directory tree ● No conflicting updates ● User is notified when a checked-out file or directory changes ● Names of objects in S3 match directory structure ● Allows back-door read-only access

Existing Systems ● s3ql – a FUSE module backed by s3 ● Doesn't handle multiple simultaneous users ● Block oriented – doesn't map to a conventional directory structure ● Distributed Version Control (e.g. git, mercurial) ● Optimistic concurrency – updates may commute, but some conflicts require human intervention to resolve ● Can check out only needed subtrees

Read of Object Time To Availability (s)