A Redundant Global Storage Architecture Jim Leek, David Schultz, Ben Schwarz {jrleek,dschultz,bschwarz}@CS

Storage as a Service Many companies already provide access to massive data sets as a service (e.g. Google) Provide access to raw storage as a service Advantages: Already know how to manage storage clusters More reliable than personal storage Available anywhere Disadvantages: Security?

Storage as a Service Clients allocate large “chunks” of storage 64 MB for now Cost based on bandwidth and total space Bandwidth is most important Disk BW increasing at 40%/yr, capacity at 60%/yr Space costs money too Making data always available consumes power Backups More to recover when a disk is lost Why backups? Because backups to WORM storage are a good way to protect against erasure of data in the case of a security compromise.

Network Architecture Storage network is a collection of chunk servers Space allocation and deallocation is centralized at the master Stateless design; simple recovery

Network/Translation Layer Master Server (lookups) Chunk Server (storage) Responsible for replicating Fault tolerant Higher bandwidth requests Buffers requests Central authority Small number Fault tolerant Low bandwidth Distributes load Block location cache Forward bytestream allocate free locateBlock read write openIOGroup closeIOGroup whoIsMaster

Filesystem Design Encrypted on the client side Always consistent Structured as a Merkle tree Copy on write Inexpensive snapshots reduce the impact of user error Block allocation/cleaning similar to LFS Local WAL to provide fast synchronous writes when required

Updating a Block

Updating a Block

Updating a Block

Updating a Block

Updating a Block

I/O Model Servers provide no transaction support Servers only see encrypted data anyway xFS designers showed how to do isolation entirely on the client side Server respects WAW dependencies Clients attach each write to an I/O group Groups are committed in order

Security Thread Model Attacks CS MIM CS Deduce data loc. Alice Timing attacks Statistical attacks Compromised CS? Alice CS e.g. file size Bob CS Improvement over existing architectures! Even chunkservers are not in-the-know

Security II Filesystem - Encrypt data prior to storing. Decrypt on retrieval. Not a fundamental limitation of OceanStore. How do we handle data that should be read by a set of people (e.g. group privileges)? One public key. Distribute multiple private keys. Use GPG! DATA = Privi(Pub(DATA)) = Privk(Pub(DATA)) Multiple private keys for member identification Or use a single private key for anonymity

Security Guarantees MACs guarantee that the server can’t change the data Hypothetical attack: server selectively ignores writes, inconsistent versions trick the client Merkle tree structure provides fork consistency Informally, if the server prevents clients A and B from seeing each other’s updates, the clients will either detect the problem or live in parallel universes from then on Freshness?

Conclusion Improvement in security and privacy over existing architectures Simple stateless server design makes recovery easier AFS and xFS have problems with recovery Virtually unconstrained write ordering should provide good performance

Questions

Related Work OceanStore Distributed object location; Google Filesystem Master/Chunkserver architecture LFS Large sequential writes WAFL WAL for synchronous semantics Tree of blocks for consistency