1. A Redundant Global Storage Architecture
Jim Leek, David Schultz, Ben Schwarz

2. Storage as a Service
Many companies already provide access to massive data sets as a service (e.g. Google)
Provide access to raw storage as a service
Advantages:
Already know how to manage storage clusters
More reliable than personal storage
Available anywhere
Disadvantages:
Security?

3. Storage as a Service Clients allocate large “chunks” of storage
64 MB for now
Cost based on bandwidth and total space
Bandwidth is most important
Disk BW increasing at 40%/yr, capacity at 60%/yr
Space costs money too
Making data always available consumes power
Backups
More to recover when a disk is lost
Why backups? Because backups to WORM storage are a good way to protect against erasure of data in the case of a security compromise.

4. Network Architecture Storage network is a collection of chunk servers
Space allocation and deallocation is centralized at the master
Stateless design; simple recovery

5. Network/Translation Layer
Master Server (lookups) Chunk Server (storage)
Responsible for replicating
Fault tolerant
Higher bandwidth requests
Buffers requests
Central authority
Small number
Fault tolerant
Low bandwidth
Distributes load
Block location cache
Forward bytestream
allocate free
locateBlock
read write openIOGroup
closeIOGroup whoIsMaster

6. Filesystem Design Encrypted on the client side Always consistent
Structured as a Merkle tree
Copy on write
Inexpensive snapshots reduce the impact of user error
Block allocation/cleaning similar to LFS
Local WAL to provide fast synchronous writes when required

7. Updating a Block

8. Updating a Block

9. Updating a Block

10. Updating a Block

11. Updating a Block

12. I/O Model Servers provide no transaction support
Servers only see encrypted data anyway
xFS designers showed how to do isolation entirely on the client side
Server respects WAW dependencies
Clients attach each write to an I/O group
Groups are committed in order

13. Security Thread Model Attacks CS MIM CS Deduce data loc. Alice
Timing attacks
Statistical attacks
Compromised CS?
Alice CS
e.g. file size
Bob CS
Improvement over
existing architectures!
Even chunkservers are not in-the-know

14. Security II
Filesystem - Encrypt data prior to storing. Decrypt on retrieval. Not a fundamental limitation of OceanStore.
How do we handle data that should be read by a set of people (e.g. group privileges)? One public key. Distribute multiple private keys. Use GPG!
DATA = Privi(Pub(DATA)) = Privk(Pub(DATA))
Multiple private keys for member identification
Or use a single private key for anonymity

15. Security Guarantees
MACs guarantee that the server can’t change the data
Hypothetical attack: server selectively ignores writes, inconsistent versions trick the client
Merkle tree structure provides fork consistency
Informally, if the server prevents clients A and B from seeing each other’s updates, the clients will either detect the problem or live in parallel universes from then on
Freshness?

16. Conclusion
Improvement in security and privacy over existing architectures
Simple stateless server design makes recovery easier
AFS and xFS have problems with recovery
Virtually unconstrained write ordering should provide good performance

17. Questions

18. Related Work OceanStore Distributed object location; Google Filesystem
Master/Chunkserver architecture
LFS
Large sequential writes
WAFL
WAL for synchronous semantics
Tree of blocks for consistency