HAIL (High-Availability and Integrity Layer) for Cloud Storage Alina Oprea Joint with Kevin Bowers and Ari Juels RSA Laboratories

Cloud storage Cloud Storage Provider Client Mostly static data: Storage server Web server Mostly static data: Back-up Archival Is my data available ? Client

Proofs of Retrievability (PORs) Cloud Storage Provider Corrects small corruption F Encoding Client k

Proofs of Retrievability (PORs) Cloud Storage Provider F F Challenge Response Requires integrity checks on server or client Detects large corruption Client k

When PORs fail F F Unrecoverable Cloud Storage Provider decoder Challenge Response Unrecoverable Client k

HAIL Goals Resilience against cloud provider failure or temporary unavailability Amazon S3 went down several times, once for 8 hours Linkup lost 45% of its customer data Use multiple cloud providers to construct a reliable cloud storage service out of unreliable components RAID (Reliable Array of Inexpensive Disks) for cloud storage Provide clients verification capabilities Efficient proofs of file availability by interacting with cloud providers

Replicate across multiple providers Amazon S3 Google EMC Atmos F F F Naïve approach F Sample and check consistency across providers Client

Roadmap Adversarial model for HAIL Small-corruption attack on replication scheme Encoding layer for each replica individually Reduce storage overhead by dispersal Increasing file lifetime with secret keys

Adversarial model Static: corrupts a fixed number b of the n total providers over time Create enough redundancy in the file to handle this (b+1 replicas) Is this realistic? Mobile (proactive): corrupts b out of n providers in each epoch Separate each server into code base and storage base At the beginning of an epoch code base of all servers is cleaned (through reboot, for instance) All servers might have residual data corruption Reactive design: check integrity and redistribute

Attack on replication scheme Amazon S3 Google EMC Atmos F F F F F F File can not be recovered after [n/b] epochs The probability that client samples the corrupted block is low Client

Replication with POR F F F F Amazon S3 Google EMC Atmos Client POR POR ECC Cons: requires integrity checks for each replica Client

Replication with POR F F F F Amazon S3 Google EMC Atmos Client Sample and check consistency across providers Client

Replication with POR F F F F Amazon S3 Google EMC Atmos Client єd єd єd Large storage overhead due to replication File lifetime still limited by [n/b] (єc/ єd) єc correction threshold of POR encoding єd detection threshold of POR Sample and check consistency across providers Client

Reduce storage overhead F decode m fragments n fragments dispersal (n,m) F Client

Dispersal code parity blocks (n,m) F F Dispersal code parity blocks Client

Dispersal code parity blocks Stripe POR encoding F Dispersal code parity blocks How to increase file lifetime? Check that stripe is a codeword in dispersal code POR encoding to correct small corruption Client

Increasing file lifetime with MACs P1 P2 P3 P4 P5 MAC MAC MAC MAC MAC Can we reduce storage overhead? Client

Integrity-protected dispersal code m hk1(m) UHF hk2(m) + PRF Reed-Solomon dispersal code Client

Integrity-protected dispersal code m + PRF MACs embedded into parity symbols Client

Current work and open problems Proofs of Retrievability Lower bounds akin to Naor and Rothblum’s lower bounds for memory checking What is the cost of file updates? HAIL K. Bowers, A. Juels and A. Oprea – “HAIL (High-Availability and Integrity Layer) for Cloud Storage”, CCS 2009 Different adversarial models Investigate alternative constructions Supporting file updates

Proofs of Retrievability (PORs) Cloud Storage Provider F F A Challenge Response Requires integrity checks on server or client Detects large corruption Client k

POR requirements F F Cloud Service Provider Client Efficient file encoding Low storage overhead Low bandwidth for challenge and response Efficient proof construction and verification Efficient file recoverability [Juels, Kaliski 07] [Shacham, Waters 08] [Dodis et al. 09] The requirements in designing a POR protocol are: That the file encoding is efficient There is low storage overhead on both client and server The ch/res protocol is efficient in both computation and bandwidth It is also efficient to recover the file from a fraction of correct server responses. There are a number of POR constructions: JK 07, SW08, Dodis et al. 09 that offer different tradeoffs in these metrics. I will not get into details here, I just wanted to point out that the POR problem has been studied a lot and there are good solutions for it. But do PORs solve our problem? Client k

Reed-Solomon parity blocks HAIL P1 P2 P3 P4 P5 F Reed-Solomon parity blocks POR encoding protects against small corruption Protects static files availability against mobile adversary Client

HAIL P1 P2 P3 P4 P5 Aggregates stripes for efficient integrity checking Periodic checking and reconstruction upon failure MACs embedded into parity symbols Client