PrincetonUniversity From application requests to Virtual IOPs: Provisioned key-value storage with Libra David Shue * and Michael J. Freedman (*now at Google)

Shared Cloud Tenant ATenant B VM Tenant C VM

Shared Cloud Storage Tenant ATenant B VM Key-Value Storage Block Storage SQL Database Tenant C VM

Unpredictable Shared Cloud Storage Tenant ATenant B VM Key-Value Storage Block Storage SQL Database Tenant C VM Disk IO-bound Tenants SSD-backed storage Key-Value Storage

Provisioned Shared Key- Value Storage Tenant A VM Tenant B VM Tenant C VM Application Requests Shared Key-Value Storage Low-level IO SSD GET/s PUT/s IOPs BW Reservation A Reservation B Reservation C (1KB normalized)

6 Libra Contributions Libra IO Scheduler - Provisions low-level IO allocations for app-request reservations w/ high utilization. - Supports arbitrary object distributions and workloads. 2 key mechanisms - Track per-tenant app-request resource profiles. - Model IO resources with Virtual IOPs.

7 Related Work Storage Type App- request s Work Conserving Medi a MaestroBlockNNHDD mClockBlockNYHDD FlashFQBlockNYSSD DynamoD B Key- Value YNSSD

Provisioned Distributed Key-Value Storage Tenant ATenant B VM Reservation A Reservation B Tenant B VM Storage Node N Storage Node 1... Shared Key-Value Storage Global Reservation Problem Local Reservation Problem [Pisces OSD1 ’12]

9 Storage Node N Provisioned Distributed Key-Value Storage Reservation A Reservation B Storage Node 1 data partitions demand... Tenant A VM Tenant B VM

10 Storage Node N Storage Node 1 Provisioned Distributed Key-Value Storage Reservation A Reservation B... Reservation A Reservation B

11 Storage Node N Storage Node 1 Provisioned Distributed Key-Value Storage... Res An Res Bn Res A1 Res B1 Reservation A Reservation B Reservation A Reservation B

12 Key-Value Protocol Persistence Engine IO Scheduler Physical Disk Retrieve K Read l337 IO operation Provisioned Local Key-Value Storage GET K GET Libra IO Scheduler

13 blah Libra Design Libra Provisioning Policy Libra IO Scheduler GET PUT How much IO to consume? Reservation Distribution Policy How much IO to provision? Persistence Engine Physical Disk DRR Determine the cost of a tenant request Provision IO allocations for tenant app-request reservations

14 Provisioning App-request Reservations is Hard Reservation Bn IO Amplification IO Interference Non-linear IO Performance 1 KB PUT ≥ 1 KB Write Variable IO throughput Underestimate provisionable IO Non-linear cost per KB Model IO with Virtual IOPs Track tenant app-request resource profiles

15 Workload-dependent IO Amplification PUT PUT K,V 3 V3V3 V2V2 FLUSH V3V3 index A M LevelDB (LSM-Tree) COMPACT V1V1 index HV V0V0 FO V3V3 A O

16 Workload-dependent IO Amplification PUT FLUSH COMPACT PUT K,V 3 V3V3 V2V2 V3V3 index A M V1V1 HV V0V0 FO V3V3 A O LevelDB (LSM-Tree)

17 Workload-dependent IO Amplification PUT FLUSH COMPACT PUT K,V 3 V3V3 V2V2 V3V3 index A M V1V1 HV V0V0 FO V3V3 A O LevelDB (LSM-Tree)

18 Workload-dependent IO Amplification GET K index A M Workload-dependent IO Amplification index HV FO K GZ

19 Libra Tracks App-request IO Consumption to Determine IO Allocations Libra IO Scheduler 5 GET 25 PUT FLUSH COMPACT Per-PUT Per-GET EWMA per-request averages Compute app- request IO profiles GET x x PUT IO = Tenant A 5 IO units PUT FLUSH Track IO consumption Provision IO allocations blah Libra Provisioning Policy IO Tenant A 500 IO/s

20 Unpredictable IO Interference Die-level parallelism, low latency IOPs 4 read/4 write tenants Shared-controller and bus contention Erase-before-write overhead FTL and read-modify- write garbage colleciton

21 Unpredictable IO Interference 21 Unpredictable IO Interference

22 Libra Underestimates IO Capacity to Ensure Provisionable Throughput Provisionable IO throughput = floor(workloads) (18 Kop/s) Provisionable IO limit

23 Libra Underestimates IO Capacity to Ensure Provisionable Throughput Provisionable IO throughput = floor(workloads) (18 Kop/s) Provisionable IO limit

24 Libra Underestimates IO Capacity to Ensure Provisionable Throughput Provisionable IO throughput = floor(workloads) (18 Kop/s) Provisionable IO limit

25 Non-linear IO Performance IOP Cost = linear decrease until 6 KB, then constant IOP Cost = constant until 6 KB, then linear increase Max BW Max IOP/s

26 Non-linear IO Performance Max BW Max IOP/s

27 Libra Uses Virtual IOPs to Model IO Resources Unifies IO cost into a single metric Captures non-linear IO performance Provides IO insulation 2 equal-allocation tenants IO Insulation = 1/2 Max Read/Write

28 Libra Uses Virtual IOPs to Model IO Resources 2 equal-allocation tenants IO Insulation = 1/2 Max Read/Write

29 blah Libra Design Libra Provisioning Policy Libra IO Scheduler Persistence Engine Physical Disk Determine the cost of a tenant request Provision IO allocations for tenant app-request reservations Charge tenant IOPs based on VOP cost Track app-request VOP consumption Provision VOPs within provisionable limit Update tenant VOP allocations

30 Evaluation Does Libra's IO resource model achieve accurate resource allocations? Does Libra's IO threshold make an acceptable tradeoff of performance for predictability in a real storage stack? Can Libra ensure per-tenant app-request reservations while achieving high utilization?

31 Interference-free Ideal Libra Achieves Accurate IO Allocations Throughput Ratio = Actual / Expected (IO Insulation) Read 1 KB even

32 Interference-free Ideal Libra Achieves Accurate IO Allocations Write KB Throughput Ratio = Actual / Expected (IO Insulation)

33 Libra Achieves Accurate IO Allocations

34 Libra Achieves Accurate IO Allocations

35 Libra Achieves Accurate IO Allocations Min-Max Ratio = Min Throughput Ratio / Max Throughput Ratio

36 Libra Trades-off Nominal IO Throughput For Predictability

37 Libra Trades-off Nominal IO Throughput For Predictability < 10th percentile covered by SLA and higher-level policies Percentile Workload10th50th80thAll 99:1 1.6%30.5%40.5%45.8% 25:75 1.4%14.9%25.0%34.7% 1:99 0.7%12.2%19.5%28.1% Unprovisionable Throughput As a Percentage of Total Throughput

38 Libra Achieves App-request Reservations Read HeavyMixedWrite Heavy 1.5x0.5x Work-conserving consumption of unprovisioned resources Fully provisioned allocations

39 Libra Achieves App-request Reservations Read HeavyMixedWrite Heavy

40 Libra IO Scheduler  Provisions IO allocations for app-request reservations w/ high utilization.  Supports arbitrary object distributions and workloads. 2 key mechanisms  Track per-tenant app-request resource profiles.  Model IO resources with Virtual IOPs. Evaluation  Achieves accurate low-level IO allocations.  Provisions the majority of IO resources over a wide range of workloads  Satisfies app-request reservations w/ high utilization. Conclusion