1. Flamingo: Enabling Evolvable HDD-based Near-Line Storage
Sergey Legtchenko, Xiaozhou Li, Antony Rowstron,
Austin Donnelly, Richard Black

2. Storing Cold Data in the Cloud
Pelican rack
(Microsoft, [OSDI 14])
8% HDDs active
576 HDDs/server
Open Compute
Cold Storage rack (Facebook)
6% HDDs active
240 HDDs/server
Custom racks: trading latency for cost
Only fraction of HDDs concurrently active
Reduced #servers (1 or 2 per rack)
Cold data: rarely accessed data
Challenge: storing cold data at low cost
Benefits
Lower capital cost
Capped resource consumption
Higher storage density

3. Designing Cold Storage Racks is Hard
Resources are constrained in the rack
Resource constraints:
1 HDD / cooling column
2 HDDs / tray
Vibration, bandwidth…
Power
Cooling
Software: co-designed, constraint-aware
Data Layout
IO Scheduler
Experience from building Pelican
Design complexity
Storage stack is brittle to design changes
Impact of resource provisioning on end performance?
Pelican: 8% HDDs active

4. Flamingo: a Tool to Help Cold Storage Rack Design
In the rest of the talk
Input
Generic
Storage Stack
Data layout
IO scheduler
parameters
Online
Constraint Solver
Data layout,
IO scheduler +
Storage stack configuration
Set of rack descriptions:
same topology
varying resource provisioning
Perf.
Analysis
(simulator)
Rack description
Constraints
Hardware properties
Resource provisioning
exploration
Performance
goals
Rack
Resource provisioning
specification

5. Rack Description
Resource Domain: set of HDDs sharing a limited resource
{D1,D2,D3,D4}: 40
{D5,D6,D7,D8}: 40
{D4,D8}: 1
{D3,D7}: 1
{D1,D5}: 1
{D2,D6}: 1
Domain A: type: power, budget: 40W
Domain B: type: vibration, budget: 1HDD D1 D2 D3 D4 D5 D6 D7 D8
Standby
Active
Spin up
power: 10W
vibration: 1
power: 20W
power: 2W
vibration: 0
HDD: operating states + resource consumption
IO-capable state
Expresses constraints:
e.g. only 1 HDD can be spinning up in A
40W
2 +
20+
20 = 44W
40W
2 +
20 = 26
can be hard or soft

6. Storage Stack Configuration
Data layout
Groups of HDDs that concurrently transition state
Minimize inter-group conflicts
Constraint
Solver
Rack description

7. Storage Stack Configuration
Data layout
Groups of HDDs that concurrently transition state
Minimize inter-group conflicts
Generic
Storage Stack
Data Layout
Spin ups/downs, IOs
Blob-store API
IO Scheduler
Blob Group of HDDs
Constraint
Solver D1 D2 D4 D5 D6 D7 D8 D3
Conflict
Conflicts minimized
vibration, budget: 1
power budget: 40
G1: {D3, D8}
G2: {D4, D7}
G3: {D1, D6}
G4: {D2, D5}
Group
definition D1 D2 D4 D5 D6 D7 D8 D3
vibration, budget: 1
power budget: 40
Each group conflicts with 2
Conflict D1 D6 D1 D5 D4 D2 D7 D2 D3 D4 D8 D4 D5 D6 D7 D8 Sd A Su
10, 1
20, 1
2, 0
power budget: 40
vibration, budget: 1
Rack description
Conflicts between groups
{D4,D8}: 1
{D3,D7}: 1
{D2,D6}: 1
{D1,D5}: 1
{D1,D2,D3,D4}: 40
{D5,D6,D7,D8}: 40
Rack description
Inter-group
constraints
{G1,G2,G3,G4}: 40
{G1,G2}: 1
{G3,G4}: 1
{G2,G1}: 1
{G1,G2}: 1
{G4,G3}: 1
{G3,G4}: 1
{G3,G4,G1,G2}: 40
{G4,G3,G2,G1}: 40

8. Resource Provisioning Exploration
N resource types: N-dimensional space of rack descriptions
Fully provisioned
Resource 1
(e.g. power) D1 D2 D4 D5 D6 D7 D8 D3 40 1
Fully provisioned
Resource 2 (e.g. vibration)

9. Resource Provisioning Exploration
N resource types: N-dimensional space of rack descriptions
Fully-provisioned rack (JBOD), per domain:
all HDDs in most resource-consuming state Sd A Su
10, 1
20, 1
2, 0 D1 D2 D4 D5 D6 D7 D8 D3
vibration, budget: 2
power budget: 80
Fully provisioned
Bottleneck resource constraint
Relaxing
Resource 1
(e.g. power)
Least-provisioned rack, per domain:
1 HDD in IO-capable state
n-1 in lowest resource-consuming D1 D2 D4 D5 D6 D7 D8 D3
vibration, budget: 1
power budget: 26
Fully provisioned
Resource 2 (e.g. vibration)

10. Resource Provisioning Exploration
N resource types: N-dimensional space of rack descriptions
Fully-provisioned rack (JBOD), per domain:
all HDDs in most resource-consuming state
Fully provisioned
Resource 1
(e.g. power)
Discrete surface
in the N-dimensional space
For Pelican: 747 rack descriptions
Bottleneck resource: vibration
Bottleneck resource: power
Fully provisioned
Resource 2 (e.g. vibration)

11. Evaluation - Pelican Pelican Simulator – Poisson workload, 1GB reads.

12. Execution Time - Pelican
9 minutes

13. Execution Time - Pelican
9 minutes
Rack name
OCP
Pelican
Rack_A
Rack_B
Rack_C
Rack_D
Rack_E
#rack descriptions
1921
747
1421
1152
973
649
683

14. Execution Time for Different Racks
9 minutes
3 hours
Rack name
OCP
Pelican
Rack_A
Rack_B
Rack_C
Rack_D
Rack_E
#rack descriptions
1921
747
1421
1152
973
649
683

15. Conclusion Cold storage racks:
Co-design: resource-constrained hardware + constraint-aware software
low cost but hard to (re)design
Flamingo simplifies design of cold storage racks
Synthesizes Data Layout and IO Scheduler parameters
Explores impact of resource provisioning on end performance
Redesign in days vs months manually