1. Unistore: Project Updates
Presenter: Wei Xie
Project Members: Jiang Zhou, Mark Reyes, Jason Noble, David Cauthron and Yong Chen
Data-Intensive Scalable Computing Laboratory(DISCL)
Computer Science Department
Texas Tech University
We are grateful to the Nimboxx and the Cloud and Autonomic Computing site at Texas Tech University for the valuable support for this project

2. Data Placement Component Characterization Component
Unistore Overview
To build a unified storage architecture (Unistore) for Cloud storage systems with the co-existence and efficient integration of heterogeneous HDDs and SCM (Storage Class Memory) devices
Prototype development based on Sheepdog and/or Ceph
Data Placement Component
Characterization Component
I/O Pattern
Random/Sequential
Read/write
Hot/cold
Workloads
Access patterns
guide
----- Meeting Notes (4/8/15 14:37) -----
Objective slide.
----- Meeting Notes (4/8/15 15:09) -----
unitore: design, idea
implement on sheepdog
title change to unistore objective and design
Prototype developement based on sheepdog distributed store for vm.
Component reflect objective.
I/O Functions
Write_to_SSD
Read_from_SSD
Write_to_HDD
Devices
Bandwidth Throughput
Block erasure
Concurrency
Wear-leveling
Placement Algorithm
Modified Consistent Hash

3. Team and Leverage Faculty: Yong Chen Post-doc researcher: Jiang Zhou
Ph.D. student: Wei Xie
Undergraduate student: Mark Reyes
Nimboxx: Jason Noble and David Cauthron
Experimental platform:
Two nodes on DISCI cluster in
CPU - 2 x 8 Core E v2, 2.60GHz
Memory - 128GB
3*500GB SAS HDD and 2*200GB SSD
Phi 5110pP Coprocessors
Used as sheepdog storage nodes

4. Background: Challenges in Data Distribution
Requirement of data distribution
Scalability
Load balance (based on capacity)
Data need to be randomly and statistical proportionally distributed according to nodes’ capacity
Handles node addition/removal
Data replication for fault-tolerance
High performance
Throughput of storage nodes need to be fully exploited
Consistent hashing and CRUSH handle the first four problems fairly well
CRUSH is a more flexible as it is able to distribute data based on the physical organization of nodes for better fault-tolerance

5. Background: Challenges in Data Distribution
Heterogeneous storage environment
Distinct throughput
NVMe SSD: 2000 or more MB/s
SATA SSD: ~500 MB/s
Enterprise HDD: ~150 MB/s
Large SSDs are becoming available, but still expensive
1.2TB NVMe Intel 750 costs $1000
1TB SATA Saumsung 640 EVO costs $500
10 or more costly than HDDs
SSDs still co-exist with HDDs as accelerator instead of replacing them

6. Background: How to Use SSDs in Cloud-scale Storage
Traditional way of using SCMs (i.e. SSD) in cloud-scale distributed storage: as cache layer
Caching/buffering generates extensive writes to SSD, which wears out the device
Need fine-tuned caching/buffering scheme
Not fully utilize capacity of SSDs
The capacity of SSDs is growing fast
Treat SSD-equipped nodes the same level as HDD-equipped nodes
No need to do cache replacement or buffer flushing
User sees the storage system with combined capacity and maximized performance
Less write to SSDs
Load-balance aware distribution and performance aware distribution are naturally conflict
SSDs are usually smaller but faster, while HDDs larger but slower
Existing data distribution algorithms do not consider this problem

7. Project Tasks: Overview
Data distribution management of Unistore
Modify the data distribution algorithm (Consistent hash) in Sheepdog or CRUSH algorithm in Ceph
Achieves load-balance, reliability and performance at the same time for heterogeneous storage
Different storage devices are unified and fully utilized
Two-mode distribution: BigData’15 Short Paper
SUORA algorithm
Tracing IO operations and workload characterization
Instrument Sheepdog for IO tracing capability
Integrate IO workload characterization component to serve as the hint for data distribution
Tracing component developed by Mark Reyes

8. Activities
Bi-weekly meeting for the team members to report progress and discuss the problems
Each student members report the recent research and development progress. Bring up new ideas or discuss current ideas
Presentation slides and meeting minutes are maintained

9. Deliverables Two-mode paper accepted by BigData15 conference
SUORA paper completed and preparing for submission
A new paper called “Tier-CRUSH” is in preparation
IO tracing and workload characterization component is being developed
Try patent filing

10. Two-Mode Data Distribution
Traditional data distribution only cares about load-balance, i.e. uses capacity-based distributor
We propose to use performance-based and capacity-based distributor at the same time
Switch between two mode is based on the use of capacity and IO workload
Read and write policy to handle two modes
Mode transition strategy to reduce data migration overhead

11. Throughput Improvement
1.8 performance gain here
Migration overhead ignored
Significant system throughput improvement in a wide range of user input

12. SUORA Algorithm
Multiple tiers, each tier represents a type of storage devices with similar characteristics (performance, capacity)
Data placed across different tiers based on hotness
Data distributed across different nodes in each tier randomly and uniformly and proportionally to capacity

13. Conclusions
Reconsider data distribution with heterogeneous storage devices with distinct performance metrics
Two-mode scheme targets at providing maximized performance while still maintaining load-balance, without drastic change to existing data distribution algorithms
Analysis shows potential of the two-mode scheme
Still need more trace-based or real world evaluation of the scheme
The proposed algorithms received positive feedback from BigData conference

14. On-going/Future Work
Starting to implement the proposed algorithms in Sheepdog or Ceph
Continue the development of IO tracing and characterization component
Writing a new paper name “Tiered-CRUSH” that extends CRUSH algorithm to support heterogeneous storage
Integrate workload characterization component and data distribution component together
Test on the experimental platform

15. http://cac.ttu.edu/, http://discl.cs.ttu.edu/
Thank You
Please visit:
Acknowledgement: The is funded by the National Science Foundation under grants IIP and IIP

16. Please take a moment to fill out your L.I.F.E. forms.
Select “Cloud and Autonomic Computing Center”
then select “IAB” role.
What do you like about this project?
What would you change?
(Please include all relevant feedback.)