Research Introduction Taeuk Kim Laboratory for Advanced System Software Sogang University 1

Contents Background Introduction of Parallel File System LADS FT-LADS What I will do

Background Large amount of Scientific data Satellite image Nuclear data Big data era Cooperative research and analyze => transferring large scale data ↑

Background Network bandwidth scale ↑ 10G ethernet, Infiniband on private network Relatively file system bandwidth ↓ Bottleneck on storage server ↑ A solution for bottleneck on file system is PFS (Parallel File System)

Parallel File System I/O processing consumes time ↑ Idea for distributing I/O workload Managing file by chunking -> object Maintaining and managing multiple storage server and parallely processing file objects with multi-threading

Parallel File System - Lustre MDS/MGS : Metadata Server, Management Server MDT : Metadata Target OSS : Object Storage Server OST : Object Storage Target

LADS There are still problems -> I/O contention on PFS Many clients use same PFS -> QoS ↓ Disk failure -> rebuilding disk from RAID Unbalanced I/O load => performance ↓ If we can avoid I/O congestion, the performance will be better LADS (Layout-Aware Data Scheduling) is motivated from above -> (Data transfer) application level solution It helps you to avoid I/O contention in OSS/OST

LADS Avoiding congestions of cluster storage OSS’s or OST’s data arrival rate > data service rate -> overflow occurs (In Lustre)

LADS In PFS, we can know the layout of objects -> Possible to distributing objects well and avoiding congeston With knowing the layout, it’s possible to schedule I/O not to be congested

LADS Transfer protocol Step1 – (src) NEW_FILE to comm’s Q, NEW_BLOCK to OST’s Q Step 2 – (sink) FILE_ID to comm’s Q Step 3 – (src) wake up N I/O threads and read data from OST and send that to sink Step 4 – (sink) receive req and read from src’s RMA buffer Step 5 – (sink) Store data objects in sink’s OST (layout-aware) Step 6 – (src) If BLOCK_DONE, release RMA buffer and goto step 3 Scheduling Layout-aware : knowing about which OST stores which data objects Congestion-aware : Defualt is RR, but when a thread is accessing an OST, another thread pass that OST and move on to next Object caching When RMA buffer is full, it uses NVM buffer

FT-LADS Disk failure situation (of facebook) New data : 500TB~900TB/day Failure : lose 100k blocks/day Reconstruction traffic : 180TB/day Recovery of disk failure is important Currently LADS doesn’t have fault tolerance mechanism

FT-LADS FT-LADS is a poster accepted by IEEE CLUSTER’16 It suggests fault tolerance mechanism for LADS File logging : remaining sent block’s log file per every files Char type logging Encoding type logging Int type logging Binary type logging : block number as a binary value Bit binary logging : n-th bit position as n-th block Transaction logging : remaining multiple files’ logs in one log file Universal logging : remaining all files’ logs in one log file

What I will do is To Support FT-LADS improvement Implementing some issues, etc To Contribute with a new idea Thinking about core-aware in NUMA

Q&A Thank you