Extending Petascale I/O with Data Services Hasan Abbasi Karsten Schwan Matthew Wolf Jay Lofstead Scott Klasky (ORNL)

Motivation I/O bottleneck Petascale data sizes Data overload Faster solution

Observations Fast Extraction Flexibility in where we execute operations Managed output to data consumer Flexible resource utilization

Compute Area  Using ADIOS for flexibility in choosing output method  Data is serialized using FFS  COD provides a processing hook within the compute application  SmartTap generates the output buffer through a user defined function  DataTap moves the data to the staging area

Staging Area  Additional resources for buffering before storage  Simple operations like aggregation  Complex analysis and compression operations  Domain specific services  Combination of extraction, processing and storage  Placement to optimize performance

Runtime Overhead