Quick introduction to the Workshop Why are we here? Quick introduction to the Workshop May 30-31, 2012 HDF5 Workshop at PSI

Ideas behind the workshop May 30-31, 2012 HDF5 Workshop at PSI

The problem Two problems for research facilities using synchrotrons and/or free electron lasers (FEL) as X-Ray sources Different binary file formats (TIFF, CBF, EDF, etc.) High rate at which the data is generated Eugen Wintersberger (HDRI, DESY) contacted The HDF Group (THG) in Fall 2011 to find out what THG could do to help THG responded with a proposal that might be of interest to other research groups and facilities May 30-31, 2012 HDF5 Workshop at PSI

Goals PSI Workshop Bring all interested parties together Educate researchers on the HDF5 current capabilities and discus future possible improvements to HDF5 Find out how The HDF Group (THG) may work with the community Collaborations Training Funding for THG to work on the HDF5 improvements 4 May 30-31, 2012 HDF5 Workshop at PSI

HDF5 Tutorial May 30-31, 2012 HDF5 Workshop at PSI

Presenters HDF5 Tutorial Elena Pourmal Dana Robinson The HDF Group developers are on stand-by to help (modulo time zone). Work can continue via telecom (GoToMeeting works well!) May 30-31, 2012 HDF5 Workshop at PSI

Focuses on the HDF5 features that can help with HDF5 Tutorial Focuses on the HDF5 features that can help with General performance Data organization File sizes Gives overview of The HDF Group proposal to address the issues brought up in the Eugen Wintersberger’s paper Presents HDF5 features under development that might be of interest to this community May 30-31, 2012 HDF5 Workshop at PSI

HDF5 Tutorial Topics Day 1 Intro to The HDF Group Company, Services, Products, Directions Quick overview of HDF5 data model and software Groups and Links HDF5 Dataset I/O performance Overview data movement during I/O for datasets with different storage layouts HDF5 filters How to write your own one Problem (and possible solution) for custom filters May 30-31, 2012 HDF5 Workshop at PSI

HDF5 Tutorial Topics Day 1 HDF5 chunking What do you need to know to make it to perform? Parallel HDF5 Overview of HDF5 parallel capabilities May 30-31, 2012 HDF5 Workshop at PSI

HDF5 new features Day 2 HDF5 new features of interest in 1.10.0 and features under consideration Shared Object Headers VFD basics Split and Family VFD issues Writing your own VFD File Image Operations Virtual Object Layer May 30-31, 2012 HDF5 Workshop at PSI

HDF5 new features Day 2 HDF5 new features of interest in 1.10.0 and features under consideration Metadata journaling Partial edge chunks SWMR and h5watch tool Page Buffering Async I/O Multithreading May 30-31, 2012 HDF5 Workshop at PSI

Workshop logistics May 30-31, 2012 HDF5 Workshop at PSI

HDF5 presentations in the mornings Workshop logistics HDF5 presentations in the mornings Discussions in the afternoons ((and during presentations) THG provides logins to Linux machine pax.hdfgroup.uiuc.edu C, Fortran 2003, H5Py examples Use h5cc, h5fc to compile h5dump to see data in HDF5 files See Dana for login information May 30-31, 2012 HDF5 Workshop at PSI

Thank You! Questions? May 30-31, 2012 HDF5 Workshop at PSI