1. Current status and future work
HDF5 and AIO
Current status and future work
May 30-31, 2012
HDF5 Workshop at PSI

2. HDF5 and AIO
Asynchronous I/O (AIO), or non-blocking I/O, is a form of input/output processing that permits other processing to continue while the transmission occurs (i.e., overlapping compute with I/O)
Current HDF5 I/O calls are synchronous or blocking
On read, call doesn’t complete until the desired data has been read from the file and written to the application buffer
On write, the call doesn’t complete until the outgoing data buffer has been transferred to OS
May 30-31, 2012
HDF5 Workshop at PSI

3. HDF5 and AIO
HDF5 may hide most of I/O overhead under application computation
Support asynchronous I/O access to data in HDF5 file:
I/O is initiated within the library in response to an API call
I/O operation completes in the background after API call has returned
Beneficial for both raw data and HDF5 metadata I/O
May 30-31, 2012
HDF5 Workshop at PSI

4. Implementation considerations
Based on POSIX Asynchronous I/O routines
APIs for applications to initiate write/read/file sync
Return immediately without waiting for requested I/O operation to complete
Facilities to:
Query OS to determine if AIO operation is complete
Stall, pending completion of AIO operation
May 30-31, 2012
HDF5 Workshop at PSI

5. Implementation considerations
Requirements of AIO library
Portability
Robustness
System libraries we tested didn’t perform well and in some cases were not even POSIX compliant!
We have been working with the “AIO-Lite” library from Argonne built on top of Pthreads
Preliminary tests show significant reduction in application I/O time for AIO
May 30-31, 2012
HDF5 Workshop at PSI

6. Current status Modifications completed to HDF5 Virtual File Layer
Define a set of new VFD calls (general enough to support different AIO implementation, e.g., POSIX AIO and MPI non-blocking calls)
Initiating of asynchronous read, write, file sync
Obtaining status of an asynchronous operation
Blocking pending completion of an asynchronous operation
Finishing an asynchronous operation
Canceling an asynchronous operation
May 30-31, 2012
HDF5 Workshop at PSI

7. Current status
Modify H5FD_class_t in H5FDpublic.h to define these calls as optional calls that file drivers may choose to implement:
herr_t (*aio_read)(H5FD_t *file, …);
herr_t (*aio_write)(H5FD_t *file, …);
herr_t (*aio_test)(hbool_t *done_ptr, …);
herr_t (*aio_wait)(void *ctlblk_ptr);
herr_t (*aio_finish)(int *errno_ptr, …);
herr_t (*aio_fsync)(H5FD_t *file, …);
herr_t (*aio_cancel)(void *ctlblk_ptr);
herr_t (*fsync)(H5FD_t *file, …);
May 30-31, 2012
HDF5 Workshop at PSI

8. Current status
Modifications completed to HDF5 Virtual File Layer (continued)
Implement VFD AIO calls as top level VFD calls (sec2, etc.)
If underlying driver supports the desired AIO operation, these functions just pass request to it
Otherwise simulate AIO by translating the required operations into functionally equivalent SIO
Modify the family and multi file drivers to implement AIO VFD calls by passing AIO VFD calls to the underlying file drivers
May 30-31, 2012
HDF5 Workshop at PSI

9. Current status
Modifications completed to HDF5 Virtual File Layer (continued)
Modify configure to enable and control the AIO extensions to file drivers
--enable/disable_aio
--enable/disable_64_bit_posix_aio
--enable/disable_posix_aio_error_recovery
Testing
Code is available from
May 30-31, 2012
HDF5 Workshop at PSI

10. Future work
Design and implement clients to use AIO for writing HDF5 metadata and raw data
Design and implement public APIs to control AIO
Those tasks are in a planning stage
May 30-31, 2012
HDF5 Workshop at PSI

11. Thank You!
Questions?
May 30-31, 2012
HDF5 Workshop at PSI