May 30-31, 2012 HDF5 Workshop at PSI May HDF5 File Image Operations Dana Robinson The HDF Group Efficient Use of HDF5 With High Data Rate X-Ray Detectors Paul Scherrer Institut

May 30-31, 2012 HDF5 Workshop at PSI Purpose Allow users to work with files like how they do on disk. No disk I/O when file images are opened, created, read from, or written to. Faster access to data. Need to be careful about minimizing your memory footprint when working with large files.

May 30-31, 2012 HDF5 Workshop at PSI Use in pipelines Processing 1 HDF5 file 1) Processing step 1 opens the HDF5 file with the core VFD. HDF5 file

May 30-31, 2012 HDF5 Workshop at PSI Process 2 Use in pipelines Process 1 2) Buffer is sent to processing step 2. This saves writing the file to the filesystem. HDF5 file IPC

May 30-31, 2012 HDF5 Workshop at PSI Useful for MPI Process 0 Process 1 Process n config file 1) Process 0 reads configuration file. 2) File image is broadcast to other processes to open in memory. Process 2

May 30-31, 2012 HDF5 Workshop at PSI New API Functions H5Pget/set_file_image H5Pget/set_file_image_callbacks H5Fget_file_image H5LTopen_file_image (high-level convenience function)

May 30-31, 2012 HDF5 Workshop at PSI H5Pget/set_file_image herr_t H5Pget/set_file_image(hid_t fapl_id, void *buffer, size_t buf_len) Requires allocating a buffer, which is passed to the function. Designed for the core VFD but most VFDs can be modified to support using an initial image file. The buffer is copied in/out of the HDF5 library, not passed by reference! We will discuss ways around this…

May 30-31, 2012 HDF5 Workshop at PSI Digression: Property Lists Used frequently in the HDF5 library. Allow us to be flexible in passing parameters to functions without breaking the API. Use "pass by value" semantics, not "pass by reference" - Designed for moving small amounts of data around. - Greatly simplifies copying and deletion of property lists.

May 30-31, 2012 HDF5 Workshop at PSI Digression: Property Lists Unfortunately, passing very large buffers by value can bring a severe performance penalty. We have a few ways around this: - Use the H5LTopen_image_file call (discussed later) - Ignore the problem if the overhead is low (small files) - Use file image callbacks to implement call-by-reference semantics (discussed later)

May 30-31, 2012 HDF5 Workshop at PSI H5Fget_file_image ssize_t H5Fget_file_image(hid_t fapl_id, void *buffer, size_t buf_len) Provides a simple way to retrieve a copy of the image of an open file. You provide the buffer and the size. You can call the function with a null buffer pointer to get the current file size. Can be used with files opened with the SEC2, STDIO, and core VFDs.

May 30-31, 2012 HDF5 Workshop at PSI H5P_get/set_file_image_callbacks herr_t H5Pget/set_file_image_callbacks(hid_t fapl_id, H5_file_image_callbacks_t *callbacks_ptr) H5_file_image_callbacks_t is a struct containing function pointers which can be invoked on: image re/allocationimage free image copy It also contains a void pointer for user-specific data and function pointers which can be invoked on: user data copyuser data free

May 30-31, 2012 HDF5 Workshop at PSI H5P_get/set_file_image_callbacks The purpose of the callback functions are twofold: 1) Allow the user more careful control over the image in memory when resources are scarce (e.g.: when handling large files). 2) Allow the user to implement pass-by-reference semantics for the buffer.

May 30-31, 2012 HDF5 Workshop at PSI H5_file_image_callbacks_t typedef struct { void *(image_malloc)(size_t size, H5_file_image_op_t file_image_op, void *udata), void *(image_memcpy)(void *dest, void *src, size_t size, H5_file_image_op_t file_image_op, void *udata), void *(image_realloc)(void *ptr, size_t size, H5_file_image_op_t file_image_op, void *udata), herr_t *(image_free)(void *ptr, H5_file_image_op_t file_image_op, void *udata), void *(udata_copy)(void *udata), herr_t *(udata_free)(void *udata), void *udata; } H5_file_image_callbacks_t;

May 30-31, 2012 HDF5 Workshop at PSI H5P_get/set_file_image_callbacks void *(image_malloc)(size_t size, H5_file_image_op_t file_image_op, void *udata), The file_image_op parameter indicates which operation is taking place when the function was invoked. The image_malloc, realloc, free, and memcpy functions must have the same semantics as their C counterparts. Note that the memcpy and free calls can indicate failure in their return values, unlike their C counterparts.

May 30-31, 2012 HDF5 Workshop at PSI H5_file_image_op_t typedef enum { H5_FILE_IMAGE_OP_PROPERTY_LIST_SET, H5_FILE_IMAGE_OP_PROPERTY_LIST_COPY, H5_FILE_IMAGE_OP_PROPERTY_LIST_GET, H5_FILE_IMAGE_OP_PROPERTY_LIST_CLOSE, H5_FILE_IMAGE_OP_FILE_OPEN, H5_FILE_IMAGE_OP_FILE_RESIZE, H5_FILE_IMAGE_OP_FILE_CLOSE, } H5_file_image_op_t; These values indicate the operation in progress when the callback was invoked.

May 30-31, 2012 HDF5 Workshop at PSI Warning!!! Creating a full set of callback functions to control the behavior of the file image is likely to be a difficult process! The high-level function H5LTopen_file_image (discussed later) should be suitable in most cases.

May 30-31, 2012 HDF5 Workshop at PSI H5LTopen_file_image hid_t H5LTopen_file_image (void *buffer, size_t buf_len, unsigned flags) High-level convenience function to simplify buffer use. Allows the library to take control of the buffer with reasonable default behavior. Suitable for most uses.

May 30-31, 2012 HDF5 Workshop at PSI H5LTopen_file_image Bitwise flags: H5LT_FILE_IMAGE_OPEN_RW Open the image file with read/write permissions. H5LT_FILE_IMAGE_DONT_COPY Pass the buffer by reference instead of by value. The library will call free on the buffer. H5LT_FILE_IMAGE_DONT_RELEASE Do not free the image when the library is done with it. Also prohibits resizing/reallocating the buffer.

May 30-31, 2012 HDF5 Workshop at PSI Reading an in-memory image H5Pset_file_image(fapl_id, buffer, buf_len);

May 30-31, 2012 HDF5 Workshop at PSI H5LTopen_file_image example hid_t file_id; unsigned flags = H5LT_FILE_IMAGE_DONT_COPY; file_id = H5LTopen_file_image(buffer, buf_len, flags);

May 30-31, 2012 HDF5 Workshop at PSI Status Complete. Appeared in HDF (May 2012). Java and Fortran equivalents are not available at this time.