May 30-31, 2012 HDF5 Workshop at PSI May Writing Your Own HDF5 Virtual File Driver (VFD) Dana Robinson The HDF Group Efficient Use of HDF5 With High Data Rate X-Ray Detectors Paul Scherrer Institut

May 30-31, 2012 HDF5 Workshop at PSI Main Idea: Users should be able to write their own VFD without having to rebuild the HDF5 library.

May 30-31, 2012 HDF5 Workshop at PSI HDF5 API Virtual File Layer VFD User-Supplied VFD disk HDF5 Library

May 30-31, 2012 HDF5 Workshop at PSI What do I have to do? 1) Implement H5FDset_fapl_foo(). This will need to work with H5Pset_driver() 2) Implement appropriate VFD functions – read, write, etc. 3) Compile your code into a library or directly into your app.

May 30-31, 2012 HDF5 Workshop at PSI "Demo" VFD The STDIO VFD is a demo driver which uses no internal HDF5 calls. Not intended for production use! Unfortunately, this means that you do not get HDF5's platform- independent wrapper functions. Can copy and paste as a basis for your own VFD.

May 30-31, 2012 HDF5 Workshop at PSI Can be separate from the library Example: Make a new "foo" driver by copying the stdio VFD To build the static library: h5cc -c H5FDfoo.c -o H5FDfoo.o ar rcs libfoovfd.a H5FDfoo.o To build the test example: h5cc -L. -lfoovfd -o test test.c

May 30-31, 2012 HDF5 Workshop at PSI static const H5FD_class_t H5FD_sec2_g = { "sec2",/*name*/ MAXADDR,/*maxaddr*/ H5F_CLOSE_WEAK,/*fc_degree*/ H5FD_sec2_term,/*terminate */ NULL,/*sb_size*/ NULL,/*sb_encode*/ NULL,/*sb_decode*/ 0, /*fapl_size*/ NULL,/*fapl_get*/ NULL,/*fapl_copy*/ NULL, /*fapl_free*/ 0,/*dxpl_size*/ NULL,/*dxpl_copy*/ NULL,/*dxpl_free*/ H5FD_sec2_open,/*open*/ H5FD_sec2_close,/*close*/ H5FD_sec2_cmp,/*cmp*/ H5FD_sec2_query,/*query*/ NULL,/*get_type_map*/ NULL,/*alloc*/ NULL,/*free*/ H5FD_sec2_get_eoa,/*get_eoa*/ H5FD_sec2_set_eoa, /*set_eoa*/ H5FD_sec2_get_eof,/*get_eof*/ H5FD_sec2_get_handle,/*get_handle */ H5FD_sec2_read,/*read*/ H5FD_sec2_write,/*write*/ NULL,/*flush*/ H5FD_sec2_truncate,/*truncate*/ NULL,/*lock */ NULL,/*unlock */ H5FD_FLMAP_SINGLE /*fl_map*/ }; Each VFD contains a struct which maps our abstract VFL calls to your VFDs functions Use NULL when not implemented

May 30-31, 2012 HDF5 Workshop at PSI Walk through VFD code files NO SLIDES FOR THIS will use actual code files via an Ubuntu VM (color!) stdio, sec2, family - Data structures: main data, fxn ptr struct, aux data - Get/Set FAPL calls (both specific and w/ set_driver()) - open/create/close - superblock fxns - cmp - alloc/free (not really)/get_eoa - read/write. Discuss large files and why we loop. - flush - query, get properties