The HDF Group HDF Tools Tutorial September 28-30, 2010HDF and HDF-EOS Workshop XIV1 Peter Cao, The HDF Group Jonathan Kim, The HDF Group

Tools Tutorial HDFView HDF5 Command-line tools September 28-30, 2010HDF and HDF-EOS Workshop XIV2

HDFView View file content Edit file content Show NPOESS/JPSS related features September 28-30, 2010HDF and HDF-EOS Workshop XIV3

HDFView Download HDF-Java 2.7 beta from Download example files from ftp://ftp.hdfgroup.uiuc.edu/pub/outgoing/WorkshopXIV/HDF5-Files/ ftp://ftp.hdfgroup.uiuc.edu/pub/outgoing/WorkshopXIV/HDF5-Files/ September 28-30, 2010HDF and HDF-EOS Workshop XIV4

HDFView View file content Open OMI-Aura_L3.he5 Open groups (try expand all) Stretch the metadata view for showing metadata Double click on “ColumnAmountO3” to show data in table Use “Open As” to show “ColumnAmountO3” in image View a subset Open weather.h5 Select from preview image or selection fields September 28-30, 2010HDF and HDF-EOS Workshop XIV5

HDFView Copy/paste a dataset and a group Create a new file: tmp.h5 Copy/paste “ColumnAmountO3” and show the image from the new file Copy the group “GRIDS” to show the whole group is copied September 28-30, 2010HDF and HDF-EOS Workshop XIV6

HDFView Create a dataset and add an attribute Create /d1 (32-bit float, 720x1440) Copy “ColumnAmountO3” to /d1 Add attribute “IMAGE_MINMAXRANGE = 123.0,447.0” Export /d1 to text file Create /d2 and import d1.txt September 28-30, 2010HDF and HDF-EOS Workshop XIV7

HDFView NPOESS/JPSS related features Object reference Region references Quality flags September 28-30, 2010HDF and HDF-EOS Workshop XIV8

HDFView Object reference 1)Open file: SVI-NPP.h5 2)Open “VIIRS-I1-SDR_Aggr” 3)Click on object ID and shows object name 4)Right mouse to click to open data as image 5)Open data in table 6)Select two cells and open them at once September 28-30, 2010HDF and HDF-EOS Workshop XIV9

HDFView Region references 1)Open “VIIRS-I1-SDR_Gran” 2)Right mouse to click to open data as image 3)Open data in table 4)Select two cells and open at once September 28-30, 2010HDF and HDF-EOS Workshop XIV10

HDFView Quality flags 1)Quality flags Land/Water Background -- 1-bit SST Skin Quality -- 2-bits SST Bulk Quality -- 2-bits Aerosol Correction -- 3-bits 2)Open QF_VIIRSI1SDR_Array 3)Use "Open As" on QF_VIIRSI1SDR_Array 4)Select first bit September 28-30, 2010HDF and HDF-EOS Workshop XIV11

HDFView Advanced features 1)Import/export images 2)Use HDF-EOS plugin September 28-30, 2010HDF and HDF-EOS Workshop XIV12

HDF5 Command-line tools h5ls h5dump h5repack h5diff h5copy September 28-30, 2010HDF and HDF-EOS Workshop XIV13 What these tools can do for you

h5ls Show file content list h5ls ami.h5 h5ls –r ami.h5 September 28-30, 2010HDF and HDF-EOS Workshop XIV14

h5dump Structure Dataset Binary XML Region references September 28-30, 2010HDF and HDF-EOS Workshop XIV15

h5dump Structure h5dump -H tools_diff1.h5 h5dump -H ami.h5 September 28-30, 2010HDF and HDF-EOS Workshop XIV16

h5dump Dataset Content h5dump -d /EnSight_model/geometry/parts/part_2/quad4 ami.h5 Information h5dump –p –H -d /EnSight_model/geometry/parts/part_2/quad4 ami.h5 Subset H5dump -d /EnSight_model/geometry/parts/part_2/quad4 -s 10,0 -c 20,2 ami.h5 September 28-30, 2010HDF and HDF-EOS Workshop XIV17

h5dump Binary h5dump -d /EnSight_model/geometry/parts/part_2/quad4 -b LE –o out.bin ami.h5 XML h5dump –x ami.h5 Region h5dump -d /Data_Products/VIIRS-I1-SDR/VIIRS-I1-SDR_Gran_0 -s 3 -c 1 SVI-NPP.h5 h5dump –R -d /Data_Products/VIIRS-I1-SDR/VIIRS-I1-SDR_Gran_0 -s 3 -c 1 SVI-NPP.h5 Packed fields September 28-30, 2010HDF and HDF-EOS Workshop XIV18

h5repack September 28-30, 2010HDF and HDF-EOS Workshop XIV19 Copies a file to a new file with different storage layouts and compression filters Remove inaccessible objects / junk spaces Change storage layout Apply compression filter

h5repack Remove inaccessible objects h5dump tools_junk.h5 h5repack tools_junk.h5 tmp.h5 Change layout h5dump -H tools_bad_layout.h5 h5repack tools_bad_layout.h5 tmp.h5 h5repack -l CHUNK=16x16 tools_bad_layout.h5 tmp.h5 Change compression h5repack -f GZIP=6 tmp.h5 tmp2.h5 More: use latest version, link storage, shared message, and etc September 28-30, 2010HDF and HDF-EOS Workshop XIV20

h5diff September 28-30, 2010HDF and HDF-EOS Workshop XIV21 Show differences between two files or two objects Dataset Whole file

h5diff September 28-30, 2010HDF and HDF-EOS Workshop XIV22 Dataset h5diff -r tools_diff1.h5 tools_diff2.h5 /g1/uint8 /x Whole file Run “h5ls –r tools_diff1.h5” and “h5ls -r tools_diff2.h5” h5diff tools_diff1.h5 tools_diff2.h5 h5diff –v tools_diff1.h5 tools_diff2.h5 Follow symlinks h5diff -r --follow-symlinks tools_diff1.h5 tools_diff2.h5 /sl1

h5copy Dataset Group September 28-30, 2010HDF and HDF-EOS Workshop XIV23 Copies objects to files…

h5copy Dataset h5copy -i tools_input.h5 -o tmp.h5 -s /g1/g2/f -d /float Group Default: h5copy -i tools_input.h5 -o tmp.h5 -s /g1 -d /g1 Parent groups: h5copy -p -i tools_input.h5 -o tmp.h5 -s /g1/g2/f -d /g1/g2/f September 28-30, 2010HDF and HDF-EOS Workshop XIV24

September 28-30, 2010HDF and HDF-EOS Workshop XIV25 Thank You! Questions/comments?