1. Dimension Scales in HDF-EOS2 & HDF-EOS5
Abe Taaheri, Raytheon IIS
HDF & HDF-EOS Workshop XIV
Champaign, IL
Sep. 29, 2010
1 of #page

2. What is a Dimension Scale ? Dimension Scales & Metadata
Dimension Scales APIs
he2
he5
Code example
Writing (he2, he5)
Reading (he2, he5)
Sample he2 and he5 files with Dimension Scales

3. What is a Dimension Scale?
It is a sequence of numbers placed along a dimension to demarcate intervals along it
HDF4
It is an array with size and name similar to its assigned dimension
Stored using a structure similar to the SDS array
One scale is assigned per dimension

4. * What is a Dimension Scale?
HDF5
A HDF5 dataset.
With additional metadata that identifies the dataset as a Dimension Scale
Typically Dimension Scales are logically associated with the dimensions of HDF5 Datasets
The meaning of the association is left to applications.
* Pedro Vicente talk, HDF/ HDF-EOS Workshop IX
1 of #page

5. 3 Dimension Scales Datasets
Example: 3D dataset
3 Dimension Scales Datasets 5
Dataset: 3D Array with 5 x 7 x 10 dimensions 7 10
1 of #page

6. More on Dimension Scale in HDF5
A dimension scale is not required to be a 1-D array, or to have a specific datatype
A dataset dimension can have more than 1 associated dimension scale
A Dimension Scale can be shared by two or more Dataset dimensions
1 of #page

7. Example: 3D dataset in HDF5
Several Dimension Scales Datasets
Dataset: 3D Array with 5 x 7 x 10 dimensions
1 of #page

8. HDF5 Dimension Scale Metadata
When the Dimension Scale is associated with a dimension of a Dataset, the association is represented by attributes of the two datasets.
The following dataset attributes are used to describe dimension scale datasets:
Attribute named “CLASS” with the value “DIMENSION_SCALE”
Optional attribute named “NAME”
Attribute references to any associated Dataset
1 of #page

9. SDsetdimstrs(): label, unit, format SDgetdimstrs() HDF5
Since Version 4.0 ?
SDsetdimscale()
SDgetdimscale()
SDsetdimstrs(): label, unit, format
SDgetdimstrs()
HDF5
Since Version 1.8
H5DSset_scale()
H5DSattach_scale(), H5DSdetach_scale()
H5DSset_label(), H5DSget_label()
A few more APIs
1 of #page

10. HDF Augmentation Tool
Augmentation Tool adds HDF5 Dimension Scales inside the file so that netCDF-4 APIs can access the data

11. Added a few routines to HDF-EOS2 to create dimension scales like HDF4
Version 2.17
Added a few routines to HDF-EOS5 to create dimension scales like those added by the HDF Augmentation Tool
Version 1.13

12. HDF-EOS2 APIs

13. Name: SWsetdimscale
Signature:
intn SWsetdimscale( int32 swathID,
char *fieldname, char *dimname, int32 dimsize,
int32 numbertype, VOIDP data)
Purpose:
Sets dimension scale for a given dimension

14. Name: SWgetdimscale
Signature:
intn SWgetdimscale( int32 swathID,
char *fieldname, char *dimname,
int32 *dimsize, int32 *numbertype,
VOIDP data)
Purpose:
Gets dimension scale for a given dimension

15. Name: SWsetdimstrs
Signature:
intn SWsetdimstrs( int32 swathID,
char *fieldname, char *dimname, char *label,
char *unit, char *format)
Purpose:
Sets the label, unit, and format strings for a given dimension

16. Name: SWgetdimstrs
Signature:
intn SWgetdimstrs( int32 swathID, char *fieldname, char *dimname, char *label,
char *unit, char *format, intn len)
Purpose:
Gets the label, unit, and format strings for a given dimension

17. Similar APIs for Dimension Scales in
Grid Fields

18. HDF-EOS5 APIs

19. Name: HE5_SWsetdimscale
Signature:
herr_t HE5_SWsetdimscale( hid_t swathID, char *fieldname, char *dimname,
const hsize_t dimsize, hid_t numbertype,
void * data)
Purpose:
Sets dimension scale for a field dimension within the swath

20. Name: HE5_SWgetdimscale
Signature:
long HE5_SWgetdimscale( hid_t swathID, char *fieldname, char *dimname,
hsize_t *dimsize, hid_t *numbertype,
void * data)
Purpose:
Gets dimension scale for a field dimension within the swath

21. Name: HE5_SWwritedscaleattr
Signature:
herr_t HE5_SWwritedscaleattr( hid_t swathID, const char *dimname, const char *attrname, hid_t ntype, hsize_t count[], void *datbuf)
Purpose:
Writes/Updates a dimension scale attribute in a specific swath

22. Name: HE5_SWreaddscaleattr
Signature:
herr_t HE5_SWreaddscaleattr( hid_t swathID, const char *dimname, const char *attrname,
void *datbuf)
Purpose:
Reads a dimension scale attribute from a specific dimension

23. Name: HE5_SWinqdscaleattrs
Signature:
Long HE5_SWinqdscaleattrs( hid_t swathID, const char *dimname, char *attrnames,
long *strbufsize)
Purpose:
Retrieve information about the attributes defined

24. Name: HE5_SWdscaleattrinfo Signature:
herr_t HE5_SWdscaleattrinfo( hid_t swathID, const char *dimname, const char *attrname,
hid_t *ntype, hsize_t *count)
count : Number of attribute elements
Purpose:
Returns information about attribute(s) in a specific dimension scale

25. Similar APIs for Dimension Scales in
Grid
Zonal Average

26. Code Examples

27. Writing Code Example (HDF-EOS2)
#include "hdf.h"
#include "HdfEosDef.h"
int main() {
intn status, i;
int swfid, SWid;
int nbands, nGeoTrack;
int bands[15]={3,6,9,12,15,18,23,26,29,32,33,34,36,37,39};
char label[16];
char unit[16];
char format[16];
float dataGeoTrack[20];
swfid = SWopen ( "SwathFile.hdf", DFACC_RDWR );
SWid = SWattach ( swfid, "Swath1“ );

28. for (i=0; i<20; i++) { dataGeoTrack[i] = (float) I; }
nGeoTrack = 20;
for (i=0; i<20; i++) { dataGeoTrack[i] = (float) I; }
status = SWsetdimscale ( SWid, "Temperature", "GeoTrack",
nGeoTrack, DFNT_FLOAT32, dataGeoTrack );
nbands = 15;
status = SWsetdimscale ( SWid, "Spectra", "Bands",
nbands, DFNT_INT32, bands );
strcpy (label, "Bands");
strcpy (unit, "none");
strcpy (format, "I2");
status = SWsetdimstrs ( SWid, "Spectra", "Bands", label, unit, format );
SWdetach ( SWid );
SWclose ( swfid );
return 0; }

29. Writing Code Example (HDF-EOS5)
#include <HE5_HdfEosDef.h>
int main() {
herr_t status = FAIL;
hid_t swfid = FAIL;
hid_t SWid1 = FAIL;
int data[15]={1,2,3,6,9,12,15,18,23,26,29,32, 33,34,35};
char label[16], unit[16], format[16];
int nbands;
hsize_t count[3];
swfid = HE5_SWopen ( "Swath.h5", H5F_ACC_RDWR );
SWid1 = HE5_SWattach ( swfid, "Swath1“ );

30. nbands = 15;
status = HE5_SWsetdimscale ( SWid1, "Spectra", "Bands", nbands, H5T_NATIVE_INT, (void *)data );
strcpy(label, "Bands Dim");
strcpy(unit, "None");
strcpy(format, "I2");
count[0]= 12;
status = HE5_SWwritedscaleattr ( SWid1, "Bands", "label", H5T_NATIVE_CHAR, count, label );
count[0]= 6;
status = HE5_SWwritedscaleattr ( SWid1, "Bands", "unit", H5T_NATIVE_CHAR, count, unit );
count[0]= 4;
status = HE5_SWwritedscaleattr ( SWid1, "Bands", "format", H5T_NATIVE_CHAR, count, format );

31. int datbuf_i1[1] = {-999};
count[0]= 1;
status = HE5_SWwritedscaleattr ( SWid1, "Bands", "MissingValue", H5T_NATIVE_INT, count, datbuf_i1 );
int datbuf_i2[3] = {-999, 0, 999};
count[0]= 3;
status = HE5_SWwritedscaleattr(SWid1, "Bands", "IntValues", H5T_NATIVE_INT, count, datbuf_i2 );
HE5_SWdetach ( SWid1 );
HE5_SWclose ( swfid );
return 0; }

32. Reading Code Example (HDF-EOS2)
#include "hdf.h"
#include "HdfEosDef.h"
Int main() {
intn status, i;
int swfid, SWid;
int *datbuf_int32;
int nbands, data_type, dimsize, dimsize1;
intn len, buffsize;
char label[16], unit[16], format[16];
swfid = SWopen ( "SwathFile.hdf", DFACC_RDWR );
SWid = SWattach ( swfid, "Swath1“ );
/* get Bands */
dimsize = SWdiminfo ( SWid, "Bands“ );
buffsize = SWgetdimscale ( SWid, "Spectra", "Bands", &dimsize1, &data_type, NULL );

33. datbuf_int32 = (int32 *) malloc(buffsize);
buffsize = SWgetdimscale ( SWid, "Spectra", "Bands", &dimsize1, &data_type, (void *)datbuf_int32 );
for (i=0; i<dimsize; i++)
{ printf("i = %d datbuf_int32 = %d\n", i, datbuf_int32[i]); }
/* get str attributes of Bands*/
len = 15;
status = SWgetdimstrs ( SWid, "Spectra", "Bands",
label, unit, format, len );
printf(" \t\tlabel = %s unit = %s format = %s \n\n", label, unit, format);
free(datbuf_int32);
datbuf_int32 = NULL;
SWdetach ( SWid );
SWclose ( swfid );
return 0; }

34. Reading Code Example (HDF-EOS5)
#include <HE5_HdfEosDef.h>
int main() {
herr_t status = FAIL;
hid_t swfid = FAIL, SWid = FAIL, numtype;
int i,j, bufsize;
int *databuff;
long nattr, strbufsize;
char *attrlist, *attrname = (char *)NULL;
size_t fldnmlen[HE5_HDFE_NAMBUFSIZE];
char *fldnm[HE5_HDFE_NAMBUFSIZE];
hid_t *ntype;
hsize_t count = 0;
void *attr;
int *attr_int;
float *attr_flt;
char *attr_char;
hsize_t dimsize;
swfid = HE5_SWopen ( "Swath.h5", H5F_ACC_RDWR );
SWid = HE5_SWattach ( swfid, "Swath1“ );

35. bufsize = HE5_SWgetdimscale ( SWid, "Spectra", "Bands", &dimsize,
bufsize = HE5_SWgetdimscale ( SWid, "Spectra", "Bands", &dimsize, &numtype, NULL );
databuff = (int *) malloc(bufsize );
bufsize = HE5_SWgetdimscale ( SWid, "Spectra", "Bands", &dimsize, &numtype, (void *)databuff );
for( i = 0; i < dimsize; i++)
{ printf ("DscaleVal[%d] = %d\n",i,databuff[i] ); }
nattr = HE5_SWinqdscaleattrs ( SWid, "Bands", NULL, &strbufsize );
attrlist = (char *) calloc(strbufsize + 1, sizeof(char));
nattr = HE5_SWinqdscaleattrs ( SWid, "Bands", attrlist, &strbufsize );
nattr = HE5_EHparsestr(attrlist, ',', fldnm, fldnmlen);

36. for( i = 0; i < nattr; i++){
attrname = (char *) calloc(fldnmlen[i] + 1, sizeof(char) );
memmove ( attrname,fldnm[i],fldnmlen[i] );
ntype = (hid_t *) calloc(1, sizeof(hid_t));
if ( strcmp ( attrname, "REFERENCE_LIST") == 0 ) continue;
status = HE5_SWdscaleattrinfo ( SWid,"Bands", attrname,
ntype, &count );
if( (int)*ntype == 0) {
attr_int = (int *) malloc(count*sizeof(int));
attr = (void *) attr_int; }
if( (int)*ntype == 10) {
attr_flt = (float *) malloc(count*sizeof(float));
attr = (void *) attr_flt; }
if( (int)*ntype == 57) {
attr_char = (char *) malloc((count+1)*sizeof(char));
attr = (void *) attr_char; }
status = HE5_SWreaddscaleattr ( SWid,"Bands", attrname, attr );
printf("\tAttribute Name: %s \n",attrname);
printf("\tNumber of attr elem: %lu \n", (unsigned long)count);
printf("\tData type of attribute: %d \n", (int)*ntype);

37. if( (int)*ntype == 0) { attr_int = (int *) attr;
for (j = 0; j < count; j++) { printf("\tAttrvalue[%d]: %d \n", j,attr_int[j]); } }
if( (int)*ntype == 10) {
attr_flt = (float *) attr;
for (j = 0; j < count; j++) { printf("\tAttrvalue[%d]: %f \n", j,attr_flt[j]); }
if( (int)*ntype == 57) {
attr_char = (char *) attr;
attr_char[count]='\0';
printf("\tAttrvalue: %s \n", attr_char);
free ( attr_char );
HE5_SWdetach ( SWid );
HE5_SWclose ( swfid );
return 0;

38. Test File content (he2) HDFView 2.4:
uses plug-in with hdf-eos5-1.11, Hdfeos2.15, HDF4.2r3, HDF

39. Test File content (he5) HDFView 2.4:
uses plug-in with hdf-eos5-1.11, Hdfeos2.15, HDF4.2r3, HDF

40. Test File content (he5) HDFView 2.6:
Without plug-in HDF4.2r4, HDF

41. Test File content (he5) HDFView 2.6:
uses plug-in with hdf-eos5-1.12, Hdfeos2.16, HDF4.2r4, HDF

42. File content (HIRDLS he5)
HIRDLS he5 Modified with HDF Augmentation Tool

43. Suggestions? Do we need new APIs ? Improvements for existing APIs ?
What else?

44. Thank You!