RSI at the HDF & HDF-EOS Workshop VI October 26, 2004 – Aurora, CO

RSI Representatives: Eddie Haskell – IDL Software EngineerEddie Haskell – IDL Software Engineer 8+ Years of IDL Experience Andy Pursch – Technical Sales Engineer Mgr. Pursch – Technical Sales Engineer Mgr. 16+ Years of IDL Experience Worked at NASA Goddard and JPL Molly Bantz – Product Marketing Specialist Bantz – Product Marketing Specialist

Agenda: RSI Corporate OverviewRSI Corporate Overview Introduction of IDL-HDF5 Write FunctionalityIntroduction of IDL-HDF5 Write Functionality Product DemonstrationProduct Demonstration

RSI Overview: Locations RSI Corporate Headquarters Boulder, ColoradoRSI Corporate Headquarters Boulder, Colorado RSI Washington D.C. OfficeRSI Washington D.C. Office RSI International OfficesRSI International Offices United Kingdom FranceItaly RSI DistributorsRSI DistributorsWorldwide

RSI Overview: History 1977 RSI founded and IDL launched1977 RSI founded and IDL launched Market for IDL grows extensively and RSI becomes a global organizationMarket for IDL grows extensively and RSI becomes a global organization 1994 RSI releases ENVI1994 RSI releases ENVI 2000 RSI acquired by KODAK2000 RSI acquired by KODAK 2004 RSS group of KODAK and RSI become part of ITT Industries2004 RSS group of KODAK and RSI become part of ITT Industries –Space Systems Division

RSI Products & Services IDL – Interactive Data Language IDL – Interactive Data Language ION, Virtual Machine, Dataminer, Wavelet Toolkit, New! DICOM Toolkit ION, Virtual Machine, Dataminer, Wavelet Toolkit, New! DICOM Toolkit ENVI – Remote Sensing Solutions ENVI – Remote Sensing Solutions FLAASH, NITF Module, RemoteView Link, ARCGIS Reader RSI’s Global Services – Solutions RSI’s Global Services – Solutions Algorithm Development, Consulting, Training, Commercialization, SBIR Support

RSI Industries Served RSI products and services provide visualization and analysis solutions for a wide variety of industries: Remote Sensing Atmospheric Science / Meteorology Atmospheric Science / Meteorology Astrophysics Astrophysics Medical Imaging Medical Imaging Engineering Engineering Mining/Oil & Gas Defense/Homeland Security Defense/Homeland Security Government Government Academic Academic Research & Development Research & Development

IDL and HDF5

IDL 6.1 HDF5 New for IDL 6.1: The HDF5 dynamically loadable module has been enhanced to include write capabilities. Write capabilities include datasets, attributes, groups, links, datatypes, and many types of data.

IDL 6.1 HDF5 Prior to IDL 6.1 the HDF5 library included only read capabilities Also included a graphical HDF5 file browser and a parse procedure that would parse an entire HDF5 file and import it into IDL.

IDL 6.1 HDF5 Added routines from the following HDF5 interfaces: Attributes: H5A Datasets: H5D File: H5F Group: H5G Reference: H5R Dataspace: H5S Datatype: H5T

IDL 6.1 HDF5 H5F Updated routine: H5F_OPEN Added WRITE keyword to open the file for reading and writing. Default is read only. New routine: H5F_CREATE file_id = H5F_CREATE(Filename) This always overwrites the file if one exists. Defaults are used for create and access properties.

IDL 6.1 HDF5 H5T New routine: H5T_IDL_CREATE Datatype_id = H5T_IDL_CREATE(Data [, MEMBER_NAMES=vector]) Creates a datatype based on the FIRST element of the IDL data (caution when creating a string datatype). Compound datatypes can be created by passing in an IDL structure. Uses H5Tcopy and H5Tcreate.

IDL 6.1 HDF5 H5T New routine: H5T_ARRAY_CREATE Datatype_id = H5T_ARRAY_CREATE(Datatype_id, Dimensions) New routine: H5T_REFERENCE_CREATE Datatype_id = H5T_REFERENCE_CREATE([/REGION])

IDL 6.1 HDF5 H5T New routine: H5T_INSERT H5T_INSERT, Datatype_id, Name, Field_id This procedure adds a new member to the end of a compound datatype. New routine: H5T_PACK H5T_PACK, Datatype_id

IDL 6.1 HDF5 H5S New routine: H5S_CREATE_SCALAR Dataspace_id = H5S_CREATE_SCALAR() Complements existing routine: H5S_CREATE_SIMPLE

IDL 6.1 HDF5 H5D New routine: H5D_CREATE Dataset_id = H5D_CREATE(Loc_id, Name, Datatype_id, Dataspace_id, [CHUNK_DIMENSIONS=vector]) CHUNK_DIMENSIONS is required for extendable datasets Default values are used for allocation time, fill time, and fill value

IDL 6.1 HDF5 H5D New routine: H5D_WRITE H5D_WRITE, Dataset_id, Data, [, MEMORY_SPACE_ID=value] [, FILE_SPACE_ID=value] Data will be converted, if possible, to the datatype that was used in the creation of the dataset

IDL 6.1 HDF5 H5A New routine: H5A_CREATE Attr_id = H5A_CREATE(Loc_id, Name, Datatype_id, Dataspace_id) Attributes are limited in size to 16KB

IDL 6.1 HDF5 H5A New routine: H5A_WRITE H5A_WRITE, Attribute_id, Data Data will be converted, if possible, to the datatype that was used in the creation of the attribute

IDL 6.1 HDF5 H5R New routine: H5R_CREATE Ref = H5R_CREATE(Loc_id, Name [, DATASPACE_ID=value]) Ref is returned as either an integer, if an object reference is returned, or a named structure, if a dataspace region reference is returned. Updated routine: H5R_DEREFERENCE

IDL 6.1 HDF5 H5A New routine: H5A_DELETE H5A_DELETE, Loc_id, Name This function should not be used when attribute identifiers are open on Loc_id as it may corrupt the file.

IDL 6.1 HDF5 H5D New routine: H5D_EXTEND H5D_EXTEND, Dataset_id, Size

IDL 6.1 HDF5 H5G New routine: H5G_CREATE Group_id = H5G_CREATE(Loc_id, Name) New routine: H5G_GET_NUM_OBJS Value = H5G_GET_NUM_OBJS(Loc_id)

IDL 6.1 HDF5 H5G New routine: H5G_GET_OBJ_NAME_BY_IDX Name = H5G_GET_OBJ_NAME_BY_IDX(Loc_id, Index) New routine: H5G_SET_COMMENT H5G_SET_COMMENT, Loc_id, Name, Comment

IDL 6.1 HDF5 H5G New routine: H5G_LINK H5G_LINK, Loc_id, Current_Name, New_Name [, /SOFTLINK] [, NEW_LOC_ID=value] Combines H5GLink and H5GLink2, calling the appropriate one as needed. New routine: H5G_UNLINK H5G_UNLINK, Loc_id, Name

IDL 6.1 HDF5 H5G New routine: H5G_MOVE H5G_MOVE, Loc_id, Src_Name, Dst_Name [, NEW_LOC_ID=value] Combines H5GMove and H5GMove2, calling the appropriate one as needed.

IDL 6.1 HDF5 H5R New routine: H5R_GET_REGION Dataspace_id = H5R_GET_REGION(Dataset_id, Reference)

IDL 6.1 HDF5 H5S New routine: H5S_SET_EXTENT_SIMPLE H5S_SET_EXTENT_SIMPLE, Dataspace_id, Dimensions [,MAX_DIMENSIONS=vector] New routine: H5S_SET_EXTENT_NONE H5S_SET_EXTENT_NONE, Dataspace_id

IDL 6.1 HDF5 H5T New routine: H5T_COMMIT H5T_COMMIT, Loc_id, Name, Datatype_id A named Datatype can be shared by objects within the same HDF5 file, but not by objects in other files.

IDL 6.1 HDF5 New routine: H5_CREATE H5_CREATE, Filename, Structure The structure is modeled after the structure returned from the existing routine H5_PARSE(). H5_CREATE may not be able to recreate a file when given the structure returned from H5_PARSE(). Limitations: Variable length arrays, data types not directly supported in IDL, order of items in the file, references, …

IDL 6.1 HDF5 H5_CREATE Structure tags: _NAME : If not supplied the name of the tag will be used (Required for top level items. Exception: if this a top level group a new group will only be created if _NAME is supplied) _TYPE : “GROUP”, “DATASET”, “ATTRIBUTE”, “DATATYPE”, “LINK” _DATA : IDL data. Full path to object if LINK. Not used for GROUPS _COMMENT : comment _LINKTYPE : “SOFT” or “HARD” LINK only, default is HARD SUBSTRUCTURES : Items in a group, attributes, etc.

IDL 6.1 HDF5 H5_CREATE Dataspaces are all defined as the full extent of the data, and datatypes are created automatically based on the type of the data. Simple Example: grey_scale = byte(bindgen(256)##(bytarr(3)+1b)) pal = {_TYPE:'Attribute', _DATA:grey_scale} dataset = {_NAME:'Hanning', _TYPE:'Dataset', $ _DATA:hanning(100,200), PALETTE:pal} H5_CREATE, 'myFile2.h5', dataset

IDL 6.1 HDF5/EOS “RSI is committed to delivering products to the scientific data format community. We will maintain a watch in order to develop the functionality necessary to support missions and new formats.” “We realize that our work is not done with HDF5-write functionality and so are beginning the work of scoping and scheduling development of HDF5-EOS.” Respectfully, Harold Cline Director of Product Management

IDL 6.1 HDF5 Currently in testing. Hopefully will be available in mid-November. Questions? Comments? Requests?