MEx Metadata extraction

IVOA Interop, Moscow, 19-Sep-2006 Metadata extraction The problem The problem Data centres need their data described homogeneously Data centres need their data described homogeneously Data in FITS files is described by heterogeneous keywords Data in FITS files is described by heterogeneous keywords

IVOA Interop, Moscow, 19-Sep-2006 Goals Bridge the gap between data producers and data centres Bridge the gap between data producers and data centres Ingest the metadata present in FITS keywords into arbitrary data models. Ingest the metadata present in FITS keywords into arbitrary data models. Enable PIs/CoIs to define the meaning of their FITS keywords Enable PIs/CoIs to define the meaning of their FITS keywords Highly configurable Highly configurable Data model Data model Mappings Mappings Extensible Extensible

IVOA Interop, Moscow, 19-Sep-2006 keyword mappingpersistence FITS files Architecture data base mapping definition keyword mapper database mapper package (FITS Files) VOTable mapper VOTable FITS mapper

IVOA Interop, Moscow, 19-Sep-2006 keyword mapping persistence Architecture: SIAP demo SIAP database mapping definition keyword mapper database mapper package (FITS Files) SIAP service

IVOA Interop, Moscow, 19-Sep-2006 Concepts: for the data centre Datatypes Datatypes e.g. image, 1d spectra,... e.g. image, 1d spectra,... Model items Model items e.g. RA, Dec, Spectral Resolution,... e.g. RA, Dec, Spectral Resolution,... Defined per datatype Defined per datatype Model items are definitions of utypes Model items are definitions of utypes Characterization: UCD, unit, required Characterization: UCD, unit, required

IVOA Interop, Moscow, 19-Sep-2006 Concepts: for the data producer Package Package File list, with datatype File list, with datatype r.UVES T00:48:15.610_0000.fits spectrum.1d 1 r.UVES T00:48:15.610_0000.fits.ps qc report ps 1 r.UVES T00:48:15.610_0013.fits spectrum.1d 2 r.UVES T00:48:15.610_0000.fits_10.ps qc report ps 2 r.UVES T00:48:15.866_0000.fits spectrum.1d 3 r.UVES T00:48:15.866_0000.fits.ps qc report ps 3 Mapping rules Mapping rules Observation.TotalExposure EXPTIME,s Coverage.Temporal.StartTime {MJD-OBS,MJD} Coverage.Temporal.StopTime {MJD-OBS,MJD}+EXPTIME,s

IVOA Interop, Moscow, 19-Sep-2006 Mapping rules Keyword/value mapping Keyword/value mapping Constant Constant Unit conversion (using CDS library) Unit conversion (using CDS library) Arithmetic expressions, string concatenation Arithmetic expressions, string concatenation Choice Choice Indirection: addressing other keys, extensions, and files Indirection: addressing other keys, extensions, and files Hard to get values: Standard computations, precomputed table Hard to get values: Standard computations, precomputed table

IVOA Interop, Moscow, 19-Sep-2006 Standard computations Populate metadata by performing (complex?) computations Populate metadata by performing (complex?) computations Defined by the data centre Defined by the data centre examples: examples: CD Matrix CD Matrix RA, Dec of Image Centre RA, Dec of Image Centre RA, Dec of Lower/Upper Bound of Coverage RA, Dec of Lower/Upper Bound of Coverage

IVOA Interop, Moscow, 19-Sep-2006 Persistence: MySQL DB