BinX and Astronomy Bob Mann Institute for Astronomy and National e-Science Centre

BinX and Astronomy The Virtual Observatory concept The Virtual Observatory concept Astronomical data formats Astronomical data formats  FITS and VOTable Possible Use Cases for BinX in the VO Possible Use Cases for BinX in the VO  Further requirements

The Virtual Observatory Federation of the world’s significant astronomical data resources, plus the tools required to exploit those data scientifically Federation of the world’s significant astronomical data resources, plus the tools required to exploit those data scientifically Based on web (later, Grid) services Based on web (later, Grid) services Coordinated projects around the world Coordinated projects around the world  UK: AstroGrid Forced discussion of interoperability issues Forced discussion of interoperability issues

Astronomical data formats

FITS – Flexible Image Transport System Near-universal in astronomy Near-universal in astronomy Extended to handle non-image data Extended to handle non-image data FITS binary table preferred format for transporting large amounts of tabular data FITS binary table preferred format for transporting large amounts of tabular data Only used in astronomy – poor leverage of generic software Only used in astronomy – poor leverage of generic software Difficult to stream FITS data – no. of rows has to be specified Difficult to stream FITS data – no. of rows has to be specified

VOTable New XML standard for tabular data New XML standard for tabular data Design allows easy translation from FITS Design allows easy translation from FITS Usual XML advantages… Usual XML advantages…  Flexibility, validation, streaming, etc …and disadvantages …and disadvantages  Verbosity – serious problem in astronomy

Three flavours of VOTable TABLEDATA TABLEDATA  Metadata and data in fully tagged XML FITS FITS  Metadata in XML, data in external FITS file, linked from VOTable document BINARY BINARY  Metadata in XML, data encoded in binary format and included in VOTable doc

How can BinX help the VO?

Conversion between FITS binary tables and VOTables Lots of legacy FITS data files around Lots of legacy FITS data files around First step in eDIKT BinX testbed First step in eDIKT BinX testbed Conversion to TABLEDATA works Conversion to TABLEDATA works  what about FITS & BINARY? Can also strip metadata from headers of non-bintable extensions (e.g. images) Can also strip metadata from headers of non-bintable extensions (e.g. images)

Aiding data transport in the VO Site 1 WAN Site 2 Site 1 WAN Site 2 Database 1 Database 2 Application VOTable Binary BinX VOTable

Site 1 WAN Site 2 Site 1 WAN Site 2 Database 1 Database 2 Application VOTable Binary BinX SAX events VOTable Delivering SAX events with BinX?

Site 1 WAN Site 2 Site 1 WAN Site 2 Database 1 Database 2 Application Binary BinX SAX events BinX Integrating BinX and databases?

Site 1 WAN Site 2 Site 1 WAN Site 2 Database 1 Database 2 Application Binary BinX SAX events BinX XQuery Querying BinX’s implied XML representation with XQuery?

Summary The requirements of the VO and the aims of BinX seem well matched The requirements of the VO and the aims of BinX seem well matched Initial results from the astronomy testbed are encouraging Initial results from the astronomy testbed are encouraging Astronomy would benefit from a number of the possible extensions to BinX: Astronomy would benefit from a number of the possible extensions to BinX:  Java version – aid integration with VO  Support for SAX as well as DOM  XQuery on implied XML representation