Workshop on How to Publish Data in VO ESAC, June 25-June 29 2007 DAL (Data Access Layer) protocols Jesus Salgado

Slides:



Advertisements
Similar presentations
European Space Astronomy Centre (ESAC) Villafranca del Castillo, MADRID (SPAIN) Pedro OSUNA ESAC ADT Team VOSpec IVOA Poona Sep 2004 VOSpec: A Tool to.
Advertisements

European Space Astronomy Centre (ESAC) Villafranca del Castillo, MADRID (SPAIN) Aurélien Stébé DALToolKit Ingestor & Server September 2006, Moscow DALToolKit.
European Space Astronomy Centre (ESAC) Villafranca del Castillo, MADRID (SPAIN) Jesús Salgado SLAP Implementations Sep 2006, Moscow, Russia Simple Line.
IVOA, Pune India September Data Access Layer Working Group Pune Workshop Summary Doug Tody National Radio Astronomy Observatory International.
IVOA, Kyoto May Simple Spectral Access SSA Query Interface Doug Tody (NRAO) Markus Dolensky (ESO) Et. al. International V IRTUAL O BSERVATORY.
SSA Query Interface M. Dolensky, ESO Data Access Layer Working Group Interoperability Workshop, Pune, India 27-Sep-2004.
IVOA, Pune September Data Access Layer Working Group SSA Overview and Status Doug Tody National Radio Astronomy Observatory International V.
IVOA Interop meeting 05/17/2006 Victoria F.Bonnarel (CDS) Generic Data discovery, Cube acces: CGPS Archive browser F.Bonnarel,T.Boch,D.Durand (CDS, CADC)
2004 Sepjcm/sao/nvo1 Spectrum Data Model Jonathan McDowell CfA US National Virtual Observatory.
SLAP: Simple Line Access Protocol v0.5
IVOA, Kyoto May Data Access Layer Working Group Working Group Report and Summary Doug Tody National Radio Astronomy Observatory International.
September 13, 2004NVO Summer School1 VO Protocols Overview Tom McGlynn NASA/GSFC T HE US N ATIONAL V IRTUAL O BSERVATORY.
September 13, 2004NVO Summer School1 VO Protocols Overview Tom McGlynn NASA/GSFC T HE US N ATIONAL V IRTUAL O BSERVATORY.
6 September 2008NVO Summer School 2008 – Santa Fe1 DAL Clients: Scripting Data Access with Python Ray Plante T HE US N ATIONAL V IRTUAL O BSERVATORY.
NVO Summer School, Santa Fe Sept Access to Spectroscopic Data In the VO Doug Tody (NRAO/US-NVO ) I NTERNATIONAL V IRTUAL O BSERVATORY A LLIANCE.
2008 NVO Summer School1 Data Access Layer Services Doug Tody (NRAO) T HE US N ATIONAL V IRTUAL O BSERVATORY.
Sept NVO Summer School1 Cone, SIAP, and OpenSkyQuery Client Development Gretchen Greene, Maria Nieto-Santisteban T HE US N ATIONAL V IRTUAL O.
NVO Summer School, Aspen Sept Data Access Layer Working Group Image and Spectral Access Doug Tody National Radio Astronomy Observatory National.
CASDA Virtual Observatory CSIRO ASTRONOMY AND SPACE SCIENCE Arkadi Kosmynin 11 March 2014.
Lessons learnt with Aladin and characterization experience for SIA2.0 F.Bonnarel, CDS (credit to Aladin developpers, CADC,ECF,ESAC, ESO VO people, DAL.
October 12, 2003ADASS NVO Tutorial1 How-To Implement Cone and SIA Services Gretchen Greene Space Telescope Science.
2003 April 151 Data Centres: Connecting to the Real World Clive Page.
S. Derriere et al., ESSW03 Budapest, 2003 May 20 UCDs - metadata for astronomy Sébastien Derriere François Ochsenbein Thomas Boch CDS, Observatoire astronomique.
Characterisation Data Model applied to simulated data Mireille Louys, CDS and LSIIT Strasbourg.
The need for a stronger IVOA review process An SDM+SSA correctness assessment Bruno Rino, Nov
DateADASS How to Navigate VO Datasets Using VO Protocols Ray Plante (NCSA/UIUC), Thomas McGlynn and Eric Winter NASA/GSFC T HE US N ATIONAL V IRTUAL.
IVOA Interop, Victoria Canada, May IVOA Data Access Layer Closing Plenary Summary, Victoria May 2006 Doug Tody (NRAO/NVO/IVOA) I NTERNATIONAL V IRTUAL.
16-17 Oct 2003IVOA Data Access Layer, Strasbourg IVOA Data Access Layer (DAL) Working Group Doug Tody National Radio Astronomy Observatory International.
29-30 April 2004NVO Team Meeting NCSA1 Data Access Layer (DAL) SSA, SIA Enhancement Doug Tody National Radio Astronomy Observatory National Virtual Observatory.
Atomic & Molecular Lines Data Model Paris Observatory and ESA/ESAC ML Dubernet, P. Osuna, M. Guanazzi, J. Salgado, E. Roueff MLD acknowledges support.
European Space Astronomy Centre (ESAC) Villafranca del Castillo, MADRID (SPAIN) Jesús Salgado SLAP Implementations May 2007, Beijing, China Simple Line.
Spectroscopy in VO, ESAC Mar Access to Spectroscopic Data In the VO Doug Tody (NRAO/US-NVO ) for the IVOA DAL working group I NTERNATIONAL.
Virtual Observatory Interfaces reused in the Virtual Atomic and Molecular Data Centre Guy Rixon Institute of Astronomy University of Cambridge September.
How to Adapt existing Archives to VO: the ISO and XMM-Newton cases Research and Scientific Support Department Science Operations.
European Space Astronomy Centre (ESAC) Villafranca del Castillo, MADRID (SPAIN) Isa Barbarisi VOSpec, new functionalities Madrid, 6-7Oct 2005 New functionalities.
European Space Astronomy Centre (ESAC) Villafranca del Castillo, MADRID (SPAIN) Jesús Salgado Spectroscopic lines in the VO context Mar 2007, ESAC, Madrid,
EÖTVÖS UNIVERSITY BUDAPEST Department of Physics of Complex Systems VO Spectroscopy Workshop, ESAC Spectrum Services 2007 László Dobos (ELTE)
IVOA Interop, SL de El Escorial, Oct IVOA DAL - Madrid DAL WG Summary October 7, 2005.
ESAVO/European Space Astronomy Centre (ESAC) Villafranca del Castillo, MADRID (SPAIN) Isa Barbarisi Beijing, May 2007 VOSpec new functionalities.
Federation and Fusion of astronomical information Daniel Egret & Françoise Genova, CDS, Strasbourg Standards and tools for the Virtual Observatories.
European Space Astronomy Centre (ESAC) Villafranca del Castillo, MADRID (SPAIN) Jesús Salgado IVOA Interop meeting Strasbourg, May 2009 (1/12) SLAP v0.9.
European Space Astronomy Centre (ESAC) Villafranca del Castillo, MADRID (SPAIN) Pedro OSUNA ESAC ADT Team VO-tech Cambridge 2004 VOSpec: A Tool to Handle.
AstroGrid Solar/STP planning meeting Agenda: Helioscope Preparing for Solar-B Time-series viewing application IVOA and time series A PPARC funded project.
European Space Astronomy Centre (ESAC) Villafranca del Castillo, MADRID (SPAIN) Applications May 2006, Victoria, Canada VOQuest A tool.
The International Virtual Observatory Alliance (IVOA) interoperability in action.
PhotDM implementation feedback | Jesus Salgado | ESAC | 18 October 2011 | IVOA Pune 2011 | Pag. 1 Photometry DM implementation feedback Jesus.
German Astrophysical Virtual Observatory Overview and Results So Far W. Voges, G. Lemson, H.-M. Adorf.
21-jun-2009 IVOA Standards Pedro Osuna ESA-VO Project Science Archives and Computer Support Engineering Unit (SRE-OE) Science Operations Department (SRE-O)
UCL DEPARTMENT OF SPACE AND CLIMATE PHYSICS MULLARD SPACE SCIENCE LABORATORY Taverna Plugin VAMDC and HELIO (part of the ‘taverna-astronomy’ edition) Kevin.
European Space Astronomy Centre (ESAC) Villafranca del Castillo, MADRID (SPAIN) Jesús Salgado AIDA Tech. Meeting Strasbourg, March 2009 (1/8) WP7 Task.
12 Oct 2003VO Tutorial, ADASS Strasbourg, Data Access Layer (DAL) Tutorial Doug Tody, National Radio Astronomy Observatory T HE US N ATIONAL V IRTUAL.
ESAVO/European Space Astronomy Centre (ESAC) Villafranca del Castillo, MADRID (SPAIN) Pedro Osuna ASVOWS Mar 2007 ESAC Astronomical Spectroscopy.
IVOA Interop, SL de El Escorial, Oct IVOA Data Access Layer Doug Tody (NRAO/NVO/IVOA) I NTERNATIONAL V IRTUAL O BSERVATORY A LLIANCE.
European Space Astronomy Centre (ESAC) Villafranca del Castillo, MADRID (SPAIN) Aurélien Stébé DALToolKit Ingestor & Server January 2008, VODay, Sofia.
VOTable agenda Current VOTable status Current VOTable status News from Applications News from Applications Questions about VOTable schema Questions about.
European Space Astronomy Centre (ESAC) Villafranca del Castillo, MADRID (SPAIN) Pedro Osuna VOSpec Kyoto May 2005 VOSpec: A Tool to Handle VO-Compatible.
Publishing Combined Image & Spectral Data Packages Introduction to MEx M. Sierra, J.-C. Malapert, B. Rino VO ESO - Garching Virtual Observatory Info-Workshop.
Coordination Portal/Data sources L2:Data access & Handling Points to be raised : - Downloading data sources/data via the Portal…keeping visibility of data.
IVOA Interop, Beijing, China, May IVOA Data Access Layer Working Group Sessions Doug Tody (NRAO/NVO ) Markus Dolensky (ESO/EuroVO) Data Access Layer.
VO Data Access Layer IVOA Cambridge, UK 12 May 2003 Doug Tody, NRAO.
IVOA Interop, Beijing, China, May IVOA Data Access Layer Working Group Sessions Doug Tody (NRAO/NVO ) Markus Dolensky (ESO/EuroVO) Data Access Layer.
JENAM 2008 Theory Standards for the Virtual Observatory SimDB + SimDAP.
Simple Image Access International VIRTUAL OBSERVATORY
Standard Query Language for VO
Data Models: Plenary-1 Jonathan McDowell May 2006
Accomplishments RSM v0.7 First draft XML Schema completed: VOResource.xsd NVO: Working prototype resource using VOResource as format for metadata exchange.
Observation/dataset datamodel: Restart non-characterization efforts?
Google Sky.
SVO - ESAVO Use case: Comparison of evolutionary synthesis models
Presentation transcript:

Workshop on How to Publish Data in VO ESAC, June 25-June DAL (Data Access Layer) protocols Jesus Salgado

Plenary Session, Page 2 Why do we need protocols?

Jesus SalgadoPlenary Session, Page 3 Why do we need protocols? (II)

Jesus SalgadoPlenary Session, Page 4 Simple Protocols  SIAP (Images), SSAP (Spectra), SLAP(Spectral Lines)  HTTP based  VOTable response  Similar interfaces; different protocols for different types of products (images, spectra, lines,…) to converge in a latter state  Not difficult to be developed

Jesus SalgadoPlenary Session, Page 5 SIAP Simple Image Access Protocol (Tody et al.)

Jesus SalgadoPlenary Session, Page 6 SIAP: IVOA standard  […] standard for retrieving image data from a variety of astronomical image repositories through a uniform interface […]  […] based primarily on two documents […]"Simple Image Retrieval: Interface Concepts and Issues", […] "Simple Cone Search specification" [...] "Simple Image Retrieval: Interface Concepts and Issues""Simple Cone Search specification""Simple Image Retrieval: Interface Concepts and Issues""Simple Cone Search specification"  Originally [...] primarily as an "image on demand" service, with images created on-the-fly [...]  […]Required query parameters: POS, SIZE, FORMAT[…]  […]Required response columns: Image_Title, POS_EQ_RA_MAIN, POS_EQ_DEC_MAIN, Image_Format […]

Jesus SalgadoPlenary Session, Page 7 SIAP Basics SIAP Servers service URLs Query Response: VOTable with list of matches Data Response: Image Coordinates/Target matches? SIAP Servers available? Give ACCREF data (real Data) Client application REGISTRY

Jesus SalgadoPlenary Session, Page 8 SIAP: facts  More than 80 SIAP services registered  The most successful IVOA protocol up to now  Easy access to images from different spectral ranges in the same VO application (e.g. Aladin)  Some examples: –HST, 2MASS, XMM, SDSS, ISO, Integral, ROSAT, FIRST, IRAS,… and many more

Jesus SalgadoPlenary Session, Page 9 SSAP Simple Spectral Access Protocol (Tody et al.)

Jesus SalgadoPlenary Session, Page 10 SSAP: A summary  […]To define a uniform interface to spectral data[…]  […]The term “Simple” in Simple Spectral Access Protocol refers to the design goal of simplicity in both implementing spectral data services and in retrieving spectroscopic data from distributed data collections[…]  […]Required query parameters: POS, SIZE, FORMAT[…]  […]Required response columns: FORMAT, ACREF, SED Object, Data Source, Coverage Metadata…]

Jesus SalgadoPlenary Session, Page 11 SSAP Basics SSA Servers service URLs Query Response: VOTable with list of matches Data Response: Spectrum ( fits, VOTable, …) Coordinates/Target matches? SSA Servers available? Give ACCREF data (real Data) Client application REGISTRY

Jesus SalgadoPlenary Session, Page 12 SSAP Basic Usage  Simple query –POS, SIZE - like cone search –Possibly refined by spectral or time bandpass, etc. –Most metadata in query response is optional  Data retrieval –URL-based –Get back a dataset (normally VOTable or FITS) –In simplest case could be wavelength, flux as text (for Spectrum) –Or external data pass-through

Jesus SalgadoPlenary Session, Page 13 SSAP Query Interface  Mandatory query parameters –POSRA, DEC (ICRS) –SIZEdiameter (decimal degrees) –FORMAT VOTable, fits, xml, text, graphics, html, external  Should other parameters be mandatory? –e.g., BANDPASS, TIME, SPECRES, APERTURE

Jesus SalgadoPlenary Session, Page 14 SSAP Query Response  Dataset Metadata –Dataset.Type Spectrum, TimeSeries, SED, etc. –Dataset.DataModel DM name, e.g., "SSA-V1.0" –DataID.Title Brief descriptive title of dataset –Dataset..Length Number of points in dataset –Dataset.SSA.SpectralAxis SpectralCoord axis (external data) –Dataset.SSA.FluxAxis Flux axis (external data) –Dataset.SpectralSISI factor and dimensions –Dataset.FluxSISI factor and dimensions –Dataset.CreationType atlas, pointed, cutout, resampled –DataID.DataSourceOriginal source of data

Jesus SalgadoPlenary Session, Page 15 SSAP 1.1– Some specialized response metadata  Characterization - Reference Frames –CoordSys.SpaceFrame.NameSpatial Coordinate frame (default ICRS) –CoordSys.SpaceFrame.Equinox Equinox (2000) –CoordSys.TimeFrame.Name Timescale (TT)  Characterization - Coverage –Char.SpatialAxis.Coverage.Location.Value  Observed position, e.g., RA DEC –Char.SpatialAxis.Coverage.Bounds.Extent  Aperture angular diameter –Char.SpectralAxis.Coverage.Location.Value  Midpoint of Spectral coord. range –Char.SpectralAxis.Coverage.Bound.Extent  Width of spectrum

Jesus SalgadoPlenary Session, Page 16 SSAP implementations  Implement service able to provide SSAP-like access –ESA’s Infrared Space Observatory (ISO) spectra available in SSAP- like form since December First ever SSAP-like server. –Other SSAP-like services that followed suite (in chronological order): IUE, HST, SDSS, FUSE, HYPERLEDA….up to 18 services (all available from VOSpec tool) (all available from VOSpec tool)VOSpec  Implementation of latest protocol specification not ready for most of them  Design and implement client able to consume SSAP spectra services –ESA’s initiative response called VOSpec: a tool for handling SSAP- like spectra in VO context

Jesus SalgadoPlenary Session, Page 17 TSAP (Theoretical spectra)  Same protocol (input/output) re-used to give support theoretical spectral servers  Both coverage input parameters and output characterization coverage metadata non- compulsory  As it will be seen in next section, discovery of server capabilities is crucial  9 services already in place

Jesus SalgadoPlenary Session, Page 18 SLAP Simple Line Access Protocol (Salgado et al.)

Jesus SalgadoPlenary Session, Page 19 SLAP (Simple Line Access Protocol) VOTable, one spectral line per row A SIAP-like URL service, would allow an easy implementation for all spectral line providers

Jesus SalgadoPlenary Session, Page 20 SLAP input parameters  If the only compulsory parameters are the minimum and maximum wavelength, the number of lines in the output result, in particular for theoretical spectral line databases, could be HUGE. –Some non-compulsory parameters added to the protocol to filter or score the output lines –CHEMICAL_ELEMENT: This parameter would constraint the search to the chemical element selected (syntax problems, but some standards found) –LEVEL_ENERGY_START & LEVEL_ENERGY_END: Parameters to specify the minimum and maximum energy for the START & FINAL level of the transition. Filter at CLIENT side –TEMPERATURE: This parameter would be used (in particular for theoretical spectral line databases) to score the lines in the output using physical models. Score at SERVER side

Jesus SalgadoPlenary Session, Page 21 SLAP extra parameters  The servers could have extra parameters that cannot be identified easily and included in the document.  For Theoretical Spectral Line services, the parameters could be used to filter/score the result due to physical models  For Observational Spectral Line services, the parameters could be project dependent  Solution adopted: –Use same approach than proposed for TSAP. All the input parameters will be DESCRIBED in a FORMAT=METADATA query. –The description should include not only the parameter names but a description, UCDs (to identify the physical meaning of the parameters), utypes whenever possible (to Line Data Model or another IVOA data model) –This FORMAT=METADATA could be requested by the registry services or by the VO clients

Jesus SalgadoPlenary Session, Page 22 FORMAT=METADATA IASD Simple Line Access Service <PARAM name="INPUT:WAVELENGTH" ucd="em.wl;stat.min" utype=”ldm:Line.wavelength” value="30"> Specify the wavelength spectral range. To be specified in micrometers. This wavelength will be interpreted as the wavelength in the vacuum of the transition originating the line <PARAM name="INPUT:CHEMICAL_ELEMENT" ucd=“phys.atmol.element" utype=”ldm:Line.initialElement.name” value="190"> Specify the chemical element that originates the line Specify the ISO observation number where this line has been identified ……………………………………..

Jesus SalgadoPlenary Session, Page 23   Output “MUST” Compulsory fields – – wavelength (ucd="em.wl”; utype=”ldm:Line.wavelength”) wavelength in the vacuum of the transition originating the line in micrometers – – title (ucd=“em.line”;utype=”ldm:Line.title”) contains a small description identifying the line.   Output “SHOULD” non-compulsory field. – – chemicalelement_name (ucd=“phys.atmol.element”;utype=”ldm:ChemicalElement.name”) contains a name of the chemical element source of this line – – initial_level & final_level (ucd=“phys.atmol.[initial/final];phys.atmol.level”; utype=”ldm:Line.[initial/final]Level) contain a full description of the initial & final levels of the transition originating the line SLAP output parameters (I)

Jesus SalgadoPlenary Session, Page 24 SLAP output parameters (II)   Summary of output COULD optional fields – – observed_wavelength (ucd="em.wl“; utype=”ldm:Line.observedWavelength”) contains the observed wavelength in the vacuum of the transition originating the line in micrometers, as it was observed. This could be used by observational spectral line databases – – score (utype=”Query.Score”) similar concept to SSAP 0.9. Easier to be implemented for physical models – – level_energy_start & level_energy_end (ucd="phys.energy; phys.atmol.[initial/final];phys.atmol.level”; utype=”ldm:Level.energy.[start/end]”) describe the energy for the starting & final levels of the transition which originates this line.

Jesus SalgadoPlenary Session, Page 25   URL-like fields – – publication_link (ucd="meta.bib”) specifies a http link that contains a bibliographic reference related to the spectral line – – Other fields could have ucd=“meta.ref.url" specifying URLs that contains extra information related to the spectral line   Non-Standard output fields – – In many occasions, extra scientifically interesting parameters could be added to the output. Implementors are encouraged to add descriptions and UCDs to the return fields to clarify the meaning of this information and utypes to the Line data Model or other existing Data Model, whenever possible.   4 services already in place; IASD, Cielo (observational), NIST, Lerma (laboratory). Plans to include ALMA in a near future SLAP output parameters (III)

Jesus SalgadoPlenary Session, Page 26