The INES Archive in the era of Virtual Observatories

Slides:



Advertisements
Similar presentations
Enrique Solano Márquez SVO Principal Investigator LAEX – CAB / INTA-CSIC The Spanish Virtual Observatory IVOA Interop., Garching, Nov 2009.
Advertisements

SCAR Data Management SSG Plenary 30 th July 2010 Kim Finney (Manager, Australian Antarctic Data Centre & Chief Officer, SCAR Standing Committee on Antarctic.
The Australian Virtual Observatory e-Science Meeting School of Physics, March 2003 David Barnes.
Jens G Jensen Atlas Petabyte store Supporting Multiple Interfaces to Mass Storage Providing Tape and Mass Storage to Diverse Scientific Communities.
CASDA Virtual Observatory CSIRO ASTRONOMY AND SPACE SCIENCE Arkadi Kosmynin 11 March 2014.
VO-DAS Chenzhou CUI Chao LIU, Haijun TIAN, Yang YANG, etc National Astronomical Observatories, CAS.
ESO-ESA Existing Activities Archives, Virtual Observatories and the Grid.
Development of China-VO ZHAO Yongheng NAOC, Beijing Nov
MAST-VizieR/NED cross correlation tutorial 1. Introduction Figure 1: Screenshot of the MAST VizieR Catalog Search Form. or enter here as object class:
Solar and STP Physics with AstroGrid 1. Mullard Space Science Laboratory, University College London. 2. School of Physics and Astronomy, University of.
NOAO/Gemini Data workshop – Tucson,  Hosted by CADC in Victoria, Canada.  Released September 2004  Gemini North data from May 2000  Gemini.
Long-Term Preservation of Astronomical Research Results Robert Hanisch US National Virtual Observatory Space Telescope Science Institute Baltimore, MD.
Leicester Database & Archive Service J. D. Law-Green, S. W. Poulton, J. Osborne, R. S. Warwick Dept. of Physics & Astronomy, University of Leicester LEDAS.
PV2013 Summary Results Data Stewardship Interest Group WGISS-37 Meeting Cocoa Beach (Florida-US) - April 14-18, 2014.
Astronomical GRID Applications at ESAC Science Archives and Computer Engineering Unit Science Operations Department ESA/ESAC.
Aus-VO: Progress in the Australian Virtual Observatory Tara Murphy Australia Telescope National Facility.
GAUDI Ground-based Asteroseismology Uniform Database Interface E. Solano Bases de données en spectroscopie stellaire. Paris.
Virtual Observatory --Architecture and Specifications Chenzhou Cui Chinese Virtual Observatory (China-VO) National Astronomical Observatory of China.
An IUE Time Series Browse Derck Massa (SGT, Inc.) Michael Van Steenberg (NASA’s GSFC)
1 Common Challenges Across Scientific Disciplines Laurence Field CERN 18 th November 2013.
The Cosmic Simulator Daniel Kasen (UCB & LBNL) Peter Nugent, Rollin Thomas, Julian Borrill & Christina Siegerist.
E. Solano Centro de Astrobiología (INTA-CSIC) I.P. Observatorio Virtual Español El Observatorio Virtual: una infraestructura básica para la investigación.
Functions and Demo of Astrogrid 1.1 China-VO Haijun Tian.
GENIUS kick-off - November 2013 GENIUS kick-off meeting WP400 – Tools for data exploitation X. Luri.
Science with the Virtual Observatory Brian R. Kent NRAO.
How to Adapt existing Archives to VO: the ISO and XMM-Newton cases Research and Scientific Support Department Science Operations.
Astronomical Spectroscopy and the Virtual Observatory ESAC, March 2007 VO tools and cross-calibration Pedro García-Lario European Space Astronomy.
F. Genova, Berlin 7, Paris, 2 December 2009 The astronomical information network.
Strasbourg astronomical Data Centre (DS) Françoise GENOVA.
E. Solano, R. Gutiérrez, B. Montesinos, C. Morales, J. García, L. Sanz LAEFF-INTA. P.O. Box 50727, Madrid (Spain) Development of a multi-mission.
Federation and Fusion of astronomical information Daniel Egret & Françoise Genova, CDS, Strasbourg Standards and tools for the Virtual Observatories.
A PPARC funded project Astronomical services: situated software vs. commodity software Guy Rixon, AstroGrid/AVO/IVOA Building Service Based Grids - GGF11.
Deepcarbon.net Xiaogang (Marshall) Ma, Yu Chen, Han Wang, John Erickson, Patrick West, Peter Fox Tetherless World Constellation Rensselaer Polytechnic.
Planetary Science Archive PSA User Group Meeting #1 PSA UG #1  July 2 - 3, 2013  ESAC PSA Introduction / Context.
March 1st, 2006Prospective PNG PNG: Databases - Virtual Observatory.
The International Virtual Observatory Alliance (IVOA) interoperability in action.
Data Archives: Migration and Maintenance Douglas J. Mink Telescope Data Center Smithsonian Astrophysical Observatory NSF
The Virtual ObservatorySVO School, October 2009 E. Solano. The Virtual Observatory Enrique Solano Spanish VO Principal Investigator LAEX-CAB (INTA-CSIC)
06-1L ASTRO-E2 ASTRO-E2 User Group - 14 February, 2005 Astro-E2 Archive Lorella Angelini/HEASARC.
Science ESAC Cluster Final Archive Pedro Osuna Head of the Science Archives and VO Team Science Operations Department CAA-CFA Review Meeting.
Sharing scientific data: astronomy as a case study for a change in paradigm Présenté par Françoise Genova.
21-jun-2009 IVOA Standards Pedro Osuna ESA-VO Project Science Archives and Computer Support Engineering Unit (SRE-OE) Science Operations Department (SRE-O)
SVO: The Spanish Virtual Observatory Enrique Solano, LAEFF / SVO Partner’s VO Project  Census of Spanish Data Centres Partner.
Introduction to the VO ESAVO ESA/ESAC – Madrid, Spain.
E. Solano. GAIA Meeting, Menorca, Oct 2009 GAIA and the Virtual Observatory Enrique Solano, LAEX/CAB (INTA-CSIC) Spanish VO Principal Investigator.
F. Genova, AstroNET meeting, Poitiers The Astrophysical Virtual Observatory.
Faculty meeting - 13 Dec 2006 The Hubble Legacy Archive Harald Kuntschner & ST-ECF staff 13 December 2006.
ESA Scientific Archives and Virtual Observatory Systems Science Archives and VO Team Research and Scientific Support Department.
IPDA Architecture Project International Planetary Data Alliance IPDA Architecture Project Report.
Click to edit Master title style Click to edit Master text styles Second level Third level Fourth level Fifth level 1 SI O S Svalbard Integrated Arctic.
Virtual Repository Progress Lars Lindberg Christensen (ESA/ESO)
Virtual Laboratory Amsterdam L.O. (Bob) Hertzberger Computer Architecture and Parallel Systems Group Department of Computer Science Universiteit van Amsterdam.
DCA WP3 Report Support to take-up and implementation of the VObs framework Christophe Arviset (ESA-VO) Paolo Padovani (ESO) Christophe Arviset (ESA),
PV 2009 Ensuring Long Term Preservation and Adding Value to Scientific and Technical Data ESAC, 1-3 December
Solar System Plasma data archiving in France
An Overview of Data-PASS Shared Catalog
Data Centres in the Virtual Observatory Age
EuroVO-DCA Support to Data Centres from non-partner countries Enrique Solano INTA- Spanish VO EuroVO-DCA WP6 coordinator.
the Virtual Observatory
Concluding Remarks Paolo Padovani Head, Virtual Observatory Project Office, ESO, Garching bei München, Germany & EURO-VO Facility Centre.
Moving towards the Virtual Observatory Paolo Padovani, ST-ECF/ESO
NA3: User Community Support Team
DATA SPHINX & EUDAT Collaboration
04/02/2019 The Use of Grid Technology in Large Scale Data Processing Collaboration Environments S.G. Ansari S.G. Ansari 04/02/2019.
Grid Application Model and Design and Implementation of Grid Services
Observing with Modern Observatories (the data flow)
Google Sky.
Monitoring the INTEGRAL satellite
GENIUS CSIC contribution Enrique Solano
X-ray high resolution spectra in the VO: the case of XMM-Newton RGS
Presentation transcript:

The INES Archive in the era of Virtual Observatories E. Solano Spanish Virtual Observatory LAEX – CAB / INTA-CSIC PV2009, ESAC, Dec 2009

Outline Introduction: Archives in Astronomy The IUE Archive Curation and preservation Added-value products Catalogues Virtual Observatory Conclusions Los centros de datos han entendido claramente que es el camino a seguir

Astronomy: a communal organisation Community-oriented instruments Public data (after short proprietary times) Community-oriented S/W Common format (FITS) Archives have become a fundamental tool for modern astrophysics

Archives in Astronomy Efficiency: One set of observations can serve many different scientific purposes, including some not considered when the observations were made.

Archives in Astronomy Periodic phenomena: Variable stars Exploitation of the time domain: Periodic phenomena: Variable stars Transient phenomena: SN, GRB

The IUE mission Jan1978 – Sep1996 > 11000 objects and > 110000 spectra. Data archiving was considered a fundamental activity from the very beginning of the IUE project.

The IUE archive: a thirthy years’ history 1978: First astronomical archive with fully calibrated data.  “Science-ready” data.

The IUE archive: a thirthy years’ history 1986: ULDA: First astronomical archive in the Internet (data access and data distribution via e-mail). More than 235 000 spectra since 1987 to 1995.

The IUE archive: Data curation The IUE Final Archive (1990’s): The problem: Along the years of the mission, different reduction systems and calibrations were used making unreliable the comparison among data observed at different epochs. Goal I: Produce a uniformly processed, fully-intercomparable archive. Improve photometric accuracy and S/N ratio by applying new image processing algorithms. Quality info. at pixel and spectrum level. Quality control on metadata. Cross-match with previous catalogues and hand-written observing logs. Goal II: Deliver IUE data to the community in a simple and efficient way.

Accessing IUE data: The INES system

The connectivity problem in the nineties Goal II: Deliver IUE data to the community in a simple and efficient way. Nowadays…

INES: The IUE Distribution System in the nineties … but in the nineties No Youtube, Google, Wikipedia or Facebook Only a few newspapers and magazines have begun to put their articles online. Your computer at home takes about 20 seconds to load each page using a modem. Costa Rica First link to the Internet in 1993. Only 12 nodes (research institutes). Speed: 64 Kbps

INES: The IUE Distribution System in the nineties A Principal Centre (LAEX) and a Mirror Site (CADC) both containing the Master Archive A number of National Hosts containing low resolution spectra.

Data preservation in the INES system Master Archive (all): 429 GB (DVD’s 3.9 GB) Fire/water proof safe. On-line archive: 129 GB. (disk) + backup (tapes)

Adding value to the archive: The INES guides Even though access to INES data is extremely easy the quantity of data in the archive is so large that the task of addressing the data efficiently in the context of an astrophysical problem is not simple. Goal of the guides: Facilitate the use of INES data for scientists with a specific astrophysical problem in mind.

Adding value to the archive: Accessing other archives Many astrophysical projects require the analysis of data covering different wavelengths of the electromagnetic spectrum.

Interoperability The ability of diverse systems providing diverse information (spectra, images, catalogues, bibliographic info.,…) to work together.

The Interoperability problem in Astronomy Data Discovery: How can astronomers find the relevant data to their scientific needs? Data access & transfer: Astronomers need to learn about different user interfaces, access and download procedures. archive 1 survey 2 survey 1 archive 2 service3 archive 3 service2 survey 3 service 1

The Interoperability problem in Astronomy Data characterisation: lack of Data Models to describe similar observations in the same way.

The Interoperability problem in Astronomy

The solution: The Virtual Observatory 2002 - INES was one of the first spectroscopic archives in the world to become VO-compliant. “All the world’s astronomical data should feel like it sits on the astronomer’s desktop” International agreement on standards. Standard semantic: UCDs Standard access protocols Standard output format Standard data models Automated discovery tools (registries) Uptake by the Data Centres  “Data Grid”. Stds Tools Science

What’s the result of all these efforts?

Summary Astronomical archives represent a fundamental tool in modern Astrophysics. IUE, a pioneer project in this field. The Virtual Observatory is a consolidated international initiative that constitutes the framework where to develop archive-related activities. INES, the IUE Archive System, is being intensively used more than 30 years after launch and more than ten years after the end of operations. In particular, VO accesses are growing every year. INES is an excellent example of how archives can extend the projects’ effective lifetime reducing very considerably the “euro per observation” cost. Los centros de datos han entendido claramente que es el camino a seguir