Presentation is loading. Please wait.

Presentation is loading. Please wait.

F. Genova, Open Archive Forum, 02/05/141 On-line information in astronomy From networking to a virtual observatory.

Published byColin Short Modified over 11 years ago

Similar presentations

Presentation on theme: "F. Genova, Open Archive Forum, 02/05/141 On-line information in astronomy From networking to a virtual observatory."— Presentation transcript:

1 F. Genova, Open Archive Forum, 02/05/141 On-line information in astronomy From networking to a virtual observatory

2 F. Genova, Open Archive Forum, 02/05/142 Why keeping data in astronomy? Long term observation of variable natural phenomena Major scientific objectives: –Variability, evolution, statistics –Observations at different wavelengths, with different techniques the physical phenomena at work

3 F. Genova, Open Archive Forum, 02/05/143 Large projects Large ground- and space-based observatories Large surveys Optimize the scientific return of big science by increasing usage in general 1year proprietary period journals: 3 years but abstracts and TOC free

4 F. Genova, Open Archive Forum, 02/05/144 A few figures: catalogues (1) Hipparcos (2 nd cent. BC), Almagest (2 nd AD) mid-19th, early 20 th : the first very large catalogues –Bonner Durchmusterung (1859-1862) 325 000 objects –Southern Durchmusterung (1886,…) 135 000 objects –Cape Photographic Durchmusterung (1895-1900) 455 000 objects

5 F. Genova, Open Archive Forum, 02/05/145 A few figures: catalogues (2) HIPPARCOS 100 000 stars, better positions Guide Star Catalogue (GSC) 1992: 20 000 000 *; 2001: 456 000 000* USNO A1, A2 500 000 000 stars

6 F. Genova, Open Archive Forum, 02/05/146 The very large surveys Sloan Digital Sky Survey 100 000 000 objects, 5 colors, π sr, z~5 DENIS, 2MASSinfrared 3 colors, 100 000 000 objects Catalogues and surveys http://vizier.u-strasbg.fr/viz-bin/VizieR

7 F. Genova, Open Archive Forum, 02/05/147 VLT ISAAC

8 F. Genova, Open Archive Forum, 02/05/148 ISO 0.6m telescope In a cryostat! 4 instruments

9 F. Genova, Open Archive Forum, 02/05/149 An increasingly complex problem Volume and complexity increase Distributed, heterogeneous information Observations, results published in journals, compilations, software, models, … Re-use data data documentation

10 F. Genova, Open Archive Forum, 02/05/1410 A major technical revolution Increased capacity to store and manage information Irruption of the WWW –Information easily accessible –Data/documentation integration –Navigation between distributed, heterogeneous information

11 F. Genova, Open Archive Forum, 02/05/1411 Useful and appealing tools But Careful work on contents and functionalities remains mandatory Information validation is critical producers and users! Maintenance of services and links

12 F. Genova, Open Archive Forum, 02/05/1412 New constraints On Agencies Data conservation, diffusion versus instrumentation, operations On the scientific community Give a sufficient priority in projects, evaluation and prospective Resources needed (scientists, engineers) On projects Available and usable data selection, organisation, « project memory »

13 F. Genova, Open Archive Forum, 02/05/1413 Networking of information in astronomy A small discipline Few commercial constraints A long term partnership to define exchange standards

14 F. Genova, Open Archive Forum, 02/05/1414 Networking of astronomical information Observations Ground- and space-based observatories Surveys Results Publications in electronic journals

15 F. Genova, Open Archive Forum, 02/05/1415 The actors Data producers Know instruments and methods preserve usable data and project memory Disciplinary centers Data and information in a given domain Journals Results Data Centers

16 F. Genova, Open Archive Forum, 02/05/1416 Centre de Données astronomiques de Strasbourg Created in 1972 –Electronic data –Expertise about data –International role –Objective: tools for science Collect, homogenize, preserve, distribute astronomical information for the usage of the whole astronomy community

17 F. Genova, Open Archive Forum, 02/05/1417 Centre de Données astronomiques de Strasbourg Value-added, reference services Selected, validated, homogenized information frompublications reference catalogues images + links Standards and tools

18 F. Genova, Open Archive Forum, 02/05/1418 The CDS hub (1) Astronomical objects identification, bibliography, data, measurements Information Federator catalogues, published tables observation logs, surveys Information Integrator images, databases, catalogues, surveys, archives, user data

19 F. Genova, Open Archive Forum, 02/05/1419 The CDS hub (2) Observations Results

20 F. Genova, Open Archive Forum, 02/05/1420 Query types Search by position What is known in a region of the sky with links to distributed information Search by criteria, e.g. –gamma ray burst seen in 1999 –published lists of objects from Chandra

21 F. Genova, Open Archive Forum, 02/05/1421 Early interoperability tools FITS- description of images, spectra - common tools to deal with observations from any telescope - maintained by an IAU WG NED and SIMBAD name resolvers

22 F. Genova, Open Archive Forum, 02/05/1422 Acces to observatory archives: by position Object name > position: Using SIMBAD

23 F. Genova, Open Archive Forum, 02/05/1423 Networking The astronomy bibliographic network Links between –electronic journals (common keyword schema) –the ADS bibliographic database –on-line services (SIMBAD, NED) –archival data bibcodee.g. 1999A&A...351.1003G

24 F. Genova, Open Archive Forum, 02/05/1424 References in an on-line article links to ADS quality check

25 F. Genova, Open Archive Forum, 02/05/1425 ADS - Abstracts - Scanned articles - Links to original on-line paper - Links to other, distributed information e.g. original observations - Also-read articles

26 F. Genova, Open Archive Forum, 02/05/1426 HST archive From observation To publication

27 F. Genova, Open Archive Forum, 02/05/1427 A&A on line From publication to SIMBAD and more

28 F. Genova, Open Archive Forum, 02/05/1428 Simbad Information About the object

29 F. Genova, Open Archive Forum, 02/05/1429 Links To high energy observations in HEASARC, To Catalogues

30 F. Genova, Open Archive Forum, 02/05/1430 Lessons learnt (1) De facto standard Bottom-up approach: Cooperation between all the actors + snowball effect Journals, ADS, data centers, archive centers Easy-to-build link but contents / validation are fundamental

31 F. Genova, Open Archive Forum, 02/05/1431 Lessons learnt (2) Standard discussed long before Internet, because two data bases needed to exchange bibliographic information Bibcode is better adapted to classical publication in journals, human readable Gateway to other bibliographical standards (e.g. DOI): correspondance table

32 F. Genova, Open Archive Forum, 02/05/1432 Data federation Tabular data in astronomy A common description for tabular data –Reference catalogues –Published tables –Large surveys –Catalogues of observations in archives ReadMe physical organization contents

33 F. Genova, Open Archive Forum, 02/05/1433 ReadMe For published tables: check of data homogeneity before publication sent by authors and checked or prepared by editorial office

34 F. Genova, Open Archive Forum, 02/05/1434 Tables published in articles are usable data!

35 F. Genova, Open Archive Forum, 02/05/1435 Narrow collaboration between CDS and journals

36 F. Genova, Open Archive Forum, 02/05/1436 The tabular data mine

37 F. Genova, Open Archive Forum, 02/05/1437 Links to observational data (images, spectra, time series), often distributed in observatory archives

38 F. Genova, Open Archive Forum, 02/05/1438 Towards data integration XML Questions –Minimize the tagging overhead –Management of binary blobs –Assess/distribute information about data quality –Unit knowledge/conversion Astrores, then VOTable

39 F. Genova, Open Archive Forum, 02/05/1439 HEARSARC Browse Astrores in action!

40 F. Genova, Open Archive Forum, 02/05/1440 OASIS

41 F. Genova, Open Archive Forum, 02/05/1441 Data integration with NGC 5236 DSS image HST observation FOV SIMBAD and NED GSC, USNO A2 IUE observations

42 F. Genova, Open Archive Forum, 02/05/1442 ALADIN access to distributed information Distributed –Image databases –Catalogues Predefined –Supercosmos –HEASARC SkyView User defined

43 F. Genova, Open Archive Forum, 02/05/1443 The Virtual Observatory « an enabling and coordinating entity to foster the development of tools, protocols, and collaborations necessary to realize the full scientific potential of astronomical databases in the coming decade » NVO White Paper, juin 2000 Science driven

44 F. Genova, Open Archive Forum, 02/05/1444 VO components (1) Network infrastructure GRID (computer AND data GRID) High capacity storage Management of very large data sets Access to distributed information Information retrieval tools Data Mining Statistical tools on very large distributed data sets

45 F. Genova, Open Archive Forum, 02/05/1445 Virtual Observatory projects A priority in Europe (OPTICON) and USA (Decadal Survey) Several conferences –Virtual Observatories of the future (CalTech) –Mining the sky (Garching) –AstroGRID workshop (Belfast) Europe: AVO; USA: NVO; AstroGRID; Australia; Germany; …

46 F. Genova, Open Archive Forum, 02/05/1446 VO components (2) Interoperability –Access to data –Query result integration Exchange standards Metadata Open question: define rules allowing people to deposit their data in the VO (quality?)

47 F. Genova, Open Archive Forum, 02/05/1447 EC RTD project PIP. Quinn (ESO) ESO, ESA, AstroGRID, CDS, Terapix, Jodrell Bank Phase A proposal Shared cost project The project (1)

48 F. Genova, Open Archive Forum, 02/05/1448 Three work areas Science Use Cases and Requirements (ST-ECF) cf Interoperability deployment and demonstration Technology needs (AstroGRID) –GRID systems –Scalable storage and computation –Databases Theproject (2)

49 F. Genova, Open Archive Forum, 02/05/1449 Interoperability prototype Interoperability deployment and demonstration Ground- and space, multiwavelength, multitechnique archives Tools VizieR (data federation)/Aladin (data integration) cross-id… ESO/CDS Data Mining Project

50 F. Genova, Open Archive Forum, 02/05/1450 Interoperability prototype Products Working prototype open for community usage Running of science cases for evaluation Science driven: results, feedback ! Improvement of federation and integration tools Counsel to archives on interoperability implementation Standards

51 F. Genova, Open Archive Forum, 02/05/1451 Interoperability WG Membership: Data managers –Study cost effective tools and standards for improving access and data exchange to/from data archives and information services (minimal workload) –Discuss the technical results of the AVO interoperability prototyping –optical/IR (EC OPTICON), radio (EC RadioNet) –International partnership beyond EU USA, Canada, Australia

52 F. Genova, Open Archive Forum, 02/05/1452 The International VO alliance Coordination between the projects A common roadmap First milestone (April 15 th, 2002) VOTable V1.0 VO meeting, Garching, June 2002

53 F. Genova, Open Archive Forum, 02/05/1453 Information retrieval Where is the data of interest for me? Practical Objective Define a minimal and controlled set of metadata To be done with archive/service managers AstroBrowse/AstroGLU; ISAIA –Knowledge/maintenance of query syntax GLU (Générateur de Liens Uniformes)

54 F. Genova, Open Archive Forum, 02/05/1454 The GLU GLU: a distributed/mirrored service directory –31 synchronized repositories, 11 countries, 5 providers GLU can build queries by adjusting automatically the syntax as required by services (coordinate conversion, splitted fields) GLU is a toolkit managing HTML pages with dynamically updated links –dealing with mirror sites –dealing with keywords describing resources Hot topic: GLU vs WSDL/UDDI

55 F. Genova, Open Archive Forum, 02/05/1455 Information retrieval (2) Proposal for query syntax ASUAstronomical Server URL Proposal for contents description UCDUniform Content Descriptor Data quality Avoid illegitimate usage of data

56 F. Genova, Open Archive Forum, 02/05/1456 UCDs = Uniform Content Descriptor Developed in the frame of the ESO/CDS Data Mining project Hierarchical knowledge tree (2 000 items) Implemented for the 100,000 columns in VizieR (cats, publ. tables, logs, surveys) To be tested and extended: access tool

57 F. Genova, Open Archive Forum, 02/05/1457 AVO/Astrogrid/NVO common collaborations VOTable (VOData as generalization?) XML schemas Metadata issues Query mechanisms Data service/resource directories Data service/resource capabilities Metadata dictionary Data models Identity, authentication Science scenarios User interfaces, portals?? AVO/Astrogrid/NVO common milestones List of topics

58 F. Genova, Open Archive Forum, 02/05/1458 Conclusion Astronomy is at the forefront for data diffusion and networking The on-line services are everyday tools for scientists Building an international virtual observatory is an important endeavour for the coming years Partnership is critical Discipline standard/tools vs generic ones is a constant debate

Download ppt "F. Genova, Open Archive Forum, 02/05/141 On-line information in astronomy From networking to a virtual observatory."

Similar presentations

Registries Work Package 2 Requirements, Science Cases, Use Cases, Test Cases Charter: Focus on science case scenarios, and use cases related specifically.

Registries Work Package 2 Requirements, Science Cases, Use Cases, Test Cases Charter: Focus on science case scenarios, and use cases related specifically.
London Registry meeting 19–20 March 2003 André Schaaff GLU – Générateur de Liens Uniformes Three sides : –GLU dictionary maintaining URLs according to.

London Registry meeting 19–20 March 2003 André Schaaff GLU – Générateur de Liens Uniformes Three sides : –GLU dictionary maintaining URLs according to.
A VO experience: Lessons learnt from accessing SIA compliant image servers with the AVO prototype F.Bonnarel on behalf of M.Allen, T.Boch, M.Dolensky,

A VO experience: Lessons learnt from accessing SIA compliant image servers with the AVO prototype F.Bonnarel on behalf of M.Allen, T.Boch, M.Dolensky,
OAF Workshop, May 13-14, 2002, Pisa.CYCLADES IST-2000-25456 CYCLADES An Open Collaborative Virtual Archive Environment www.ercim.org/cyclades Umberto Straccia.

OAF Workshop, May 13-14, 2002, Pisa.CYCLADES IST CYCLADES An Open Collaborative Virtual Archive Environment Umberto Straccia.
September 13, 2004NVO Summer School1 VO Protocols Overview Tom McGlynn NASA/GSFC T HE US N ATIONAL V IRTUAL O BSERVATORY.

September 13, 2004NVO Summer School1 VO Protocols Overview Tom McGlynn NASA/GSFC T HE US N ATIONAL V IRTUAL O BSERVATORY.
8 September 2008NVO Summer School 2008 – Santa Fe1 Publishing Data and Services to the VO Ray Plante Gretchen Greene T HE US N ATIONAL V IRTUAL O BSERVATORY.

8 September 2008NVO Summer School 2008 – Santa Fe1 Publishing Data and Services to the VO Ray Plante Gretchen Greene T HE US N ATIONAL V IRTUAL O BSERVATORY.
CASDA Virtual Observatory CSIRO ASTRONOMY AND SPACE SCIENCE Arkadi Kosmynin 11 March 2014.

CASDA Virtual Observatory CSIRO ASTRONOMY AND SPACE SCIENCE Arkadi Kosmynin 11 March 2014.
ESO-ESA Existing Activities Archives, Virtual Observatories and the Grid.

ESO-ESA Existing Activities Archives, Virtual Observatories and the Grid.
BPS - 5th Ed. Chapter 141 Introduction to Inference.

BPS - 5th Ed. Chapter 141 Introduction to Inference.
CS 245Notes 141 CS 245: Database System Principles Notes 14: Coping with Limited Capabilities of Sources Hector Garcia-Molina.

CS 245Notes 141 CS 245: Database System Principles Notes 14: Coping with Limited Capabilities of Sources Hector Garcia-Molina.
Why, what were the idea ? 1.Create a data infrastructure, 2.Data + the knowledge products that are produced on the basis of data a) Efficiant access to.

Why, what were the idea ? 1.Create a data infrastructure, 2.Data + the knowledge products that are produced on the basis of data a) Efficiant access to.
F. Genova, Strategy, CDS SC meeting, 2011/06/29-30 Strategy.

F. Genova, Strategy, CDS SC meeting, 2011/06/29-30 Strategy.
Chapter 2. Slide 1 CULTURAL SUBJECT GATEWAYS CULTURAL SUBJECT GATEWAYS Subject Gateways  Started as links of lists  Continued as Web directories  Culminated.

Chapter 2. Slide 1 CULTURAL SUBJECT GATEWAYS CULTURAL SUBJECT GATEWAYS Subject Gateways  Started as links of lists  Continued as Web directories  Culminated.
Desperately Trying to Cope with the Data Explosion in Astronomical Sciences Ray Norris CSIRO Australia Telescope National Facility.

Desperately Trying to Cope with the Data Explosion in Astronomical Sciences Ray Norris CSIRO Australia Telescope National Facility.
14 October 2003ADASS 2003 – Strasbourg1 Resource Registries for the Virtual Observatory R.Plante (NCSA), G. Greene (STScI), R. Hanisch (STScI), T. McGlynn.

14 October 2003ADASS 2003 – Strasbourg1 Resource Registries for the Virtual Observatory R.Plante (NCSA), G. Greene (STScI), R. Hanisch (STScI), T. McGlynn.
Long-Term Preservation of Astronomical Research Results Robert Hanisch US National Virtual Observatory Space Telescope Science Institute Baltimore, MD.

Long-Term Preservation of Astronomical Research Results Robert Hanisch US National Virtual Observatory Space Telescope Science Institute Baltimore, MD.
Long-Term Preservation of Astronomical Research Results Robert Hanisch US National Virtual Observatory Space Telescope Science Institute Baltimore, MD.

Long-Term Preservation of Astronomical Research Results Robert Hanisch US National Virtual Observatory Space Telescope Science Institute Baltimore, MD.
GAUDI Ground-based Asteroseismology Uniform Database Interface E. Solano Bases de données en spectroscopie stellaire. Paris.

GAUDI Ground-based Asteroseismology Uniform Database Interface E. Solano Bases de données en spectroscopie stellaire. Paris.
VOTable: Tabular Data for Virtual Observatory François Ochsenbein Roy Williams Clive Davenhall, Daniel Durand, Pierre Fernique, Robert Hanisch, David Giaretta,

VOTable: Tabular Data for Virtual Observatory François Ochsenbein Roy Williams Clive Davenhall, Daniel Durand, Pierre Fernique, Robert Hanisch, David Giaretta,
S. Derriere et al., ESSW03 Budapest, 2003 May 20 UCDs - metadata for astronomy Sébastien Derriere François Ochsenbein Thomas Boch CDS, Observatoire astronomique.

S. Derriere et al., ESSW03 Budapest, 2003 May 20 UCDs - metadata for astronomy Sébastien Derriere François Ochsenbein Thomas Boch CDS, Observatoire astronomique.

Similar presentations

Ads by Google