Metadata in the TAP context (1) The Problem: learn about which tables, tablesets,... are available from a TAP server for each of the tables / tablesets,

Slides:



Advertisements
Similar presentations
Registries Work Package 2 Requirements, Science Cases, Use Cases, Test Cases Charter: Focus on science case scenarios, and use cases related specifically.
Advertisements

May 18, 2006IVOA Interoperability Meeting Fine-grained vs. Coarse-grained Registries or How much detail about a resource should be stored in a registry?
28 October 2008 IVOA Interoperability Meeting -- Baltimore T HE I NTERNATIONAL V IRTUAL O BSERVATORY ALLIANCE TAP/VOTable Registry Interface Reg 1 – G.
SSA Query Interface M. Dolensky, ESO Data Access Layer Working Group Interoperability Workshop, Pune, India 27-Sep-2004.
SimDB as a TAP service various TIG members (IVOA.IVOATheorySimDB)IVOA.IVOATheorySimDB.
September 13, 2004NVO Summer School1 VO Protocols Overview Tom McGlynn NASA/GSFC T HE US N ATIONAL V IRTUAL O BSERVATORY.
September 13, 2004NVO Summer School1 VO Protocols Overview Tom McGlynn NASA/GSFC T HE US N ATIONAL V IRTUAL O BSERVATORY.
DB glossary (focus on typical SQL RDBMS, not XQuery or SPARQL)
Data Definition and Integrity Constraints
Chapter 10: Designing Databases
UCD inheritance in VOTables François Ochsenbein. 11 May 2003 François Ochsenbein Summary 1. Alternative propositions 2. impact of the notion of column.
1Key – Report Creation with DB2. DB2 Databases Create Domain for DB2 Test Demo.
Copyright © 2011 Pearson Education, Inc. Publishing as Pearson Addison-Wesley Chapter 3 The Basic (Flat) Relational Model.
Chapter 3. 2 Chapter 3 - Objectives Terminology of relational model. Terminology of relational model. How tables are used to represent data. How tables.
1 Lecture 13: Database Heterogeneity Debriefing Project Phase 2.
1 Relational Algebra and Calculus Yanlei Diao UMass Amherst Feb 1, 2007 Slides Courtesy of R. Ramakrishnan and J. Gehrke.
The Relational Model Codd (1970): based on set theory Relational model: represents the database as a collection of relations (a table of values --> file)
IMS1907 Database Systems Summer Semester 2004/2005 Lecture 2 Relational DBMS Software An Overview of Microsoft Access.
RIZWAN REHMAN, CCS, DU. Advantages of ORDBMSs  The main advantages of extending the relational data model come from reuse and sharing.  Reuse comes.
Indexing XML Data Stored in a Relational Database VLDB`2004 Shankar Pal, Istvan Cseri, Gideon Schaller, Oliver Seeliger, Leo Giakoumakis, Vasili Vasili.
Midterm 1 Concepts Relational Algebra (DB4) SQL Querying and updating (DB5) Constraints and Triggers (DB11) Unified Modeling Language (DB9) Relational.
3 The Relational Model MIS 304 Winter Class Objectives That the relational database model takes a logical view of data That the relational model’s.
CSC2012 Database Technology & CSC2513 Database Systems.
T HE I NTERNATIONAL V IRTUAL O BSERVATORY ALLIANCE VAO Registry Relational Schema: Updates and New Interface(s) Theresa Dower Registry WG 16 May 2013 IVOA.
VOTable: Tabular Data for Virtual Observatory François Ochsenbein Roy Williams Clive Davenhall, Daniel Durand, Pierre Fernique, Robert Hanisch, David Giaretta,
The GAVO Cross-Matcher Application Hans-Martin Adorf, Gerard Lemson, Wolfgang Voges GAVO, Max-Planck-Institut für extraterrestrische Physik, Garching b.
DateADASS How to Navigate VO Datasets Using VO Protocols Ray Plante (NCSA/UIUC), Thomas McGlynn and Eric Winter NASA/GSFC T HE US N ATIONAL V IRTUAL.
1 11/3/05CS360 Windows Programming Databases and Data Representation.
FEN  Concepts and terminology  Operations (relational algebra)  Integrity constraints The relational model.
Michael Cafarella Alon HalevyNodira Khoussainova University of Washington Google, incUniversity of Washington Data Integration for Relational Web.
Copyright © Curt Hill The Relational Model of Database Basic organization and terms.
Lecture2: Database Environment Prepared by L. Nouf Almujally & Aisha AlArfaj 1 Ref. Chapter2 College of Computer and Information Sciences - Information.
Databases. Not All Tables Are Created Equal Spreadsheets use tables to store data and formulas associated with that data The “meaning” of data is implicit.
 Agenda 2/20/13 o Review quiz, answer questions o Review database design exercises from 2/13 o Create relationships through “Lookup tables” o Discuss.
FEN Introduction to the database field:  The Relational Model Seminar: Introduction to relational databases.
Databases Shortfalls of file management systems Structure of a database Database administration Database Management system Hierarchical Databases Network.
Prepared By Prepared By : VINAY ALEXANDER ( विनय अलेक्सजेंड़र ) PGT(CS),KV JHAGRAKHAND.
T-SQL: Simple Changes That Go a Long Way DAVE ingeniousSQL.com linkedin.com/in/ingenioussql.
30 October 2008 IVOA Interoperability Meeting -- Baltimore T HE I NTERNATIONAL V IRTUAL O BSERVATORY ALLIANCE VOTable interface with Registry Joint Apps/DM/Registry.
CSE314 Database Systems Lecture 3 The Relational Data Model and Relational Database Constraints Doç. Dr. Mehmet Göktürk src: Elmasri & Navanthe 6E Pearson.
IS201 Agenda: 09/19  Modify contents of the database.  Discuss queries: Turning data stored in a database into information for decision making.  Create.
The relational model A data model (in general) : Integrated collection of concepts for describing data (data requirements). Relational model was introduced.
UCL DEPARTMENT OF SPACE AND CLIMATE PHYSICS MULLARD SPACE SCIENCE LABORATORY Taverna Plugin VAMDC and HELIO (part of the ‘taverna-astronomy’ edition) Kevin.
Discussed in Kyoto Schema changes for the next version (Gerard Lemson)  will be included in VOTable1.2 Schema changes for the next version (Gerard Lemson)
The Relational Model. 2 Relational Model Terminology u A relation is a table with columns and rows. –Only applies to logical structure of the database,
20 Copyright © 2008, Oracle. All rights reserved. Cache Management.
VOEvent and the Registry Introducing VOEventStream and VOEventService Roy Williams Caltech.
VOTable agenda Current VOTable status News from Applications Referring STC (as a data model example) Relations between s Questions about VOTable schema.
Jennifer Widom Relational Databases The Relational Model.
 Review quiz. Answer questions.  Discuss queries: ◦ What is a query? Turning data stored in a database into information for decision making. ◦ You: Completed.
FEN Introduction to the database field: The development process Seminar: Introduction to relational databases Development process: Analyse.
SEMI-STRUCTURED DATA (XML) 1. SEMI-STRUCTURED DATA ER, Relational, ODL data models are all based on schema Structure of data is rigid and known is advance.
Lecture 5 Data Model Design Jeffery S. Horsburgh Hydroinformatics Fall 2012 This work was funded by National Science Foundation Grant EPS
Rationale Databases are an integral part of an organization. Aspiring Database Developers should be able to efficiently design and implement databases.
IT 5433 LM3 Relational Data Model. Learning Objectives: List the 5 properties of relations List the properties of a candidate key, primary key and foreign.
Introduction to Database Development CH2. CH2. Introduction to DB Development Database n Components of Database Systems (Figure 2-1) –User data –Metadata.
Is non-unicity of utype/ucds a problem? The problem: is it possible to query some IVOA database directly from UCDs or Utypes? Select * from ivoa_table.
Teacher Workshop Database Design Pearson Education © 2014.
Chapter 2: Relational Model
Lecture 2 The Relational Model
Associative Query Answering via Query Feature Similarity
The Relational Model Textbook /7/2018.
Data Model.
Contents Preface I Introduction Lesson Objectives I-2
Query Functions.
Chapter 2: Intro to Relational Model
Relational Database Design
Chapter 2: Intro to Relational Model
Utypes for Model Referencing
Presentation transcript:

Metadata in the TAP context (1) The Problem: learn about which tables, tablesets,... are available from a TAP server for each of the tables / tablesets, discover their contents: descriptions, characterisation, keywords relations between tables (keys) statistics (rows, region covered) column contents: description, units, utypes, UCDs, datatypes, domains,... part or all is known / cached in Registries

Metadata in the TAP context (2) What are metadata ? Essentially 3 parts: semantics (science metadata): describes role of tables (e.g. observational data, model results, etc) and column contents (units, detailed explanations, UCDs, utypes, references...) logistics (implementation metadata): organisation, datatypes, primary/foreign keys, unicity, domain... statistics (statistical properties): summary of table(set)s like row counts, sky regions for source catalogs, column min/max/,...

Why a metadata access ? Find out table(set)s that could match my requirements. Typical examples: which table(set)s contain redshifts in a given region of the sky ? (statistics) which table delivers calibrated fluxes in some IR band (semantics) Some parts (almost all?) exist in Registries – is it necessary to access metadata at the TAP server ? YES because: more up-to-date (delay of propagation to Registries) metadata generally richer than what's in the Registry

How to access metadata ? Essentially two possible ways: by methods (functions) which return some IVOA- defined object (e.g. VOResource objects or identifiers). Several methods may exist: getResource(), keywordSearch(),... by a set of (system) tables similar to what is defined in the INFORMATION_SCHEMA can be queried like other tables can accept complex constraints

INFORMATION_SCHEMA (1) is a standardized system tableset introduced in SQL92 typical contents: information_schema.schemata = list of tablesets information_schema.tables = list of tables information_schema.columns = list of columns information_views = list of views (stored queries) etc... information_schema.tables = list of tablesinformation_schema.tables = list of tables

INFORMATION_SCHEMA (2) BUT does not exist in all relational DBs not identical among different DBs is aimed to describe DB schema only (tables, columns, keys, protections,...), i.e. logistics most important (semantics) parameters are missing! For example: no notion of units, UCDs, etc... in columns no notion of date of last modification in tables information_schema.tables = list of tablesinformation_schema.tables = list of tables

INFORMATION_SCHEMA (3) Therefore... information_schema alone can't be the solution! alternatives: definition of some IVOA standard set of meta tables containing the required 3 sets of metadata, similar to what exists in VizieR (METAcat, METAtab, METAcol) define tables which complement what exists in information_schema (many joins!)... keep just the methods... information_schema.tables = list of tablesinformation_schema.tables = list of tables

Metadata Query Output Result of a search for tables satisfying some constraint may be returned in several forms: a list of tuples of type information_schema.tables (or IVOA-defined table of tables) expressed as a VOTable (each row describes one table) a set of VOResource identifiers or objects (limited set of metadata!) empty VOTable(s) with/without all FIELD (s)