IVOA Interoperalibity 20031 JVO Query Language Naoki Yasuda (NAOJ/Japanese VO)

Slides:



Advertisements
Similar presentations
SSA Query Interface M. Dolensky, ESO Data Access Layer Working Group Interoperability Workshop, Pune, India 27-Sep-2004.
Advertisements

A.Micol IVOA Registry REGISTRY WG Mar 2003 A Science Case (and 1000 Questions) for the IVOA Registry.
September 13, 2004NVO Summer School1 VO Protocols Overview Tom McGlynn NASA/GSFC T HE US N ATIONAL V IRTUAL O BSERVATORY.
Metadata in the TAP context (1) The Problem: learn about which tables, tablesets,... are available from a TAP server for each of the tables / tablesets,
CASDA Virtual Observatory CSIRO ASTRONOMY AND SPACE SCIENCE Arkadi Kosmynin 11 March 2014.
1 If we modify the machine for the language from F12 p. 47 we can easily construct a machine for the language Observation Turing machine for the language.
1Key – Report Creation with DB2. DB2 Databases Create Domain for DB2 Test Demo.
Construction of the Japanese Virtual Observatory (JVO) Abstract : The National Astronomical Observatory of Japan (NAOJ) has been operating several large.
László Dobos 1,2, Tamás Budavári 2, Nolan Li 2, Alex Szalay 2, István Csabai 1 1 Eötvös Loránd University, Budapest,
0 General information Rate of acceptance 37% Papers from 15 Countries and 5 Geographical Areas –North America 5 –South America 2 –Europe 20 –Asia 2 –Australia.
Design and Implementation of JVO SkyNode Yuji SHIRASAKI National Astronomical Observatory of Japan IVOA Small Projects Meeting, 2004 Oct 1.
20 Spatial Queries for an Astronomer's Bench (mark) María Nieto-Santisteban 1 Tobias Scholl 2 Alexander Szalay 1 Alfons Kemper 2 1. The Johns Hopkins University,
3-1 Chapter 3 Data and Knowledge Management
A Guide to MySQL 7. 2 Objectives Understand, define, and drop views Recognize the benefits of using views Use a view to update data Grant and revoke users’
A Guide to SQL, Seventh Edition. Objectives Understand, create, and drop views Recognize the benefits of using views Grant and revoke user’s database.
Copyright © 2011 Pearson Education, Inc. Publishing as Pearson Addison-Wesley Chapter 2 Overview of Database Languages and Architectures.
TAP service at ESAC - VOTAP Carlos Rios Diaz Science Archives Team (SAT) ESA-ESAC, Madrid, Spain VOTAP.
1 Distributed and Parallel Databases. 2 Distributed Databases Distributed Systems goal: –to offer local DB autonomy at geographically distributed locations.
Supported by the National Science Foundation’s Information Technology Research Program under Cooperative Agreement AST with The Johns Hopkins University.
László Dobos, Tamás Budavári, Alex Szalay, István Csabai Eötvös University / JHU Aug , 2008.IDIES Inaugural Symposium, Baltimore1.
11/27/2003IVOA Small Projects Meeting China-VO Data Access Service Based on OGSA Jian Sang National Astronomical Observatory of China Chinese Virtual.
The Japanese Virtual Observatory (JVO) Yuji Shirasaki National Astronomical Observatory of Japan.
Astronomical Data Query Language Simple Query Protocol for the Virtual Observatory Naoki Yasuda 1, William O'Mullane 2, Tamas Budavari 2, Vivek Haridas.
Database System Concepts and Architecture Lecture # 2 21 June 2012 National University of Computer and Emerging Sciences.
Yuji SHIRASAKI National Astronomical Observatory of Japan Current Status of Japanese Virtual Observatory (JVO)
2. Database System Concepts and Architecture
Prototype system of the Japanese Virtual Observatory The Japanese Virtual Observatory (JVO) aims at providing easy access to federated astronomical databases.
JVO JVO Portal Japanese Virtual Observatory (JVO) Prototype 2 Masahiro Tanaka, Yuji Shirasaki, Satoshi Honda, Yoshihiko Mizumoto, Masatoshi Ohishi (NAOJ),
RELATIONAL FAULT TOLERANT INTERFACE TO HETEROGENEOUS DISTRIBUTED DATABASES Prof. Osama Abulnaja Afraa Khalifah
Relational Databases Database Driven Applications Retrieving Data Changing Data Analysing Data What is a DBMS An application that holds the data manages.
Database Design and Management CPTG /23/2015Chapter 12 of 38 Functions of a Database Store data Store data School: student records, class schedules,
MIS 327 Database Management system 1 MIS 327: DBMS Dr. Monther Tarawneh Dr. Monther Tarawneh Week 2: Basic Concepts.
Dimu' Rumpak © 2009 by Prentice Hall 1 Getting Started Didimus Rumpak, M.Si. Database Concepts Chapter 1 1.
P Structured Query Language for Virtual Observatory Yuji Shirasaki National Astronomical Observatory of Japan, and Masahiro Tanaka (NAOJ), Satoshi.
1 Computing Challenges for the Square Kilometre Array Mathai Joseph & Harrick Vin Tata Research Development & Design Centre Pune, India CHEP Mumbai 16.
VO Query Language GSFC XML Group Ed Shaya Brian Thomas Kirk Borne.
Oct. 1, 2004IVOA Small Projects Meeting1 Development of JVO prototype system and its application to Astrophysics Portal System : M. Tanaka Data Service.
EASYASK NLP to SQL Translator Aditya Khandekar Aditya Khandekar Ashish Jain Ashish Jain Kunal Dabir Kunal Dabir Praveen Awasthy Praveen Awasthy.
Japanese Virtual Observatory Project Abstract : The National Astronomical Observatory of Japan (NAOJ) started the Japanese Virtual Observatory (JVO) project.
Distributed database system
Copyright © 2011 Pearson Education, Inc. Publishing as Pearson Addison-Wesley Chapter 2 Database System Concepts and Architecture.
Progress Report of VOQL WG May 15 (Thu) Masatoshi Ohishi (Japan)
ADQL/s Syntax (Proposal) -- towards unification of ADQL, SIAP, SSAP, SXAP Yuji SHIRASAKI National Astronomical Observatory.
The VAO is operated by the VAO, LLC. Scalable Cross-Comparison Requirements Review.
Japanese Virtual Observatory Project Abstract : The National Astronomical Observatory of Japan (NAOJ) started the Japanese Virtual Observatory (JVO) project.
Japanese Virtual Observatory (JVO) National Astronomical Observatory of Japan (NAOJ) Contact Address: Demo 1: GRID-based database federation.
Pan-STARRS PS1 Published Science Products Subsystem Presentation to the PS1 Science Council August 1, 2007.
12 Oct 2003VO Tutorial, ADASS Strasbourg, Data Access Layer (DAL) Tutorial Doug Tody, National Radio Astronomy Observatory T HE US N ATIONAL V IRTUAL.
JVO portal service Yuji Shirasaki National Astronomical Observatory of Japan.
William O’Mullane/ Tannu Malik - JHU IVOA Cambridge May 12-16, 2003 SkyQuery.Net SKYQUERY Federated Database Query System (using WebServices)
May 2006IVOA Victoria, Canada1 VOQL Where do we stand? What is left? Yuji Shirasaki JVO Maria A. Nieto-Santisteban JHU T HE US N ATIONAL V IRTUAL O BSERVATORY.
Three color composite image Color-Color diagram SED SED relative to R-band Download FITS Searching for a cosmic string through the gravitational lens effect:
Annotation of “special structures” in astronomy Bob Mann Institute for Astronomy and National e-Science Centre University of Edinburgh.
IVOA Small Projects Meeting Application to the science S. Honda, Y. Shirasaki, M. Tanaka and JVO team National Astronomical Observatory of Japan.
2004/03/08Japan-France Grid Computing Workshop in Paris Japanese Virtual Observatory Project Masatoshi Ohishi National Astronomical Observatory of Japan.
ISC321 Database Systems I Chapter 2: Overview of Database Languages and Architectures Fall 2015 Dr. Abdullah Almutairi.
Introduction to Spatial Computing CSE 555
Databases (CS507) CHAPTER 2.
Standard Query Language for VO
Cross-matching the sky with database server cluster
Sky Query: A distributed query engine for astronomy
Copyright © 2011 Pearson Education, Inc. Publishing as Pearson Addison-Wesley Chapter 2 Database System Concepts and Architecture.
May 14 (Wed) Masatoshi Ohishi (Japan)
Database.
2018/11/23 An Application for the Data Grid : A Prototype of the Japanese Virtual Observatory System Masatoshi Ohishi National Astronomical Observatory.
Current Status of Japanese Virtual Observatory
2018/11/27 An Application for the Data Grid : A Prototype of the Japanese Virtual Observatory System Masatoshi Ohishi National Astronomical Observatory.
Progress Report of VOQL WG
Google Sky.
Efficient Catalog Matching with Dropout Detection
Presentation transcript:

IVOA Interoperalibity JVO Query Language Naoki Yasuda (NAOJ/Japanese VO)

IVOA Interoperalibity Basic Functions of JVOQL Query and retrieve required data –Describe query condition –Describe what to retrieve –Describe analyses on retrieved data Federate distributed databases –Describe queries for multi databases in one unit Federate catalogs (tables) and images –Need not to discriminate catalogs and images

IVOA Interoperalibity Requirement for JVOQL Platform independent –VO will be constructed as heterogeneous system –Query must be transferable among systems Sufficient to describe astronomical queries –QL must describe what astronomers want to do Easy to understand for human and machine –Necessary for debugging and programming

IVOA Interoperalibity Format of JVOQL JVOQL adopted SQL as a base language –SQL is a well-defined query language optimized for relational database –SQL is easy to understand for human and machine –Backend queries can be done by DBMS Astronomy specific functions are defined as extensions for SQL –SQL is not sufficient enough for astronomical queries XMATCH (Cross Matching) AREA (Image cutout / Celestial area limit)

IVOA Interoperalibity Extension 1 : Cross-Matching Astronomical objects will be identified mainly based on its celestial position (and distance from the earth). Distributed databases can be related through celestial positions. Similar to ‘join’ in SQL –where A.id = B.id –where XMATCH(A, B) < 3 arcsec

IVOA Interoperalibity XMATCH XMATCH(A, B, !C, …) < 3 arcsec –Select objects whose celestial positions in catalog A and catalog B coincide within the tolerance of 3 arcsec but not found in catalog C. Catalog ACatalog B Catalog C

IVOA Interoperalibity Extension 2 : Image cutout In most cases, astronomers request images associated with objects selected from catalogs. –Cutout area will be defined as celestial area. Celestial area definition can be used to limit searching celestial area. Image cutout can be easily expressed like SQL –select A.ra, A.dec, A.BOX(POINT(A.ra, A.dec, J2000), 3 arcmin, 3 arcmin)

IVOA Interoperalibity AREA POINT(long, lat, coord) will define a point on the celestial sphere. –coord could be J2000, B1950, GAL, … AREA will be defined based on POINT –BOX(POINT(…), width, height[, PA]) CIRCLE(POINT(…), radius) OVAL(POINT(…), rad1, rad2[, PA]) TRIANGLE(POINT(…), POINT(…), POINT(…))

IVOA Interoperalibity Sample Query create view myEROtable as select s.ra, s.dec, s.Rmag, t.Kmag, sr.BOX(POINT(s.ra, s.dec, J2000), 1 arcmin, 1 arcmin) as Rimage, tk.BOX(POINT(s.ra, s.dec, J2000), 1 arcmin, 1 arcmin) as Kimage from SUBARU s, 2MASS t, SUBARU.R sr, 2MASS.K tk where XMATCH(s,t) < 3 arcsec and (s.Rmag-t.Kmag) > 6 mag and BOX(POINT(201.0 deg, 27.4 deg, J2000), 1 deg, 1 deg)

IVOA Interoperalibity Sample Query create view myEROtable as select s.ra, s.dec, s.Rmag, t.Kmag, sr.BOX(POINT(s.ra, s.dec, J2000), 1 arcmin, 1 arcmin) as Rimage, tk.BOX(POINT(s.ra, s.dec, J2000), 1 arcmin, 1 arcmin) as Kimage from SUBARU s, 2MASS t, SUBARU.R sr, 2MASS.K tk where XMATCH(s,t) < 3 arcsec and (s.Rmag-t.Kmag) > 6 mag and BOX(POINT(201.0 deg, 27.4 deg, J2000), 1 deg, 1 deg) common constraint multiple DB constraints

IVOA Interoperalibity select count(*) from SUBARU s where BOX(POINT(201.0, 27.4, J2000), 1.0, 1.0) select count(*) from 2MASS t Where BOX(POINT(201.0, 27.4, J2000), 1.0, 1.0) count 1 count 2 First count the number of objects which satisfies common constraints According to the counts, server access sequence will determined count 1count 2 > 2MASS SUBARU count 1count 2 < 2MASS SUBARU

IVOA Interoperalibity select s.ra, s.dec, s.Rmag from SUBARU s where BOX(POINT(201.0, 27.4, J2000), 1.0, 1.0) select t.Kmag from 2MASS t where XMATCH(s,t) < 3 arcsec and (s.Rmag-t.Kmag) > 6 mag select sr.BOX(POINT(s.ra, s.dec, J2000),…) as Rimage from SUBARU.R sr select tk.BOX(POINT(s.ra, s.dec, J2000),…) as Kimage from 2MASS.K tk Output Table A Output Table B Output Table C Result Table

IVOA Interoperalibity Extension of AREA Usual queries are for catalogs There is a need for query for area like –Search for area where V band, I band, and K band observations are available. –Search for area where observed in B band more than 3 times and their intervals are more than 1 month. –… These queries will expand the usefulness of astronomical data archives.

IVOA Interoperalibity Sample Queries select s.a, t.a,... from SUBARU.R s, 2MASS.K t,... where (s.AREA() OVERLAP t.AREA()) as a select x.a, y.a from SUBARU.B x, SUBARU.B y where (DIFF(x.obs_date, Y.obs_date) > 30 days) and (s.AREA() OVERLAP t.AREA())) as a

IVOA Interoperalibity s.AREA()t.AREA() AREA info OVERLAP cutout request Divide into pieces st a1 a2 a3