TAP Standards and Feedback

Slides:



Advertisements
Similar presentations
Phoenix We put the SQL back in NoSQL James Taylor Demos:
Advertisements

Recommendations for a Table Access Protocol Ray Plante, Tamas Budavari, Gretchen Greene, John Goode, Tom McGlynn, Maria Nieto-Santistaban, Alex Szalay,
The VAO is operated by the VAO, LLC. VOSpace and TAP Deoyani Nandrekar-Heinis, Johns Hopkins University.
9 September 2005NVO Summer School Aspen Astronomical Dataset Query Language (ADQL) Ray Plante T HE US N ATIONAL V IRTUAL O BSERVATORY.
A Guide to SQL, Seventh Edition. Objectives Use joins to retrieve data from more than one table Use the IN and EXISTS operators to query multiple tables.
MULTIPLE-TABLE QUERIES
Keys, Referential Integrity and PHP One to Many on the Web.
CASJOBS: A WORKFLOW ENVIRONMENT DESIGNED FOR LARGE SCIENTIFIC CATALOGS Nolan Li, Johns Hopkins University.
ICS 421 Spring 2010 Performance Tuning Asst. Prof. Lipyeow Lim Information & Computer Science Department University of Hawaii at Manoa 4/15/20101Lipyeow.
Introduction to Structured Query Language (SQL)
Virtual techdays INDIA │ 9-11 February 2011 SQL 2008 Query Tuning Praveen Srivatsa │ Principal SME – StudyDesk91 │ Director, AsthraSoft Consulting │ Microsoft.
Database Systems More SQL Database Design -- More SQL1.
Introduction to Structured Query Language (SQL)
Introduction to Oracle9i: SQL1 Views. Introduction to Oracle9i: SQL2 Chapter Objectives Create a view, using CREATE VIEW command or the CREATE OR REPLACE.
Architecting a Large-Scale Data Warehouse with SQL Server 2005 Mark Morton Senior Technical Consultant IT Training Solutions DAT313.
SQL Basics. SQL SQL (Structured Query Language) is a special-purpose programming language designed from managing data in relational database management.
GIS Concepts ‣ What is a table? What is a table? ‣ Queries on tables Queries on tables ‣ Joining and relating tables Joining and relating tables ‣ Summary.
Physical Database Design & Performance. Optimizing for Query Performance For DBs with high retrieval traffic as compared to maintenance traffic, optimizing.
Physical Database Design Chapter 6. Physical Design and implementation 1.Translate global logical data model for target DBMS  1.1Design base relations.
Ashwani Roy Understanding Graphical Execution Plans Level 200.
A Guide to MySQL 5. 2 Objectives Use joins to retrieve data from more than one table Use the IN and EXISTS operators to query multiple tables Use a subquery.
Using Special Operators (LIKE and IN)
1 Database Management Systems: part of the solution or part of the problem? Clive Page 2004 April 28.
Oracle Data Integrator Transformations: Adding More Complexity
7 1 Chapter 7 Introduction to Structured Query Language (SQL) Database Systems: Design, Implementation, and Management, Seventh Edition, Rob and Coronel.
SQL 101 – Class 1 Lee Turner. Agenda 1. This is your life – SQL A brief history of SQL What SQL is and what it is not Normalization 2. Some Super Simple.
1 Chapter 10 Joins and Subqueries. 2 Joins & Subqueries Joins – Methods to combine data from multiple tables – Optimizer information can be limited based.
Programming in R SQL in R. Running SQL in R In this session I will show you how to: Run basic SQL commands within R.
Information Building and Retrieval Using MySQL Track 3 : Basic Course in Database.
ADQL/s Syntax (Proposal) -- towards unification of ADQL, SIAP, SSAP, SXAP Yuji SHIRASAKI National Astronomical Observatory.
Chapter 13 Views Oracle 10g: SQL. Oracle 10g: SQL2 Objectives Create a view, using CREATE VIEW command or the CREATE OR REPLACE VIEW command Employ the.
Concepts of Database Management Seventh Edition Chapter 3 The Relational Model 2: SQL.
SQL/Lesson 7/Slide 1 of 32 Implementing Indexes Objectives In this lesson, you will learn to: * Create a clustered index * Create a nonclustered index.
Deoyani Nandrekar Beijing May 2007Virtual Observatory India VOCat Developed By Deoyani and Mohasin.
Physical Database Design Purpose- translate the logical description of data into the technical specifications for storing and retrieving data Goal - create.
Recent spatial work by Jim Gray and Alex Szalay Bob Mann.
A Guide to SQL, Eighth Edition Chapter Five Multiple-Table Queries.
January 23, 2016María Nieto-Santisteban – AISRP 2003 / Pittsburgh1 High-Speed Access for an NVO Data Grid Node María A. Nieto-Santisteban, Aniruddha R.
Lecture 3 With every passing hour our solar system comes forty-three thousand miles closer to globular cluster 13 in the constellation Hercules, and still.
Lecture 15: Query Optimization. Very Big Picture Usually, there are many possible query execution plans. The optimizer is trying to chose a good one.
Virtual techdays INDIA │ august 2010 Filtered Indexes – The unexplored index … Vinod Kumar M │ Microsoft India Technology Evangelist – DB and BI.
7 1 Database Systems: Design, Implementation, & Management, 7 th Edition, Rob & Coronel 7.6 Advanced Select Queries SQL provides useful functions that.
Quiz Which of the following is not a mandatory characteristic of a relation? Rows are not ordered (Not required) Each row is a unique There is a.
LM 5 Introduction to SQL MISM 4135 Instructor: Dr. Lei Li.
Retele de senzori Curs 2 - 1st edition UNIVERSITATEA „ TRANSILVANIA ” DIN BRAŞOV FACULTATEA DE INGINERIE ELECTRICĂ ŞI ŞTIINŢA CALCULATOARELOR.
Spatial Searches in the ODM. slide 2 Common Spatial Questions Points in region queries 1.Find all objects in this region 2.Find all “good” objects (not.
Concepts of Database Management, Fifth Edition Chapter 3: The Relational Model 2: SQL.
IFS180 Intro. to Data Management Chapter 10 - Unions.
More SQL: Complex Queries, Triggers, Views, and Schema Modification
Indexes By Adrienne Watt.
Prepared by : Moshira M. Ali CS490 Coordinator Arab Open University
Multiple Table Queries
Query-by-Example (QBE)
Advanced Queries in MS Access
Informix Red Brick Warehouse 5.1
Cross-matching the sky with database server cluster
COMP 430 Intro. to Database Systems
CHAPTER 5: PHYSICAL DATABASE DESIGN AND PERFORMANCE
Query Execution Expectation-Reality Denis Reznik
Database management concepts
Lecture 12: Data Wrangling
Please thank our sponsors!
G-CORE: A Core for Future Graph Query Languages
SQL: Structured Query Language
Writing Basic SQL SELECT Statements
Some issues in databse-directory integration
Efficient Catalog Matching with Dropout Detection
PSoup: A System for streaming queries over streaming data
LSST, the Spatial Cross-Match Challenge
All about Indexes Gail Shaw.
Presentation transcript:

TAP Standards and Feedback Deoyani Nandrekar-Heinis Johns Hopkins University

Points of Discussion Upload tables Authentication VOSpace integration ADQL and spatial queries Other issues

Upload Tables Scope of tables Life of uploaded table Access Usage and query edit

Authentication Upload access to only authorized user User query and result access

VOSpace Integration Store results in user’s vospace Store job.xml and result in vospace

ADQL and Spatial Queries Generic Queries? Spatial constraints e.g. Using HTM ids Complex Queries Parsing Difficulties Suggestions and Options

ADQL Example ADQL (Spatial) SELECT o.ra, o.dec FROM photoobjall as o WHERE CONTAINS( POINT('J2000', o.ra, o.dec), Region('CIRCLE J2000 180 0 0.3')) = 1 SDSS SQL Query: Select o.ra, o.dec From photoobjAll as o inner join dbo.fSphGetHtmRanges(dbo.fSphSimplifyString('CIRCLE J2000 180 0 0.3')) h on o.HtmID between h.HtmIdStart and h.HtmIdEnd Where dbo.fSphRegionContainsXYZ(dbo.fSphSimplifyString('CIRCLE J2000 180 0 0.3'), o.cx ,o.cy,o.cz) = 1

ADQL and Spatial Queries SELECT a.objid, a.ra, a.dec, b.objid, b.ra, b.dec FROM CatalogA AS a INNER JOIN CatalogB AS b ON CONTAINS(CIRCLE(a.ra, a.dec, 0.1), POIN T(b.ra, b.dec)) WHERE (some condition) circle has to be calculated for every row in A in order to evaluate the query HTM based optimization (not sufficient?) Solution:: temporary tables, generate the HTM ranges, put an index on them and then do the BETWEEN query.

ADQL and Spatial Queries CREATE TABLE ##htmtemp (ObjID bigint, HtmStart bigint, HtmEnd bigint) -- + create the indices here: PK on ObjID, cover index on HtmStart/End INSERT ##htmtemp WITH (TABLOCK) (ObjID, HtmStart, HtmEnd) SELECT ObjID, htm.Start, htm.End FROM CatalogA CROSS APPLY dbo.fHtmCoverCircleEq(Ra, Dec, 0.1) AS htm SELECT a.objid, a.ra, a.dec, b.objid, b.ra, b.dec FROM CatalogA AS a INNER JOIN ##htmtemp htm ON htm.ObjID = a.ObjID INNER JOIN CatalogB AS b ON dbo.fGetHtmEq(b.Ra, B.Dec) BETWEEN htm.HtmStart AND htm.HtmEnd WHERE ...

ADQL and Spatial Queries CREATE TABLE ##htmtemp (ObjID bigint, HtmStart bigint, HtmEnd bigint) -- + create the indices here: PK on ObjID, cover index on HtmStart/End INSERT ##htmtemp WITH (TABLOCK) (ObjID, HtmStart, HtmEnd) SELECT ObjID, htm.Start, htm.End FROM CatalogA CROSS APPLY dbo.fHtmCoverCircleEq(Ra, Dec, 0.1) AS htm SELECT a.objid, a.ra, a.dec, b.objid, b.ra, b.dec FROM CatalogA AS a INNER JOIN ##htmtemp htm ON htm.ObjID = a.ObjID INNER JOIN CatalogB AS b ON dbo.fGetHtmEq(b.Ra, B.Dec) BETWEEN htm.HtmStart AND htm.HtmEnd WHERE ... Multiway joins are slow!!

Special Spatial Objects bright in Optical and invisible in UV Query: SELECT ...FROM SDSS AS a INNER JOIN Galex AS b ON NOT CONTAINS(CIRCLE(a.ra, a.dec, 0.1), POINT(b.ra, b.dec)) Ugly where clauses WHERE CONTAINS(CIRCLE(POINT(a.ra, a.dec), 0.5), POINT(b.ra, b.dec) OR     CONTAINS(CIRCLE(POINT(a.ra + 0.2, a.dec + 0.2), 0.5), POINT(b.ra,b.dec) Multiple conditions where clause WHERE CONTAINS(CIRCLE(POINT(a.ra, a.dec), 0.5), POINT(b.ra, b.dec) AND type= galaxy OR     CONTAINS(CIRCLE(POINT(a.ra, a.dec), 0.5), POINT(b.ra, b.dec) AND type= star

Suggestions Spatial region: only constant values Spatial constraints: Outside where clause CROSS MATCH : handling separately

Thank You !