The COMPASS (Catalogs of Objects and Measure Parameters for All Sky Surveys) Database Overview Gretchen Greene, Brian McLean, David Wolfe, and Charles.

Slides:



Advertisements
Similar presentations
Trying to Use Databases for Science Jim Gray Microsoft Research
Advertisements

Viewing and Features ShowSky - a Jini aware Applet/API astronomical archive discovery tool Object Design and Implementation Guide Star Catalog-II Jini.
Footprint Service Specification NVO Summer School 2008 Gretchen Greene (thanks to Tamas Budavari and Francois Bonnarel) T HE US N ATIONAL V IRTUAL O BSERVATORY.
Overview of Current and Forthcoming GALEX Search Capabilities and Data Products Current Search Options New GALEX Fluxes gPhoton.
Development of China-VO ZHAO Yongheng NAOC, Beijing Nov
The Pan-STARRS Archive at STScI
HIGH PROPER MOTION WHITE DWARF CANDIDATES GSCII Annual Meeting October CBBS, Stevensville (MD) by Daniela Carollo Osservatorio Astronomico.
MAST-VizieR/NED cross correlation tutorial 1. Introduction Figure 1: Screenshot of the MAST VizieR Catalog Search Form. or enter here as object class:
Web + VO + Database Technologies = HLA Footprints STScI: Gretchen Greene, Steve Lubow, Brian McLean, Rick White and the HLA Team JHU: Alex Szalay and Tamas.
1 Software & Grid Middleware for Tier 2 Centers Rob Gardner Indiana University DOE/NSF Review of U.S. ATLAS and CMS Computing Projects Brookhaven National.
Databases Chapter Distinguish between the physical and logical view of data Describe how data is organized: characters, fields, records, tables,
A Web service for Distributed Covariance Computation on Astronomy Catalogs Presented by Haimonti Dutta CMSC 691D.
16 months…. The Visibility Information Exchange Web System is a database system and set of online tools originally designed to support the Regional Haze.
Building a Framework for Data Preservation of Large-Scale Astronomical Data ADASS London, UK September 23-26, 2007 Jeffrey Kantor (LSST Corporation), Ray.
Leicester Database & Archive Service J. D. Law-Green, J. P. Osborne, R. S. Warwick X-Ray & Observational Astronomy Group, University of Leicester What.
Hubble Legacy Archive Lee Quick - TIPS meeting July 19, 2012 Goals Data History Current Work Demo.
6 th Annual GSC-II Project Meeting C B A S GSC-II and DSS-II Projects 2000 Annual Report Brian McLean 19 th October 2000.
Digitized Sky Survey Update Brian McLean : Archive Sciences Branch / Operations and Engineering Division.
Database System Concepts and Architecture Lecture # 3 22 June 2012 National University of Computer and Emerging Sciences.
Cool white dwarfs in the Sloan & SuperCOSMOS Sky Surveys Nigel Hambly, Wide Field Astronomy Unit, IfA, University of Edinburgh.
Why Build Image Mosaics for Wide Area Surveys? An All-Sky 2MASS Mosaic Constructed on the TeraGrid A. C. Laity, G. B. Berriman, J. C. Good (IPAC, Caltech);
Data Management Subsystem: Data Processing, Calibration and Archive Systems for JWST with implications for HST Gretchen Greene & Perry Greenfield.
LSST Scheduler status Francisco Delgado Sr. Software Engineer Telescope & Site.
MASSACHUSETTS INSTITUTE OF TECHNOLOGY NASA GODDARD SPACE FLIGHT CENTER ORBITAL SCIENCES CORPORATION NASA AMES RESEARCH CENTER SPACE TELESCOPE SCIENCE INSTITUTE.
Dec 2, 2014 MAST Data Discovery Portal Tom Donaldson Tony Rogers.
Database System Concepts and Architecture
EdSkyQuery-G Overview Brian Hills, December
Functions and Demo of Astrogrid 1.1 China-VO Haijun Tian.
Innovations in the Multimission Archive at STScI (MAST) M. Corbin, M. Donahue, C. Imhoff, T. Kimball, K. Levay, P. Padovani, M. Postman, M. Smith, R. Thompson.
Using the Open Metadata Registry (openMDR) to create Data Sharing Interfaces October 14 th, 2010 David Ervin & Rakesh Dhaval, Center for IT Innovations.
DATABASE MANAGEMENT SYSTEMS IN DATA INTENSIVE ENVIRONMENNTS Leon Guzenda Chief Technology Officer.
26 October 2005HST Calibration Workshop1 The National Virtual Observatory and HST T HE US N ATIONAL V IRTUAL O BSERVATORY Robert Hanisch US National Virtual.
Seminars on formation and evolution of the Galaxy Feb 12, 2002 The construction of the GSC2.2 Catalog Mario G. Lattanzi Osservatorio Astronomico di Torino.
Using the NSA Presentation to NOAO Users Committee October 5, 2005.
Copyright © 2015 McGraw-Hill Education. All rights reserved. No reproduction or distribution without the prior written consent of McGraw-Hill Education.
Patrizia Ferrero 3rd Integral Bart Work Shop Chocerady - November 1-3, Fast dissemination of GRB afterglow information Patrizia Ferrero (IASF-BO,
GSC-II Plate Processing Pipeline Status C. Loomis, C. Sturch, F. Guglielmetti.
Footprint Service Specification IVOA Interop Meeting Trieste 2008 Gretchen Greene and Tamas Budavari.
P Structured Query Language for Virtual Observatory Yuji Shirasaki National Astronomical Observatory of Japan, and Masahiro Tanaka (NAOJ), Satoshi.
Federation and Fusion of astronomical information Daniel Egret & Françoise Genova, CDS, Strasbourg Standards and tools for the Virtual Observatories.
1 BILLION “STARS” Barolo 2001 Attilio Ferrari Istituto Nazionale di Astrofisica.
7 th Annual GSC-II Project Meeting C B A S GSC-II and DSS-II Projects 2001 Annual Report Brian McLean 22 nd October 2001.
1 Computing Challenges for the Square Kilometre Array Mathai Joseph & Harrick Vin Tata Research Development & Design Centre Pune, India CHEP Mumbai 16.
GSC2 Maintenance GSC2 Annual meeting Database administrative tasks Database production tasks Identification and correction of errors Processing.
LSST Scheduler construction plan Francisco Delgado Sr. Software Engineer Telescope & Site.
Common Archive Observation Model (CAOM) What is it and why does JWST care?
Nov Common Archive Observation Model What is CAOM and why should MAST use it? Brian McLean.
STScI MSR template Page 1 Space Telescope Science Institute Guide Star Discussion August 27, 2007.
Data Archives: Migration and Maintenance Douglas J. Mink Telescope Data Center Smithsonian Astrophysical Observatory NSF
Nov JWST Data Management Systems Gretchen Greene.
Pan-STARRS PS1 Published Science Products Subsystem Presentation to the PS1 Science Council August 1, 2007.
MGL/OATo 7th GSC2 Meeting, Barolo (I), 22 Oct 2001 The GSC2.2 Catalog: Global Properties Mario G. Lattanzi Osservatorio Astronomico di Torino.
NGST/NEXUS Operations Jane Morrison October 19, 2000 Nexus NGST.
Photometric Calibration Jorge F. García Yus GEMINI Observatory Barolo 2001.
GSC2.2 Classification GSC II Annual Meeting October 2001.
Lecture 3 With every passing hour our solar system comes forty-three thousand miles closer to globular cluster 13 in the constellation Hercules, and still.
Distributed Archives Interoperability Cynthia Y. Cheung NASA Goddard Space Flight Center IAU 2000 Commission 5 Manchester, UK August 12, 2000.
Budapest Group Eötvös University MAGPOP kick-off meeting Cassis 2005 January
26th October 2005 HST Calibration Workshop 1 The New GSC-II and it’s Use for HST Brian McLean Archive Sciences Branch.
5-7 May 2003 SCD Exec_Retr 1 Research Data, May Archive Content New Archive Developments Archive Access and Provision.
Physical Oceanography Distributed Active Archive Center THUANG June 9-13, 20089th GHRSST-PP Science Team Meeting GHRSST GDAC and EOSDIS PO.DAAC.
Wide-field Infrared Survey Explorer (WISE) is a NASA infrared- wavelength astronomical space telescope launched on December 14, 2009 It’s an Earth-orbiting.
Spatial Searches in the ODM. slide 2 Common Spatial Questions Points in region queries 1.Find all objects in this region 2.Find all “good” objects (not.
© 2017 by McGraw-Hill Education. This proprietary material solely for authorized instructor use. Not authorized for sale or distribution in any manner.
JWST FGS Guide Star Studies
Copyright © 2011 Pearson Education, Inc. Publishing as Pearson Addison-Wesley Chapter 2 Database System Concepts and Architecture.
Planning Observations
COMPASS Database SPACE TELESCOPE SCIENCE INSTITUTE Gretchen Greene
Google Sky.
Footprint Service Specification
Presentation transcript:

The COMPASS (Catalogs of Objects and Measure Parameters for All Sky Surveys) Database Overview Gretchen Greene, Brian McLean, David Wolfe, and Charles Loomis Space Telescope Science Institute Antonio Volpicelli, Roberto Morbidelli and Allesandro Spagna Osservatorio Astronomico di Torino October 22, 2001

Introduction: Database Functionality lKey Component in the GSC-2 catalog production –Online access to processed plate data –Identification of unique sources –Cross-indexing of multiple observations Overlapping photographic plates Multiple epochs –Global recalibration capability –GSC 2 is “exported” from COMPASS lSuccessful demonstation of Object-Oriented technology and the selection of commercial software, Objectivity/DB lOne of the first large-scale astronomical archives – Estimated 2 billion objects, 10 billion sources/observations – 4 TB catalog derived from 8 TB image archive! lCollaborative project with OATo and SDSS JHU Science Archive –OATo assisted with schema development, QA (CIQA), plate- matching and much more! –SDSS/STScI joint development of the HTM Indexing Hierarchical Triangulated Mesh Level 3 Leafnode S213

Infrastructure: How is the Database designed? Page = Federated Database = Database = Container = Class/Basic Object HTM Region Database RegionCatalogContainers Plate Database [ COMPASS Federated Database ] Plate Source Objects GSPC2 SourceIndex RegionInfoContainer PlateInfoContainer GSH for each plate for each plate section within region for each of regions Section of reference catalog within region Tycho2 lObject model or design is based on hierarchical systems for efficiency –Federation of Databases and Containers lPlate data information stored in 2 forms for access –MetaData and Observations within each HTM region

Highlights: What happened in 2001 Completion of Loading and Cross-Matching Observations: All-Sky, 2 bands Reference catalogs: Successfully loaded GSPC2 and Tycho2 into COMPASS, utilize real-time object-matching based on GSC1 Fully Online RAID System: Has been upgraded to 4TB using NT client/server configuration. Export of GSC 2.2 Public Release: The export of ~1 billion objects from the database with best observation position selection, magnitude and classification filtering for accuracy. Steady State Production Operations: Catalog production pipeline, recalibration, and administration of multi-Terabyte data volume HST Bright Object Protection (BOP): GSC-2.2 incorporation into Observation Planning Tools and target acquisition development (APT) – special release ShowSky JAVA Visualization Tool Public Release Multiple Catalog viewing, including SDSS, 2MASS, Tycho, DSS, GSC…. Parameter Filtering HTM grid field view with level

GSC2.2 Export Catalog Fields: what will change with 2.3? Field/Column NumberParameter 1GSC ID2 2Right Ascension 3Declination 4Mean Epoch 5RA Error 6Dec Error 7RA Proper Motion 8Dec Proper Motion 9RA Proper Motion Error 10Dec Proper Motion Error 11F Magnitude 12F Magnitude Error 13J Magnitude 14J Magnitude Error 15V Magnitude 16V Magnitude Error 17N Magnitude 18N Magnitude Error 19Classification 20Semi Major Axis 21Eccentricity 22Position Angle 23Status Flag Define new requirements and evaluate current parameters early enough to avoid schedule slips need time to develop modifications to export task Coordination of DB administation with science goals How do we want to query the catalog? move toward mining.

Export Catalog: Current configuration and moving toward GSC 2.3 lSource identification –names from GSC2.2 release should be preserved now that the catalog has been incorporated into telescope operations, is this always possible? lCatalog column parameters, new ones? Changes to existing? –Position currently is based on observation closest to plate center for the same bandpass Astrometric residuals were verified to be a factor of 2 improved. –Reclassification: further definition of non-star into galaxy, blend, etc. lPlate identification –epoch is non-unique distinction of plate source lMulti-plate calibrated parameters, proper motion: –API development will benefit from experience of previous catalog pipeline tasks, e.g. Plate loading and object/source cross matching. –Minimize the large scale transaction time, perform dynamically if possible. Will this parameter be changing

Multi-Archive at STScI: CASB coordination with MAST MAST develops and maintains the public data release WWW user interface for SDSS and the STScI WWW pages for GSC-2 – individual development sites provide more project specific information -CASB -FNAL -JHU Unified view of STScI archives –Growth in the large scale catalog archives –Cross correlation of mission data now possible, development of large scale correlations between ground and space missions in progress.

Moving toward Multi-catalog archive: SDSS and GSC-2 joint operation at STScI Data Archive Server is mirrored parallel to GSC-2 on hybrid LINUX/NT dual server. New Develop of MAST/GALEX archive based on GSC-2/SDSS archive interoperation SDSS population into COMPASS for high speed cross-matching utilizing HTM -no spatial remapping New Database Engine?? There is a possibility that SDSS will migrate from Objectivity to another commerical system, e.g. SQL server.

Future Developments: GSC-2 enhancements to support access for science and operations l2MASS population into COMPASS –GSC MASS will be input catalog of NGST guidance system –2MASS also utilizes HTM for their file system lSDSS population into COMPASS lDirect public access to COMPASS observations extended for higher speed access –HTMIndex container using sorted level10 source indexing as in export catalog similar to the export catalog infrastructure lCorrect Internal database page file limitations lGSC2Query Enhancements lGSC-2, SDSS, GALEX, 2MASS =>>>> NVO (National Virtual Observatory) –Still very early in conceptual stage, but nevertheless it is real, 10M NSF grant is currently issued for collaboration between STScI, Caltech, JHU + others.