EOVSA Catalog Design and VSO Integration D. Zarro 1 EOVSA Kickoff Meeting 24-26 September 2012.

Slides:



Advertisements
Similar presentations
Theo van Veen, Koninklijke Bibliotheek The European Library: opportunities for new services.
Advertisements

GTS MetaData Generation data GTS data bases GTS Switch Volume C1 Central Support Office Information Classes white-list Metadata Synchronization.
Databases. A database program can be used to:  sort a file into a different order  Maintain contact with clients  search through the records for a.
CASDA Virtual Observatory CSIRO ASTRONOMY AND SPACE SCIENCE Arkadi Kosmynin 11 March 2014.
Hinode/EIS Data Products and Archive Access Jian Sun (MSSL)
CAPTURE SOFTWARE Please take a few moments to review the following slides. Please take a few moments to review the following slides. The filing of documents.
Introducing UCLA’s ERDb (Electronic Resources Database) Anita Colby UCLA Science & Engineering Library.
Solar and STP Physics with AstroGrid 1. Mullard Space Science Laboratory, University College London. 2. School of Physics and Astronomy, University of.
Hinode IAS2007-Nov-14 Hinode/EIS Data Products and Archive Access Jian Sun (MSSL)
CPSC 231 Managing Files of Records (D.H.) 1 Learning Objectives Concept of key - primary and secondary keys. Sequential versus direct access. RRN Use of.
CONTENT: A model for collaborative database building Trevor Bond Alan Cornish Washington State University Libraries.
Crawler-Based Search Engine By: Bryan Chapman, Ryan Caplet, Morris Wright.
1 CS 502: Computing Methods for Digital Libraries Lecture 4 Identifiers and Reference Links.
Information systems and databases Database information systems Read the textbook: Chapter 2: Information systems and databases FOR MORE INFO...
Instructional Technology & Design Office or Zotero & Mendeley Workshop Presented by Aisha Conner-Gaten.
STEREO Science Center SWG – Dec 2004 William Thompson NASA Goddard SFC Code Greenbelt, MD
GEOSS Common Infrastructure: A practical tour Doug Nebert U.S. Geological Survey September 2008.
CSI315CSI315 Web Development Technologies Continued.
Mastering Windows Network Forensics and Investigation Chapter 12: Windows Event Logs.
What’s New in VRS? GUGM May 15, 2008 Presenter: Kelly P. Robinson GIL Service Georgia State University
 A databases is a collection of data organized to make it easy to search and easy to retrieve in a useful, usable form.
A Framework for Relationship Discovery Among Files of Different Types Michal Ondrejcek, Jason Kastner and Peter Bajcsy National Center for Supercomputing.
MASSACHUSETTS INSTITUTE OF TECHNOLOGY NASA GODDARD SPACE FLIGHT CENTER ORBITAL SCIENCES CORPORATION NASA AMES RESEARCH CENTER SPACE TELESCOPE SCIENCE INSTITUTE.
Functions and Demo of Astrogrid 1.1 China-VO Haijun Tian.
Data Management Subsystem Jeff Valenti (STScI). DMS Context PRDS - Project Reference Database PPS - Proposal and Planning OSS - Operations Scripts FOS.
Innovations in the Multimission Archive at STScI (MAST) M. Corbin, M. Donahue, C. Imhoff, T. Kimball, K. Levay, P. Padovani, M. Postman, M. Smith, R. Thompson.
Introduction to Apache OODT Yang Li Mar 9, What is OODT Object Oriented Data Technology Science data management Archiving Systems that span scientific.
Access to CARBOOCEAN and related data. Data is standardized and homogenised (parameters, metadata, etc.) quality checked well documented international.
EOVSA Data and Database System Gordon Hurford and Jim McTiernan EOVSA Prototype Readiness Review 24-Sept-2012.
Data Structure & File Systems Hun Myoung Park, Ph.D., Public Management and Policy Analysis Program Graduate School of International Relations International.
Chapter 6 Server-side Programming: Java Servlets
Search Engines. Search Strategies Define the search topic(s) and break it down into its component parts What terms, words or phrases do you use to describe.
© Anselm Spoerri Web Design Information Visualization Course Prof. Anselm Spoerri
Digital Volcanoes and Data Flows Carol Hamilton 1VALA 2012.
2: Getting started Is this a serial? –Mode of issuance Which issue should I use? –Basis for identification of the resource What sources within this issue?
Introduction to metadata
26 Mar 04 1 Application Software Practical 5/6 MS Access.
Nov Common Archive Observation Model What is CAOM and why should MAST use it? Brian McLean.
NQuery: A Network-enabled Data-based Query Tool for Multi-disciplinary Earth-science Datasets John R. Osborne.
1 Guide to Oracle10G CHAPTER 7: Creating Database Reports 7.
VSO Status Update HDMC Meeting 9 June Vision To allow solar physicists to identify and search for data even if they don't know it exists. Make.
Forms 5.02 Understand database queries, forms, and reports.
1 MS Access. 2 Database – collection of related data Relational Database Management System (RDBMS) – software that uses related data stored in different.
Plasma Node demonstrators 1/Registry : –Set of XML descriptors of planetary plasma data (MAPSKP, VEX, MEX ) –Compliant with the SPASE data model
The World Wide Web. What is the worldwide web? The content of the worldwide web is held on individual pages which are gathered together to form websites.
EOVSA Pipeline Processing System J. McTiernan EOVSA Prototype Review 24-Sep-2012.
WebDat: A Web-based Test Data Management System J.M.Nogiec January 2007 Overview.
Publishing Combined Image & Spectral Data Packages Introduction to MEx M. Sierra, J.-C. Malapert, B. Rino VO ESO - Garching Virtual Observatory Info-Workshop.
VO-enabled spectroscopy tools Ivo Busko Science Software Branch STScI.
Managing your research: Citation Management Systems Scott Johnston University of Victoria.
Astro-Wise prompt ● awe is a shell with uses such as: – Python – data reduction (parallel, “dpu”) – interactive analysis (eclipse) – calculator – plotting.
EOVSA Data and Database System J. McTiernan EOVSA Technical DesignMeeting 7-Nov-2011.
ORGANIzED AT WORK PLACE. FreeMind is a free mind mapping aplication written in java. FreeMind is licensed under the GNU General Public License.  It provides.
Database Essentials Lesson 1. Software Orientation.
Discovery and Metadata March 9, 2004 John Weatherley
2b. Create an Access Database Lingma Acheson Department of Computer and Information Science IUPUI CSCI N207 Data Analysis with Spreadsheets 1.
Storage and File Organization
IDL/SSW Analysis Software for SWAP and LYRA
Tables & Relationships
Module 11: File Structure
Indexing Goals: Store large files Support multiple search keys
CPSC 231 Managing Files of Records (D.H.)
Electronic Integrating Resources
Building A Web-based University Archive
Product Retrieval Statistics Canada / Statistique Canada Title page
PDAP Query Language International Planetary Data Alliance
Chapter 9 Web Services: JAX-RPC, WSDL, XML Schema, and SOAP
The Plant Database.
David Cyphert CS 2310 – Software Engineering
Presentation transcript:

EOVSA Catalog Design and VSO Integration D. Zarro 1 EOVSA Kickoff Meeting September 2012

Build on IDL Database  Heritage: IUE, UIT, SOHO/CDS, Hinode/EIS  IDL-based, with supporting routines in SSW  OS- and Platform-independent (Windows, Unix/Linux, Mac OS X ) 2

Catalog Design Considerations  What data products will be searched (images, spectra, lightcurves)?  What metadata will be stored in catalog (observing ids, engineering parameters)?  What metadata will be searchable (e.g. time, frequency)? 3

#items TL_ID I*4 Timeline ID from planning software OBSTITLE C*80 Observing program title TARGET C*40 Target of observation STUD_ACR C*40 Study Acronym OBS_DEC C*240 Description of observation OBS_NUM I*2 Sequential number of observation JOP_ID I*2 Joint Observing Program ID. EIS_SC C*120 Description of EIS science objective OBSERVER C*60 Name of chief observer DATE_OBS R*8 Date/time of beginning of observation DATE_END R*8 Date/time of end of observation NOAA_NUM I*2 NOAA Active Region Number (from planning software) NOAA_AR I*2 NOAA Active Region Number (calculated) SCI_OBJ C*80 Science objective XCEN R*4 Actual pointing of center of image in solar X YCEN R*4 Actual pointing of center of image in solar Y FOVX R*4 Width of field of view in solar X FOVY R*4 Width of field of view in solar Y DATE C*80 Date the FITS file was written FILENAME C*80 Name of FITS file study data was retrieved from. 4 Example EIS Catalog entries

 Archive of searchable data products  Catalog writer/updater  Catalog reader/browser 5 Catalog Requirements

Catalog Population Pipeline EOVSA Processed Data Archive SSW IDL EOVSA Catalog VSO Registry 6 Continuous process: 1.Extract metadata from data file headers. 2.Populate catalog with metadata. 3.Push catalog to SSW. 4.Pull search entries into VSO. => Search catalog via VSO clients and/or IDL directly.

 SSW/IDL database reader returns data structures with fields that match metadata entries 7 IDL> list_eis_main,’1-may-11’,’2-may-11’,obs ** Structure EIS_MAIN_LIST, 28 tags, length=304, data length=296: TL_ID LONG OBSTITLE STRING 'HOP 185. AR magnetism with SST. TARGET STRING 'Active Region ' OBS_DEC STRING 'Switch to returning AR on NE limb. JOIN_SB STRING 'E,S,X ' OBS_NUM INT 0 JOP_ID INT 185 STUDY_ID LONG 360' EIS_SC STRING 'UNKNOWN DATE_OBS DOUBLE e+09 DATE_END DOUBLE e+09 SCI_OBJ STRING 'AR XCEN FLOAT YCEN FLOAT FOVX FLOAT FOVY FLOAT DATE STRING ' FILENAME STRING 'eis_l0_ _ fits.gz

 VSO/IDL client returns selected metadata and file URL locations 8 IDL> files=vso_search('1-may-08','10-may-08',inst='eis',/url) Records Returned : SDAC_EIS : 206/206 _ _ fits.gz _ _ fits.gz _ _ fits.gz _ _ fits.gz