Sep 4, 2008NVOSS 20081 VO From the Command-Line Mike Fitzpatrick NOAO.

Slides:



Advertisements
Similar presentations
Using the SQL Access Advisor
Advertisements

Syracuse PBT - Tax Year Introduction & AARP Major 2008 Changes.
INSTITUTE FOR CYBER SECURITY April Access Control and Semantic Web Technologies Ravi Sandhu Executive Director and Endowed Chair Institute for Cyber.
Registries Work Package 2 Requirements, Science Cases, Use Cases, Test Cases Charter: Focus on science case scenarios, and use cases related specifically.
The NVO Portal Tom McGlynn, NASA/GSFC Gretchen Greene, ST ScI John Good, IRSA/IPAC Roy Williams, CACR/Caltech Mike Fitzpatrick, NOAO T HE US N ATIONAL.
The VAO is operated by the VAO, LLC. Alternative Protocols for Discovery & Access Mike Fitzpatrick NOAO.
GL10 – December 8-9, Grey literature in French digital repositories: a survey J. Schöpfel (University of Lille 3) C. Stock (INIST-CNRS)
NVO Summer School VO Protocols and Jargon Overview Tom McGlynn NASA/GSFC T HE US N ATIONAL V IRTUAL O BSERVATORY.
NVO Summer School, September Desktop Integration with VO VOClient, DALClient NVO Summer School, Aspen Sept 2006 Doug Tody (NRAO), Mike Fitzpatrick.
Sep 3, 2008NVOSS VO Analysis Using Local Utilities Mike Fitzpatrick NOAO.
September 13, 2004NVO Summer School1 VO Protocols Overview Tom McGlynn NASA/GSFC T HE US N ATIONAL V IRTUAL O BSERVATORY.
September 13, 2004NVO Summer School1 VO Protocols Overview Tom McGlynn NASA/GSFC T HE US N ATIONAL V IRTUAL O BSERVATORY.
Discovery and Exploration in the VO Chris Miller NOAO/CTIO La Serena, Chile T HE US N ATIONAL V IRTUAL O BSERVATORY.
2008 NVO Summer School1 Finding Services in the NVO Registry Gretchen Greene T HE US N ATIONAL V IRTUAL O BSERVATORY.
Sep 6, 2008NVOSS Desktop Messaging Mike Fitzpatrick NOAO.
NVO Summer School, Santa Fe Sept Access to Spectroscopic Data In the VO Doug Tody (NRAO/US-NVO ) I NTERNATIONAL V IRTUAL O BSERVATORY A LLIANCE.
The NVO Data Discovery Portal Tom McGlynn NASA/GSFC.
Sep 3, 2008NVOSS Mobile VO Mike Fitzpatrick NOAO.
Sept NVO Summer School1 Cone, SIAP, and OpenSkyQuery Client Development Gretchen Greene, Maria Nieto-Santisteban T HE US N ATIONAL V IRTUAL O.
8 September 2008NVO Summer School 2008 – Santa Fe1 Publishing Data and Services to the VO Ray Plante Gretchen Greene T HE US N ATIONAL V IRTUAL O BSERVATORY.
Sep 6, NVO Summer School1 Using IRAF in the VO Mike Fitzpatrick, NOAO T HE US N ATIONAL V IRTUAL O BSERVATORY.
1 Hyades Command Routing Message flow and data translation.
18 Copyright © 2005, Oracle. All rights reserved. Distributing Modular Applications: Introduction to Web Services.
Masterclass Introduction to hands-on Exercise Aim of the exercise Find out what happens in proton-proton collisions at the LHC as seen by the ATLAS.
Masterclass Introduction to hands-on Exercise Aim of the exercise Identify electrons, muons, neutrinos in the ATLAS detector Types of Events (particles.
1 Euro-VO Technology Forum, 23 September 2009, Annotations at the CDS Web 2.0: annotations at the CDS Brice GASSMANN Sébastien DERRIERE Thomas BOCH.
Server Access The REST of the Story David Cleary
Copyright Josep Torrellas 2003,20081 Cache Coherence Instructor: Josep Torrellas CS533 Term: Spring 2008.
Windows Server 2008 Chapter 7
10/04/20081 TWG of ESF Committee 10 April 2008 Franck Sébert Head of unit DG EMPL/I/1 Relations with Control Authorities Action plan to strengthen the.
Rule September The following presentation is for educational purposes only and is not a substitute for the statute and Division rules.
EPI809/Spring Chapter 10 Hypothesis testing: Categorical Data Analysis.
Cognos Web Services Business Intelligence. SOA SOA (Service Oriented Architecture) The SOA approach involves seven key principles: -- Coarse -grained.
Database System Concepts and Architecture
31242/32549 Advanced Internet Programming Advanced Java Programming
CASDA Virtual Observatory CSIRO ASTRONOMY AND SPACE SCIENCE Arkadi Kosmynin 11 March 2014.
Patterns and Algebra in Stages 3 and 4 Judy Anderson The University of Sydney AIS Conference 2008.
Executional Architecture
HORIZONT 1 XINFO ® The IT Information System HORIZONT Software for Datacenters Garmischer Str. 8 D München Tel ++49(0)89 /
Chapter 12 Working with Forms Principles of Web Design, 4 th Edition.
02/12/ a tutorial on Markov Chain Monte Carlo (MCMC) Dima Damen Maths Club December 2 nd 2008.
Modular – Flexible – Networked
UK Higher Education library statistics The role of SCONUL.
Introduction Peter Dolog dolog [at] cs [dot] aau [dot] dk Intelligent Web and Information Systems September 9, 2010.
National Center for Supercomputing Applications 259 th fastest computer in the world Michael Remijan NCSA –Research Programmer –Web-based Distributed Programming.
Updated SkyMouse and Suggestions to VO Service Developers HuaPing SUN, Chenzhou CUI China-VO National Astronomical Observatory of China.
A Smart On-line Astronomical Information Collector Chenzhou CUI, Huaping SUN China-VO National Astronomical Observatory of China.
Virtual Observatory Single Sign-on U.S. National Virtual Observatory National Center for Supercomputing Applications Ray Plante, Bill Baker.
WSRF Supported Data Access Service (VO-DAS)‏ Chao Liu, Haijun Tian, Dan Gao, Yang Yang, Yong Lu China-VO National Astronomical Observatories, CAS, China.
DateADASS How to Navigate VO Datasets Using VO Protocols Ray Plante (NCSA/UIUC), Thomas McGlynn and Eric Winter NASA/GSFC T HE US N ATIONAL V IRTUAL.
Functions and Demo of Astrogrid 1.1 China-VO Haijun Tian.
1 Apache. 2 Module - Apache ♦ Overview This module focuses on configuring and customizing Apache web server. Apache is a commonly used Hypertext Transfer.
Introduction to Linux ( I ) Sidney Fong 4 th Feb 2006.
1 Overview of the Application Hosting Environment Stefan Zasada University College London.
Federation and Fusion of astronomical information Daniel Egret & Françoise Genova, CDS, Strasbourg Standards and tools for the Virtual Observatories.
您身边的 天文 Chenzhou CUI China-VO National Astronomical Observatory of China.
The International Virtual Observatory Alliance (IVOA) interoperability in action.
Data Archives: Migration and Maintenance Douglas J. Mink Telescope Data Center Smithsonian Astrophysical Observatory NSF
Virtual Observatories, Press Release Images, and Web Services Dr. Frank Summers Space Telescope Science Institute November 3, 2005.
August 2003 At A Glance The IRC is a platform independent, extensible, and adaptive framework that provides robust, interactive, and distributed control.
8 Chapter Eight Server-side Scripts. 8 Chapter Objectives Create dynamic Web pages that retrieve and display database data using Active Server Pages Process.
12 Oct 2003VO Tutorial, ADASS Strasbourg, Data Access Layer (DAL) Tutorial Doug Tody, National Radio Astronomy Observatory T HE US N ATIONAL V IRTUAL.
April , 2006 HEASARC Users Group Tom McGlynn The HEASARC On-line Services Tom McGlynn.
Publishing Combined Image & Spectral Data Packages Introduction to MEx M. Sierra, J.-C. Malapert, B. Rino VO ESO - Garching Virtual Observatory Info-Workshop.
STAR Scheduling status Gabriele Carcassi 9 September 2002.
1 Copyright © 2008, Oracle. All rights reserved. Repository Basics.
Fundamental of Databases
Development Environment
Introduction to Programming the WWW I
Google Sky.
Presentation transcript:

Sep 4, 2008NVOSS VO From the Command-Line Mike Fitzpatrick NOAO

Sep 4, 2008NVOSS Introduction The Portals discussed yesterday show typical VO-access modes using the web. However, this isnt easily scriptable (same is true for GUI applications). –Saving results is done manually –Data not easily saved for analysis in other environments What we need are general VO client applications we can script

Sep 4, 2008NVOSS VOClient Overview Hides details of VO implementation from applications Provides: –Hi-level, easy-to-use interface between applications and the distributed VO framework, –Client-side VO interfaces returning higher- level structures (like tables) rather than a service API, –Equivalent functionality in all supported languages Caches entire result until queried by app

Sep 4, 2008NVOSS Architecture Modeled after database server VOClient Daemon –Multi-threaded –Java access to VO svcs –Layered on DAL, Registry and services API –C-based, custom and SWIG bindings –Communicates w/ voclientd using proprietary RPC protocol Connection Oriented –Allows voclientd to maintain state on behalf of client Python IRAF, IDL, etc. Java VOCLIENTD dalclient registry, etc. Other…

Sep 4, 2008NVOSS API Example voc_initVOClient (config_opts) voc_closeVOClient (shutdown_flag) query = voc_regQuery (term, orValues) voc_regAddSearchTerm (query, term, orValue) voc_regRemoveSearchTerm (query, term) count = voc_regGetSTCount (query) str = voc_regGetQueryString (query) res = voc_regExecute (query) str = voc_regExecuteRaw (query)

Sep 4, 2008NVOSS From API to Apps VOSESAME Name resolver using Sesame service (Simbad/NED) Configurable output, multiple inputs VOREGISTRY VO Registry search and resource resolution VODATA Uses capabilities of both of the above VO catalog and image access (spectra in progress)

Sep 4, 2008NVOSS VO-CLI in NVOSS Software in NVOSS_HOME$vo-cli directory To build: % cd $NVOSS_HOME/vo-cli % make install This installs binaries in the package bin directory, to use these tools you must install to some directory in your path like the $NVOSS_HOME/bin % cp bin/* $NVOSS_HOME/bin % cp voclientd $NVOSS_HOME/bin % cp -r voclient.jars $NVOSS_HOME/bin

Sep 4, 2008NVOSS VOSesame: Name Resolution Usage: vosesame [ ] [ | [...] ] -a print all information about the object -d print position in decimal degrees (default) -e print position errors -h print help summary -n print object name -t print object type -s print position as sexagesimal coordinates -v print verbose warning output -i invert search to print unresolved objects -f force Sesame svc invocation and ignore cached values -o specify output file (default: stdout) -A output an ASCII table of results (default) -C output a CSV table of results -T output a TSV table of results -H output a table heading

Sep 4, 2008NVOSS VOSesame: Examples 1) Print the coordinates of NGC4456 decimal degrees % vosesame ngc ) Print the sexagesimal coordinates of multiple objects include the type: % vosesame -st m31 00:42: :16:09.4 LINER 3) Print the decimal coordinates of those same objects listed in the file 'myobjs.txt', output as CSV, include a header, and print the id, coords, type: % vosesame -CHndt myobjs.txt #Name,DRA,DDEC,Type, m31, , ,LINER m51, , ,Seyfert_2 m99, , ,HII_G : : : :

Sep 4, 2008NVOSS VOSesame: Examples Calculator Mode: % vosesame -nd m98 m [EOF] Print the sexagesimal and decimal values for multiple user coords: % vosesame -s -c 12:30: :00:0.0 -c :30: :00: :30: :03:00.0

Sep 4, 2008NVOSS VORegistry: Registry Query/Resolve Command-line query of the VO Registry Two Major Modes: –Search (typical keyword search) –Resolve (ShortName/Identifier to something else) List full resource record List table metadata Count matching records Arbitrary ADQL searches Constrain searches –Bandpass, Service Type or ContentLevel

Sep 4, 2008NVOSS VORegistry: Examples 1) Find all services with radio data of Abell clusters. % voregistry -b radio abell 2) Find all resources that mention Keck, how many are image services? % voregistry -count keck % voregistry -t image -count keck

Sep 4, 2008NVOSS VOData: Data Access Multi-threaded queries (hundreds/min) Simple names for resources (e.g. 2mass- xsc and not ivo://irsa.ipac/2MASS-XSC) Various output formats (XML/ASCII/CSV/KML…) Not complete in terms of what VOClient API provides Callable from variety of scripting languages Interacts with NVO Portal Targeted for power users

Sep 4, 2008NVOSS VOData: Usage vodata [ ] [ [[ [ ]]] ] vodata [ ] [ ]

Sep 4, 2008NVOSS VO-CLI For all tasks, a -h flag will print help The voclientd will be started automatically Web interface allows you to experiment with options. Note the command being executed is printed. So, what can we actually do with this??

Sep 4, 2008NVOSS Scavenger Hunt 1) How many image services currently provide data for the GOODS survey?

Sep 4, 2008NVOSS Scavenger Hunt 1)How many image services currently provide data for the GOODS survey? Solution: % voregistry -t image goods % voregistry -count -t image goods

Sep 4, 2008NVOSS Scavenger Hunt 2) Where can you get an HST 4350A image of the center of the HDF?

Sep 4, 2008NVOSS Scavenger Hunt 2) Where can you get an HST 4350A image of the center of the HDF? Solution: % vodata -O hdf -eu HST.GOODS.Cutout \ "Hubble Deep Field" 0.01 the file 'hdf.urls' will contain the acrefs, last image is the F435 possible to use the "-get" flag to simply download them all, or elsepass the hdf.urls in to download

Sep 4, 2008NVOSS Scavenger Hunt 3) How many Bautz-Morgan type II Abell clusters are within 20 degrees of the south pole?

Sep 4, 2008NVOSS Scavenger Hunt 3) How many Bautz-Morgan type II Abell clusters are within 20 degrees of the south pole? Solution: % vodata -V -O - ivo://nasa.heasarc/abell | \ stilts tpipe ifmt=votable in=- \ cmd='select "bmtype == \"II\""' omode="count"

Sep 4, 2008NVOSS Scavenger Hunt 4) How many of them are in the ROSAT FSC (within 10')

Sep 4, 2008NVOSS Scavenger Hunt 4) How many of them are in the ROSAT FSC (within 10') Solution: Regenerate the data query, but save results: % vodata -V -O - ivo://nasa.heasarc/abell | \ stilts tpipe ifmt=votable in=- \ cmd='select "bmtype == \"II\""' \ ofmt=votable out=bmII.xml Submit to Inventory, looking for ROSAT catalogs: % vodata -I -rm 10 any bmII.xml | grep ROSAT total of only 4 clusters match. Query the RASS FSC for the data: % vodata -O - -A -rm 10 RASS/FSC bmII.xml We see that one of the clusters (A3628) matches to 5 distinct X- ray sources, but in this tool we didn't crossmatch to the Abell number. (This could be done easily using STILTS, however)

Sep 4, 2008NVOSS Scavenger Hunt 5) How many chromospherically active binaries are EUV sources … seen by the ROSAT WFC? …seen by EUVE?

Sep 4, 2008NVOSS Scavenger Hunt 5) How many chromospherically active binaries are EUV sources? … seen by the ROSAT WFC? …seen by EUVE? Solution: Do a registry query to find the catalog and download it as an all-sky table: % voregistry chromospherically active binary % vodata -V -O sources ChrAcBin Send it to Inventory and grep for results: % vodata -I -rs 10 any source.xml | egrep -i "rosat|euv" Second EUVE catalog shows 30 matches, WFC 2RE shows 9 matches

Sep 4, 2008NVOSS Scavenger Hunt 6) How many stars with known exoplanets have been observed by ST? Solution: - Try to find a catalog in the Registry: % voregistry -R exoplanet - Access the entire table and send it to Inventory: % vodata -O planets -V J/ApJ/646/ % vodata -I -rm 1 any planets_table2.xml | grep HST

Sep 4, 2008NVOSS Scavenger Hunt 7) Write a script that will show you the rough number of HST, Chandra and Spitzer observations at a given location.

Sep 4, 2008NVOSS Scavenger Hunt 7) Write a script that will show you the rough number of HST, Chandra and Spitzer observations at a given location. Solution 1: vodata -n hst $1 $2.02 2>/dev/null | grep hst vodata -n spitzer $1 $2.2 2>/dev/null | grep spitzer vodata -n chandra $1 $2.2 2>/dev/null | grep chandra When given 3c273 as the object this gives: hst 238 C Hubble Space Telescope spitzer 23 C Spitzer Space Telescope Observation Log chandra 35 I Chandra X-Ray Observatory Data Archive

Sep 4, 2008NVOSS Scavenger Hunt 7) Write a script that will show you the rough number of HST, Chandra and Spitzer observations at a given location. Solution 2: #!/bin/csh -f vodata -n hst,chandra,spitzer $1 $2 $3 |& grep -v -e "#"

Sep 4, 2008NVOSS