Manipulating VOTables with TOPCAT and STILTS

Slides:



Advertisements
Similar presentations
The NVO Portal Tom McGlynn, NASA/GSFC Gretchen Greene, ST ScI John Good, IRSA/IPAC Roy Williams, CACR/Caltech Mike Fitzpatrick, NOAO T HE US N ATIONAL.
Advertisements

Applications Interest Group Mark Allen, Tom McGlynn (lead)
May IVOA Interop Meeting1 STScI/JHU Registry Status Gretchen Greene Wil OMullane T HE US N ATIONAL V IRTUAL O BSERVATORY.
Applications Interest Group Tom McGlynn (based on Mark Allens summary from Cambridge)
M. C. Leoni 1, T. A. McGlynn 2, R. D. Williams 3 1 European Southern Observatory, Germany 2 HEASARC, Greenbelt - MD, USA 3 Caltech, Pasadena - LA, USA.
STILTS – Command-line tools for table/VOTable manipulation TOPCAT VO features – VO features in the GUI tabular data exploration tool SPLAT-VO – SSAP functionality.
The VAO is operated by the VAO, LLC. Web SAMP in VAO Data Discovery Tool Dean Hinshaw, NASA/HEASARC.
NVO Summer School VO Protocols and Jargon Overview Tom McGlynn NASA/GSFC T HE US N ATIONAL V IRTUAL O BSERVATORY.
Sep 3, 2008NVOSS VO Analysis Using Local Utilities Mike Fitzpatrick NOAO.
September 13, 2004NVO Summer School1 VO Protocols Overview Tom McGlynn NASA/GSFC T HE US N ATIONAL V IRTUAL O BSERVATORY.
September 13, 2004NVO Summer School1 VO Protocols Overview Tom McGlynn NASA/GSFC T HE US N ATIONAL V IRTUAL O BSERVATORY.
2008 NVO Summer School1 Finding Services in the NVO Registry Gretchen Greene T HE US N ATIONAL V IRTUAL O BSERVATORY.
The NVO Data Discovery Portal Tom McGlynn NASA/GSFC.
September 7, 2005NVO Summer School1 Building a SkyNode Server Tom McGlynn NASA/GSFC T HE US N ATIONAL V IRTUAL O BSERVATORY.
Sep 8, NVO Summer School1 WCSFixer A Web-Enabled IRAF Plate Solver Mike Fitzpatrick, NOAO T HE US N ATIONAL V IRTUAL O BSERVATORY.
Sept NVO Summer School1 Cone, SIAP, and OpenSkyQuery Client Development Gretchen Greene, Maria Nieto-Santisteban T HE US N ATIONAL V IRTUAL O.
8 September 2008NVO Summer School 2008 – Santa Fe1 Publishing Data and Services to the VO Ray Plante Gretchen Greene T HE US N ATIONAL V IRTUAL O BSERVATORY.
Sept NVO Summer School1 The NVO DataScope: Internals Tom McGlynn NASA/GSFC T HE US N ATIONAL V IRTUAL O BSERVATORY.
DDI for the Uninitiated ACCOLEDS /DLI Training: December 2003 Ernie Boyko Statistics Canada Chuck Humphrey University of Alberta.
WEB DESIGN TABLES, PAGE LAYOUT AND FORMS. Page Layout Page Layout is an important part of web design Why do you think your page layout is important?
Google Chrome & Search C Chapter 18. Objectives 1.Use Google Chrome to navigate the Word Wide Web. 2.Manage bookmarks for web pages. 3.Perform basic keyword.
FM Web Scraping FMPUG: Dallas Chapter Taylor Made Services: FileMaker Presentation March 6, 2009 Dallas Texas.
GLAST LAT ProjectOnline Peer Review – July 21, Integration and Test A. T. Kavelaars 1 GLAST Large Area Telescope: I&T Integration Readiness.
JMeter Workshop Friday 1 December 2006 Anthony Colebourne IT Services The University of Manchester.
PaperScope: Visually Exploring the ADS Mark Holliman VOTECH Web Developer University of Edinburgh ADASS XVII, London,
Chapter 7 Managing Data Sources. ASP.NET 2.0, Third Edition2.
VOView Rendering VOTables using XSLT and JavaScript Dean Hinshaw HEASARC GSFC/NASA.
DEiXTo.
Understanding and Managing WebSphere V5
Form Handling, Validation and Functions. Form Handling Forms are a graphical user interfaces (GUIs) that enables the interaction between users and servers.
M. Taimoor Khan * Java Server Pages (JSP) is a server-side programming technology that enables the creation of dynamic,
CS413: Java Programming language Applications Applets
MySQL GUI Administration Tools Rob Donahue Manager, Distributed Systems Development May 7th, 2001 Rob Donahue Manager, Distributed Systems Development.
Using the SAS® Information Delivery Portal
Chapter 17 - Deploying Java Applications on the Web1 Chapter 17 Deploying Java Applications on the Web.
WSRF Supported Data Access Service (VO-DAS)‏ Chao Liu, Haijun Tian, Dan Gao, Yang Yang, Yong Lu China-VO National Astronomical Observatories, CAS, China.
Functions and Demo of Astrogrid 1.1 China-VO Haijun Tian.
Last News of and
UCSC All rights reserved. No part of this material may be reproduced and sold. 1 IT1202-Fundamentals Of Programming (Using JAVA) Interacting with.
T HE I NTERNATIONAL V IRTUAL O BSERVATORY ALLIANCE VAO Registry Status Spring 2014 Theresa Dower Registry WG IVOA Interoperability Meeting – Madrid 2014.
UWG 2013 Meeting PO.DAAC Web Services Demo. What are PO.DAAC Web Services?
INSERT BOOK COVER 1Copyright © 2011 Pearson Education, Inc. Publishing as Prentice Hall. Exploring Microsoft Office Excel 2010 by Robert Grauer, Keith.
DLI Boot Camp 2011 Finding Statistics: Tools and Techniques Jean Blackburn Vancouver Island University Library SDA.
FLUKA GUI Status FLUKA Meeting CERN, 10/7/2006.
Copyright 2007, Information Builders. Slide 1 Scaling Large HTML Reports With Active Cache Mark Nesson,Vashti Ragoonath June 2008.
View_hdf Kam-Pui Lee Science Applications International Corporation CERES Data Management Team Linda Hunt Computer Sciences Corporation Atmospheric Sciences.
© Copyright SELA software & Education Labs Ltd Baruch Hirsch St.Bnei Brak Israel
Starlink VOTable software Author: Mark Taylor Open source Java software for table manipulation STIL:
Mozilla. Why mozilla Main Components Browser features Loads very quickly Personal toolbar with your locations Can turn off pop-up windows good control.
Devanshu Bawa Customization Specialist Logo Business Solutions.
April , 2006 HEASARC Users Group Tom McGlynn The HEASARC On-line Services Tom McGlynn.
Introduction to CADStat. CADStat and R R is a powerful and free statistical package [
Coordination Portal/Data sources L2:Data access & Handling Points to be raised : - Downloading data sources/data via the Portal…keeping visibility of data.
Evanthia Hatziminaoglou, ESO - Garching Virtual Observatory Info-Workshop, SOFIA January 2008 VO Tools Overview.
In part from: Yizhou Sun 2008 An Introduction to WEKA Explorer.
丁建文 國立高雄應用科大資管系副教授 兼任計網中心軟體發展組組長 跨平台行動應用軟體開發技術 : HTML5 & Mobile JavaScript Framework 暨南大學.
Installing and Using Evolve Evolve is written in Java and runs on any system with Java 1.6 Download Evolve from Unzip.
Bulk Loading Documents* into Windchill
Big Data Analytics and Machine Intelligence Capability Team
Emergent Information Technologies, Inc.
Essential tools for implementing and testing websites
Getting Started with R.
Good Morning  Please be sure to take care of your belongings.
Bulk Loading Documents* into Windchill
CFS Community Day Core Flight System Command and Data Dictionary Utility December 4, 2017 NASA JSC/Kevin McCluney December 4, 2017.
The Celera Genome Browser: A Tool for Visualizing and Annotating the Human Genome
HDF and HDF-EOS Tools George Schwenke Raytheon STX
Python I/O.
Embedding Graphics in Web Pages
Google Sky.
Presentation transcript:

Manipulating VOTables with TOPCAT and STILTS Tom McGlynn

What are TOPCAT and STILTS? STILTS is a powerful library that enables users to filter and convert tables in VOTable and other popular formats. TOPCAT is a user interface built on top of STILTS with extensive GUI and graphics capabilities. All written by Mark Taylor (Bristol University/UK) originally as part of the Starlink project. This demo uses TOPCAT but the underlying functionality is available through the STILTS library (in Java). There are some other standalone commands built on STILTS (e.g., votcopy). There is a TOPCAT chapter in the NVO book.

Starting TOPCAT In the standard distribution as command ‘topcat’ Note capabilities to download data from various VO and other data sources. Remember to run setup first! Files used in this presentation: Download these (Twiki doesn’t let TOPCAT read them directly) http://nvo-twiki.stsci.edu/twiki/pub/Main/NVOSS2008Sched/spocs.csv http://nvo-twiki.stsci.edu/twiki/pub/Main/NVOSS2008Sched/spocs.xml http://nvo-twiki.stsci.edu/twiki/pub/Main/NVOSS2008Sched/exostars.xml This URL can be used directly http://heasarc.gsfc.nasa.gov/cgi-bin/W3Browse/getvotable.pl?name=messier

Starting and loading a URL

Table loaded successfully

Getting to and from VOTables Converting to VOTables from CSV Load spocs.csv (available on Twiki) Save as spocs.xml (VOTable) Can covert into many formats. Converting VOTables to CSV Load http://heasarc.gsfc.nasa.gov/cgi-bin/W3Browse/getvotable.pl?name=messier Save as CSV Portal tools may work better in handling data in non-standard formats.

Convert to a CSV file

After loading CSV convert to VOTable

The table browser

Filtering data using TOPCAT Show SPOCS table (view/tabledata) Create a subset (view/row subsets/+) Enter name=Sunlike Enter criterion Teff>5720 && Teff<5820 && log_g>4.34 && log_g<4.54 && M_H>-0.1 && M_H <0.1 Should create subset with ~30 rows. Show with Views/Table data/Apply Subset/Sunlike We can use column names or $number. A sample can be saved in any of the supported formats.

Creating a subset

Histograms Click on histogram Note which subsets are showing and click on all Change column to VMAG Note both all and sunlike data.

Histograms

Plots X/Y display Ensure that full sample is shown. Change parameters to Teff/Age Zoom around sun location Create new sample from plot (third box from right) Note region selection tool (fourth from right)

X-Y plots

Creating a subset graphically

3-D Graphics/Synthetic columns 3-d displays (change third axis to something interesting in box) Creating new columns: column metadata/+ Name = AppMag Expr = Vmag – 5*log10(Dist/10) [note striping error] Note function browser New columns can be used anywhere and show up on output

3-D plots

Creating a new column

Correlations using TOPCAT Load SPOCS table EXOStars table Click on double match Cross-correlate by position: Find all the SPOCS stars that have exoplanets

Cross-correlations

Viewing the correlation

Saving the correlated data.

Many more capabilities Direct access to VO enabled services Density plots Column data conversions Hundreds of functions usable in creating columns PLASTIC/SAMP communication with other tasks Joining tables Stacked plots or spherical plots Table statistics …