IPAC Firefly Development Roadmap Xiuqin Wu IPAC, Caltech
What is Firefly (1) A web UI framework for web applications Built in astronomy images and catalog searches Visualization of data FITS image visualization Tabular table display 2D XY plot 10/28/15 IPAC Firefly, ADASS XXV
What is Firefly (2) Firefly tools APIs External task launcher JavaScript API Python API External task launcher To enable developer to write special task code to do data analysis at the server end 10/28/15 IPAC Firefly, ADASS XXV
History of Firefly (1) 2009 Spitzer Heritage Archive design, presented two invited talks at ADASS XIX, Japan GWT technology Multiple browsers support Java code reuse 2010 Spitzer Heritage Archive first release 10/28/15 IPAC Firefly, ADASS XXV
History of Firefly (2) 2011 WISE and Planck image services Multiple images display WCS match of multiple images Solar system objects track overlay 2012 IRSA new Finderchart 2013 IRSA Viewer with JS API IRSA images and catalog searches display LCOGT image display 2014 PTF image service 10/28/15 IPAC Firefly, ADASS XXV
WISE image service 10/28/15 IPAC Firefly, ADASS XXV
New Finderchart 10/28/15 IPAC Firefly, ADASS XXV
Catalog search results 10/28/15 IPAC Firefly, ADASS XXV
New Projects and Firefly 2014 IPAC formed Science User Interface and Tools group for LSST IRSA is responsible for ZTF data archive Open source under BSD 3-clause license In github 10/28/15 IPAC Firefly, ADASS XXV
Current Firefly Capabilities
Firefly Architect 10/28/15 IPAC Firefly, ADASS XXV
Client Components FITS image visualization Tabular data display 2D XY plot All three work together to give a coherent presentation of data 10/28/15 IPAC Firefly, ADASS XXV
Without filter Same data With filter 10/28/15 IPAC Firefly, ADASS XXV
Server Manages all the search requests Queue the requests Invoke the right services Returns the results to the client in the right format Maintains the state for the user Cache the results properly for performance Manipulate the images for visualization, PNG image generation Sorts/filters the tables for display Does the necessary calculation for YX plots. 10/28/15 IPAC Firefly, ADASS XXV
Database and Data Services Built-in services to access all IRSA catalogs and many images Base class to easily extend to add new data services Direct SQL search through JDBC 10/28/15 IPAC Firefly, ADASS XXV
Future Development Plan Convert GWT client code to JavaScript JS API to support customizable UI Remote Python API to interact with Firefly New functions to enable more data analysis New ways to enable research collaboration TAP support Python widget 10/28/15 IPAC Firefly, ADASS XXV
GWT client code to JavaScript Mature development frameworks, Backbone.js, AngularJS, Ember.js, and ReactJS. Native look look and feel Improved syntax check and debug capabilities Eslint Karma, Mocha, Chai Better browser support and performance Integrated with GWT through JsInterop WebGL Position us to take advantage of advanced industry development, Google/Facebook … 10/28/15 IPAC Firefly, ADASS XXV
JS API - Extensible/Customizable UI <!doctype html> <html> <head> <title>Demo of Firefly Tools</title> </head> <body> <!-- need a div id to plot to --> <div id="myID" style="width: 350px; height: 350px;"></div> <script type="text/javascript"> { // this function must exist, called when fftools loads var onFireflyLoaded= function() { var primaryViewer= firefly.makeImageViewer('myID'); primaryViewer.plot({ "URL" : "http://someHost.org/someFile.fits", "ZoomType" : "TO_WIDTH"}); }; }</script><!-- script name is fftools.nocache.js --> <script src='fftools.nocache.js'> </script> </body> </html> 10/28/15 IPAC Firefly, ADASS XXV
Remote Python API setenv PYTHONPATH /where-git-repository-is/firefly/src/fftools/python/displayfrom FireflyClient import *fc = FireflyClient( 'localhost:8080') fc.launchBrowser() data = '/your/path/yourFits.fits’ fitsPathInfo= fc.uploadFile(data) fc.showFits(fitsPathInfo) 10/28/15 IPAC Firefly, ADASS XXV
Enable Data Analysis More table features More charts More image operations Pluggable backend 10/28/15 IPAC Firefly, ADASS XXV
More Table Features SQL like filtering capability Cascading filtering among related tables Table A: Coadd table has a coaddID column Table B: Single frames table also has coaddID column Selection on Table A will make the filter on Table B , shows all the single frames in the coadd Aggregate result for large dataset Several density plots Only some sample rows Statistics formulas 10/28/15 IPAC Firefly, ADASS XXV
More Different Charts Histograms Density plots Bar charts Statistics formulas/algorithm 3D view of the data 10/28/15 IPAC Firefly, ADASS XXV
More Image Operations Display large image smoothly Multiple resolution images Overlay image on another image Mask bits Improve and add more color algorithms More meaning to the color 10/28/15 IPAC Firefly, ADASS XXV
Pluggable Back End (customized data analysis) Write your own code for data analysis Configure the Firefly server to run it as external task Use JS API to add it as extra function One use case Select a region of image Run customized code on it to get some statistics Display the result and plot a histogram 10/28/15 IPAC Firefly, ADASS XXV
Enable Collaboration Two people working on the same data at the same time, from different location Firefly channel, using webSocket ws4py Provide support to user workspace 10/28/15 IPAC Firefly, ADASS XXV
Easy Installation and Deployment Java JDK1.8 http://java.com pip install ws4py Download fftools-exec.war at this URL https://github.com/lsst/firefly/releases/ get the Python file FireflyCLient.py from https://github.com/lsst/firefly/blob/master/src/fftools/python/display/FireflyClient.py 10/28/15 IPAC Firefly, ADASS XXV
IPAC LSST Group Job Opening https://jobs.caltech.edu/postings/3764 10/28/15 IPAC Firefly, ADASS XXV
Backup Slides 10/28/15 IPAC Firefly, ADASS XXV
Web Application 10/28/15 IPAC Firefly, ADASS XXV
Python Controls Display 10/28/15 IPAC Firefly, ADASS XXV
Customized Server Extension 10/28/15 IPAC Firefly, ADASS XXV
More Requirement More Firefly API functions to control the display Current API does not expose all the visualization functions Customized functions at server side to do specific data analysis Extend the data analysis functions 10/28/15 IPAC Firefly, ADASS XXV