JAS3 – Current Status and Prospects by Victor Serbo, SLAC

ACAT December 2003 JAS3 – Current Status and Prospects, V. Serbo Contents Introduction to JAS3 (Java Analysis Studio) Basic: Opening Files, Plots, Printing, etc. Web Browser, Editor, Java Compiler, Scripting Miscellaneous: Spreadsheet, Configuration Save/Restore, Preferences, etc. Analysis AIDA – Abstract Interfaces for Data Analysis JAIDA – Java Implementation of AIDA AIDA JNI – using JAIDA from C++ Analysis with JAIDA and JAS3 JAS3 Plugins What is Plugin, examples Demo: AmbientPlugin and Geant4 Plugin Demo: JAIDA Web Interface

ACAT December 2003 JAS3 – Current Status and Prospects, V. Serbo JAS3 Overview A completely new version of JAS Design based on Application Shell, into which many (optional) modules can be plugged  Highly customizable for different application domains –HEP/Astrophysics/Other –Data analysis/Online Monitoring/GRID analysis –Experiment/User specific modules  Modules can be updated independently of shell –Possible to release bug fixes fast Includes support for programming in many languages  Scripting: Pnuts, Python, …. –Command prompt  Java (compiled) Analysis (histograms, tuples, fitting) based on AIDA standard Not technically backwards compatible with JAS2 But migration is straightforward.

ACAT December 2003 JAS3 – Current Status and Prospects, V. Serbo JAS3 Lightning Tour Tour designed to give you an overview of the capabilities of JAS3, you can try them out for yourself: Welcome Page, gives initial info and links to example scripts and programs Memory monitor

ACAT December 2003 JAS3 – Current Status and Prospects, V. Serbo Opening Files Use file menu Drag from explorer

ACAT December 2003 JAS3 – Current Status and Prospects, V. Serbo Plot Manipulation and Printing Can send individual plots or full page direct to printer Or save as PS, EPS, PDF, SWF, SVG, PNG, GIF… Or copy/paste into Word, PowerPoint etc.

ACAT December 2003 JAS3 – Current Status and Prospects, V. Serbo Java Editor, Compiler and Loader Built-in Java compiler Tree shows loaded programs Built-in editor for writing analysis code JAS3 allows any Java program to be loaded. This example “main routine” is taken directly from the AIDA manual

ACAT December 2003 JAS3 – Current Status and Prospects, V. Serbo Miscellaneous Features Save/Restore configuration User Preferences Plugin Manager

ACAT December 2003 JAS3 – Current Status and Prospects, V. Serbo JAS3 Spreadsheet Simple spreadsheet plugin for  Displaying results  Calculations  Simple Plots Supports reading/writing .csv files  Excel files Cut/Paste with Excel etc Coming Soon…  Scripting interface  GUI for building plots  User defined functions –Java, scripting

ACAT December 2003 JAS3 – Current Status and Prospects, V. Serbo JAS3 Scripting JAS3 has multi-language OO scripting support Command line, Console, Editor Major components (e.g. AIDA) have scripting interfaces Currently have plugins to support Pnuts Pnuts – syntax almost identical to Java, fast, well documented and feature complete Python Python  Use Jython (the Java implementation of Python)  Limitation: can not call Python libraries written in C More scripting languages can be added  not restricted to Java implementations (e.g. could use C-Python, JPE)

ACAT December 2003 JAS3 – Current Status and Prospects, V. Serbo Pnuts Language Currently support Pnuts scripting language Complete and well documented  Fast (although not as fast as compiled Java) Syntax very similar to Java; can compile script into Java bytecode or convert it to Java code Can easily call compiled Java classes from scripts – best of both worlds

ACAT December 2003 JAS3 – Current Status and Prospects, V. Serbo JAS3 Scripting II Can also write and run scripts Console allows direct interaction with scripting language

ACAT December 2003 JAS3 – Current Status and Prospects, V. Serbo AIDA - Abstract Interfaces for Data Analysis AIDA is a standard set of interfaces for creating and manipulating histograms, n-tuples and related data analysis objects, like Plotter, Fitter, etc. AIDA defines storage XML format. It has been created cooperatively by a group of developers working on high-energy physics data analysis tools. The goal of the AIDA project is to provide the user with a powerful set of interfaces which can be used regardless of which analysis tool they are using. Only define “protocol” for analysis objects, no internal details AIDA defines behavior, analysis tool provides implementation C++ and Java versions of interfaces are generated from the AID files.

ACAT December 2003 JAS3 – Current Status and Prospects, V. Serbo AIDA Examples IHistogram1D - IAxis axis() - void fill(double x) - void fill(double x, double w) - double mean() - double rms() - double binEntries(int index) - double binError(int index) - …. For more details on AIDA, see: IAxis - int bins() - double lowerEdge() - double upperEdge() - double binLowerEdge(int index) - double binUpperEdge(int index) - double binWidth(int index) - int coordinateToIndex(double x) - ….

ACAT December 2003 JAS3 – Current Status and Prospects, V. Serbo JAIDA JAIDA = Java implementation of AIDA Complete implementation of AIDA interfaces  With a few exceptions –Support for plot styles is somewhat limited –Some missing features in ITree, like cp, mv Uses Java “service provider mechanism” for dynamic loading of services at Runtime  Can be used to add support for new fitters, stores etc.  Have defined “developer level” interfaces for IOptimizer, IStore etc Uses Java Expression Library (JEL) for runtime compilation of expressions (for tuple evaluators and functions)

ACAT December 2003 JAS3 – Current Status and Prospects, V. Serbo JAIDA – Features continued… Fitting support for multiple fitting engines (optimizers)  UNCMIN (pure Java)  Minuit (needs minuitAdapter native library) Support for multiple fitting methods  LeastSquares, Chi2, BinnedMaximumLikelihood and UnbinnedMaximumLikelihood Stores Read/Write support for AIDA XML format (with built-in gzip) Read support for HBOOK/PAW files (needs hbookAdapter native library) Read support for Root files (write support hopefully coming soon) Future plans to add:  SQL interface, binary store format, text file support … Plotting Two implementations of IAnalysisFactory  BatchAnalysisFactory – dummy plotter, ignore calls to IPlotter  AnalysisFactory – Full plotter functionality Multi-Threaded – plots update in real-time as they are filled  No need to call interact() or refresh() Support for embedding plotter  In other Java GUI’s  In Java servlets for web based applications. Graphics export in many formats including:  PDF, EPS, SVG, SWF, PNG, GIF, JPG

ACAT December 2003 JAS3 – Current Status and Prospects, V. Serbo AIDA-JNI C++ interface uses AIDA-JNI C++ program AIDA AIDA- JNI AIDA JAIDA (Java) AIDA-JNI partially generated by AID, partly by hand Does support full AIDA functionality

ACAT December 2003 JAS3 – Current Status and Prospects, V. Serbo Using JAIDA with Java

ACAT December 2003 JAS3 – Current Status and Prospects, V. Serbo Using JAIDA with Python

ACAT December 2003 JAS3 – Current Status and Prospects, V. Serbo Using JAIDA with C++

ACAT December 2003 JAS3 – Current Status and Prospects, V. Serbo Future Plans for JAIDA Complete implementation of missing features Continue to track improvements to AIDA standard Add support for function arithmetic Web Interface Support for “Remote AIDA” Support for more storage formats Need good binary IO format.. either AIDA standard or roll our own Future AIDA developers’ meetings

ACAT December 2003 JAS3 – Current Status and Prospects, V. Serbo JAS3 and AIDA JAS3 uses JAIDA - supports all AIDA functionality, including Histograms (includes arithmetic, projections, etc.) Clouds (unbinned histograms, scatterplots) Plotter Tuples Fitting – AIDA interfaces allow for multiple fitters  Uncmin -- pure java minimizer  Minuit -- Fortran called by Java Native Interface (JNI) IO  AIDA XML, PAW, Root JAS3 supports user interaction with AIDA in three ways Scripting (Pnuts, Python etc) Compiled (Java) code GUI – Plotting, Fitting, Cuts etc.

ACAT December 2003 JAS3 – Current Status and Prospects, V. Serbo Graphical Interface to AIDA Histograms, Clouds, Tuples all presented in AIDA tree.aida files,.hbook files,.root files all presented as AIDA objects Drag items onto page, or use (popup) menus

ACAT December 2003 JAS3 – Current Status and Prospects, V. Serbo HistogramProfile ScatterPlot XY Data (More appropriate for smaller data sets) Tuple Explorer - Plots Works with any tuple, read from file or dynamically created

ACAT December 2003 JAS3 – Current Status and Prospects, V. Serbo Tuple Explorer – Define Columns

ACAT December 2003 JAS3 – Current Status and Prospects, V. Serbo Tuple Explorer - Cuts

ACAT December 2003 JAS3 – Current Status and Prospects, V. Serbo Tuple Explorer - Tabulate

ACAT December 2003 JAS3 – Current Status and Prospects, V. Serbo To be used with record loop Tuple Explorer – Record Source

ACAT December 2003 JAS3 – Current Status and Prospects, V. Serbo Record Sources Opening record (or event) based files causes the run control toolbar to appear Supports random access and “tagged” data sets (mainly for event displays)

ACAT December 2003 JAS3 – Current Status and Prospects, V. Serbo JAS3 Plugins JAS3 is based on the Freehep Application Framework, which provides a basic application framework into which extension modules, or "plugins" can be loaded. The application framework provides general purpose functionality, such as top level GUI (windows, menus etc.), help system, storing and retrieving user preferences etc. All of the data analysis specific functionality of JAS3 is provided by a set of extension modules. Modules communicate with each other via service interfaces and notification events. JAS3 comes with a standard set of service interfaces and notification events, but new plugins can also define their own interfaces and events.

ACAT December 2003 JAS3 – Current Status and Prospects, V. Serbo JAS3 Plugin II Plugin functionality: Do some plugin-specific task Use services provided by other plugins Provide services to other plugins Modify JAS3 menus and pop-up menus “Core” Plugins – provide basic services Simple: File Opener, Memory Status, Status Bar, … More Complex: Web Browser, Plotter, Console, XMLIO, … “Extensions” – separate packages Included with JAS3 distribution  AIDA, TupleExplorer, JAVA Compiler and Editor, Spreadsheet, … Specific to particular Experiment  BaBar Ambient DB Browser: AmbientPlugin, Geant4 Plugin, etc. Partial Tutorial on writing a plugin:

ACAT December 2003 JAS3 – Current Status and Prospects, V. Serbo JAS3 (Java) JAS3 (Java) JAS3 Plugin Demo I AmbientPlugin: Access BaBar Ambient DB via CORBA Present Time History as an AIDA object (DataPointSet) BaBar Ambient DB BaBar Ambient DB CORBA Server (C++) AmbientPlugin (Java CORBA Client)

ACAT December 2003 JAS3 – Current Status and Prospects, V. Serbo JAS3 (Java) JAS3 (Java) JAS3 Plugin Demo II Gean4 Plugin Geant4 Application (C++) JNI Layer Java RMI Server Geant4 Plugin (Java RMI Client)

AIDA Web Interface

ACAT December 2003 JAS3 – Current Status and Prospects, V. Serbo Goals Provide access to Histogram plots without requiring any software installation Putting results on to the web for collaborators etc.  For example for viewing results of “quality control” plots Viewing “live” histograms on the web for  Online monitoring  Monitoring progress of batch jobs Add as much functionality as possible in a “generic” way so that it can be used by others (with minimal work) Complete “out-of-the-box” web application  Packaged as a.war file for use with any J2EE app server –(e.g. Tomcat) Configurable via a simple.ini file (or better via a web interface)

ACAT December 2003 JAS3 – Current Status and Prospects, V. Serbo Architecture.root.paw.aida Application Server (Tomcat) Web Application JAIDA AIDA Servlet SQL … Remote AIDA C++ App Java App

ACAT December 2003 JAS3 – Current Status and Prospects, V. Serbo Features Current Feature set Display of “AIDA Tree” for selection of histograms Displays plots as.png or.gif images in browser  Allows (limited) control of image properties Allows (limited) control over plot properties Allows comparison to reference plots (including Kolmagorov test) Allows export of plots to PDF, EPS, SVG, SWF, GIF, PNG, etc. Planned features Extend to support n-tuples as well as histograms  Dynamic definition of new columns and cuts (evaluators, filters) Web based fitting (including function definition) Support multiple plots per page  Thumbnail browser Extend which plot options can be configured through web Allow users to retain settings via cookies

ACAT December 2003 JAS3 – Current Status and Prospects, V. Serbo Current Interface Can try from the web: aida/welcome.html

ACAT December 2003 JAS3 – Current Status and Prospects, V. Serbo Status Currently released JAS3 version (0.7.4 next week) AIDA functionality is quite solid Compiler, Loader, Record Loop all quite recently added,  Certainly still some rough edges Documentation limited but available Built-in example scripts and programs Tutorial on web If you used JAS2 you will find some functionality not yet ported to JAS3 Remote (client/server) access to data. 3D Lego/Surface plots

ACAT December 2003 JAS3 – Current Status and Prospects, V. Serbo JAS3 Conclusion JAS3 is a powerful and flexible Analysis Tool JAS3 uses Plugin Architecture Easy to add new functionality Easy to fix problems with existing modules – do not have to wait for new release of all libraries JAS3 release is out – try it! It is still a beta version, so bugs happen We need your input to make JAS3 better!  Report bugs, problems, suggestions at  Do not have to be on JAS team to be JAS developer – ideas/code for general-purpose plugins are always welcome Try writing JAS3 plugins for your own needs: for Experiment and Home

ACAT December 2003 JAS3 – Current Status and Prospects, V. Serbo JAS3 Links, More Info Report Issues – JAS – Java Analysis Studio - JAS3 – JAIDA – AIDA – FreeHEP - FreeHEP Java Libraries - WIRED –