JAS3 - A general purpose data analysis framework for HENP and beyond Tony Johnson, Victor Serbo, Max Turri, Mark Dönszelmann, Joseph Perl SLAC

March 2003 JAS3 Contents JAS History JAS3 Overview JAS3 Plugins JAS3 & AIDA  JAIDA and AIDA-JNI JAS3 & Scripting JAS3 Tuple Explorer JAS3 Spreadsheet JAS3 Status Future Plans

March 2003 JAS3 JAS History First version of JAS2 released in Incremental improvements released over time. Most recent change, addition of PDF, EPS, SVG, EMF vector graphics Offline Analysis

March 2003 JAS3 JAS3 History – Use Cases With WIRED event display Online Monitoring

March 2003 JAS3 JAS3 History – Use Cases Custom Applications Web Servlets

March 2003 JAS3 JAS 3 History – Use Cases JAS and the GRID

March 2003 JAS3 JAS History JAS2 Limitations Imposed an (HEP specific) event-analysis model Often used for online monitoring, often used outside HEP  Not really designed for this use Hard to integrate with C++ code No scripting Opportunities Realized a lot of overlap with WIRED project  Created FreeHEP Java library to –Pull out (refactor) common parts of JAS, WIRED –Implement new common requirements for JAS and WIRED Vector Graphics, Application Framework, Utilities (see M.Dönzselmann talk) Formed collaboration with other similar tool developers to create AIDA package.

March 2003 JAS3 JAS3 Overview A completely new version of JAS Design based on Application Shell, into which many (optional) modules can be plugged  Highly customizable for different application domains –HEP/Astrophysics/Other –DST analysis/Online Monitoring/GRID analysis –Experiment/User specific modules  Modules can be updated independently of shell –Possible to release bug fixes fast Includes support for programming in many languages  Scripting: Python, Pnuts, Dynamic Java, …. –Command prompt  Java (compiled) Analysis (histograms, tuples, fitting) based on AIDA standard Not technically backwards compatible with JAS2 But migration is straightforward.

March 2003 JAS3 JAS3 Plugins JAS3 based on FreeHEP Application Framework Functionality provided by “Plugin” modules  Modules may provide services, or use services provided by other plugins –Loose coupling between modules allows for adding/removing/extending modules without breaking other modules  Modules are hierarchical, range from coarse to fine grained Examples of Plugins AIDA (provides core analysis interfaces, histograms, tuples, fitting)  Fitters: Minuit, uncmin,...  IO: XML, Paw, Root, … Tuple Browser Event Processing Scripting:  Pnuts, Jython, Compilers: Java Editor WIRED Spreadsheet

March 2003 JAS3 JAS3 and AIDA JAS3 has adopted AIDA for analysis See talk by Max Turri in this session AIDA allows us to leverage experience and skill of other developers AIDA is functionally more complete than JAS2 analysis package AIDA allows JAS to exchange data with other AIDA tools AIDA provides bridge to C++ programs (e.g. Geant4) AIDA encourages creativity and innovation JAS3 HEP Analysis tools based on JAIDA JAIDA = Java implementation of AIDA  JAIDA is part of FreeHEP library  Usable as standalone library for any Java Application AIDAJNI = Interface between C++ and Java AIDA  Allows C++ programs to use JAIDA, JAS3

March 2003 JAS3 JAS3, AIDA and C++ C++ program AIDA C++ AIDA Implementation AIDA- JNI AIDA JAIDA JAS3.aida file (XML) Java program AIDA

March 2003 JAS3 JAS3 and AIDA JAS3 supports all AIDA functionality, including Histograms (includes arithmetic, projections, etc.) Clouds (unbinned histograms, scatterplots) Plotter Tuples Fitting – AIDA interfaces allow for multiple fitters  Uncmin -- pure java minimizer  Minuit -- Fortran called by Java Native Interface (JNI) IO  AIDA XML, PAW, Root JAS3 supports user interaction with AIDA in three ways Scripting (Pnuts, Python etc) Compiled (Java) code GUI – Plotting, Fitting, Cuts etc.

March 2003 JAS3 JAS3 Scripting JAS3 has multi-language OO scripting support Command line, Console, Editor Major components (e.g. AIDA) have scripting interfaces Currently have plugins to support Pnuts Pnuts – syntax almost identical to Java, fast, well documented and feature complete Python Python (using Jython) More scripting languages can be added  not restricted to Java implementations (e.g. could use C-Python, JPE)

March 2003 JAS3 JAS 3 -- Simple Example

March 2003 JAS3 JAS3 – Histogram Arithmetic

March 2003 JAS3 JAS3 -- Fitting

March 2003 JAS3 JAS3 Tuple Explorer GUI for tuple manipluation Supports int, float, double, String, Date, Objects Expression evaluator for defining new columns  Expressions compiled on fly Cuts can be defined  Using expression evaluator  Numeric expressions –Sliders for varying cuts Works with “nested” tuples  E.g. Event, Tracks, Hits Interfaced to AIDA tuples Can be generated on fly Read from file (PAW, XML, etc)

March 2003 JAS3 JAS3 Spreadsheet Simple spreadsheet plugin for  Displaying results  Calculations  Simple Plots Plugins for reading/writing .csv files  Excel files (pure Java) Cut/Paste with Excel etc Working on  Scripting interface  GUI for building plots  User defined functions –Java, scripting

March 2003 JAS3 JAS3 Status JAS3 0.5 is available for download Supports full AIDA functionality, scripting, tuple explorer, spreadsheet. Exceeds JAS2 is some areas, but does not yet support full JAS2 functionality in all areas (hence 0.5) Having frequent 0.x releases as we move towards 1.0 Plugins under development for JAS3 1.0  WIRED event display  Record/Event loop (in collaboration with IceCube experiment)  More IO plugins, Root, StdHEP, SQL database etc.  Publication quality plots – using FreeHEP vector graphics Longer term plans Log book, web publishing/collaboration plugin Distributed computing support (using SOAP or Corba in addition to RMI) Grid interface (in collaboration with PPDG – CS11 and Tech-X) Geant4 interface for small experiments/beam tests/medical users  Combine JAS3, Wired, HepRep, AIDA, GAG and MOMO (with Hajime Yoshida)

March 2003 JAS3 JAS3 Links, More Info JAS – Java Analysis Studio - JAS3 – JAIDA – AIDA – FreeHEP - FreeHEP Java Libraries - WIRED – Come and see the other JAS3/FreeHEP related talks at CHEP –