1. Java Root IO Part of the FreeHEP Java Library Tony Johnson (tony_johnson@slac.stanford.edu) tony_johnson@slac.stanford.edu Mark Dönszelmann (duns@slac.stanford.edu) duns@slac.stanford.edu Victor Serbo (serbo@slac.stanford.edu) serbo@slac.stanford.edu Max Turri (turri@slac.stanford.edu) turri@slac.stanford.edu Presented at Computing in High Energy Physics Interlaken, Switzerland, September 2004

2. Contents  Goals  Java IO library for Root –Implementation –Test Suite –Benchmarks –Networked Root IO (rootd/xrootd) –Demo tools –Root Plugin for AIDA  Examples of Use –AIDATLD Web Interface –JAS3 –WIRED4  Future plans  Links for more information

3. Java Root IO: Goals  Goals –Pure Java package for reading Root Files  Will be extended to writing later –High Performance  We want to do data analysis so we need very good performance –Should not need to know about objects ahead of time (no need for dll’s,.so files etc.)  But should provide full support for reading user- defined objects –Easy to use –Provide access to Root data, not C++ code

4. Implementation  Use “TStreamerInfo” objects inside Root files (since Root 3.0) to decode contents of file  Dynamically generate Java proxies for each Root object read  Each proxy has custom “streamer” –Build as Java bytecode using BCEL (http://jakarta.apache.org/bcel) http://jakarta.apache.org/bcel –Converted at runtime to machine code by HotSpot VM  Uses java.nio package (new since Java 1.4) –Standard Java package for high-performance binary IO

5. Implementation Details TStreamerInfo Java Interface Accessor method for each data member.java file Java Proxy Custom version based on how object is stored in file Implements Interface Generator

6. Example of Use

7. Test Suite  Most classes are dynamically generated from info in Root file, so are fairly robust against changes  Need bootstrap process, in particularly need enough custom Java classes to read the streamer info, and interpret it –Relatively few classes, but  This tends to change from one Root release to another  Typically have to make small changes in the bootstrap code for each new Root release  We have test suite that creates a set of test files from each new root release, and verifies that our library can read the files. –Currently test 3.00.06, 3.01.06, 3.02.07, 3.03.07, 3.05.07, 3.10.02, 4.00.08  In addition we have a set of Root files collected from users, and the test suite verifies that we can (continue) to read these files with each release.

8. Java Root IO - Benchmark  1.4GHz AMD Athlon, Windows XP, 512MByte DDR, ATA100 disk..  S= Split level = 0,1,2  C= Compression = 0,1  Each test run 5 times in succession, lines show range in results –in later runs file is typically cached in memory == faster if task IO limited  Task: Loop over all events, loop over all tracks, sum px for all tracks. 5000 event files created with “event” test program distributed with Root (200-400 MByte). CSms/Event 0012.3 - 4.23 0110.7 -- 2.75 021.70 -- 0.20 1010.5 -- 5.12 119.6 -- 4.77 122.7 -- 0.25

9. Network Root IO (x)rootd  Recently added ability for Java library to talk directly to rootd or xrootd server –Defined “root:” protocol handler for URL class. –Defined Authenticator for authenticating to rootd –Extended RootFileReader to accept (any) URL argument to constructor  If URL corresponds to file: read the file directly  If URL corresponds to root: protocol use xrootd/rootd (random access)  If URL corresponds to http: and web server supports http 1.1 “ranges”, use extended http  Otherwise buffer URL contents into memory and read from there (suitable for small files only).

10. Authentification Issues  Authentification can be set programmatically, or via GUI Authenticator  Rootd supports many authentification schemes 1. Anonymous (c.f. anonymous ftp) 2. User/Password (clear text) 3. User/Password (RSA encrypted) 4. SCP 5. SSH like 6. GSI certificate 7. UID/GID Currently FreeHep library supports only 1 and 2Currently FreeHep library supports only 1 and 2 Work underway on 3 (but some compatibility problems between Root and Java RSA implementations)Work underway on 3 (but some compatibility problems between Root and Java RSA implementations) Java libraries exist for all other methods, so can be supported in future.Java libraries exist for all other methods, so can be supported in future.

11. Network Root IO: Example of Use

12. Demo Apps: Root Object Browser

13. Demo Apps: Root Histogram Browser

14. Root Plug-in for JAIDA  JAIDA = Java implementation of AIDA  AIDA = Abstract Interfaces for Data Analysis = Histograms + Tuples + Fitting + Plotting + IO data analysis package –(See Monday’s talk by Victor Serbo)  Root plugin for JAIDA makes it possible to read root files via the AIDA interfaces and treat the objects contained in the root file as AIDA objects –Useful if:  You like AIDA interfaces better than root interfaces  You want to use language not supported by Root  You want to analyze data in mixed formats  You want to work in J2EE, J2ME, Applet etc.

15. Root Plug-in for JAIDA: Example

16. Root Java IO: Examples of Use AIDATLD Web Interface JAS3WIRED4

17. AIDATLD + Root  AIDATLD = AIDA Tag Library for use in JSP pages –Trivial to put “live” histograms on web –Directly supports Root files

18. AIDATLD Example: GLAST System Tests Web Interface

19. JAS3 + Root  JAS3 = Modular, Extensible, Data Format Independent data analysis system.  Among many other formats JAS3 supports Root –Contents of Root files can be accessed directly, or via Root->AIDA interface  Full access to AIDA functionality for fitting, projections, plotting etc. –Can operate on Root data  using GUI  using scripting (Python, Pnuts)  using compiled Java code

20. JAS3 Access to Root Files

21. JAS3 + Root: Scripting

22. Root + WIRED4  Build yourself a super cool event display for your custom Root file in under 1 hour!

23. Future Plans  Continue to track new Root releases –Root network protocol  More work on authentification –Root file writing  Most of work already done by Petr Vokac/Julius Hrivnac  Easier than reading because only need to support one root protocol version  Support Root XML format –Will be much slower, but perhaps more robust  Use LCGDICT system to allow Java access to C++ methods as well as data? –Probably also slower, but more flexible

24. More Info  Freehep Java Library –http://java.freehep.org/ http://java.freehep.org/  Root IO component of library –http://java.freehep.org/lib/freehep/doc/root/ http://java.freehep.org/lib/freehep/doc/root/  AIDATLD –http://aidatld.freehep.org/ http://aidatld.freehep.org/  JAIDA –http://java.freehep.org/jaida/index.html http://java.freehep.org/jaida/index.html  JAS3 –http://jas.freehep.org/jas3 http://jas.freehep.org/jas3