ElVis web services and systems Matt Milano Mentor - Eliot Feibush

Scientific Graphics

3 Data Monitoring Integrated with Job Monitoring

Web Services

ElVis Software Package Display program – 45,000 lines of Java code – 300 classes – Summer ’08: Over 200 updates to the code Application Programming Interface (API) – Fortran – C/C++ – Java

08/08/08 Contribution – Matthew Milano NSTX web service – HTML parsing Balloon Transp/rplot Global Status Panel Printing Preferences.java

08/08/08 NSTX web service Completely new feature Integrating NSTX-generated Graphs with ElVis through eml Many components many people many languages (perl, html, java, idl)‏

08/08/08 NSTX before ElVis Web-interface portal for retrieving graphs Completely static png images of graphs or datasheets Original idea was to have NSTX tools launch ElVis, mentor suggested this “easier” solution – many html implementations existed in ElVis.

08/08/08 Original web interfaces follows flowchart Goal – change as little as possible while still getting all new benefits of ElVis Webtools overview IDL graph creator

08/08/08 NSTX with ElVis Multiple graph windows Interactive data exploration Compare Signals through copy and paste Zoom and save graphs Print the graph at any resolution/aspect ratio Interactive log plots

08/08/08 NSTX challenges Html forms which use javascript in java – not good Java has no javascript support FormView class supplies rudimentary form support, but creates inaccessible java elements Original task – no similar quick how-to's ElVis help, URL loading all use html Eml is already loaded through protocol 95% of task accomplished in 5% of the time – remaining 5% proved onerous

08/08/08 NSTX solutions Having a standard format for graphs made writing to ElVis an easy task for the NSTX developers (Bill Davis) Create a recursive parsing method Get root htmlDocument element and iterate through all children elements Check tag type (using an element toString) and add all auto-created java elements to Vector (a list of objects) Catch Button during iteration and overload with custom action listener Remove original button action listeners

08/08/08 NSTX – input fields Iterate through all members of Vector Override FormView class to make createComponent() publicly visible Apply Overrided class to all elements of Vector with “text” identifiers Attempt to cast into Vector Remove all failed (null) attempts

08/08/08 NSTX – action handler Add ElVis-generated filename to URL string Open new UrlConnection to perl server with rendered url string (over 1000 characters)‏ Iterate through Vector and Vector, get text, and assemble into URL string (to create URL as if $_GET[] variable had been used)‏

08/08/08 NSTX – perl server Developed by Bill Davis Perl server runs in web-accessible directory (/p/fgweb) and receives requests in the form of GET Modified perl server to remove automatically generated names and accept ElVis-generated names Moved server, changed output style and directory

08/08/08 NSTX – end result Nicely parsed, accessible HTML form turned into Java elements dynamically Process can be applied to any HTML form Pretty picture here

08/08/08 Used at several institutes world-wide. New physics models being added but still using legacy plotting code. Access analysis runs stored in database. Scripted to formulate graphs of fusion data. ElVis API sends data to visualization client with minimal changes to Fortran code. ElVis: A Portal for Scientific Graphics E. Feibush, D. McCune, L. Randerson Transport Analysis Legacy Code Multiple Graph Windows 1. Visualization Client 5. Compute Services Control Server Firewall Explore Data Interactively Animation & Whiteboard Security Panel Data Monitoring Verify Input - Monitor Results Subset of total output is stored as URL. Client polls the URL, appends new data to graphs. Animate multiple variables Data highlighter tool displays colored traces in data space for each time step in animation. Zoom & scroll with view. Compare experiments to simulation data. Java applet. Runs in browser. Requests data from server running legacy Fortran application. Multi-Tier Architecture Distributes Functionality 3. Portal Server Java & Jython programs import ElVis graph classes and gain all encapsulated methods for interaction & display. Applications Requests Graph Objects Credentials Register sessions. Manage credentials. Create graphs. Collaboration among clients. ElVis XML 6. Experiment Data Server Access from anywhere on Internet. Receives requests from visualization clients. Forwards servlet requests through PPL firewall. Returns graph objects. Command Line I/O Controller Master Pseudo-Terminal Slave Certificates stored on remote myProxy server. Convenient for user: enter name and password in security panel instead of managing files. MDSplus Database Special interprocess communication channel sends text between master & slave processes. Master forks slave which execs legacy plotting code. Buffering of standard I/O prevents trivial Linux pipe. Globus job-submit Acquired Data Analysis Runs Encapsulated Methods User Files 2. HTTP Web Server Visualization Servlet ElVis API 4. myProxy Server X.509 Certificates

08/08/08 rplot & transp server-side processes Existing, heavily-used pppl analysis programs which are older than me. Rplot and transp utilities run on a separate server and are accessible with a jython interpreter within ElVis ElVis itself is not tied to the transp or rplot services – the jython interpreter must be used to send commands to clterm.c which then executes commands on the server. If a user runs a command that interrupts the jython process, server does not receive any new data, and thus keeps defunct processes alive.

08/08/08 End server-side programs Transp and rplot can be killed by specific commands Commands must be sent when an ElVis process interrupts/destroys jython process (ie window close event, exiting ElVis)‏ Add methods which send commands, call them when action is triggered, thereby sending the signals right as ElVis exits.

08/08/08 Problem Clterm and 4 others (out of 5) live Transp has no problems Solution: – Created shell script to kill all old pshrxxx processes. – Sort through all processes, regex matching on date, execution runtime, and executing user

08/08/08 Balloon web service usability enhancements Fixed input checking – added a toUpperCase method to remove case-sensitive errors Made input checking occur earlier in transp retrieval and refined method to check for each field's accuracy instead of check if the first field was accurate Added status panel, allowing the user to see server-side console output.

08/08/08 Balloon: plasma state, GEQ Plasma state and GEQ files can also be used for/results in a balloon run – field created but no implementation Created utility to copy plasma state/GEQ files Added utility to ElVis servlet, made generic so other processes may use it.

08/08/08 Global Status Panel Challenge: can we put a JinternalPane in ElVis which contains the console output instead of printing it to the standard output window? Can we make it so that other classes can write to it? Can we also log console output using a similar method? There are over 300 classes in ElVis – we don’t want to have to edit 300 files!

08/08/08 Use Java logger class to create new ByteStream PrintStream log Redirect System.err and System.out to new PrintStream Create new Preferences.java entry for saving log, and remembering log numbers (max 10 logs)‏ Allows anyone who experiences bugs to send us their log files for troubleshooting Global Status Panel

08/08/08 Print to pdf/ps Feature already existed Tarun Pondicherry implemented aspect ratios into main ElVis graph windows last summer Aspect ratios not properly preserved in printing – code attempted to cast all aspect layouts to 4:3 or 3:4 Resulted in unrenderable and unreadable graphs.

08/08/08 Print to ps Built-in java libraries make job a little easier Create publicly visible variables with the ratio of height to width, modified at every window move Inject public ratio variables instead of setting a hard-locked 4:3, define custom orientation based on image size.

08/08/08 Print to pdf - complications No built-in java libraries Not enough time to pursue nuances of open- source libraries during initial implementation Bad programming practice in open-source library – entire class is within constructor, pdf printing treated as a static method rather than an object. Constant aspect ratio re-definition – occurs at least 6 times within the class

08/08/08 Print to pdf solution Create (the only non-constructor) method to recalculate offsets and resize image Continually use this method every time aspect ratio was hard-coded in.

08/08/08 Remembering user selection First assignment – worked with Michael Shmulevich Have ElVis remember last URL, last opened file, last log file, last printed file, and last saved file Used static strings to store data during an ElVis run

08/08/08 Keeping preferences across sessions Re-wrote existing file Preferences.java Changed loading method to load entire file into string array – Easy to loop through – Slightly more memory efficient than an ArrayList or Vector. Created individual save methods for each element – Load into memory and loop through existing file in search of key (ex. load_file=)‏ – Modify existing entry if found, if none is found append key and entry to the end of existing file.

08/08/08 Preferences.java – log preferences Printing methods – print multiple aspect ratios Balloon Service – use GEQ and Plasma State, see logged progress Rplot and transp – exit cleanly Elvis servlet – generic copy method NSTX web tools – completely new feature to request NSTX shot/signal graphs In-ElVis console: ability to see console output when running in applet mode or just without cli Creation Summary

08/08/08 Acknowledgements Eliot Feibush – my direct mentor Bill Davis – author of original NSTX web tools Andrew Zwicker, Christine Ritter, James Morgan – PPPL science education program Eric Zatz – a key member of the ElVis dev team

Questions?

Ben – API, EML Mike Movie Making User Enhancements Tutorials Eric Zatz -CPPG summer undergrad ElVis Mark-Up Language Matt System & Services Eliot - Mentor Doug – Mentor CPPG Group Head