31 January 2007Craig E. Ward1 Large-Scale Simulation Experimentation and Analysis Database Programming Using Java

31 January 2007Craig E. Ward2 Agenda Background on problem presented by the simulations. Architecture of the Scalable Data Grid (SDG) solution. Java technology choices. Future Directions for SDG Questions

31 January 2007Craig E. Ward3 Simulation Environment Joint Semi-Autonomous Forces (JSAF) environment in Urban Resolve experiments. Distributed across multiple workstations and two 128-node clusters. Goal of simulating 1,000,000 entities.

31 January 2007Craig E. Ward4 Urban Resolve 2015 Sites

31 January 2007Craig E. Ward5 Logging Simulation Data Data loggers captured simulation data from JSAF environment Before analysis, data moved to central store. Experiments generating multiple terabytes of data. Data difficult to manage and analyze.

31 January 2007Craig E. Ward6 Scaleable Data Grid SDG is a system for distributed collection and analysis of large amounts of data. Initial prototype deployment for JSAF environment in Urban Resolve 2015 experiment. Goals Make the analysis of the data more useful. Make the management of data more convenient.

31 January 2007Craig E. Ward7 SDG n-tier Architecture Java applications Services between nodes Services to client simulations Web application User Interface Database engine MySQL

31 January 2007Craig E. Ward8 Multidimensional Cubes Use the raw data to create multidimensional cubes of dimensions of interest. Dimensions “roll up” into categories of greater abstraction. Detection Cube

31 January 2007Craig E. Ward9 SDG Cube Implementation Concepts Cube Description What the possible ranges are for indices (coordinates) and measures in a Cube. Cube View A selection of a subset of the possible indices and measures of interest in a query. Cube A instantiation of a Cube that matches a Cube View. Comprised of Elements with indices and measures.

31 January 2007Craig E. Ward10 Distributed Communication Communication between processes and nodes with Remote Method Invocation (RMI). Cube Manager classes register handles with RMI servers Managers return handles to Cubes Descriptions and Views are serialized and passed to clients.

31 January 2007Craig E. Ward11 Distributed Communications

31 January 2007Craig E. Ward12 JDBC Cubes do not exist in their entirety outside of the database tables. Tables store simple elements of the Cubes. Measures at the coordinates in relational table. Elements are very simple Java objects.

31 January 2007Craig E. Ward13 Cube Generator Processing

31 January 2007Craig E. Ward14 Threaded Processing Interface to simulations using sockets and pooled Thread objects. Internal mechanisms use Java 5 Executor classes and thread-safe queues from the concurrent processing package. Generator Query Aggregation

31 January 2007Craig E. Ward15 SDG Web Application HTML Forms for query selection. Display of results of queries. Java Server Pages Create HTML pages. JavaScript Manipulate DOM to display form. Java servlets Interface between backend and JSP pages.

31 January 2007Craig E. Ward16 Java Technologies used JDBC Interaction with the database. RMI Communication between nodes. Threads “Traditional” and new Java 5 concurrent classes. Sockets Interface to simulations.

31 January 2007Craig E. Ward17 Technologies Not Used Java Data Objects Complicates build process. Hibernate Another product to configure and distribute. Java EE Too complex. Clustered JDBC (Sequoia) Requires that tables be uniform.

31 January 2007Craig E. Ward18 Future SDG Development (?) Enhance web application with JSF, Ajax. Caching of cube view results. Develop ad hoc query capability. Implement an algebra for manipulating cubes. Dynamic update of cube descriptions. Dynamic reconfiguration of communication links.

31 January 2007Craig E. Ward19 Questions & Comments