Gordon Erlebacher Florida State University Vlab Framework Gordon Erlebacher Florida State University

Typical workflow workflow job submission Elasticity code input data collection ? data analysis and visualization Defines a workflow Phonon code … more of the same …

VLAB design principles All services are available at multiple locations (redundancy) No single point of failure Automatic resource discovery Development of client interfaces and services are decoupled User interaction is through web browsers (and other specialized client software) Assume the system will be (eventually) be used by hundreds of users on thousands of systems Independence of the languages used to write the services

VLAB requirements Collaboration Ease of use Flexibility Fault tolerance (hardware failures) Diversity of tools analysis, visualization, data reduction, storage, etc .

IT team (US component) Gordon Erlebacher (FSU) Dave Yuen (UMN) (client interfaces) Marlon Pierce (IU) (portals) Bijay Karki (LU) (visualization) Stefano Cozzini (Italy) (scheduling) Kirsten Kleese van Dam (UK) (metadata)

Mantle Convection 5003 grid Ra=1010

Mantle Convection Ra = 5 x 108 Streamlines Isosurface Line Integral Convolution Ra = 5 x 108 Produced with Amira

ElasViz

Visualization Algorithm Challenges Strike balance between High- resolution versus interactive speed Batch versus interactivity Intelligent compression Time-dependent datasets Interactive navigation through a Terabyte dataset across a distributed system? How to share datasets from > 10003 time-dependent simulations

Client devices University of Minnesota

Clients Clustering of earthquake events WEB-IS Mantle convection HTML interface to AMIRA Clients Clustering of earthquake events WEB-IS Mantle convection

VLAB components Portals (user front end) Services (tasks) Schedulers (schedule task execution) Data/metadata transport mechanisms Middleware to connect 1-4 based on NaradaBrokering

Message structure (publish-subscribe) Topic Headers Body: SOAP message

Message Flow with SOAP Client specfies remote commands Service executes remote commands Remote call is encoded in soap message Return values are encoded in soap message

Messaging with SOAP through Naradabrokering Client Service Narada- Brokering SOAP SOAP Wrapped SOAP Wrapped SOAP Client Proxy Service Proxy

Grid – Collaboration NaradaBrokering www.naradabrokering.com

Workflows Services have multiple inputs/outputs Connect services together to form a workflow conditionals loops Elasticity code input job submission data collection data analysis and visualization ? Phonon code

Schedulers Take a high level task definition and break it into subtasks Submit these subtasks to appropriate services Return results to the subscribers (role of HPSEARCH)

Grid – Collaboration NaradaBrokering www.naradabrokering.com

Everything is a service Data input Job submission Analysis tools Visualization tools Data storage Scheduling

Services Can be written in any language with an interface to SOAP Executed via remote calls Defined in terms of an interface (WSDL)

Simulation Services Create “smart” input files Job submissions hide details of computational resources, concentrate on requirements for successful run(s)

Storage Services Retrieve files from storage Supercomputer File servers Archival storage Process file (on one or multiple servers) Extract subset Filter subset Transform subset Send processed file for computing or visualization services

Visualization Services Offscreen, hardware-accelerated, remote visualization Wrap existing visualization software Amira VTK Specialized applications (e.g. elasviz -- Karki 2005) Movie creators

Portals Front end for users Reconfigurable Used for job submission visualization data analysis monitoring collaboration

Summary Challenging goals Potential payback is enormous Accomplishments Initial portals to Espresso input Leveraging of existing software for job submission In the process of converting visualization tools to visualization Initial version of Naradabrokering-based framework is in progress

10 months in the future … Implement test workflow data input and submission of many elastic codes collection and visualization of data basic collaboration between two (or more) individuals