Gordon Erlebacher Florida State University Vlab Framework Gordon Erlebacher Florida State University
Typical workflow workflow job submission Elasticity code input data collection ? data analysis and visualization Defines a workflow Phonon code … more of the same …
VLAB design principles All services are available at multiple locations (redundancy) No single point of failure Automatic resource discovery Development of client interfaces and services are decoupled User interaction is through web browsers (and other specialized client software) Assume the system will be (eventually) be used by hundreds of users on thousands of systems Independence of the languages used to write the services
VLAB requirements Collaboration Ease of use Flexibility Fault tolerance (hardware failures) Diversity of tools analysis, visualization, data reduction, storage, etc .
IT team (US component) Gordon Erlebacher (FSU) Dave Yuen (UMN) (client interfaces) Marlon Pierce (IU) (portals) Bijay Karki (LU) (visualization) Stefano Cozzini (Italy) (scheduling) Kirsten Kleese van Dam (UK) (metadata)
Mantle Convection 5003 grid Ra=1010
Mantle Convection Ra = 5 x 108 Streamlines Isosurface Line Integral Convolution Ra = 5 x 108 Produced with Amira
ElasViz
Visualization Algorithm Challenges Strike balance between High- resolution versus interactive speed Batch versus interactivity Intelligent compression Time-dependent datasets Interactive navigation through a Terabyte dataset across a distributed system? How to share datasets from > 10003 time-dependent simulations
Client devices University of Minnesota
Clients Clustering of earthquake events WEB-IS Mantle convection HTML interface to AMIRA Clients Clustering of earthquake events WEB-IS Mantle convection
VLAB components Portals (user front end) Services (tasks) Schedulers (schedule task execution) Data/metadata transport mechanisms Middleware to connect 1-4 based on NaradaBrokering
Message structure (publish-subscribe) Topic Headers Body: SOAP message
Message Flow with SOAP Client specfies remote commands Service executes remote commands Remote call is encoded in soap message Return values are encoded in soap message
Messaging with SOAP through Naradabrokering Client Service Narada- Brokering SOAP SOAP Wrapped SOAP Wrapped SOAP Client Proxy Service Proxy
Grid – Collaboration NaradaBrokering www.naradabrokering.com
Workflows Services have multiple inputs/outputs Connect services together to form a workflow conditionals loops Elasticity code input job submission data collection data analysis and visualization ? Phonon code
Schedulers Take a high level task definition and break it into subtasks Submit these subtasks to appropriate services Return results to the subscribers (role of HPSEARCH)
Grid – Collaboration NaradaBrokering www.naradabrokering.com
Everything is a service Data input Job submission Analysis tools Visualization tools Data storage Scheduling
Services Can be written in any language with an interface to SOAP Executed via remote calls Defined in terms of an interface (WSDL)
Simulation Services Create “smart” input files Job submissions hide details of computational resources, concentrate on requirements for successful run(s)
Storage Services Retrieve files from storage Supercomputer File servers Archival storage Process file (on one or multiple servers) Extract subset Filter subset Transform subset Send processed file for computing or visualization services
Visualization Services Offscreen, hardware-accelerated, remote visualization Wrap existing visualization software Amira VTK Specialized applications (e.g. elasviz -- Karki 2005) Movie creators
Portals Front end for users Reconfigurable Used for job submission visualization data analysis monitoring collaboration
Summary Challenging goals Potential payback is enormous Accomplishments Initial portals to Espresso input Leveraging of existing software for job submission In the process of converting visualization tools to visualization Initial version of Naradabrokering-based framework is in progress
10 months in the future … Implement test workflow data input and submission of many elastic codes collection and visualization of data basic collaboration between two (or more) individuals