1. e-Viz Towards an Adaptive Framework for Visualization on the Grid

2. http://www.eviz.org/ e-Vizzers UW Swansea UW Bangor U Manchester U Leeds e-Viz is a three year joint research project funded by UK EPSRC Four partner universities: –University of Leeds –(Ken Brodlie, Jason Wood) –University of Manchester –(John Brooke, Mark Riding) –University of Wales, Bangor (Nigel John, Chris Hughes) –University of Wales, Swansea –(Min Chen, David Chisnall, Mark Jones, Nicolas Roard)

3. http://www.eviz.org/ What problem are we trying to solve? Data Choose Visualization Algorithm Choose Software to Implement Algorithm Choose Hardware to run on Which are applicable to the data? Which is the most suitable? Whats fastest? Availability? Domain Scientist Visualization Specialiste-Scientist

4. http://www.eviz.org/ Why are we trying to solve this? Because not all potential Grid visualization users are experts in visualization and Grid technologies.

5. http://www.eviz.org/ Why are we trying to solve this? Visualization Experience Grid Experience Application Scientists Visualization Specialists Grid & Visualization Developers e-Scientists

6. http://www.eviz.org/ Putting Visualization on the Grid Can the Grid help? – its supposed to be: –heterogeneous in architecture –seamless and transparent in use –fault tolerant in operation –capable of adapting to the changing environment in order to provide the best service.

7. http://www.eviz.org/ What can the Grid offer? Heterogeneity: architectures SMPNUMADesktopSpecialised ComputationRendering Cluster

8. http://www.eviz.org/ What can the Grid offer? Heterogeneity: operating systems LinuxIrixAIXSolarisBSDHP/UXWindowsMacOS

9. http://www.eviz.org/ What can the Grid offer? Heterogeneity: visualization software AVS IRIS Explorer OpenDXVTKrtrtSimianVolumizerMatlab

10. http://www.eviz.org/ What can the Grid offer? Heterogeneity: Implications –There are a wide variety of architectures, operating systems and software applications used in visualization. –Each has its own particular rewards and benefits, but we cant expect every Grid user to be knowledgeable and proficient with each, or even to know which is the most appropriate for a given task. –Want to be able to offer the power afforded by each application to all potential users of visualization.

11. http://www.eviz.org/ What can the Grid offer? Seamlessness and transparency: Implications –End users should only have to learn one interface, but still be able to benefit from the features offered by a wide range of applications. –But each visualization application has its own user interface… –… and every operating system looks and behaves differently

12. http://www.eviz.org/ What can the Grid offer? Fault tolerance: Implications –A distributed system is one in which the failure of a computer you didnt even know existed can render your own computer unusable – Leslie Lamport –A Grid system is one in which the failure of a computer you didnt even know existed goes by unnoticed –Users should be able to rely on the Grid

13. http://www.eviz.org/ What can the Grid offer? Fault Tolerance - User Scenario … keyboard, please

14. http://www.eviz.org/ What can the Grid offer? Adaptation: Implications –The world is dynamic and ever changing; so is the Grid. –Network loads and status –Queues on HPC machines –Runtime limits on HPC machines –CPU loading, both remotely and locally –A Grid system should adapt to cope with changes

15. http://www.eviz.org/ How can we solve this problem?

16. http://www.eviz.org/ What is Visualization? VisualizationDataImage

17. http://www.eviz.org/ What is Visualization? FilterMapRenderDataImage

18. http://www.eviz.org/ What is Visualization? FilterMapRenderDataImageControls

19. http://www.eviz.org/ What is Visualization? VisualizationDataImageControls

20. http://www.eviz.org/ Common Abstract Interface VisualizationDataImageControls

21. http://www.eviz.org/ Common Abstract Interface VisualizationDataImageControls Computational Steering Frame Transport Format Conversion

22. http://www.eviz.org/ Common Abstract Interface VTKDataImageControls Computational Steering Frame Transport Format Conversion AVS/Express IRIS Explorer

23. http://www.eviz.org/ Common Abstract Interface DataImageControls Computational Steering Frame Transport Format Conversion VTKrtrtAVSSimian

24. http://www.eviz.org/ Common Abstract Interface Feasibility –can we create equivalent visualizations using different software packages?

25. http://www.eviz.org/ Equivalent Visualizations?

26. http://www.eviz.org/ Equivalent Visualizations?

27. http://www.eviz.org/ Abstraction Need to describe the pipeline itself – need an abstract visualization description language –gViz project gives us skML Visualization Computational Steering Frame Transport Format Conversion

28. http://www.eviz.org/ Abstraction Format Conversion –Most visualization software applications can already read a wide range of data formats Visualization Computational Steering Frame Transport Format Conversion

29. http://www.eviz.org/ Abstraction Computational Steering Visualization Computational Steering Frame Transport Format Conversion

30. http://www.eviz.org/ Abstraction Computational Steering –RealityGrid or gViz APIs can be used to control a running pipeline –Visualization applications must be instrumented to expose their steerable pipeline parameters –Currently instrumented VTK and RTRT (real time ray tracer) –GUI created, which is really a specialised Computational Steering client -Reads in a pipeline description and dynamically configures itself to show appropriate widgets

31. http://www.eviz.org/ Abstraction Frame Transport Visualization Computational Steering Frame Transport Format Conversion

32. http://www.eviz.org/ Abstraction Frame Transport –Images can be rendered locally or remotely – in either case, they need to be displayed on the users own machine, as well as any collaborators –Have created a library to compress remote images and transport them back to the client for display. Visualization applications can be modified to use the library –Range of image compression codecs –Local rendering supported via a rendering overlay –Linux and Windows clients

33. http://www.eviz.org/ What do we still need? Decision making –What visualizations are applicable for a given input data type –Which of the available hardware and software is most suitable to implement such a pipeline –User may not know the answer to these questions – provide assistance

34. http://www.eviz.org/ Architecture Software StoreClientBrokerData StoreHPV Machine WSRF Globus gViz e-Viz

35. http://www.eviz.org/ Architecture (Current) ClientBrokerHPV Machine Software StoreData Store WSRF Globus gViz e-Viz

36. http://www.eviz.org/ Demonstration 1 gViz Pollution Demonstrator - shows –Active simulations as a data source –Computational steering of an active visualization –Common interface to servers

37. http://www.eviz.org/ Demonstration 1

38. http://www.eviz.org/ Demonstration 2 Volume Render Demonstrator – shows –Heterogeneous access to multiple servers –(Almost) seamless switching between servers –Fault tolerance through redundancy

39. http://www.eviz.org/ Demonstration 2

40. http://www.eviz.org/ The Knowledge Gap The Broker has a knowledgebase which it will use to make informed decisions on pipeline choices. Its empty!

41. http://www.eviz.org/ How to Distribute? FilterMapRenderDataImage

42. http://www.eviz.org/ How to Distribute? FilterMapRenderDataImage

43. http://www.eviz.org/ How to Distribute? FilterMapRenderDataImage

44. http://www.eviz.org/ How to Distribute? FilterMapRenderDataImage

45. http://www.eviz.org/ How to Distribute? FilterMapRenderDataImage

46. http://www.eviz.org/ When to Distribute? Depends on –Where the dataset resides –How big the dataset is –Network bandwidth –Available machines –Chosen algorithm –Queue status –CPU capabilities –Graphical capabilities –…..

47. http://www.eviz.org/ Other problems Single point of failure in the broker

48. http://www.eviz.org/ Conclusions e-Viz has created a framework for visualization on the Grid that is: –Heterogeneous in architecture –Seamless and transparent in use –Fault tolerant –Adaptive (potentially) But much more to be done! (project runs another 16 months)

49. http://www.eviz.org/ For more information See a live demo: –Tuesday 22 nd, 14:00-16:30, White Rose Grid stand Project Web Site: –http://www.eviz.orghttp://www.eviz.org email: –mark.riding@manchester.ac.uk