1. Collaborative Virtual Environments For Scientific Visualization: AMMI Lab. Contributions to The WestGrid Project Dr. Pierre Boulanger Advanced Man-Machine Interface Laboratory University of Alberta http://www.cs.ualberta.ca/ammi Dr. Pierre Boulanger Advanced Man-Machine Interface Laboratory University of Alberta http://www.cs.ualberta.ca/ammi Department of Computing Science University of Alberta Department of Computing Science University of Alberta

2. Western Canada Grid SFU UBC/Triumf/NEWMIC UofA UofC UofL Banff Center Computational Resources Computational Resources Grid Storage Grid Storage Advanced Collaborative Environment Advanced Collaborative Environment Immersive Visualization Research Centre Immersive Visualization Research Centre Graphics Workstation Graphics Workstation

3. MACI Cluster Located at the UofA

4. The UofA MACI Cluster

5. The Grid From a Services Viewpoint Resource-specific implementations of basic services : E.g., Transport protocols, name servers, differentiated services, CPU schedulers, public key infrastructure, site accounting, directory service, OS bypass Resource-independent and application-independent services : E.g., authentication, authorization, resource location, resource allocation, events, accounting, remote data access, information, policy, fault detection Distributed Computing Applications Toolkit Grid Fabric (Resources) Grid Services (Middleware) Application Toolkits Data- Intensive Applications Toolkit Collaborative Applications Toolkit Remote Visualization Applications Toolkit Problem Solving Applications Toolkit Remote Instrumentation Applications Toolkit Applications Chemistry Biology Cyber Cell Cosmology Nanotechnology Environment Physics Earth Dynamo

6. Definition of Virtual Reality “ A virtual reality system is an interface between a man and a machine capable of creating a real-time sensory experience of real and artificial worlds through the various human sensory channels. These sensory channels for man are: Vision, Audition, Touch, Smell, and Taste.” Burdea, 1993 “ A virtual reality system is an interface between a man and a machine capable of creating a real-time sensory experience of real and artificial worlds through the various human sensory channels. These sensory channels for man are: Vision, Audition, Touch, Smell, and Taste.” Burdea, 1993

7. The Three I’s of Virtual Reality Immersion Interaction Imagination

8. Lets Start With An Example Collaboration: Moritz Heimpel, Institute for Geophysical Research, Department of Physics, UofA Pierre Boulanger, Advanced Man Machine Interface Lab., Department of Computing Science, UofA The Problem: Simulation and 3D Visualization of The Planetary Dynamo Problem Collaboration: Moritz Heimpel, Institute for Geophysical Research, Department of Physics, UofA Pierre Boulanger, Advanced Man Machine Interface Lab., Department of Computing Science, UofA The Problem: Simulation and 3D Visualization of The Planetary Dynamo Problem

9. The Planetary Dynamo Problem The Earth’s magnetic field Spatial structure, an approximate dipole Time history, geomagnetic reversals Inversion for outer core flow structure Inner core differential rotation Planetary dynamos Origin and structure of magnetic fields The Earth’s magnetic field Spatial structure, an approximate dipole Time history, geomagnetic reversals Inversion for outer core flow structure Inner core differential rotation Planetary dynamos Origin and structure of magnetic fields

10. Core Geometry: The Radius Ratio  = r i /r o Earth (  = 0.35) Mercury (  ~ 0.75) Ganymede (  ~0.2) Io (  ~ 0.50 ) Jupiter (c ~ 0.85) Earth (  = 0.35) Mercury (  ~ 0.75) Ganymede (  ~0.2) Io (  ~ 0.50 ) Jupiter (c ~ 0.85)

11. Equations of Motion

12. Some Non-Dimensional Parameters & Typical Values for the Numerical Simulations NumberDefinition Value Magnetic ReynoldsRm = VD/  Ekman numberE = /(  D 2 ) 10 -3 -10 -4 Rayleigh numberRa =  g o  TD 3 /(  ) 10 5 -10 7 Prandtl numberPr = /  Magnetic PrandtlPm =  Radius Ratio  = r i /r o 0.1 – 0.9 NumberDefinition Value Magnetic ReynoldsRm = VD/  Ekman numberE = /(  D 2 ) 10 -3 -10 -4 Rayleigh numberRa =  g o  TD 3 /(  ) 10 5 -10 7 Prandtl numberPr = /  Magnetic PrandtlPm =  Radius Ratio  = r i /r o 0.1 – 0.9

13. Comparison of Earth Magnetic Field With Dynamo Model (U. Christensen et al., 1999)

14. Model Geometry LEFT:  = 0.35 (Earth’s core); RIGHT:  = 0.75

15. Numerical Dynamo: c = 0.35, E = 10 -4, Pm = 1, Ra = 10Ra c

16. Single Plume Dynamo, c = 0.15 Temperature Isosurfaces Magnetic field lines Radial Magnetic Field at CMB

17. Initial Magnetic Field Lines, Field Magnitude Slice and Equatorial Temperature Slice  = 0.35 Ra = 4 Ra c E = 10 -3 P = 1 P m = 5 Weak initial magnetic field  = 0.35 Ra = 4 Ra c E = 10 -3 P = 1 P m = 5 Weak initial magnetic field

18. Velocity & Magnetic Field  = 0.35 Ra = 4 Ra c E = 10 -3 P = 1 P m = 5 Weak initial magnetic field  = 0.35 Ra = 4 Ra c E = 10 -3 P = 1 P m = 5 Weak initial magnetic field

19. Steady Dynamo: Various Visualizations a) Vorticity isosurfaces & velocity streamlines b) Volume vorticity & magnetic fieldlines c) Velocity magnitude d) Z – magnetic field a) Vorticity isosurfaces & velocity streamlines b) Volume vorticity & magnetic fieldlines c) Velocity magnitude d) Z – magnetic field

20. Real time visualization: Motivation Understanding the variation of radius ratio could be a key for understanding the magnetic fields of planetary dynamos Dynamos of intermediate shell thickness are surprisingly Earth-like. Thick shell dynamos typically have single plume flow field. Thin shell dynamos have weaker dipole fields and are characterized by smaller scale flow and magnetic field scaling. Construction of real-time visualization system will aid interpretation of field structures. Understanding the variation of radius ratio could be a key for understanding the magnetic fields of planetary dynamos Dynamos of intermediate shell thickness are surprisingly Earth-like. Thick shell dynamos typically have single plume flow field. Thin shell dynamos have weaker dipole fields and are characterized by smaller scale flow and magnetic field scaling. Construction of real-time visualization system will aid interpretation of field structures.

21. Project CyberCell Fundamentally, biology is a visual science. Up until very recently, nearly everything in biology was described in terms of what could be seen or inferred through the naked eye. Therefore, the principle objective of all of Project CyberCell's simulation efforts is to produce visually and informationally rich reconstructions of cellular activity. Fundamentally, biology is a visual science. Up until very recently, nearly everything in biology was described in terms of what could be seen or inferred through the naked eye. Therefore, the principle objective of all of Project CyberCell's simulation efforts is to produce visually and informationally rich reconstructions of cellular activity.

22. Project CyberCell In other words we want to create spatially and temporally correct models of what really happens inside a cell. http://www.projectcybercell.com/ The Institute for Biomolecular Design (IBD) was established in 1998 as a $25.6 million investment In other words we want to create spatially and temporally correct models of what really happens inside a cell. http://www.projectcybercell.com/ The Institute for Biomolecular Design (IBD) was established in 1998 as a $25.6 million investment

23. Project CyberCell The objective of Project CyberCell is to develop an accurate simulation of a living cell within the virtual environment of a computer, one that can be manipulated at different levels of molecular resolution, and, that can respond, adapt and evolve to exploit this virtual environment. Project CyberCell has selected the bacterium E. coli as its model. The objective of Project CyberCell is to develop an accurate simulation of a living cell within the virtual environment of a computer, one that can be manipulated at different levels of molecular resolution, and, that can respond, adapt and evolve to exploit this virtual environment. Project CyberCell has selected the bacterium E. coli as its model.

24. Project CyberCell

25. Collaboration and Visualization Tools are Essential for WestGrid WestGrid is predicated on creating unique regional collaborations between a diverse group of researchers distributed across a wide geographical area. It is essential that WestGrid researchers have access to innovative suites of tools that enable collaborations and natural interactions (as if in the same room) regardless of physical distance. WestGrid is predicated on creating unique regional collaborations between a diverse group of researchers distributed across a wide geographical area. It is essential that WestGrid researchers have access to innovative suites of tools that enable collaborations and natural interactions (as if in the same room) regardless of physical distance.

26. It is these communications tools that will help create distinctive research capabilities and identities. By allowing researchers to interact both casually and formally—quickly and easily visualizing and manipulating data—the infrastructure will enhance investigations in all disciplines. It is these communications tools that will help create distinctive research capabilities and identities. By allowing researchers to interact both casually and formally—quickly and easily visualizing and manipulating data—the infrastructure will enhance investigations in all disciplines. Collaboration and Visualization Tools are Essential for WestGrid

27. Shared virtual environments, advanced multi-point video conferencing, shared applications well beyond white-boarding, new display technologies, and video streaming will be used together and separately for research collaborations and visualizations, as well as for resource management and training.

28. Others Systems Developed Around the World DOE Advanced Photon Source TIDE: the Tele-Immersive Data Explorer NASA Virtual Wind Tunnel The Collaborative Image Based Rendering Viewer (CIBR View) Argon Access Grid EVL: The Collaborative Continuum National Tele-Immersion Initiative DOE Advanced Photon Source TIDE: the Tele-Immersive Data Explorer NASA Virtual Wind Tunnel The Collaborative Image Based Rendering Viewer (CIBR View) Argon Access Grid EVL: The Collaborative Continuum National Tele-Immersion Initiative

29. tomographic reconstruction real-time collection real-time collection wide-area dissemination wide-area dissemination desktop & VR clients with shared controls Advanced Photon Source Virtualized Reality Allow On-line Instrumentation archival storage archival storage DOE X-ray grand challenge: ANL, USC/ISI, NIST, U.Chicago—source: Carl Kesselman

30. TIDE: the Tele-Immersive Data Explorer Electronic Visualization Laboratory University of Illinois at Chicago, USA National Center for Data Mining, University of Illinois at Chicago Electronic Visualization Laboratory University of Illinois at Chicago, USA National Center for Data Mining, University of Illinois at Chicago TIDE is a CAVERNsoft-based collaborative, immersive environment for querying and visualizing data from massive and distributed datastores. www.evl.uic.edu/cavern www.ncdm.uic.edu www.evl.uic.edu/cavern www.ncdm.uic.edu

31. NASA Virtual Wind Tunnel NASA Virtual Wind Tunnel (VWT) is an application of virtual reality interface technology to the visualization of the results of modern computational fluid dynamics simulations. The highly interactive three-dimensional nature of virtual reality provides an intuitive exploration environment for the analysis of the complex structures arising in time varying fluid flow simulations. A variety of standard visualization techniques are supported in the virtual wind tunnel. These visualizations are controlled via a direct manipulation paradigm by visualization control tools. http://www.nas.nasa.gov/Software/VWT/

32. Tele-Immersive Image Based Rendering Electronic Visualization Laboratory, University of Illinois at Chicago, USA Lawrence Berkeley National Laboratory, USA Electronic Visualization Laboratory, University of Illinois at Chicago, USA Lawrence Berkeley National Laboratory, USA The Collaborative Image Based Rendering Viewer (CIBR View) is a CAVERNsoft-based tool for viewing animated sequences of image-based renderings from volume data. CIBR View was designed to allow DOE scientists to view volume renderings composed of 2D image slices. www.evl.uic.edu/cavern/cibr

33. The Access Grid

34. Access Grid does for people what the computational Grid does for machines The Access Grid project focus is to enable groups of people to interact with Grid resources and to use the Grid technology to support group to group collaboration at a distance Distributed Lectures and seminars Remote participation in panel discussions Virtual site visits meetings Complex distributed grid based demonstrations Access Grid does for people what the computational Grid does for machines The Access Grid project focus is to enable groups of people to interact with Grid resources and to use the Grid technology to support group to group collaboration at a distance Distributed Lectures and seminars Remote participation in panel discussions Virtual site visits meetings Complex distributed grid based demonstrations

35. EVL: The Collaborative Continuum Passive stereo VR display Camera array for image based panorama Wireless tablet PCs + camerasvelcroedto wall for private video or persistentpostits Wireless mobile Plasma Touch screen Persistent flip notes 5Ghz 40Mbps 802.11a Tiled display (LCD tiles for high resolution, or plasma screens) Electronic Visualization Laboratory (EVL), University of Illinois at Chicago

36. National Tele-Immersion Initiative

37. Three WestGrid Collaborative Environments As part of WestGrid there will be three main hardware and software configurations: A desktop grid interface An access grid room An advanced immersive collaborative environment As part of WestGrid there will be three main hardware and software configurations: A desktop grid interface An access grid room An advanced immersive collaborative environment

38. CNS/ AMMI Lab. Passive Stereo Immersive Display and Access Grid Room First AMMI Lab. Contribution to WestGRID First AMMI Lab. Contribution to WestGRID

39. CNS/ AMMI Lab. Collaboration Room

40. System Configuration Control Computer Audio Computer Video Computer 2D Display Computer Gentner XAP 800 MM100 KVM Switch Network Switch CA*net 3 Local Network 3D Display Computers

41. UofA Low Cost Passive Stereo Projection Unit Screen Projectors Graphic Server Stereo Glasses Low Cost VR Wall

42. Passive Stereo Two projectors are used for the single screen one for each eye's view. Differently polarizing filters are placed in front of each projector lens. Users wear polarizing glasses where each lens only admits the light from the corresponding projector. Two projectors are used for the single screen one for each eye's view. Differently polarizing filters are placed in front of each projector lens. Users wear polarizing glasses where each lens only admits the light from the corresponding projector. Right Eye Left Eye Circular Polarizers

43. UofA Collaborative Room Extension PC Based Passive VR Display System PC Based Passive VR Display System Access Grid Immersive Communication Device Immersive Communication Device

44. UofA Research Component in WestGrid Visualization Project Immersive Video Display over High Speed Network Virtual Meeting Place Project Distributed Solution Server Advanced Collaborative Tools for the Analysis of Physical or Cellular Simulations Network issues with Immersive Collaborative Environments Immersive Video Display over High Speed Network Virtual Meeting Place Project Distributed Solution Server Advanced Collaborative Tools for the Analysis of Physical or Cellular Simulations Network issues with Immersive Collaborative Environments

45. UofA Advanced Collaborative Immersive Environments 3D Sound Rendering Input Sensors 3D Graphic Rendering Massive Storage Haptic Rendering New VizRoom High Speed Network CA*net 4 High Speed Network CA*net 4

46. UofA Upgrade of VizRoom to AG Master Control Computer 3D Audio Server S1 S2 S3 Sn Speakers AG Audio Capture Server M1 M2 M3 M4 Microphones AG Video Server C1 C2 C3 Cn Video Cameras Interface Server Joystick Inertial Tracker 3D Tracker Other Interface Front Display Pipe Left Display Pipe Right Display Pipe Active Stereo Projectors SGI ONYX 2 with 6 CPUs and 3 Graphic Pipes

47. Immersive Desktop Video Display and Visualization over High Speed Network Decoder H323 Control PC Control PC Network Controller DTI Glassless Stereo Display CLCR Side By Side MPX Encoder H323 Decoder H323 Control PC Control PC Network Controller DTI Glassless Stereo Display CLCR Side By Side MPX Encoder H323 CA*net 3

48. The UofA Virtual Meeting Place Project Goal: the main goal of this project is to create a general man-machine interface allowing engineers and scientists to communicate their design and visualize their data over the internet, producing the equivalent of a virtual meeting place. Collaborative Object And Data Manipulations and Interactions Collaborative Object And Data Manipulations and Interactions Virtual Actuators Virtual Actuators Live Stereo Texture and sound CAD Model or Scientific Data CAD Model or Scientific Data

49. UofA Virtual Meeting Project

52. Virtual Avatars Based on Stereo Textures Left Image Right Image Stereo Texture Background Extraction Movie

53. WestGrid Solution Server There is a need to develop a solution server that will allow WestGrid members to minimally modify their code and display the result of the simulations on a advanced immersive visualization environment The code is under development at the UofA (Physics and Computing Science Department) The code will allow: Truly distributed Simulation and Visualization It will allow to separate simulation time from real-time visualization requirements Allow multi-users to interact with the simulator Will allow real-time modifications of boundary conditions and simulation parameters There is a need to develop a solution server that will allow WestGrid members to minimally modify their code and display the result of the simulations on a advanced immersive visualization environment The code is under development at the UofA (Physics and Computing Science Department) The code will allow: Truly distributed Simulation and Visualization It will allow to separate simulation time from real-time visualization requirements Allow multi-users to interact with the simulator Will allow real-time modifications of boundary conditions and simulation parameters

54. Visualization vs Simulation: Software Architecture MAGIC Simulation Program Share Memory Solution #1 Solution #2 Solution #3. Solution #n ---------------------- Simulation Parameters Solution Server Visualization Program Storage Server Solution Formatter VTK Agents Stored Solutions TCP/IP Connection HIPPI-Net Processor #1 Local Formatted Solution Memory Server Control Commands Server Status Solution Parameters Diagnostics Server Control Commands Server Status Solution Parameters Diagnostics Formatted Solutions Processor #2 Processor n+1 to n+m Based on Cavern Soft G2

55. Visualization Toolkit - VTK Allow3D vis & image processing Hundreds of algorithms Object oriented (C++) Other language bindings for RPD (Tcl/Tk, Python, Java) Unix/Linux, Windows Threads, MPI support Active user community Open source www.kitware.com/vtk.html Allow3D vis & image processing Hundreds of algorithms Object oriented (C++) Other language bindings for RPD (Tcl/Tk, Python, Java) Unix/Linux, Windows Threads, MPI support Active user community Open source www.kitware.com/vtk.html

56. Advanced Collaborative Tools for the Analysis of Physical Simulations In this project we will analyze and explore new types of interactive tools to explore the results of physical simulations. We will explore how sound can help in the perception of a field such as the magnetic field by correlating the position of a 3D wand with a sound generator. We will explore various particle tracer techniques to display vorticity and turbulent flow We will try to relate Sound, Haptics and Visual cues to give physicists a better understanding of complex fields In this project we will analyze and explore new types of interactive tools to explore the results of physical simulations. We will explore how sound can help in the perception of a field such as the magnetic field by correlating the position of a 3D wand with a sound generator. We will explore various particle tracer techniques to display vorticity and turbulent flow We will try to relate Sound, Haptics and Visual cues to give physicists a better understanding of complex fields

57. Network issues with Immersive Collaborative Environments Timing is essential in the operation of distributed virtual environment applications, since the perception of changes in the virtual environment is based on the timely delivery of messages, informing all the participants of the changes made by a user. Any action issued by any participant must reach the other participants within 200 ms. This task, challenging as it is, becomes even more challenging, when dealing with virtual environments with a large number of participants. Timing is essential in the operation of distributed virtual environment applications, since the perception of changes in the virtual environment is based on the timely delivery of messages, informing all the participants of the changes made by a user. Any action issued by any participant must reach the other participants within 200 ms. This task, challenging as it is, becomes even more challenging, when dealing with virtual environments with a large number of participants.

58. UofA WestGrid Local Network Configuration SGI ONYX2 3 Graphic Pipes 6 CPUs Vis-server V3.0 BigBangWidth Switch AMMI Lab Access Grid AMMI Lab Virtualized Reality Servers BigBangWidth Switch UofA Computer Networking Services Access Grid New WestGrid Super-Computer BigBangWidth Switch 1Gb/s 100Mb/s 1Gb/s 100Mb/s CanNet*4 Gigapop 10Gb/s Netera Dedicated ( Using One of the Lambda) Optical Line To Calgary NewMic/SFU Vancouver Netera/UofC Calgary UofL Lethbridge Banff Center Banff

59. WED Virtualized Reality Server Configuration I (Video Avatar) PC Server For Camera Bank 1 PC Server For Camera Bank 2 PC Server For Camera Bank 3 PC Server For Camera Bank 4 Display Client #1 BBW Switch BBW Switch Display Client #2

60. WED Virtualized Reality Server Configuration II (Virtual Guards) Tour PC Server For Camera Bank 2 PC Server For Camera Bank 1 PC Server For Camera Bank 3 PC Server For Camera Bank 4 BBW Switch BBW Switch Display Client #1 Display Client #2

61. Heritage Visual Area Networking SGI ONYX2 3 Graphic Pipes 6 CPUs Vis-Server V3.0 BigBangWidth Switch BigBangWidth Switch 1Gb/s CanNet*4 Gigapop Netera Dedicated (Using One Lambda) Optical Line To Calgary U of C/ Netera BBW Switch U of T BBW Switch SFU BBW Switch UofA Computing Science UofA Computer Networking Services Vis Client Vis Client Vis Client

62. Network issues with Immersive Collaborative Environments The high level of dynamicity in group structure and topology increases the complexity of the problem. Participants might join and leave the session dynamically. The requirement that the developed DVE applications will be supporting collaborative functions, makes timing even more important. The high level of dynamicity in group structure and topology increases the complexity of the problem. Participants might join and leave the session dynamically. The requirement that the developed DVE applications will be supporting collaborative functions, makes timing even more important.

63. Network issues with Immersive Collaborative Environments In this project, will attempt to: Identify and quantify the end-system and network parameters that are crucial to the operation of the virtual reality application; Develop technology to support the strict requirements of these applications; Use and improve CAVERNsoft*G2, a C++ toolkit for building collaborative networked applications. In this project, will attempt to: Identify and quantify the end-system and network parameters that are crucial to the operation of the virtual reality application; Develop technology to support the strict requirements of these applications; Use and improve CAVERNsoft*G2, a C++ toolkit for building collaborative networked applications.

64. High-bandwidth connectivity The University of Alberta is proposing to work with industrial partner, BigBangwidth, to experiment and test advanced network connectivity solutions within the WestGrid network. The purpose of the project is to provide researchers, high-bandwidth access to the WestGrid network to enable effective grid computing applications, especially visualization This project is part of a 2M$ WED grant that will be hopefully funded at the beginning of next year. The University of Alberta is proposing to work with industrial partner, BigBangwidth, to experiment and test advanced network connectivity solutions within the WestGrid network. The purpose of the project is to provide researchers, high-bandwidth access to the WestGrid network to enable effective grid computing applications, especially visualization This project is part of a 2M$ WED grant that will be hopefully funded at the beginning of next year.

65. High-bandwidth connectivity

66. The BigBangwidth BroadLAN System

67. Objectives The main categories of technical measurements to be tested in this project will include: Raw Bangwidth Baseline Performance benchmarking Demonstration of removal of network congestion with BroadLAN Demonstration of automated traffic control using BroadLAN Demonstration of distributed network control and security Demonstration of distributed application functionality Final Demonstration of GRID digital information network The main categories of technical measurements to be tested in this project will include: Raw Bangwidth Baseline Performance benchmarking Demonstration of removal of network congestion with BroadLAN Demonstration of automated traffic control using BroadLAN Demonstration of distributed network control and security Demonstration of distributed application functionality Final Demonstration of GRID digital information network

68. Cave to Cave Visualization Using BigBangWidth Switch NewMic VizServer CanNet3 UofA Visualization Software

69. Proposed Demonstrations One or Two Collaborative Visualization demonstrations for 4D Simulation over the Grid using immersive display, BroadLAN switch, and UofA collaborative Visualization Software. First Demonstration with the UofA Department of Physics: Earth Dynamo. Second demonstration with IBD CyberCell: the bacterium E. coli simulation One or Two Collaborative Visualization demonstrations for 4D Simulation over the Grid using immersive display, BroadLAN switch, and UofA collaborative Visualization Software. First Demonstration with the UofA Department of Physics: Earth Dynamo. Second demonstration with IBD CyberCell: the bacterium E. coli simulation

70. Proposed Time Line for Key Demos Integration and testing of CNS Room and VizRoom to Access Grid: April to end of May 2003 Development of the first solution server prototype: Now to June 2003 Integration and testing of BigBandWidth switch with UofA MACI computers and the VizRoom: March-April 2003 Integration of solution server to VizRoom Software: June-September 2003. First Demo of Earth Dynamo Simulation: Beginning of October 2003 Integration and testing of CNS Room and VizRoom to Access Grid: April to end of May 2003 Development of the first solution server prototype: Now to June 2003 Integration and testing of BigBandWidth switch with UofA MACI computers and the VizRoom: March-April 2003 Integration of solution server to VizRoom Software: June-September 2003. First Demo of Earth Dynamo Simulation: Beginning of October 2003

71. Towards Wide Area Teleimmersion Convergence of Virtual Reality, Collaboration Technology and Active Spaces VR Audio Visual Haptics VR Audio Visual Haptics VR Audio Visual Haptics