1. Workshop on Commodity-Based Visualization Clusters Integration of a Commodity Cluster into an Existing 4-Wall Display System Douglas B. Maxwell, Aaron Bryden, Greg S. Schmidt, Ian Roth, Sue-Ling Chen Virtual Reality Laboratory, Naval Research Laboratory

2. Workshop on Commodity-Based Visualization Clusters 2 General Overview Introduction Project Rationale Related Work System Overview  Cluster Design Strategies  Graphics Cluster Requirements  NRL VR-Lab Hardware Overview System Implementation  Software  Physical Installation Testing and Evaluation Summary and Conclusions

3. Workshop on Commodity-Based Visualization Clusters 3 Introduction  Not all virtual reality applications today require the power or expense of single large visualization “supercomputers”.  Increasingly, low cost commodity consumer electronics and computing technology are becoming powerful enough to present an acceptable level of graphics performance.  Already, commodity PCs are driving virtual reality workbenches with stereo and tracking options.

4. Workshop on Commodity-Based Visualization Clusters 4 Project Rationale Why should anyone use a commodity cluster when the capability already exists with solutions from SGI and Sun?  Rapid evolution of commodity computers and consumer electronics has both driven down costs and accelerated obsolescent computing cycles.  A general rule to follow for buying graphics capability from SGI is to budget $250,000 per graphics pipe.  Our experimental cluster costs less than $1000 per node.

5. Workshop on Commodity-Based Visualization Clusters 5 Project Rationale We present an experiment that integrates a commodity cluster into an existing 4-wall display system—a Surround-Screen Visualization System (SSVR) from Mechdyne Corporation. The objective is to attain active stereo visualization on multiple walls using genlocking, swap-locking and data- locking capabilities – all with acceptable stereo frame rates. The cluster was designed from the beginning to attempt to replace aging SGI computing equipment used to drive our current 4-wall display system.

6. Workshop on Commodity-Based Visualization Clusters 6 Related Work Siggraph 2002 - Course Notes 47 ”Commodity Clusters for Immersive Projection Environments” Cornell Theory Center Deploys the First Ever Immersive Display Using the Windows 2000 Cluster CAVELib™ http://noel.feld.cvut.cz/magi/ Net Juggler - The Scalable High Performance Virtual Reality Project http://netjuggler.sourceforge.net

7. Workshop on Commodity-Based Visualization Clusters 7 System Overview Cluster Design Strategies  Client/Server  Master/Slave Graphics Cluster Requirements  Genlocking  Swap Locking  Data Locking NRL VR-Lab Hardware Overview

8. Workshop on Commodity-Based Visualization Clusters 8 Cluster Design Strategies Client/Server Consists of a single node of a cluster serving data to the graphics rendering clients. Advantages:  Many applications may embed a server that works with the same rendering client nodes.  Flexible Disadvantage:  High network bandwidth consumption. Master/Slave Consists of multiple nodes, where each node of the cluster locally stores and runs an identical copy of the application. Advantages:  Small amounts of data is transferred; Network bandwidth is less of a concern. Disadvantage:  Inflexible

9. Workshop on Commodity-Based Visualization Clusters 9 Graphics Cluster Requirements Genlocking is the process of synchronizing the video frames from each node in a graphics cluster so that they produce a fluid, coherent image. Swap Locking is the process of synchronizing the frame buffer rendering and swapping. Data Locking is the process of synchronizing the views to maintain consistency across all screens.

10. Workshop on Commodity-Based Visualization Clusters 10 NRL VR-Lab Hardware Overview 3 node cluster graphics cluster in master/slave configuration using commodity PC’s. Generic PC configuration:  AMD Duron Processors @ 1.2Ghz  512 Mb ram  NVidia GeForce 4 Ti4600 w/ 128 Mb video memory Networking:  10/100 NICs  100BaseT Switch

11. Workshop on Commodity-Based Visualization Clusters 11 NRL VR-Lab Hardware Overview Video  Insertion of video output from cluster into existing 4-wall display achieved by an Extron CrossPoint 124 matrix video switcher. Misc.  Genlocking and data locking are handled in software through the parallel ports.  A special purpose communication box was fabricated from COTS hardware. This box also outputs a genlocking signal to a set of Crystal Eyes infrared emitters.

12. Workshop on Commodity-Based Visualization Clusters 12 NRL VR-Lab Hardware Overview Commodity Cluster Configuration Diagram

13. Workshop on Commodity-Based Visualization Clusters 13 System Implementation - Software Software used to deal with active stereo and scene synchronization needs of the graphics cluster:  Real Time Application Interface (RTAI) kernel patch applied to Red Hat 7.2. RTAI allows for low latency and task completion timing.  SoftGenLock enables active stereo in clusters. SoftGenLock and RTAI are used in concert to provide a software active stereo solution.

14. Workshop on Commodity-Based Visualization Clusters 14 RTAI Kernel Module - Active Stereo The RTAI kernel module detects the vertical refresh and changes a pointer in the video card memory. Simulated Frame Buffer Contents

15. Workshop on Commodity-Based Visualization Clusters 15 System Implementation – Software  Genlock/data lock is achieved by synchronizing the machines through the parallel ports.  The RTAI kernel module reads and writes to two pins on the parallel port.  The master tells all the other nodes in the cluster when to draw. Parallel Port Signal Adapter

16. Workshop on Commodity-Based Visualization Clusters 16 System Implementation – Software  SoftGenLock does not synchronize applications between the nodes of a cluster; It provides data lock and stereo only.  It is the responsibility of the application to synchronize the viewing frustum and animations in the application.  SoftGenLock only uses VGA registers, it potentially can work with any graphics card.

17. Workshop on Commodity-Based Visualization Clusters 17 Physical Installation – Setup Issues for Consideration  Adequate HVAC, power and network access where the cluster will be set up.  Set up the cluster and display system in separate rooms, since noise levels from fans, drives, etc. may be distracting.  Prepare to deal with a lot of issues associated with cables.  Create a master power switch to turn off all units at once.

18. Workshop on Commodity-Based Visualization Clusters 18 Testing and Evaluation  Commercial Software  Software Design

19. Workshop on Commodity-Based Visualization Clusters 19 Commercial Software  VR Juggler –Open source virtual reality tools (LGPL) –VR interoperable Works on multiple computing platforms Works with multiple display systems Works with multiple tracking and interaction devices –Works with advanced 3D visualization tools Open GL Open Scene Graph library SGI OpenGL Performer –Free Software (LPGL) –www.vrjuggler.orgwww.vrjuggler.org  Open GL

20. Workshop on Commodity-Based Visualization Clusters 20 Software Design VBU Software  Displays bathymetry and has capability to load simple models  Utilizes VR Juggler, OpenGL  Works on SGI and Linux platforms  Works with standard 4-wall CAVE, workbench, desktop VBU Software on Linux DesktopBathymetry Uncertainty Using Glyphs

21. Workshop on Commodity-Based Visualization Clusters 21 Software Design VBU Software  Adapted to work with pc-cluster (3 machines)  Works on 3 walls of CAVE 3-PC Cluster Running VBU Software VBU Software Running on 3 Walls of CAVE Driven by 3-PC Cluster

22. Workshop on Commodity-Based Visualization Clusters 22 Summary and Conclusions  The use of a graphics PC cluster is now becoming a viable low-cost alternative to the use of single large visualization supercomputers.  Our cluster experiment explored the feasibility of phasing out and replacing existing expensive single large computing hardware.  The results show we can make this kind of transition in the near future, and we believe our experiences will be motivation for others to follow suit.

23. Workshop on Commodity-Based Visualization Clusters 23 Contact Information Douglas B Maxwell, MSME Advanced Systems Division Weapon Countermeasure Control Branch Naval Undersea Warfare Center maxwelldb@npt.nuwc.navy.mil Gregory Schmidt, Ph.D. Virtual Reality Laboratory Naval Research Laboratory schmidt@ait.nrl.navy.mil

24. Workshop on Commodity-Based Visualization Clusters 24 Acknowledgement Thanks are extended to Dr. Larry Rosenblum for the resources and support necessary to make this project a success.