1. High Performance VR Jérémie Allard, ID/IMAG Jeremie.Allard@imag.fr
Commodity Clusters for Immersive Projection Environments
High Performance VR
Jérémie Allard,
ID/IMAG

2. Goal
From a bunch of PC toward a high performance plateform for intensive VR applications

3. VR Cluster How-To (1)
Our new cluster
First step: install the OS

4. OS Requirements on a Cluster
Distributed environment
Fast installation/update on several nodes
Shared directory (NFS)
User accounts (NIS)
Remote shell service (SSH)
Network access (DHCP, DNS)

5. Linux Cluster Distribution
Based on MandrakeLinux
Preconfigured solution to easily deploy and manage a cluster

6. Installation Simple and Fast Installation Install one node as a server
Install one slave
Automatic cloning to install all other slaves (200+ PCs in 20 min.)
Cluster ready!

7. Tools Lots of integrated tools: Communication: MPICH, LAM-MPI
Computing: ATLAS, LAPACK, SCALAPACK, PETSC
Job Managment: OpenPBS
Monitoring: Ganglia
Administration: Ka-tools, pconsole, urpmi –parallel
VR/Graphics: Povray, Net Juggler

8. VR Cluster How-To (2) Cluster installed
Next step: Connect the projectors and stereo system

9. SoftGenLock: Multi-Display Active Stereo
Alternate projection of the right and left eye images synchronized with shutter glasses
Multiple projectors: glasses shutters must be flipped when all the displays are in the vertical blanking period
genlock
Vertical blanking
Projector 1
Left
Right
Left
Projector 2
Left
Right
Left
Left eye
Right eye
Time

10. SoftGenLock: Software Implementation
Software approach to enable active stereo and genlock on Linux clusters
Synchronize the video signals
Sequentially display left and right images
Send signal to the glasses (LCD shutters)
Works with potentially any graphics card
New pixel-clock based algorithm (SoftGenLock 2.0)
Master/Slave synchronization using the parallel port

11. SoftGenLock: Master/Slave Algorithm
For each frame:
On the master
send a flip signal to the glasses and the slaves
flip the displayed image
On the slaves
read the master signal
accelerate or slowdown the video signal

12. SoftGenLock: Custom Parallel Port Network
Slaves
Glasses
+ ground connections
Master
Can be a simple tweaked cable for a few nodes:

13. SoftGenLock: Requirements
Environments supported
Linux with a standard 2.4 kernel
Useful for testing but genlock may be unstable (jittering)
Real-time kernel: RT-Linux, RTAI (preferred)
Best performance (no jitter)
Require to patch the kernel
Future 2.5/2.6 kernels
expected to have less jittering and much better scheduling precision

14. VR Cluster How-To (3)
Displays ready
Next step: VR Toolkit

15. VR Toolkits: VR Juggler
Open Source Platform for VR
Abstraction of the I/O devices
Direct access to graphics API:
OpenGL, Performer, Open Inventor, OpenSG
Take advantage of the latest shader technologies
Linux, Windows, IRIX, Solaris, HP Unix
VR Juggler 1.0 released in april 2001, 2.0 currently in active development (first alpha in march 2003)

16. On-line Reconfiguration
VR Juggler
Simulator Mode
Performance
Monitoring
PC + mouse + keyboard
On-line Reconfiguration
Configuration Files
VJ Application
Tracker + Displays + …
vjUser, vjButton,…

17. VR Juggler Configuration system
Configuration data are organized in ConfigChunks
JugglerUser Name "User1" headPos { "VJHead" } interocular_distance { "0.229" } end
Configuration editor GUI:

18. VR Cluster How-To (4)
VR toolkit installed
Next step: cluster support

19. Clustering Solutions for VR Juggler
Data Lock at
Input Event Level:
Cluster Juggler
Net Juggler
Scene Graph Level:
OpenSG
Graphics Primitive Level:
WireGL

20. Net Juggler http://netjuggler.sf.net
Extends VR Juggler 1.0 for clusters
Abstract the cluster into one system
Only few modifications to VR Juggler applications source code
One configuration for the whole cluster
One operation to launch the application (using command line or GUI)

21. On-line Reconfiguration
Net Juggler
Simulator Mode
Performance
Monitoring
Net Juggler
On-line Reconfiguration
Configuration Files (add each PC role)
VJ Application
PC1 -> Tracker + left display
PC2 -> Front display
PC3 -> Wand + Right display

22. Net Juggler communication layer (data lock)
Net Juggler Implementation
Configuration events
Tracker event
Wand events
Net Juggler communication layer (data lock)
VJ Application
VJ Application
VJ Application
Net Juggler swaplock

23. Low bandwidth required Fast Ethernet Network sufficient
Net Juggler Communications
Data in transit (device + config events):
Small amount
Independent of the scene’s graphics complexity
Communication Layer implementation:
Data aggregation
Optimized collective communications
Scalable
Low bandwidth required
Fast Ethernet Network sufficient

24. Net Juggler application launching GUI: NjRun
Set/Load/Save a configuration
Actual nodes of the cluster
Template configuration files : hosts are defined as
Application executable
Generate the configuration files and the mpi script
Launch the application
Kill the application

25. Cluster Juggler / VR Juggler 2
Currently (july 2003) in development
Data Lock: Input event distribution scheme
Directly integrated in VR Juggler 2.0
Based on TCP
Heterogeneous cluster support

26. Net Juggler vs Cluster Juggler
Both use similar distribution schemes
Net Juggler is based on VR Juggler 1.0 and MPI
Collective communication for high scalability
Applications can use MPI based distributed computations
Cluster Juggler is based on VR Juggler 2.0 and TCP
Heterogeneous clusters
ApplicationData: API for broadcasting application-specific chunk of data

27. VR Cluster How-To (5) VR toolkit with cluster support
Next step: performance intensive applications

28. adhoc parallelization
Developing applications for Net Juggler/Cluster Juggler
Performance issues
Replication on all nodes:
Low communication overhead
data and computations duplicated
fps on one PC = fps on a X nodes PC cluster
Performance improvements: distribute data and computations
Application dependant bottleneck
adhoc parallelization
Tradeoff between distribution and extra communication costs

29. Distributed Interactive Fluid Simulation
Fluids simulation: Stam’s Navier-Stokes equation solver
2D grid distributed on the nodes + parallel Navier-Stokes equation solver
Implementation using the PETSc math library (based on MPI)
Node 1
Node 2
Node 3
Node 4

30. NjFluid Results: Dual PIII-800 MHz GeForce 2 GTS 64 MB
100 Mbits/s Fast Ethernet Network
128x128 grid
20 fps
21 fps
Solver distributed on 4 nodes
22 fps
24 fps
Solver distributed on 6 nodes
8fps
8 fps
Solver duplicated
4 displays
1 display

31. Everything is Open-Source!
Summary
We combined several tools
Cluster OS: CLIC
Stereo system: SoftGenLock
VR Toolkit: VR Juggler
Cluster support: Net Juggler
Distributed computations: PETSC
Everything is Open-Source!

32. VR Cluster How-To (6)
High performance VR plateform
Next step: invent!

33. Advanced Applications
Real-time video processing and scene reconstruction
Simulation-based design
Rich virtual environments
Complex virtual environments
MOVI, INRIA
Jin Tech, VRAC

34. Toward Rich Virtual Environments
Integrating multiple simulations in a graphical intensive world
Fluid, Grass/Wind, Fur
Better immersion
Interactive simulations for industrial design
Fast workflow
Require advanced tools
on going work at ID/IMAG