1 The Rendering Pipeline

CS788 Topic of HCI 2 Outline  Introduction  The Graphics Rendering Pipeline  Three functional stages  Example  Bottleneck  Optimization  The Haptics Rendering Pipeline

3 Introduction  For smooth simulations  Need to display at least 24, or, better 30 frame/secs  Total latency is not over 100msec  Low latency and fast graphics require a VR engine that has a powerful computer architecture  These architectures are designed around a rendering pipeline  Rendering  The process of converting the 3D geometrical models populating a virtual world into a 2D scene presented to the user

4 The Graphics Rendering Pipeline

5 Three functional stages  Graphics rendering has three functional stages. FIFO buffer Application stage Geometry stage Rasterizer stage FIFO buffer Output buffer User input Video controller

6 Application stage  Implemented in software  Run on several CPUs  Read the world geometry database & the user’s input  Basic operations  take care of user’s input  acceleration algorithm  Output: 3D geometric primitives (polygons, meshes) Geometry stage Rasterizer stage Application stage CPU1 CPU2 User input Video controller

7 Geometry stage  Implemented in software or hardware  Run on the geometry engines  Basic operations  Output: 2D geometric primitives (2D polygons) Application stage Rasterizer stage Geometry stage GE1 GE2 User input Video controller Model transformation Light computation Scene projection Clipping Screen mapping

8 Rasterizer stage  Implemented in hardware  Convert the vertex information output into pixel information needed by the video display  Basic operations:  Scan conversion (rasterization)  Z-buffering  Anti-aliasing  Texture mapping  Output: pixel values Application stage Geometry stage Rasterizer stage RU1 RU2 User input Video controller

9 Bottleneck  One of the three stages will be the slowest => bottleneck stage  CPU-limited  In the application stage  Transform-limited  In the geometry stage  Fill-limited  In the rasterizer stage

10 Optimization (1/2)  Application stage : CPU-limited  Replace the CPU with a faster one or add another CPU  Reduce CPU’s load  Reduce the scene complexity by using 3D models with a lower polygonal count  Optimize the simulation software  Geometry stage : Transform-limited  Need to look at the computation load assigned to the GEs  Reduce the number of virtual light sources  Use the Simple shading mode  Use the type of polygon for which its rendering hardware was optimized

11 Optimization (2/2)  Rasterizer stage : fill-limited  Reduce the number of pixel in the displayed image  Reduce the size of the display window  Reduce the window resolution

12 The Haptics Rendering Pipeline

13 The Haptics Rendering Pipeline User input Haptic interface Collision detection stage Force computation stage Tactile computation stage CPU1 CPU2 Compute force Force smoothing Force mapping Haptic texturing

14 The stages of the Haptics Rendering Pipeline (1/2)  Collision detection stage  Load the physical characteristics of the 3D objects from the database  Perform collision detection to determine which virtual objects collide  Force computation stage  Compute the collision forces  Force smoothing  Force mapping

15 The stages of the Haptics Rendering Pipeline (2/2)  Tactile computation stage  Render the touch feedback component of the simulation  The computed effects are added to the force vector send to the haptics output display  The haptics rendering pipeline has a much less standardized architecture compared to its graphics counterpart.

16 Thank you

CS788 Topic of HCI 17 Conceptual Model of VR H-sensor perception cognition motion control H-effector P-effector P-sensor L-effector L-sensor sensing action Human avatar virtualagent Logical devices for displacements, angles, events. virtual object V-effector V-sensor

CS788 Topic of HCI 18 Functional model displaying (Sec. 3-4,5,6,7) rendering (chap. 7) simulation (Sec. 6-3,4) Interaction (chap. 5) Virtual perception (chap. 4) Sensing (Sec. 3-3) VW Authoring (Sec. 6-2) VW DB (Sec. 6-1)