1. Introduction to Simulation and VR Week 5 Human Dynamics in a Virtual World

2. Recap Calculate Geometry Draw Wire Frame Render Surfaces Enhance Surfaces and lighting Sensor input and output Initialize world

3. Human Dynamics Users described as participants basic interaction involves control of camera (viewpoint) exploratory navigation / locomotion Walk-through or Fly-Thru (sic!)systems More advanced environment allow interaction Touch, selection, manipulation referred to as direct manipulation

4. Components of interaction VR model Simulation of body Interaction with virtual body Object pair collision General collision detection

5. VR Model Goal of Being There Presence or Telepresnce Held and Durlach 1992, Draper 1998 Must model expectations -> realism Ideal VR model must Immerse participant in visual, audio, touch, smell and taste Humans can process several audio streams and can focus and segrgate on one - Wenzel 1992

6. VR model - Immersion Surrounds body fills visual field extensive inclusive (replaces reality) Vivid human body in CAVE actual body can obscure projection of virtual objects In HMD body must be represented

7. VR model - HCI Mouse and keyboard has two problems gulf of execution gulf of evaluation Hutchins 1986 Direct Manipulation paradigm Tracked HMD is simplest form 0- 1 to 1 mapping, Low cognitive overhead Using mouse - must map actions to different translations

8. VR Model - Interaction Immersion and tracking rely on registration Registration implies that motion of limbs accurate Better appreciation of 3D environment Cannot lose interaction - reduces gulf of execution Gulf of evaluation reduced when whole virtual body used - Slater and Usoh 1994, Mine 1997

9. Simulation of Body Body model is the description of the interface eyes are visual interface, ears are audio interface geometric description drawn from egocentric point of view description of hand and fingers forms basis of grasping simulation for picking up objects (Boulic 1996)

10. Simulation of Body - Building the body The more points representing the body the more realistic the movement Up to 90 points for motion-capture in animation Standard for human skeleton (H-Anim 1999) More typically head, Torso, Both hands Inferred movement from limited points Inverse kinematics problem - infinite possibilities of movement in virtual environment, consistent restraint Elbow position in 4- Tracker system (Badler, 1993)

11. H-Anim Humanoid Sacroiliac L MidtarsalL AnkleL KneeL Hip R MidtarsalR AnkleR KneeR Hip L WristL ElbowL Shoulder R WristR ElbowR Shoulder vl5 Skullbase

12. Different sensing methods

13. Simulation Of body - tracking the participant Choice of system depends on 5 factors accuracy, resolution, range, lag, update rate Many different tracking technologies Meyer 1992 frequency and time ultrasonic time-of-flight measurement Pulsed Infra-red GPS Optical Gyroscopes Phase difference

14. Simulation Of body - tracking the participant Spatial Scan Outside-in Inside-out Inertial sensing mechanical gyroscope Accelerometer Mechanical Linkages Direct - Field Sensing

15. Head tracking

17. Accelerometer Fast Track Phantom

18. How the tracker works Distance detection Transmitter Receiver

19. How the tracker works Orientation detection Transmitter Receiver

20. Head tracking

21. Latency Filtering, keep steady Jitter

23. Mechanical Linkages

24. Sensors in joints detect position 3D viewer updates Robot applies force to joints Force is felt on hand

26. Phantom

27. Phantom working θ1θ1 θ2θ2 θ3θ3 Virtual pencil

28. Phantom working θ1θ1 θ2θ2 θ3θ3 Virtual pencil

29. Phantom working θ1θ1 θ2θ2 θ3θ3 Virtual pencil

30. Phantom working θ1θ1 θ2θ2 θ3θ3 Virtual pencil Apply force

31. Phantom working θ1θ1 θ2θ2 θ3θ3 Virtual pencil Apply force Motors lock

32. Phantom working θ1θ1 θ2θ2 θ3θ3 Virtual pencil

33. Interaction with virtual Body Limitations mean reliance on metaphors for object manipulation (grasping and moving) locomotion (movement) Limitations in haptics mean that restraint on the virtual environment exists

34. Object Manuipulation World Body BObject O Hand HObject P World Body BObject O Hand H Object P Grasping Releasing

35. Object Manipulation Hand posture may not be tracked - makes grasping difficult Must establish a point at which union is deemed to have taken place Moved by repositioning in the scene graph Robinett and Holloway 1992

36. Locomotion Tracker has a limited range Must use locomotion metaphor to move greater distances Locomotion is on an even plane, virtual terrain may not be Collision detection can be employed to raise or lower the participant accordingly

37. Transformations employed in object manipulation Calculate relative transformation from hand to object M R M R = (M B.M H ) -1.M O.M P M B :Transformation from body to world co-ords M H :Transformation from hand to body co-ords M O :Transformation from Object O to world co-ords M P :Transformation from Object P to Object O co-ords After manipulation new local transformation of Object M p’ is M p’ = M O -1.M B.M H.M R

38. Locomotion Tracker has a limited range Must use locomotion metaphor to move greater distances Locomotion is on an even plane, virtual terrain may not be even Collision detection can be employed to raise or lower the participant accordingly

39. Fly in direction of aim Fly in direction of pointing Fly in direction of gaze Fly in direction of torso Directions of locomotion

40. Object Pair Collision Detection Vital component of interaction Describe Exhaustive Test for when two object intersect (process hungry) Try to aviod doing exhaustive test igf possible

41. Exhaustive Test Assume all objects as collection of triangles (polygons) Object 1 consists of m triangles Object 2 consists of n triangles Use triangle intersection test to test all possible pairs of of intersections This requires n.m triangle-triangle tests

42. Triangle Intersection Test Moller 1997 comparison of triangles A and B They do not intersect if all vertices in A lie to one side of plane of B and V.V Otherwise plane of A intersects plane of B on L Find line intersection of L with A ( L A ) and L with B ( L B ) A and B intersect only if L A and L B overlap A B LBLB L LALA

43. Basic Rejection Tests Simplest tests based on distance Each scene object has a bounding sphere. Two objects cannot overlap if distance between two centres is > than sum of the radii Better test id the separating plane test. If a plane can be drawn such that all points of one object lie on one side and all points of the other on the reverse, cannot collide. Key ids to find a good separating plane Bounding Box range test

44. General Collision Detection Detecting collision between a set of n objects generates n 2 possible pairs of objects requiring testing for overlap Use spatial partitionong to discard as may pairs as possible and use object pair collision tests on remaining pairs Uniform Space Subdivision

45. Space Subdivision

46. Recap Immersion requires body representation Track body Robot arm - inverse kinematics Monitor collisions Force Feedback Uses inverse kinematics Scene manipulation