1. VR Interfaces – Navigation, Selection and UI Elements By David Johnson

2. VR Interfaces How do we tell the computer to do things? How do we select things? How do we navigate around?

3. VR Interface Challenges Intuitive –Make interaction work like the real world –Minimize cognitive overhead Augmentation –Give users new capabilities

4. Quick UI review Norman’s Principles of Design –Make things visible –Provide a good conceptual model Affordance Mapping Constraints Feedback

5. Visibility

6. Good Conceptual Model A good conceptual model allows us to predict the effects of our actions Without a good model we operate blindly –Simply follow rules without understanding a reason –No understanding of cause or effect –No recourse when something breaks Fridge/freezer controls Thermostat

7. Affordances

8. Mapping

9. Constraints Prevent you from doing what you shouldn’t do –Grey out selections that don’t apply at the current time

10. Feedback Examples Clicker on your turn signal Animated icon while waiting for a web page to load

11. Why is usability important? Poor usability results in –anger and frustration –decreased productivity in the workplace –higher error rates –physical and emotional injury –equipment damage –loss of customer loyalty

12. 2D Interfaces Dominant computer interface uses a mouse and graphical elements Xerox Star (1981)

13. 2D Interfaces Why is it a WIMP interface? –Windows –Icons –Menus –Pointer Xerox Star (1981)

14. 3D Interfaces Need to map 2D interfaces to 3D Hopefully, create whole new expressive interfaces

15. 3D equivalent of a Mouse? Mouse –2D positioning –Buttons to hold or click

16. 6DOF mouse Flying mouse Fledarmaus The Bat How do you clutch/ratchet? –In 2D, picking up disables tracking

17. Menus in Virtual Space Cannot easily overlay menus “Float” menus in space –Select by raycasting –Keep near user’s head Jacoby, Ellis 1992

18. Menus in Virtual Space Ring Menus –JDCAD 1993 Liang –ISAAC 1995, Mark Mine –Rotate hand to move selection point

19. Menus in Virtual Space Darken, 1994 Overlaid menus Speech selection

20. Menus in Virtual Space Pen and Tablet –Track a tablet and pen –Put 2D menus on tablet –“Haptic Hand”

21. Menus in Virtual Space Bowman Pinch Gloves –Select with thumb to finger –High-level menu on ND hand –Secondary menu on D hand –First tries Scrolling menu using pinches or More items on pinkie

22. TULIP Menus Three-Up, Labels in Palm Virtually raise hands Rotate menus Put ‘next’ groups on palms Users preferred over floating and tablet menus –Perhaps slower

23. Menus in Virtual Spaces ‘Virtual tricorder’ Wloka 1995

24. Gestures Symbolic –Cultural meaning (O.K. sign) Deictic –Pointing, direct viewer’s attention Iconic –Showing an example path with hand Pantomimic –Act out the activity

25. Gestures GIVEN (1992) –Neural net recognition –20 gestures Fly, grab, etc. Mine –“Physical mnemonics” Pull-down menus from near head Delete by throwing over shoulder

26. Numerical Input Mark Mine A digit at a time Sliders too imprecise

27. Text Input Bowman Pinch glove Thumb to home row finger Hands in/out to go down/up row Rotate to hit extra keys

28. Basic Navigation Tasks Exploration –Untargeted movement –Build internal map Positioning –Move to known location Maneuvering –Precise positioning of viewpoint –Typically short motions

29. Natural Interfaces Walking Biking Snowboard Swimming Issues?

30. Walking workaround Redirected walking –moviemovie

31. Flying Interfaces Flying Magic carpet Guided navigation –River analogy Issues?

32. Steering Interfaces Pointing –Expressive –Hand shake Torso Gaze-directed –simple Physical device

33. Hand-based Interfaces Colin Ware (1990s) –World-in-hand –Eye-in-hand

34. Point-to-point Travel Select a point in a scene –Computer picks path Teleport –Bowman et al. found significant spatial disorientation from teleport

35. World in Miniature User holds dynamic map in one hand Navigation is reduced to object positioning

36. WIM Setup Physical props – clipboard and interface ball

37. Two-handed Flying Mark Mine

38. Fundamental Operations in a UI Select an object Manipulate an object –Translate –Rotate What are some techniques in 2D interfaces?

39. From the Beginning Sutherland and Vickers –Sorcerer’s Apprentice (1972) Track stylus Selection of vertices –Intersection of cube at tip of stylus

40. Pointing: Put That There 1979 Ray from tracked hand Speech interface movie

41. Pointing: JD-CAD (1993) “Laser gun” from hand –Tracker noise –Harder to select far away Spotlight –Add a cone to the ray –Select objects based on Distance from cone axis Distance from hand

42. Silk Cursor Replace wireframe selection box with translucent box –Visual cues to containment

43. Pointing: Aperture Spotlight from eye Cone angle based on distance from hand to eye Selection modified by hand orientation

44. Pointing: Flexible Pointer Two-handed Hand direction bends pointer Can select occluded objects movie

45. Hand: GoGo Interaction (1996) Go-Go uses Non-Linear mapping between virtual and real hand Control-display ratio Stretch go-go variation Pros: –Extended reach when needed –Direct manipulation Cons: –Reach still limited by arm length –Precision suffers when reach is extended (low level of control) Movie

46. Image Plane Techniques Point or gesture at an objects projection onto the viewing plane –“head-crusher” Kids in the hall –“Sticky finger” Similar to ray casting Pros: –Very intuitive –Allows user to reach objects at an arbitrary distance Cons: –Limited by the need for line of sight –Can be fatiguing –Virtual hand may obscure small objects

47. Two Handed and Body-Centered Interaction What can you do with two hands? What if you use your body as a reference point? Mine, Mark, Frederick P. Brooks Jr., and Carlo Sequin (1997). Moving Objects in Space: Exploiting Proprioception in Virtual-Environment Interaction. Proceedings of SIGGRAPH 97, Los Angeles, CA.Moving Objects in Space: Exploiting Proprioception in Virtual-Environment Interaction. “Scaled-space grab”

48. HOMER technique Hand-Centered Object Manipulation Extending Ray-Casting Select: ray-casting Manipulate: hand Translation proportional to initial object distance Time

49. World in Miniature User holds dynamic map in one hand Objects can be moved in map What about fine positioning? What about selection of small objects?

50. Voodoo Dolls User creates map with image plane selection

51. Summary Ergonomics an issue Usability still low Standard GUI elements translate poorly