VR Interfaces – Navigation, Selection and UI Elements By David Johnson

VR Interfaces How do we tell the computer to do things? How do we select things? How do we navigate around?

VR Interface Challenges Intuitive –Make interaction work like the real world –Minimize cognitive overhead Augmentation –Give users new capabilities

Quick UI review Norman’s Principles of Design –Make things visible –Provide a good conceptual model Affordance Mapping Constraints Feedback

Visibility

Good Conceptual Model A good conceptual model allows us to predict the effects of our actions Without a good model we operate blindly –Simply follow rules without understanding a reason –No understanding of cause or effect –No recourse when something breaks Fridge/freezer controls Thermostat

Affordances

Mapping

Constraints Prevent you from doing what you shouldn’t do –Grey out selections that don’t apply at the current time

Feedback Examples Clicker on your turn signal Animated icon while waiting for a web page to load

Why is usability important? Poor usability results in –anger and frustration –decreased productivity in the workplace –higher error rates –physical and emotional injury –equipment damage –loss of customer loyalty

2D Interfaces Dominant computer interface uses a mouse and graphical elements Xerox Star (1981)

2D Interfaces Why is it a WIMP interface? –Windows –Icons –Menus –Pointer Xerox Star (1981)

3D Interfaces Need to map 2D interfaces to 3D Hopefully, create whole new expressive interfaces

3D equivalent of a Mouse? Mouse –2D positioning –Buttons to hold or click

6DOF mouse Flying mouse Fledarmaus The Bat How do you clutch/ratchet? –In 2D, picking up disables tracking

Menus in Virtual Space Cannot easily overlay menus “Float” menus in space –Select by raycasting –Keep near user’s head Jacoby, Ellis 1992

Menus in Virtual Space Ring Menus –JDCAD 1993 Liang –ISAAC 1995, Mark Mine –Rotate hand to move selection point

Menus in Virtual Space Darken, 1994 Overlaid menus Speech selection

Menus in Virtual Space Pen and Tablet –Track a tablet and pen –Put 2D menus on tablet –“Haptic Hand”

Menus in Virtual Space Bowman Pinch Gloves –Select with thumb to finger –High-level menu on ND hand –Secondary menu on D hand –First tries Scrolling menu using pinches or More items on pinkie

TULIP Menus Three-Up, Labels in Palm Virtually raise hands Rotate menus Put ‘next’ groups on palms Users preferred over floating and tablet menus –Perhaps slower

Menus in Virtual Spaces ‘Virtual tricorder’ Wloka 1995

Gestures Symbolic –Cultural meaning (O.K. sign) Deictic –Pointing, direct viewer’s attention Iconic –Showing an example path with hand Pantomimic –Act out the activity

Gestures GIVEN (1992) –Neural net recognition –20 gestures Fly, grab, etc. Mine –“Physical mnemonics” Pull-down menus from near head Delete by throwing over shoulder

Numerical Input Mark Mine A digit at a time Sliders too imprecise

Text Input Bowman Pinch glove Thumb to home row finger Hands in/out to go down/up row Rotate to hit extra keys

Basic Navigation Tasks Exploration –Untargeted movement –Build internal map Positioning –Move to known location Maneuvering –Precise positioning of viewpoint –Typically short motions

Natural Interfaces Walking Biking Snowboard Swimming Issues?

Walking workaround Redirected walking –moviemovie

Flying Interfaces Flying Magic carpet Guided navigation –River analogy Issues?

Steering Interfaces Pointing –Expressive –Hand shake Torso Gaze-directed –simple Physical device

Hand-based Interfaces Colin Ware (1990s) –World-in-hand –Eye-in-hand

Point-to-point Travel Select a point in a scene –Computer picks path Teleport –Bowman et al. found significant spatial disorientation from teleport

World in Miniature User holds dynamic map in one hand Navigation is reduced to object positioning

WIM Setup Physical props – clipboard and interface ball

Two-handed Flying Mark Mine

Fundamental Operations in a UI Select an object Manipulate an object –Translate –Rotate What are some techniques in 2D interfaces?

From the Beginning Sutherland and Vickers –Sorcerer’s Apprentice (1972) Track stylus Selection of vertices –Intersection of cube at tip of stylus

Pointing: Put That There 1979 Ray from tracked hand Speech interface movie

Pointing: JD-CAD (1993) “Laser gun” from hand –Tracker noise –Harder to select far away Spotlight –Add a cone to the ray –Select objects based on Distance from cone axis Distance from hand

Silk Cursor Replace wireframe selection box with translucent box –Visual cues to containment

Pointing: Aperture Spotlight from eye Cone angle based on distance from hand to eye Selection modified by hand orientation

Pointing: Flexible Pointer Two-handed Hand direction bends pointer Can select occluded objects movie

Hand: GoGo Interaction (1996) Go-Go uses Non-Linear mapping between virtual and real hand Control-display ratio Stretch go-go variation Pros: –Extended reach when needed –Direct manipulation Cons: –Reach still limited by arm length –Precision suffers when reach is extended (low level of control) Movie

Image Plane Techniques Point or gesture at an objects projection onto the viewing plane –“head-crusher” Kids in the hall –“Sticky finger” Similar to ray casting Pros: –Very intuitive –Allows user to reach objects at an arbitrary distance Cons: –Limited by the need for line of sight –Can be fatiguing –Virtual hand may obscure small objects

Two Handed and Body-Centered Interaction What can you do with two hands? What if you use your body as a reference point? Mine, Mark, Frederick P. Brooks Jr., and Carlo Sequin (1997). Moving Objects in Space: Exploiting Proprioception in Virtual-Environment Interaction. Proceedings of SIGGRAPH 97, Los Angeles, CA.Moving Objects in Space: Exploiting Proprioception in Virtual-Environment Interaction. “Scaled-space grab”

HOMER technique Hand-Centered Object Manipulation Extending Ray-Casting Select: ray-casting Manipulate: hand Translation proportional to initial object distance Time

World in Miniature User holds dynamic map in one hand Objects can be moved in map What about fine positioning? What about selection of small objects?

Voodoo Dolls User creates map with image plane selection

Summary Ergonomics an issue Usability still low Standard GUI elements translate poorly