1. Chapter 9 System Control
Rowan Gudmundsson

2. Human Factors
Some human factors we have to consider when designing a 3D UI:
Visibility
Focus switching
Feedback

3. Visibility
Occlusion
Legibility

4. Focus Switching
UI decoupling
Physical devices

5. Feedback
Haptic
Auditory
Visual

6. Breadth and Depth Breadth (number of categories)
Depth (depth of sub-categories)

7. Ergonomic Issues Control position
Shape, size, and location of controls
Optimal grip, pose for user

8. System Factors Resolution, size, and luminance
Secondary visual devices
Input devices
Noise, device constraints

9. Classification Physical controllers Graphical menus Voice commands
Buttons
Switches
Graphical menus
Adapted 2D menus
1 DOF menus
3D Widgets
Voice commands
Gestural commands
Mimic gestures
Symbolic gestures
Sign language
Speech connected hand gestures
Whole-body interaction

10. Classification continued ...
Tools
Physical tools
Virtual tools
Tangibles
Multimodal techniques

11. Physical Controllers Buttons (usually to switch states) Examples:
External joysticks (flight simulators)
Controllers (consoles)
Mice (personal computers)

12. Graphical Menus
Adapted 2D menus
1-DOF menus
3D widgets

13. Adapted 2D Menus The 2D menus that we’re all used to but in 3D
Projected as a billboard into 3D space

14. Adapted 2D Menus ... Pros: Familiar Simple to use
Good for complex functionality
Cons:
Obvious occlusion issues
Difficult to navigate for some selection techniques

15. 1-DOF Menus Menu usually attached to the player in some way
Controlled by a single motion with a button selection

16. 1-DOF Menus Continued ... Pros: Natural for the user
Attached directly to the hand
Cons:
Ergonomic constraints
Simple functionality

17. 3D Widgets Tied directly to the object being manipulated
Allows for complex options for a specific interaction
2 Different types:
Collocated (context-sensitive)
Non-context-sensitive

18. 3D Widgets Continued ... Pros:
Allows for complex options for specific interactions
Integration into the 3D space
Cons:
Discoverability issues
Difficult to implement
Sometimes complex interactions are needed

19. Design and Implementation Issues
Placement
Selection
Representation and structure

20. Menu Placement Things to consider: Amount of occlusion introduced
Ability for the user to access the menu
Spatial reference point
world-ref , object-ref, head-ref, body-ref, device-ref.

21. Menu Selection Things to consider: Controlling degrees of freedom
Intuitive methods of selection

22. Menu Representation and Structure
Things to consider:
Size and spacing issues
Functionally grouping items
Control coding (textures, colors, dimensions, symbols, etc.)
Environment issues (luminance, contrast, etc.)

23. Voice Commands
Can be very effective in applications which require the use of both hands
Very low focus shift
Graphically invisible to the user
Downsides:
Very robust speech recognition engine for seamless integration
Influenced by background noise
Can be tiring for the user
Requires confirmation to verify intent

24. Gestural Commands Broken down into 2 categories:
Postures (static poses)
Gestures (full motion metaphor)

25. Gestural Commands Continued ...
Mimic gestures
Not connected to speech but instead convey a concept
Symbolic gestures
Gestures as used in daily life to express things such as insults or praise
Sweeping
Gestures for navigation like scrolling
Sign language
Set of postures or gestures to convey particular words or numbers
Speech connected hand gestures
Gestures people use when speaking to further convey emotion or to emphasise a point
Surface based gestures
Gestures made on 2D surfaces (eg. pinch-zoom)
Whole-body interaction
Mimic or symbolic gestures using more than just hands and arms

26. Gestural Commands Continued ...
Pros:
Natural to the user
Can be used to convey complex interactions
Cons:
Calibration may be required
Requires a machine learning or heuristic back-end
Like speech, a confirmation is usually needed to verify intent

27. Tools Physical tools Tangibles Virtual tools
Advantages of using tools:
Users can operate them by touch which frees up visual attention
May feel more natural to the user
More defined behaviour

28. Physical Tools Real world “props” which have a specific purpose
Largely used in public installations like theme parks or museums
Very easy to use but have limited functionality
Pirates of the caribbean - Disney quest

29. Tangibles A superset of physical tools
More abstract than physical tools
Tool may have buttons or gestures associated with it to allow for more functionality

30. Virtual Tools Tools which have no physical manifestation
Representation of a specific action set in the virtual environment
Allows developer to group actions

31. Multimodal Techniques
Combines previous techniques to give a more complete UX.