Cognitive Issues in Virtual Reality Wickens, C.D., and Baker, P., Cognitive issues in virtual environments, in Virtual Environments and Advanced Interface Design, Barfield and Furness, pp Summarized by Geb Thomas

The Main (Cognitive ) Features of VR 3D viewing vs. 2D viewing Dynamic vs. static displays Closed-loop vs. open-loop interaction Inside-out vs. outside in frame of reference Multimodal interaction

A Comparison

Uses of Virtual Reality What cognitive issues lie behind each application How do these play into the user’s perceptual strengths and weaknesses

On-Line Performance Direct manipulation capabilities in a remote environment Principle issues: –Closed-loop perceptual motor performance –Situation awareness –Low workload and cognitive effort

Off-line Training and Rehearsal Lumbar injection Maneuvering a space craft Rehearsing a dangerous mission Key consideration: –Effective transfer of training

On-line comprehension Reaching understanding, comprehension or insight Scientist interacting with a database Key: –perceiving relations –perceiving constraints –perceiving constancies

Off-line Knowledge Acquisition Useful for education Key issue: –knowledge transfer

How to Cognitively Engineer VR Select features that aid and do not disrupt cognitive needs –closed-loop performance may not help on-line performance but is key to understanding Different parts of the visual system are involved in perceptual-motor coordination and navigation than are involved with perceptual understanding of spatial location –Ambient vs. focal vision

The Operator Brings Wide sensory bandwidth Limited perceptual bandwidth Constraints on attention Constraints on working memory High level of natural perceptual-motor coordination Large repertoire of facts and knowledge

Search Find an object of interest Object of the search may be concrete instance or an abstraction of the rendering in VE A map often facilitates searching –Minimize map clutter –Flexible frame of reference –Tie or link map and VE

Navigation Challenging because of the removal of constraints Speed and flexibility can cause loss of situational awareness May help to partially automate movement, such as logarithmically control speed Metaphors matter –flight in mazes –“in-hand” for objects

Perceptual Biases Gibson and ecological perception, regularly spaced texture, level surfaces for gradients, slant and optical flow. Ellis, McGreevy et al. –Virtual space effect because of minification or magnification from FOV –2D-3D effect perceived rotation of vectors towards viewing plane –Display enhancement

Visual Motor Coupling and Manipulation Gain Time delay Control order Target view decoupling Field of view

Field of View Wider provides greater situational awareness Wider distorts perceived position Wider provide better sense of motion Wider can promote motion sickness

Perception and Inspection For navigation, spatial relations predominate For inspection, light, shadow, motion parallax Schematic figures for learning

Learning Procedural learning –Lower realism Perceptual motor skill learning –Active participation in control loop is important –Do dynamic simulation help? Spatial Learning and Navigation Rehearsal –rotating frame can inhibit map building –Active control loop is not always a benefit –Head-mounted displays may be a hindrance Conceptual learning –Multi-modal, active

HF Guidelines in Learning Consistency Redundancy Visual Momentum –Consistent representations –Graceful transitions –Highlight anchors