Ubiquitous Computing and Augmented Realities Material from Authors of Human Computer Interaction Alan Dix, et al

Overview Breaking the tradition of the computer box ubiquitous computing virtual reality augmented reality information visualization

Introduction Ubiquitous computing (pervasive computing) single user --> groups and larger organizations traditional desk top --> computational power in the environment user search out computer --> interface locates and serves user

Three challenges 1. Defining appropriate physical interaction experience 2. Discovering general application features 3. Theories for designing and evaluation human experience with ubicomp

Challenge 1: Input Textual input --> pointing devices for variety of data types Explicit input --> implicit input (natural interactions with environment) tipping a PDA to move between pages walking into a space announces one’s presence Better support for natural human communication (handwriting, gestures, etc.)

Challenge 1: Input Large scale touch interactive surfaces (capacitive coupling) RFID sensors (security tags) Infrared range finders Incorporation into artifacts

Challenge 1: Output Concerns must address technology and aesthetics Size of displays Multiple displays Multiple modalities Ambient forms of communication

Challenge 2: Output Size of displays Small (inch)- mobile devices Medium (foot)- traditional desktop displays Large (yard) multiple low-resolution projected displays display wall coordinate interaction between them

Challenge 1: Output Ambient displays peripheral output information is medium to low priority aesthetic appeal Examples: Dangling String (detects network traffic) Audio Aura (auditory cues eg. ) Kandinsky (collage of images from keywords)

Challenge 1: Seamless integration Merge computation artifacts with world of physical artifacts Augmented reality Use objects in physical world to manipulate electronic artifacts

Challenge 2: Application themes To make computing disappear, need a combination of a large range of services Context-aware computing Capture and record experiences

Challenge 2: Context Aware Computing Location aware appliances simple context - user location (where and who) provide services like automatic cell forwarding updated maps of user locations GPS navigation systems tour guide systems need more context When - relative changes of time, vary from routine What - perceiving and interpreting human activity (Cookies) Why

Challenge 2: Automated capture and access Augment inefficiency of human record taking Record multiple streams of activity Old idea : Vannevar Bush and Memex to store artifacts we come in contact with in everyday lives and the associations that we create between them Applications Meeting capture at Xerox Parc

Understanding interaction Everyday computing Continuous interaction orchestrates daily routines, communicate with others, and manages information Focus on activities (eg. writing), not tasks Problems:  no clear start or end  interruptions  concurrent activities

Understanding Interaction In Ubiquitous Computing Integrate numerous devices in one setting Need knowledge in the world (rather than in the head) Need models that consider nature of relationship between internal cognitive processes and the outside world Three main theories serve as guides Activity Theory Situated Action Distributed Cognition

Understanding Interaction: Activity Theory Oldest of three and closest to tradition theory Recognizes goals (objects), actions, and operations BUT goals and actions are fluid based on changing state of world emphasizes transformational properties of artifacts

Understanding Interaction: Situated Action Reject the pre-planned goals giving rise to actions Emphasizes improvisational aspects of human behavior Knowledge in the world continues to shape interpretation and execution of task System aims to add knowledge to assist in shaping user’s actions

Understanding Interaction: Distributed Cognition Also de-emphasizes internal human cognition View humans as a part of larger system Emphasize information is encoded in objects and how that information is translated/transmitted by different users

Challenge 3: Evaluation Challenges Develop assessment and evaluation techniques for ubicomp Predict how novel technologies serve human need Observe authentic use and co-evolution of human activities and novel technologies

Challenge 3: Evaluation Challenges Little research to date on this topic understanding is on structuring activities and tasks not clear when we move away from structured work ubicomp systems not yet reliable and robust to support continuous activity --> how to evaluate? Essentially we have living laboratories

Virtual and Augmented Reality VR refers to computer-generated simulation of a world or a subset of it immersive (cave) desktop (command and control situations) augmented reality where virtuality and reality meet

VR Technology: Immersive VR Challenges individual devices must work together render realistic images in real time computationally expensive faceted, limited texture, flat lighting Applications architectural tours wind tunnel protein chemistry

Desktop VR Lower cost alternative to immersive VR Applications flight simulators interactive games (DOOM, MYST, QUAKE) Uses VRML (Virtual Reality Markup Language) creates virtual worlds which can be linked

Command and Control Operator is in a physical environment with real controls and sees through windows or video screens cockpit flight simulator VR games at arcades

Augmented Reality Electronic images are projected over the real world - virtuality and reality meet Use semi-transparent goggles or wearable computers read while walking around images projected on a paper desktop Problem: Registration - correctly aligning physical and virtual world

Current and Future Applications of VR TO date Military simulations and militaristic games Medical training virtual operation on real patient Treatment of phobias Future VR holidays (holodeck)

Information and Data Visualization Scientific and technical data virtual reality 3D maps Structured information Hierarchies and networks Time and interactivity Use of color to determine patterns

Information and Data Visualization Scientific and technical data virtual wind tunnel investigate patterns of air movement  investigate turbulence cheaper than model making engineers can get close without disturbing air flow relief map to represent rainfall (3D) More abstract (3D) data

Information and Data Visualization Structured information Hierarchies and networks split over several pages avoidance of crossing lines when node selected, system rotates view shadows can give clues about hidden parts of trees

Information and Data Visualization Time and interactivity Time is 3rd dimension Model temporal data Model movement and changes Replay at various speeds Several slices of object can be displayed of use ‘slider’ to progress through object Use color to show degree of matching of patterns