Making Sense of Sensing Systems Five Questions for Designers and Researchers Madhumita Behera Susanne Edevåg Weiwei Liu Magnus Lorentzon Joel Sandlund

Making Sense of Sensing Systems Five Questions for Designers and Researchers When I adress a system, how does it know I am adressing it? When I ask a system to do something, how do I know it is attending? When I issue a command, how does the system know what it relates to? How do I know the system understands my command, and is correctly executing my intended action? How do I recover from mistakes?

Making Sense of Sensing Systems Five Questions for Designers and Researchers The Topic of the Paper When designing traditional GUIs, solutions to common interaction issues are provided by copying existing ideas and using standardized toolkits. In ubiquitous computing, there seldom are any given solutions to these kind of problems. Inspired by HHI, the paper presents a framework for adressing common design challenges in ”sensing systems”.

Making Sense of Sensing Systems Five Questions for Designers and Researchers The Topic of the Paper When adressing these issues, the paper explores the possibility of applying theories of human-human interaction (HHI) to the science of HCI: Signals used to.. - adress each other - ignore each other - show intention to initiate conversation - show availability for communication - show understanding of what has been said - …

Making Sense of Sensing Systems Five Questions for Designers and Researchers Five Interaction Challenges - Adress: Directing communication to a system - Attention: Establishing that the system is attending - Action: Defining what is to be done with the system - Alignment: Monitoring system response - Accident: Avoiding or recovering from errors or misunderstanings.

Making Sense of Sensing Systems Five Questions for Designers and Researchers 1. Adress How do I adress one (or more) of many possible devices? The GUI solution: Keyboard, mouse, etc.

Making Sense of Sensing Systems Five Questions for Designers and Researchers 1. Adress In ubiquitous computing: Analyses of HHI show that humans make use of a formidable array of non-verbal mechanisms to accomplish or avoid addressing people. This poses a problem in UbiComp when using ambient modes of input, such as gestures. - How to disambiguate signal-to-noise? - How to disambiguate intended target system? - How to not adress the system?

Making Sense of Sensing Systems Five Questions for Designers and Researchers 1. Adress Three example projects

Making Sense of Sensing Systems Five Questions for Designers and Researchers 1. Adress The Listen Reader An interactive children’s storybook with a soundtrack that the reader plays by sweeping hands over pages. Embedded RFID tags sense which page is open, and capacitive field sensor measure human proximity to the pages.

Making Sense of Sensing Systems Five Questions for Designers and Researchers 1. Adress Augmented Objects Uses sensors and RFID tags. Wave at pickup sensors to initiate action. Digital Voices A computer-to-computer interaction mechanism that uses audile sound as the communication medium. A user can address a suitably equipped system using another Digital Voices enabled device, as long as the devices can ”hear” one another. Moreover, the user hears the communication as it occurs.

Making Sense of Sensing Systems Five Questions for Designers and Researchers 1. Adress Problems Failure to to communicate with the system if the sensing fails for any reason. Avoiding unintended communication with devices the user does not want to interact with. Simply getting too close can lead to accidental address which can be dangerous, e.g. a voice activated car phone triggered accidentally could compete for a driver’s attention with serious consequences. However, auidtory feedback from Digital Voices informs the user which devices are responding and helps them to decide whether the response is appropriate.

Making Sense of Sensing Systems Five Questions for Designers and Researchers 2. Attention Is the system ready and attending to my actions? The GUI solution: - Flashing cursor - Cursor move when the mouse is moved - Watch icon

Making Sense of Sensing Systems Five Questions for Designers and Researchers 2. Attention Design Challenges How to embody appropriate feedback to show the system’s attention? How to direct feedback to users attention zone?

Making Sense of Sensing Systems Five Questions for Designers and Researchers 2. Attention Problems Limited operations.. i.e. system is ignoring to draw attention of the user Failure to execute action.. i.e. absence of visible mode to make user to interact Wrong response

Making Sense of Sensing Systems Five Questions for Designers and Researchers 2. Attention Example: Conference Assistance System uses a sensing technology to identify a user and supply information about the user like location, session arrival and departure time. Supplies these information to the other conference attendees. The system always attends the user in range. Gives no feedback to tell the user that its actions are being monitored.

Making Sense of Sensing Systems Five Questions for Designers and Researchers 2. Attention Example: Audio Video Media Space Monitors placed next to camera in the public places to tell the inhabitants that they are on camera. By seeing their own images the people can say that the system is attending them.

Making Sense of Sensing Systems Five Questions for Designers and Researchers 3. Action How do I effect a meaningful action, control its extent and possibly specify a target or targets for my action. The GUI solution Direct manipulation, menu selection, accelerator keys

Making Sense of Sensing Systems Five Questions for Designers and Researchers 3. Action Exposed challenges How to identify and select a possible object for action How to identify and select an action and bind it to the object(s) How to avoid unwanted selection How to handle the complex operations?

Making Sense of Sensing Systems Five Questions for Designers and Researchers 3. Action Example: Sensor Chair Gesture-based Sound Control System From MIT Media Lab “Brain Opera” On Multiple Actions Multiple Objects

Making Sense of Sensing Systems Five Questions for Designers and Researchers 3. Action Example: SenseTable Augmented system supporting dynamic binding & unbinding of actions to objects From MIT Media Lab On Using Physical Traits of Objects for Multiple actions & Objects

Making Sense of Sensing Systems Five Questions for Designers and Researchers 4. Alignment How do I know that the system is doing the right thing? The GUI solution Echoing input text and formatting, wire-frame, outlines, progress bars, highlighting changes in a document, listing sent messages and so on.

Making Sense of Sensing Systems Five Questions for Designers and Researchers 4. Alignment Challenges Q: Show to make system state perceivable? A: Graphical and audio information in the space of action Q: Show to direct timely and correct feedback? A: Slow down the machines time frame to the same as people's time frame.

Making Sense of Sensing Systems Five Questions for Designers and Researchers 5. Accident How do I avoid mistakes? The GUI solution Words spelling correction Printjobs has cancel buttons Saving over old files -> popupbox

Making Sense of Sensing Systems Five Questions for Designers and Researchers 5. Accident Challenges Q: How do I stop or cancel a system action in progress? Q: In order to correct mistakes they have to be visible in time, before it is too late. No good solution..

Making Sense of Sensing Systems Five Questions for Designers and Researchers Questions Can you see any of these problems in your projects? If so, how will you plan to solve them?