1. To Err is Human Computational Limits to Human Thinking : Implications for the Design of Human Centered Interfaces Raj Reddy Carnegie Mellon University Pittsburgh, PA 15213 January 21, 2010 www.rr.cs.cmu.edu

2. Outline of Talk  Limitations of Current Interfaces from the Perspective of Cognitive Science  Missing Science of Multimedia Interfaces  Interfaces for Enabling Access to the Five Billion people at the Bottom of the Pyramid

3. Outline of Talk  Limitations of Current Interfaces from the Perspective of Cognitive Science  Missing Science of Multimedia Interfaces  Interfaces for Enabling Access to the Five Billion people at the Bottom of the Pyramid

4. Human Strengths and Human Limitations Lessons from Cognitive Science  Strengths: People  communicate using speech and natural language  tolerate errorful, ambiguous and imprecise input  exploit vast amounts of knowledge  learn from the environment  Limitations: People  make errors  tend to forget  become impatient  tend to be lazy

5. To Err Is Human Problem: –Catastrophic loss of data: “I didn’t mean to do that  ” –Unanticipated side-effects: “how did that happen??” Causes: –Sensory, Cognitive and Motor overload –Information Overload: “Like being in a traffic jam” –Timing Errors: Simplify the task Present Solutions: –Partial Undo –Dialog Boxes Future Opportunities: Unlimited Undo Anytime Anywhere Abort DWIM (Do What I Mean)

6. To Forget Is Human Problem: –A non-expert occasional user can’t be expected to remember the details Causes: –Forgetting is the loss of indexing structure –Redundancy in the indexing structure is the key Present Solutions: –Recognition vs. recall: GUIs and Menus –On-line manuals Future Opportunities: Use of color, fonts, voice responses for focusing attention Intelligent Help “How do I” and “What if” MultiMedia documentation “Reference librarian” agent

7. To be Impatient Is Human Problem: –Time to get the answer in interactive problem solving Long Start-up and Shutdown Times Network operations: Unpredictable retrieval and browsing times Present Solutions: –Hour glasses and wheels No idea how long it will take –Progress bars Future Opportunities: Updates in the background Learn from experience: self-aware systems Look ahead retrieval and computation Hurry-up algorithms Keystroke model

8. To be Lazy Is Human Problem: –Most people use a minimal subset of functionality in Word, PowerPoint, etc. –Most people avoid tasks requiring too much cognitive effort PGP - too much work FTP - too complex Causes –Principle of Least Effort Future opportunities: Advice giving agents that look over your shoulder Just-in-time learning Gentle slope systems: Easy Things should be Easy to Do Agents (wizards!) that know about PGP, FTP, or whatever

9. Outline of Talk  Limitations of Current Interfaces from the Perspective of Cognitive Science  Missing Science of Multimedia Interfaces  Interfaces for Enabling Access to the Five Billion people at the Bottom of the Pyramid

11. Missing Science of Multimedia Interfaces Lessons from Flash Forward Video SILKy Interfaces Speech, Image, Language and Knowledge Forgiving Interfaces: Tolerate error and ambiguity – “You Bet” --- “Did That Mean Yes?” Self Improving Interfaces: Incremental non-intrusive knowledge acquisition “Darn” --- “Would you Care to Define It?” Intelligent Help, Advice and Tutoring – “ Enthusiasm Detected --- Care to View Large Screen?” Intelligent Agents – Mail Serve: Junk Mail – Communications: Telephone Tag – Intelligent Tutoring Tools – Agent Creation Technology

12. Outline of Talk  Limitations of Current Interfaces from the Perspective of Cognitive Science  Missing Science of Multimedia Interfaces  Interfaces for Enabling Access to the Five Billion people at the Bottom of the Pyramid

13. User Interfaces for Rural Environments Low cost PC: target of less than $100 A Multifunction Information Appliance: PC tvt –PC tvt : TV, PVR, Video Phone, IP Phone and PC PCs for consumption, not creation –For Most People in a Village, Entertainment and Communication are of Greater Importance than PC Functionality

14. PC tvt UI Design for Use by Illiterate Persons An Illiterate person needs a more powerful PC than a Microsoft employee –If not e-mail, use voice-mail –Replace Text Help by Video Help Radically simple design –One minute learning time –Two click model –Three modes of communication: Video, Audio and Text –Both Synchronous and Asynchronous All-Iconic interfaces Multiple input modalities –TV-remote, Speech I/O, Keyboard, Mouse or Cell phone

15. Mapping TV Remote to Iconic Screen Layout OK 1 23 4 56 7 89 BACK 0 MENU 9 OK 12 3 4 6 7 9

16. TV MenuVideo Phone Menu OK 2 3 4 5 6 7 9 0

17. The Unfinished Research Agenda  Add Features to Assist in Overcoming Human Limitations such as “Unlimited Undo”, Instant On and Off, and Multimedia Help.  WIMPy to SILKy: Move Towards Intelligent Multimedia Interfaces  Eliminate Language Divide and Literacy Divide in Future Systems: Interfaces that Enable Access to People at the Bottom of the Pyramid