Common Sense Computing MIT Media Lab Interaction Challenges for Agents with Common Sense Henry Lieberman MIT Media Lab Cambridge, Mass. USA

Common Sense Computing MIT Media Lab Agents with Common Sense Some AI interfaces are now beginning to make use of large knowledge bases of Common Sense Explicitly collected Cyc, Open Mind, ThoughtTreasure Distilled from other sources Semantic Web, Wikipedia, Web resources Web mining, other resource mining

Common Sense Computing MIT Media Lab Interaction Challenges Finding opportunities for applying Common Sense in interfaces Setting users' expectations Making interfaces fail-soft Taking advantage of user interaction

Common Sense Computing MIT Media Lab Interaction Challenges Making better mistakes Get lots of knowledge, but not too much Common sense inference vs. mathematical inference Debugging Evaluating Common Sense interfaces

Common Sense Computing MIT Media Lab Interaction challenges for AI/Common Sense Many interaction challenges for Common Sense interfaces are the same as for AI in general But some are unique or critical for Common Sense… Can't be sure what will be known Reasonable, rather than right Know a little about everything, not much about anything Don't miss the obvious Try not to make stupid mistakes

Common Sense Computing MIT Media Lab Open Mind Common Sense Asks the Web community to contribute English sentences expressing Common Sense knowledge "The Wikipedia version of Cyc" (#2 after Cyc) Launched in 2001 by Push Singh Now contains ~800 Kilofacts Freely available / Open Source Some versions in other languages/cultures Brazilian Portuguese, Korean, Japanese, Chinese

Common Sense Computing MIT Media Lab Open Mind Common Sense English sentences parsed by POS Tagger Pattern-directed mining of 22 relations (isA, PartOf, UsedFor…) Strong focus on easy integration with applications Semantic Net (ConceptNet: Liu, Eslick) Natural Language toolkit (MontyLingua: Liu) Tools for: Context, Analogy, Affect and more

Common Sense Computing MIT Media Lab What do we mean by “Common Sense”? Simple statements about everyday life Things fall down, not up A wedding has a bride and a groom You go to a restaurant to eat And…the ability to use that knowledge when appropriate

Common Sense Computing MIT Media Lab Open Mind Common Sense - Push Singh & colleagues

Common Sense Computing MIT Media Lab ConceptNet - Liu, Singh, Eslick

Common Sense Computing MIT Media Lab Common Sense projects Predictive typing Speech recognition -- disambiguation & error correction Storytelling with photo libraries Searching social networks Macro recording using Common Sense generalization World construction for video games Phrasebook for tourist information Debugging problems in e-commerce interactions

Common Sense Computing MIT Media Lab Common Sense projects Video editing based on story structures Goal-oriented interfaces for consumer electronics Mining Common Sense from the Web Multi-lingual and multi-cultural Common Sense; translation Games for acquiring, verifying and using Commonsense knowledge Commonsense "Captchas" Understanding imprecise qualities such as affect ShapeNet and Expectation-driven Image Recognition Understanding sensor data using Commonsense

Common Sense Computing MIT Media Lab Opportunities for using Common Sense Find UI situations that are underconstrained Ordinary system would either take no action or do something arbitrary Then, give user some reasonable choices Provide intelligent defaults Make the most likely thing easiest to do

Common Sense Computing MIT Media Lab Opportunities for using Common Sense Recognize users' likely goals Help users map from goals to actions Sanity checking In the case of trouble, help users debug

Common Sense Computing MIT Media Lab Opportunities for using Common Sense Find situations where every little bit helps A little bit of knowledge is better than none A little bit of knowledge about a lot of things can be more useful than a lot of knowledge about a few things

Common Sense Computing MIT Media Lab Setting users' expectations Avoid direct question-answer interfaces Right or wrong. Only one shot. Better to cast system in role of advisor Making suggestions, help Adapting interface to most likely uses Remove unnecessary steps in the interface The user only expects intelligent behavior only once in a while

Common Sense Computing MIT Media Lab Take advantage of user interaction Repurpose input that the user gives you for other reasons Every time the user tells the interface something, they're telling you what their interests are -- learn from it

Common Sense Computing MIT Media Lab Make Common Sense interfaces "fail-soft" There should no dire consequences of being wrong or not knowing what to do Don't interfere if the user wants to use the application without interaction with the agent If the relevant knowledge is missing, incomplete or wrong, the user is no worse off than without the agent

Common Sense Computing MIT Media Lab Make better mistakes Common Sense approaches have the advantage that when they make mistakes, they tend to make plausible mistakes Statistical approaches can make arbitrary mistakes Better mistakes improve user trust in interfaces

Common Sense Computing MIT Media Lab Evaluation of Common Sense interfaces Evaluation is tough because Depends on what's in the knowledge base Depends on limited-depth and other kinds of approximate inference Standardized tasks don't test breadth of coverage Try to relativize testing to coverage Start with easier cases, then move to "typical" or "hard" cases

Intelligent User Interfaces Location: Canary Islands, Spain Dates: January 2008 Deadline: late September 2007