1. Adding Common Sense into Artificial Intelligence Common Sense Computing Initiative Software Agents Group MIT Media Lab

2. Why do computers need common sense? Conversation works because of unspoken assumptions People tend not to provide information they consider extraneous (Grice, 1975) Understanding language requires understanding connections

3. What can computers do with common sense? Understand the context of what the user wants Fill in missing information using background knowledge Discover trends in what people mean, not just what they say But how do we collect it?

4. A Brief Outline What is OMCS? What is ConceptNet? Using AnalogySpace for Inference Using Blending for Intuition OMCS Applications

5. Open Mind Common Sense Project Collecting common sense from internet volunteers since 2000 We have over 1,000,000 pieces of English language knowledge from 15,000 contributors Multilingual –Additional resources in Chinese, Portuguese, Korean, Japanese, and Dutch –In-progress: Spanish and Hungarian Users consider 87% of statements used in ConceptNet to be true

6. “A coat is used for keeping warm.” “People want to be respected.” “The sun is very hot.” “The last thing you do when you cook dinner is wash your dishes.” “People want good coffee.” What kind of knowledge?

7. Where does the knowledge come from? Contributors on our Web site (openmind.media.mit.edu) Games that collect knowledge

10. What is ConceptNet? A semantic network representation of the OMCS database (Liu and Singh, 2004) Over the years, used for: affect sensing, photo and video storytelling, text prediction, goal-oriented interfaces, speech recognition, task prediction, … ConceptNet 4.0 – Over 300,000 connections between ~80,000 concepts – Natural language processing tools to help line up your data with ConceptNet

11. An Example

12. Creation of ConceptNet A shallow parser turns natural language sentences into ConceptNet assertions 59 top-level patterns for English, such as “You would use {NP} to {VP}” {NP} and {VP} candidates identified by a chart parser

13. Representation Statement: expresses a fact in natural language Assertion: asserts that a relation exists between two concepts Concepts: sets of related phrases –identified by lemmatizing (or stemming) and removing stop words Relations: one of 25: –IsA, UsedFor, HasA, CapableOf, Desires, CreatedBy, AtLocation, CausesDesire, …

14. Example

15. Reliability Reliability increases when more users affirm that a statement is true –by entering equivalent statements independently –by rating existing statements on the Web Each assertion gets a weight according to how many users support it

16. Polarity Allows predicates that express true, negative information: “Pigs cannot fly” Negated assertions are represented by negative weights Reliability and polarity are independent

17. AnalogySpace Technique for learning, reasoning, and analyzing using common sense AnalogySpace can: –generalize from sparsely-collected knowledge –confirm or question existing knowledge –classify information in a knowledge base in a variety of ways Can use the same technique in other domains: businesses, people, communities, opinions

18. AnalogySpace Overview Finds patterns in knowledge Builds a representation in terms of those patterns Finds additional knowledge using the combination of those patterns Uses dimensionality reduction via Singular Value Decomposition

19. Input to the SVD Input to SVD: matrix of concepts vs. features Feature: a concept, a relation, and an open slot, e.g., (..., MadeOf, metal) Concepts × features = assertions

20. The Input Matrix For consistency, we scale each concept to unit Euclidean magnitude

21. Running the SVD

22. The Truncated SVD Truncating the SVD smoothes over sparse data.

24. Good vs. Bad

27. Reasoning with AnalogySpace Similarity represented by dot products of concepts (AA T ) –Approximately the cosine of their angle

28. Reasoning with AnalogySpace Predictions represented by dot products of concepts with features

29. Contributors are in the loop

30. Ad-hoc Categories

31. What can we use common sense for? A “sanity check” on natural language Text prediction Affect sensing Recommender systems “Knowledge management”

32. Common Sense in Context We don’t just use common sense to make more common sense Helps a system make sense of everyday life –Making connections in domain-specific information –Understanding free text –Bridging different knowledge sources

33. Digital Intuition Add common sense intuition Using similar techniques to make connections and inference between data sets Create a shared “Analogy”Space from two data sets using Blending

34. Blending Two data sets are combined in a way to maximize the interaction between the data sets They are weighted by a factor: C = (1 – f)A + fB

35. Blending Creates a New Representation With f = 0 or 1, equivalent to projecting one dataset into the other’s space In the middle, representation determined by both datasets.

36. No overlap = no interaction A’s singular valuesB’s singular values

37. Overlap -> Nonlinear Interaction (Veering)

38. Overlap -> Nonlinear Interaction

39. SVD over Multiple Data Sets Convert all data sets to matrices Find a rough alignment between the matrices –Some rows or features Find a blending factor –Maximize veering or interaction Run the AnalogySpace process jointly

40. Blends of Multiple Data Sets You can blend more than two things –Simple blending heuristic: scale all your data so that their largest singular vectors are equal

41. Applications Inference over domain specific data Word sense disambiguation Data visualization and analysis Finance

42. Tools we Distribute The OMCS database ConceptNet Divisi In development: the Luminoso visualizer

43. The Common Sense Computing Initiative Web: http://csc.media.mit.edu/http://csc.media.mit.edu/ Email: conceptnet@media.mit.educonceptnet@media.mit.edu Thank you!