Commonsense Reasoning in and over Natural Language Hugo Liu, Push Singh Media Laboratory of MIT The 8 th International Conference on Knowledge- Based Intelligent Information & Engineering Systems (KES ’ 04)

2 Abstract ConceptNet is a very large semantic network of commonsense knowledge suitable for making various kinds of practical inferences over text. ConceptNet is a very large semantic network of commonsense knowledge suitable for making various kinds of practical inferences over text. To meet the dual challenge of having to encode complex higher- order concepts, and maintaining ease-of-use, we introduce a novel use of semi-structured natural language fragments as the knowledge representation of commonsense concepts. To meet the dual challenge of having to encode complex higher- order concepts, and maintaining ease-of-use, we introduce a novel use of semi-structured natural language fragments as the knowledge representation of commonsense concepts.

3 Introduction What is ConceptNet? What is ConceptNet? The largest freely available, machine-useable commonsense resource. The largest freely available, machine-useable commonsense resource. Structured as a network of semi-structured natural language fragments. Structured as a network of semi-structured natural language fragments. Consists of over 250,000 elements of commonsense knowledge. Consists of over 250,000 elements of commonsense knowledge. Inspired dually by the range of commonsense concepts and relations in Cyc, and by the ease-of-use of WordNet. Inspired dually by the range of commonsense concepts and relations in Cyc, and by the ease-of-use of WordNet.

4 Introduction The work of the author for ConceptNet The work of the author for ConceptNet 1. Extending WordNet ’ s lexical notion of nodes to a conceptual notion of nodes  Semi-structured natural language fragments according to an ontology of allowable syntactic patterns. 2. Extending WordNet ’ s small ontology of semantic relations to include a richer set of relations appropriate to concept-level nodes. 3. We supplement the ConceptNet semantic network with some methodology for reasoning. 4. We supplement the ConceptNet semantic network with a toolkit and API which supports making practical commonsense inferences about text.

5 Fig 1. An excerpt from ConceptNet ’ s semantic network of commonsense knowledge

6 Origin of ConceptNet ConceptNet is mined out of the Open Mind Commonsense (OMCS) corpus. ConceptNet is mined out of the Open Mind Commonsense (OMCS) corpus.Open Mind CommonsenseOpen Mind Commonsense A collection of nearly 700,000 semi-structured English sentences of commonsense facts. A collection of nearly 700,000 semi-structured English sentences of commonsense facts. An automatic process which applies a set of ‘ commonsense extraction rules ’. An automatic process which applies a set of ‘ commonsense extraction rules ’. A pattern matching parser uses roughly 40 mapping rules. A pattern matching parser uses roughly 40 mapping rules.

7 Structure of Concept ConceptNet nodes are fragments semi-structured to conform to preferred syntactic patterns. ConceptNet nodes are fragments semi-structured to conform to preferred syntactic patterns. 3 general classes: 3 general classes: Noun Phrases: things, places, people Noun Phrases: things, places, people Attributes: modifiers Attributes: modifiers Activity Phrases: actions and actions compounded with a NP or PP Activity Phrases: actions and actions compounded with a NP or PP ConceptNet edges are described by an ontology of 19 binary relations. ConceptNet edges are described by an ontology of 19 binary relations. The syntactic and/or semantic type of the arguments are not formally constrained. The syntactic and/or semantic type of the arguments are not formally constrained.

8 Table 2. Semantic relation types currently in ConceptNet

9 Methodology for Reasoning over Natural Language Concepts Computing Conceptual Similarity Computing Conceptual Similarity Flexible Inference Flexible Inference Context finding Context finding Inference chaining Inference chaining Conceptual analogy Conceptual analogy

10 Computing Conceptual Similarity (1/3) 1. The concept is decomposed into first order atomic concepts to compute its meaning. Ex: “ buy good cheese ” → ” buy ”, “ good ”, “ cheese ” 2. Each atom is situated within the conceptual frameworks of several resources. WordNet WordNet Longman ’ s Dictionary of Contempory English (LDOCE) Longman ’ s Dictionary of Contempory English (LDOCE) Beth Levin ’ s English Verb Classes Beth Levin ’ s English Verb Classes FrameNet FrameNet

11 Computing Conceptual Similarity (2/3) 3. Within each resource, a similarity score is produced for each pair of corresponding atoms. (verb matching verb, etc) The similarity score is inversely proportional to inference distance in WordNet, LDOCE, or FrameNet ’ s inheritance structure. The similarity score is inversely proportional to inference distance in WordNet, LDOCE, or FrameNet ’ s inheritance structure. In Levin ’ s Verb Classes, the similarity score is proportional to the percentage of alternation classes shared. In Levin ’ s Verb Classes, the similarity score is proportional to the percentage of alternation classes shared. 4. The weighted sum of the similarity scores is produced for each atom using each of the resources. Weight on each resource is proportional to the predictive accuracy of that resource. Weight on each resource is proportional to the predictive accuracy of that resource. 5. Weight on a atom is proportional to the relative importance of the different atom type.

12 Computing Conceptual Similarity (3/3) Computing conceptual similarity using lexical inferential distance is very difficult, so we can only make heuristic approximations. Computing conceptual similarity using lexical inferential distance is very difficult, so we can only make heuristic approximations. Table 3. Some pairwise similarities in ConceptNet

13 Flexible Inference One of the strengths of representing concepts in natural language is the ability to add flexibility and fuzziness to improve inference. One of the strengths of representing concepts in natural language is the ability to add flexibility and fuzziness to improve inference. Inferences in semantic networks are based on graph reasoning methods like spreading activation, structure mapping, and network traversal. Inferences in semantic networks are based on graph reasoning methods like spreading activation, structure mapping, and network traversal. Basic spreading activation Basic spreading activation activation_score(B) = activation_score(A)*weight(edge(A,B))

14 Flexible Inference – Context Finding Determining the context around a concept, or around the intersection of several concepts is useful. Determining the context around a concept, or around the intersection of several concepts is useful. The contextual neighborhood around a node is found by performing spreading activation from that source node. The contextual neighborhood around a node is found by performing spreading activation from that source node. Pairwise similarity of nodes leading to a more accurate estimation of contextual neighborhood. Pairwise similarity of nodes leading to a more accurate estimation of contextual neighborhood.

15 Flexible Inference – Inference Chaining Inference chain is a basic type of inference on a graph: traversing the graph from one node to another via some path. Inference chain is a basic type of inference on a graph: traversing the graph from one node to another via some path. A temporal chain between “ buy food ” and “ fall asleep ” : A temporal chain between “ buy food ” and “ fall asleep ” : “ buy food ”  “ have food ”  “ eat food ”  “ feel full ”  “ feel sleepy ”  “ fall asleep ” The pairwise conceptual similarity is particularly crucial to the robustness of inference chaining. The pairwise conceptual similarity is particularly crucial to the robustness of inference chaining. ex: “ buy steak ” instead of “ buy food ” (Liu, 2003) used inference chaining for affective text classification. (Liu, 2003) used inference chaining for affective text classification.

16 Flexible Inference – Conceptual Analogy Structural analogy is not just a measure of semantic distance. Structural analogy is not just a measure of semantic distance. Ex: “ wedding ” is much more like “ funeral ” than “ bride ” Structure-mapping methods are employed to generate simple conceptual analogies. Structure-mapping methods are employed to generate simple conceptual analogies. We can emphasize functional similarity versus temporal similarity by biasing the weights of particular semantic relations. We can emphasize functional similarity versus temporal similarity by biasing the weights of particular semantic relations.

17 Some Applications of ConceptNet MAKEBELIEVE MAKEBELIEVE A story-generator that allows a person to interactively invent a story with the system. A story-generator that allows a person to interactively invent a story with the system. GloBuddy GloBuddy A dynamic foreign language phrasebook. A dynamic foreign language phrasebook. AAA AAA A profiling and recommendation system that recommends products from Amazon.com by using ConceptNet to reason about a person ’ s goals and desires. A profiling and recommendation system that recommends products from Amazon.com by using ConceptNet to reason about a person ’ s goals and desires.

18 Conclusion ConceptNet is presently the largest freely available commonsense resource, with a set of tools to support several kinds of practical inferences over text. ConceptNet is presently the largest freely available commonsense resource, with a set of tools to support several kinds of practical inferences over text. ConceptNet maintains an easy-to-use knowledge representation and incorporates more complex higher-order commonsense concepts and relations. ConceptNet maintains an easy-to-use knowledge representation and incorporates more complex higher-order commonsense concepts and relations. A novel methodology for computing the pairwise similarity of concepts is presented. A novel methodology for computing the pairwise similarity of concepts is presented. ConceptNet has been widely used in a number of research projects. ConceptNet has been widely used in a number of research projects.