1. KNOWLEDGE REPRESENTATION Ziyang Lin March 17, 2011

2. Different Forms of Knowledge 2  Knowledge can be represented in different forms  “Apple is red”  “1+2=3”  “The probability of raining in the afternoon is 20%”  Difference forms of knowledge give agents different reasoning ability  What can be the best way to represent knowledge?

3. Overview 3  Knowledge in Unified Learning Model (3-1)  Knowledge Representation and Ontologies (3-17)  Example: Knowledge in AI system Cyc (3-17)

4. Overview 4  Knowledge in Unified Learning Model (3-1)  Knowledge Representation and Ontologies (3-16)  Example: Knowledge in AI system Cyc (3-16)

5. Knowledge in Unified Learning Model 5  What is Knowledge in ULM  The second core component in ULM  Everything we know or can do that is stored in the neurons in our long-term memory  The micro-architecture of the brain  Resides in the firing potentials and interconnections of neurons  Some integrated cluster or network of neurons that form a chunk

6. Knowledge in Unified Learning Model 6  Retrieval and Connection of long term memory  Retrieval Neurons fire (matched chunk fire) when a pattern match occurs in working memory The firing potential increases (learning by repetition) If a chunk doesn’t fire, the firing potential decreases (may never be retrievable) “What is retrieved is what matches best”- retrieve the chunk with highest firing potential  Connection A chunk is an interconnected cluster of neurons and it is a single entity Chunks are also connected Example: ball->football game->watch games with friends

7. Knowledge in Unified Learning Model 7  Different types of knowledge  Episodic Knowledge  Semantic Knowledge Declarative Knowledge Objective Knowledge Symbolic Knowledge Procedural Knowledge

8. Knowledge in Unified Learning Model 8  Episodic Knowledge  Episodic knowledge (episodic memory) refers to our knowledge of our own lives  Also known as autobiographical memory  Episodic knowledge is attended to automatically in working memory (Auto Attention)  Usually does not last long because live seldom repeats  Example: what did you eat yesterday? One week ago? One year ago?  Stable elements lasts longer  Example: who was you advisor yesterday? One week ago? One year ago?

9. Knowledge in Unified Learning Model 9  Semantic knowledge  All knowledge that is not episodic, not about our own life  Example: 2 + 2 = 4 is the knowledge for everyone  Is what we teach in school  Begins as isolated information and is built into larger chunks (learning math)  We hardly learn it automatically  Learning episodic knowledge is effortless but learning semantic knowledge requires effort

10. Knowledge in Unified Learning Model 10  Semantic Knowledge  Declarative Knowledge Referred to as “fact" and “concept” Objective Knowledge (the actual things) knowledge of objects and actions in the outside world as we experience it through the senses long-term memory representation of sensory input Symbolic Knowledge socially transmitted Example: meaning of words, theoretical explanation not a copy of a visual, auditory, or other sensory stimulus

11. Knowledge in Unified Learning Model 11  Building a Chunk  Frequency of characters of a concept is maintained in the chunk  Neurons operate according to the law of large numbers  More repetition -> higher frequency -> better understanding of the actual knowledge  Starting from an initial teaching closer to the actual knowledge saves time of learning

12. Knowledge in Unified Learning Model 12  Semantic Knowledge  Procedural Knowledge Knowledge that produces action Involve creating knowledge structures that turn movement into coordinated action sequences to achieve specific results Involve creating coordinated sequences of cognitive actions or thinking that do not have specific physical movement as their outcome purposeful or goal directed in the form of “if then” relationship

13. Knowledge in Unified Learning Model 13  Building a Procedure  A chain of actions is strengthened if the result is acceptable.  Example: If something hits my head and it is water, then it is raining. If it is raining, then I will get wet. If I don’t want to get wet, then I need an umbrella. If I don’t have an umbrella, then find one. If I have an umbrella, then find the opening mechanism. If I have found the opening mechanism and I recognize how to operate it, then open it. If I have found the opening mechanism and I don’t recognize it, then figure out how to open and open.

14. Knowledge in Unified Learning Model 14  Automaticity – Goal of Procedural Learning  At the beginning, we know that we are learning and thinking  Working Memory is allocated for each step in the chain of actions  At the end, we just allocate wm for the first condition chunk in the chain and the entire chain fires without further wm involvement  Example: You learn to drive with much attention and later you can drive with less attention while talking to the others

15. Knowledge in Unified Learning Model 15  Building Larger Knowledge Network  Building Declarative Network  Building Procedural Network

16. Knowledge in Unified Learning Model 16  Building Declarative Network  Hierarchical structuring of knowledge  Example: cat and dog are mammals  Frequency of common characters are counted at higher level chunk  Relation between higher and lower chunks is strengthened if they are retrieved together  Relation between sibling chunks is lowered if they are not retrieved together.

17. Knowledge in Unified Learning Model 17  Building Procedural Network  Sequences (chain) of actions are connected by “connector” to avoid building sequence which is too long  At the “connector”, we evaluate the result of the sequence and decide whether to connect to the next sequence  Example: when we catch the ball in a basketball game, we may wait and see whether we should pass or shoot

18. Knowledge in Unified Learning Model 18  Situated Knowledge and Transfer  Situated knowledge is the one linked to context in specific components in environment and it can not be transferred to other environment  In order to build complex connections, we need to remove the situated components and generalize the knowledge

19. Knowledge in Unified Learning Model 19  Situated Knowledge and Transfer  The best way to transfer the knowledge is to practice in the real world situations and working memory will try to connect the chunks and procedures using pattern matching

20. Knowledge in Unified Learning Model 20  Problem Solving and Critical Thinking  When the sensory input dose not trigger any pattern match  We can search for new sensory input  Or create new “knowledge” in working memory in real time

21. Knowledge in Unified Learning Model 21  Incidental Learning  The “knowledge” created dynamically in problem solving and critical thinking can be part of the long term knowledge if it shows up repetitively in the future  But this “knowledge” may not be optimal  Once it is stored in the long term memory, it can hardly be removed  Hands-on instructional approach in teaching helps incidental learning

22. Knowledge in Unified Learning Model 22  Knowledge and Working Memory Interaction: Expanding Capacity  The capacity limit in WM concerns Temporary working memory storage Focused attention Processing in temporary storage  Each slot may contain a chunk of declarative knowledge or a condition of procedural knowledge

23. Knowledge in Unified Learning Model 23  Knowledge and Working Memory Interaction: Expanding Capacity  When neuron fires a pattern match for one slot, all the related knowledge will be accessible  Not all the knowledge is put into the WM, only the desired result is pulled out to replace the content in the WM  A slot can be considered the entry to all the knowledge in the brain  The capacity is just the total number of the entries  Example: I see a baseball flying towards me (sensory input in WM) and I could catch it, throw it back to my teammate (knowledge fired in the neurons)

24. Knowledge in Unified Learning Model 24  Basic Knowledge Processes  1. If knowledge in long-term memory is retrieved, its strength is increased (the repetition effect).  2. If a knowledge chunk is retrieved, all other chunks to which it is connected are retrieved and all the connections between them are strengthened (the spreading activation effect).  3. If parts of retrieved knowledge match to working memory contents, they are strengthened; if parts of retrieved knowledge do not match to working memory contents, they are weakened or inhibited (chunk building – the law of large numbers).  4. Learning personal, episodic knowledge of one’s life is easy; learning semantic(non-episodic) knowledge is hard (the Fourth Rule of the ULM).

25. Knowledge in Unified Learning Model 25  Basic Knowledge Processes  5. If an action is successful, its connection to the knowledge of the situation in which it occurred is strengthened; if an action is unsuccessful, its connection to the knowledge of the situation is weakened or inhibited (proceduralization – the practice effect).  6. If knowledge has been retrieved, new information in working memory will be connected to this knowledge (ULM Learning Principle 2: The Prior Knowledge Effect).  7. Any active knowledge in long-term memory is accessible to working memory (ULM Principle 2: Working memory capacity is increased by prior knowledge)

26. Question? 26  What could be the benefit of implementing ULM Knowledge in agent reasoning?  What could be the disadvantage?

27. Overview 27  Knowledge in Unified Learning Model (3-1)  Knowledge Representation and Ontologies (3-17)  Example: Knowledge in AI system Cyc (3-17)

28. Knowledge Representation and Ontology 28  In Artificial Intelligence, knowledge representation studies the formalization of knowledge and its processing within machines.  Many important research projects begin in 1970s and early 1980s  For example, Cyc project (to be discussed later) is one of the most significant projects and it is still ongoing today

29. Knowledge Representation and Ontology 29  Major Forms of Representing Knowledge  Semantic Networks A graph whose nodes represent concepts and whose edges represent relations between these concepts Example: Business Trips  Rules Reflect the notion of consequence in the form of IF-THEN- constructs  Logic Both forms, semantic networks as well as rules, have been formalized using logic to give them a precise semantics

30. Knowledge Representation and Ontology 30  Example of Sematic Network

31. Knowledge Representation and Ontology 31  Ontologies  Originally denotes the study of existence in philosophy  In information systems, ontologies are conceptual models of what “exists” in some domain  The structural frameworks for organizing information in the form of Semantic Network  Recent research in knowledge representation is mostly driven by the research of Ontologies (Semantic Network)

32. Knowledge Representation and Ontology 32  An “ontology” is a formal explicit specification of a shared conceptualization of a domain of interest  Formality  Explicitness  Being shared  Conceptuality  Domain Specificity

33. Knowledge Representation and Ontology 33  Example

34. Knowledge Representation and Ontology 34  Types of Ontologies

35. Knowledge Representation and Ontology 35  Applications of Ontologies  Information integration  Information retrieval  Knowledge management and community portals  Expert systems

36. Knowledge Representation and Ontology 36  Ontology Languages  Formal languages used to construct ontologies  Some Traditional ontology languages CycL used in Cyc Frame Logic  Some Popular ontology languages in recent research Web Ontology Language (OWL) Resource Description Framework (RDF) Semantic Web Rule Language (SWRL) Web Service Modeling Language (WSML)

37. Knowledge Representation and Ontology 37

38. Question? 38  Can we use ontologies or other forms of knowledge representation to store the knowledge for an agent?  If some of the knowledge is stored as ontologies, how can the agents teach each other?

39. Overview 39  Knowledge in Unified Learning Model (3-1)  Knowledge Representation and Ontologies (3-17)  Example: Knowledge in AI system Cyc (3-17)

40. Knowledge in Cyc 40  What is Cyc?  An artificial intelligent system that assemblies knowledge and reasons like human  The name is from "encyclopedia”  The goal is to break the "software brittleness bottleneck" once and for all by constructing a foundation of basic "common sense" knowledge

41. Knowledge in Cyc 41  Components in Cyc:  The Cyc Knowledge Base  The Cyc Inference Engine  The CycL Representation Language  The Natural Language Processing Subsystem  Cyc Semantic Integration Bus  Cyc Developer Toolsets

42. Knowledge in Cyc 42  A Brief Look at Knowledge in Cyc  http://cyc.com/cyc/technology/whatiscyc_dir/mapt est

43. Knowledge in Cyc 43  Foundations of Knowledge Representation  Logic Expressive for reasoning Simplified X -> Y Efficient Precise X is at the bank? Riverbank? Financial institution? Calculus of Meaning Not (All men are taller than all women) Not (All A are F than all B) CycL is the language implementation

44. Knowledge in Cyc 44  Foundations of Knowledge Representation  Constants denote specific individuals or collections #$Dog  Formulas is a relation applied to some arguments Sentence Formula with a Truth Function (x is true or false) Example: (#$isa #$Tom #$Person) Non-atomic Term Formula with a Function-Denotational Example: (#$GovernmentFn #$France)

45. Knowledge in Cyc 45  Foundations of Knowledge Representation  Arity – number of arguments Example:(#$arity #$performedBy 2) (#$performedBy #$Fishing #$Jack)  Argument Type (#$argIsa #$performedBy 1 #$Action) (#$argGenl #$penaltyForInfraction 2 #$Event)  Logical Connectives #$and #$or #$implies #$ not  Quantifiers #$forAll #$thereExists

46. Knowledge in Cyc 46  Foundations of Knowledge Representation  Collections and Individuals A collection is a kind or class Collections have instances Example: I am human An individual is a single thing Individuals may have parts Example: I have hands

47. Knowledge in Cyc 47  Foundations of Knowledge Representation

48. Knowledge in Cyc 48  Foundations of Knowledge Representation  Microtheory A set of assertions Assertion Γ ├ Σ Σ can be derived from Γ Assertions in a microtheory share common compoments (assumption, topic … ) The assertions within a microtheory must be mutually consistent Assertions in different microtheories may be inconsistent

49. Knowledge in Cyc 49  Foundations of Knowledge Representation  Advantages of Microtheory Focus on the development of the Cyc knowledge base Shorter and simpler assertions Cope with global inconsistency in the KB

50. Knowledge in Cyc 50  Foundations of Knowledge Representation  Microtheory Predicates #$ ist (#$ist MT FORMLA) (#$ist #$CyclistsMt (#$isa #$Lenat #$Person)) #$genlMt (#$genlMt MT-1 MT-2) means that every assertion which is true in MT-2 is also true in MT-1. (#$genlMt #$TransportationMt #$NaivePhysicsMt)

51. Knowledge in Cyc 51  Foundations of Knowledge Representation

52. Question? 52  What are the pros and cons of an agent with logic based knowledge?

53. References 53  Shell, D.F., Brooks, D.W., Trainin, G., Wilson, K.M., Kauffman, D.F., Herr, L.M. The Unified Learning Model. 2010 ISBN: 978-90-481-3214-0  http://www.opencyc.org http://www.opencyc.org  Studer,R., Grimm,S., Abecker,A. Semantic Web Services: Concepts, Technology and Applications, Springer, Berlin, 2007, pp. 51-106.