1. cse@buffalo S.C. Shapiro Knowledge Representation and Reasoning Stuart C. Shapiro Professor, CSE Director, SNePS Research Group Member, Center for Cognitive Science Fellow, AAAI Chair, ACM/SIGART, 1991-1995 President, KR., Inc., 1998-2000

2. cse@buffalo S.C. Shapiro Introduction

3. cse@buffalo S.C. Shapiro Long-Term Goal Theory and Implementation of Natural-Language-Competent Computerized Cognitive Agent and Supporting Research in Artificial Intelligence Cognitive Science Computational Linguistics.

4. cse@buffalo S.C. Shapiro Research Areas Knowledge Representation and Reasoning Cognitive Robotics Natural-Language Understanding Natural-Language Generation.

5. cse@buffalo S.C. Shapiro Goal A computational cognitive agent that can: –Understand and communicate in English; –Discuss specific, generic, and “rule-like” information; –Reason; –Discuss acts and plans; –Sense; –Act; –Remember and report what it has sensed and done.

6. cse@buffalo S.C. Shapiro Cassie A computational cognitive agent –Embodied in hardware –or Software-Simulated –Based on SNePS and GLAIR.

7. cse@buffalo S.C. Shapiro GLAIR Architecture Knowledge Level Perceptuo-Motor Level Sensory-Actuator Level NL Vision Sonar Motion Proprioception Grounded Layered Architecture with Integrated Reasoning SNePS

8. cse@buffalo S.C. Shapiro SNePS Knowledge Representation and Reasoning –Propositions as Terms SNIP: SNePS Inference Package –Specialized connectives and quantifiers SNeBR: SNePS Belief Revision SNeRE: SNePS Rational Engine Interface Languages –SNePSUL: Lisp-Like –SNePSLOG: Logic-Like –GATN for Fragments of English.

9. cse@buffalo S.C. Shapiro Interaction with Cassie English (Statement, Question, Command) (Current) Set of Beliefs [SNePS] (Updated) Set of Beliefs [SNePS] Actions [SNeRE] (New Belief) [SNePS] English sentence expressing new belief answering question reporting actions Answer [SNIP] GATN Parser GATN Generator Reasoning Clarification Dialogue Looking in World Reasoning

10. cse@buffalo S.C. Shapiro Example Cassies & Worlds

11. cse@buffalo S.C. Shapiro Cassie, the BlocksWorld Robot

12. cse@buffalo S.C. Shapiro FEVAHR: Award-Winning Embodied Cassie Project

13. cse@buffalo S.C. Shapiro FEVAHRWorld Simulation

14. cse@buffalo S.C. Shapiro UXO Remediation Cassie Cassie Corner flag NonUXO object Corner flag UXO Battery meter Corner flag Drop-off zone Field Safe zone Recharging Station

15. cse@buffalo S.C. Shapiro Crystal Space Environment

16. cse@buffalo S.C. Shapiro UB Virtual Site Museum The 9 th -Century BC Northwest Palace at Nimrud-Iraq is the best preserved and documented of all the Assyrian palaces. Its audience halls were originally created as the backdrop for differing royal activities. Completely immersive re-creation of this palace with animated characters and interactive story boards. T. Kesavadas & S. Paley Modeling of King - Animation in Real time VR

17. cse@buffalo S.C. Shapiro Sample Research Issues Intensional Entities

18. cse@buffalo S.C. Shapiro Intensional Entities 1 Rather than represent “objects in the world,” represent mental entities. Includes Imaginary and Fictional Entities. Multiple mental entities may correspond to one world object. –Intensional entities may be co-extensional. –But must be kept separate.

19. cse@buffalo S.C. Shapiro Intensional Entities 2 : The morning star is the evening star. I understand that the morning star is the evening star. : The evening star is Venus. I understand that Venus is the evening star. : Clark Kent is Superman. I understand that Superman is Clark Kent.

20. cse@buffalo S.C. Shapiro Intensional Entities 3 : Lois Lane saw Clark Kent. I understand that Lois Lane saw Clark Kent. : Did Lois Lane see Superman? I don't know. : Did Lois Lane see Clark Kent? Yes, Lois Lane saw Clark Kent. Note Open World Assumption.

21. cse@buffalo S.C. Shapiro Intensional Entities 4 : Superman went to the morning star. I understand that Superman went to Venus. : Did Clark Kent go to Venus? Yes, Superman went to Venus.

22. cse@buffalo S.C. Shapiro Intensional Entities 5 : Buck Rogers went to the evening star. I understand that Buck Rogers went to Venus. : Who went to Venus? Buck Rogers went to Venus and Superman went to Venus.

23. cse@buffalo S.C. Shapiro Intensional Entities 6 The morning star The evening star Venus Superman Clark Kent Buck Rogers Lois Lane Go to See

24. cse@buffalo S.C. Shapiro Sample Research Issues Complex Categories

25. cse@buffalo S.C. Shapiro Complex Categories 1 Noun Phrases: {N | Adj}* N Understanding of the modification must be left to reasoning. Example: orange juice seat Representation must be left vague.

26. cse@buffalo S.C. Shapiro : Kevin went to the orange juice seat. I understand that Kevin went to the orange juice seat. : Did Kevin go to a seat? Yes, Kevin went to the orange juice seat. Complex Categories 2

27. cse@buffalo S.C. Shapiro : Pat is an excellent teacher. I understand that Pat is an excellent teacher. : Is Pat a teacher? Yes, Pat is a teacher. : Lucy is a former teacher. I understand that Lucy is a former teacher. Complex Categories 3

28. cse@buffalo S.C. Shapiro : `former' is a negative adjective. I understand that `former' is a negative adjective. : Is Lucy a teacher? No, Lucy is not a teacher. Complex Categories 4

29. cse@buffalo S.C. Shapiro PseudoRepresentation of Complex Categories Isa(B30, CompCat(orange, CompCat(juice, seat))) Isa(Pat, CompCat(excellent, teacher)) Isa(Lucy, CompCat(former, teacher))

30. cse@buffalo S.C. Shapiro Sample Research Issues Possession

31. cse@buffalo S.C. Shapiro Possession 1 “One man’s meat is another man’s poison.”

32. cse@buffalo S.C. Shapiro : Richard's meat is Henry's poison. I understand that Henry's poison is Richard's meat. : Edward ate Richard's meat. I understand that Edward ate Richard's meat. : Did Edward eat Henry's poison? Yes, Edward ate Henry's poison. Possession 2

33. cse@buffalo S.C. Shapiro : Did Edward eat Henry’s meat? I don’t know. : Did Edward eat Richard's poison? I don’t know. Possession 3 Moral: Possession is a three-place relation.

34. cse@buffalo S.C. Shapiro PseudoRepresentation of Possession Has(Richard, meat, B35) Has(Henry, poison, B37) Equiv(B35, B37)

35. cse@buffalo S.C. Shapiro Sample Research Issues Propositions about Propositions

36. cse@buffalo S.C. Shapiro Propositions about Propositions 1 Propositions are “first-class” mental entities. They can be discussed, just like other mental entities. And must be represented like other mental entities.

37. cse@buffalo S.C. Shapiro : That Bill is sweet is Mary's favorite proposition. I understand that Mary's favorite proposition is that Bill is sweet. : Mike believes Mary's favorite proposition. I understand that Mike believes that Bill is sweet. Propositions about Propositions 2

38. cse@buffalo S.C. Shapiro : That Mary's favorite proposition is that Bill is sweet is cute. I understand that that Mary's favorite proposition is that Bill is sweet is cute. Propositions about Propositions 3

39. cse@buffalo S.C. Shapiro Representing Propositions Representation of Proposition –Not by a Logical Sentence –But by a Functional Term –Denoting a Proposition.

40. cse@buffalo S.C. Shapiro PseudoRepresentation of Propositions about Propositions Has(Mary, CompCat(favorite, proposition), HasProp(Bill, sweet)) Believes(Mike, HasProp(Bill, sweet)) HasProp(Has(Mary, CompCat(favorite, proposition), HasProp(Bill, sweet)), cute)

41. cse@buffalo S.C. Shapiro Sample Research Issues Conditional Plans

42. cse@buffalo S.C. Shapiro Conditional Plans If a block is on a support then a plan to achieve that the support is clear is to pick up the block and then put the block on the table. all(x, y) ({Block(x), Support(y), On(x, y)} &=> {GoalPlan(Clear(y), Snsequence(Pickup(x), Put(x, Table)))}) STRIPS-like representation: No times

43. cse@buffalo S.C. Shapiro Use of Conditional Plan GoalPlan(Clear(B), Snsequence(Pickup(A), Put(A, Table))) Remember (cache) derived propositions.

44. cse@buffalo S.C. Shapiro Use of Conditional Plan GoalPlan(Clear(B), Snsequence(Pickup(A), Put(A, Table))) ??? SNeBR to the rescue!

45. cse@buffalo S.C. Shapiro Sample Research Issues Indexicals

46. cse@buffalo S.C. Shapiro Representation and Use of Indexicals Words whose meanings are determined by occasion of use E.g. I, you, now, then, here, there Deictic Center *I : SNePS term representing Cassie *YOU: person Cassie is talking with *NOW: current time.

47. cse@buffalo S.C. Shapiro Analysis of Indexicals (in input) First person pronouns: *YOU Second person pronouns: *I “here”: location of *YOU Present/Past relative to *NOW.

48. cse@buffalo S.C. Shapiro Generation of Indexicals *I : First person pronouns *YOU : Second person pronouns *NOW : used to determine tense and aspect.

49. cse@buffalo S.C. Shapiro Come here. Use of Indexicals 1

50. cse@buffalo S.C. Shapiro Come here. I came to you, Stu. I am near you. Use of Indexicals 2

51. cse@buffalo S.C. Shapiro Who am I? Your name is ‘Stu’ and you are a person. Who have you talked to? I am talking to you. Talk to Bill. I am talking to you, Bill. Come here. Use of Indexicals 3

52. cse@buffalo S.C. Shapiro Come here. I found you. I am looking at you. Use of Indexicals 4

53. cse@buffalo S.C. Shapiro Come here. I came to you. I am near you. I found you. I am looking at you. Use of Indexicals 5

54. cse@buffalo S.C. Shapiro Who am I? I talked to Stu and I am talking to you. Your name is ‘Bill’ and you are a person. Who are you? I am the FEVAHR and my name is ‘Cassie’. Who have you talked to? Use of Indexicals 6

55. cse@buffalo S.C. Shapiro Sample Research Issues Time

56. cse@buffalo S.C. Shapiro Motivating Joke 9:30:00 AM (Door-to-Door Salesman): May I interest you in a brush? 9:30:02 AM (Homeowner): Not now. 9:30:03 AM (Salesman): Now?

57. cse@buffalo S.C. Shapiro A Personal Sense of Time *NOW contains SNePS term representing current time. *NOW moves when Cassie acts or perceives a change of state.

58. cse@buffalo S.C. Shapiro The Pacemaker PML process periodically increments variable COUNT. *COUNT = some PML integer. Reset to 0 when NOW moves. Provides bodily “feel” of passing time.

59. cse@buffalo S.C. Shapiro Quantizing Time Cannot conceptualize fine distinctions in time intervals. So quantize, e.g. into half orders of magnitude (Hobbs, 2000).

60. cse@buffalo S.C. Shapiro Movement of Time with Pacemaker NOW COUNTn hom 0 KL PML t1 t2 q ! beforeafter time duration !

61. cse@buffalo S.C. Shapiro The Vagueness of “now” I’m now giving a talk. I’m now on sabbatical. I’m now living in East Amherst. I’m now at UB. Multiple now’s at different granularities.

62. cse@buffalo S.C. Shapiro NOW-MTF NOW Semi-lattice of times, all of which contain *NOW, any of which could be meant by “now” Finite---only conceptualized times of conceptualized states Maximal Temporal Frame based on *NOW

63. cse@buffalo S.C. Shapiro Moving NOW with MTF NOW

64. cse@buffalo S.C. Shapiro Current

65. cse@buffalo S.C. Shapiro Current Students Bharat Bhushan, M.S. Candidate Preferential Ordering of Beliefs for Default Reasoning Debra T. Burhans, Ph.D. Candidate A Question-Answering Interpretation of Resolution Refutation Frances L. Johnson, Ph.D. Candidate Belief Revision in a Deductively Open Belief Space John F. Santore, Ph.D. Candidate Distinguishing Perceptually Indistinguishable Objects

66. cse@buffalo S.C. Shapiro For More Information URL: http://www.cse.buffalo.edu/~shapiro/http://www.cse.buffalo.edu/~shapiro/ Group: http://www.cse.buffalo.edu/sneps/http://www.cse.buffalo.edu/sneps/