1. © 2002 Franz J. Kurfess Introduction 1 CPE/CSC 481: Knowledge-Based Systems Dr. Franz J. Kurfess Computer Science Department Cal Poly

2. © 2002 Franz J. Kurfess Introduction 2 Course Overview u Introduction u Knowledge Representation u Semantic Nets, Frames, Logic u Reasoning and Inference u Predicate Logic, Inference Methods, Resolution u Reasoning with Uncertainty u Probability, Bayesian Decision Making u Expert System Design u ES Life Cycle u CLIPS Overview u Concepts, Notation, Usage u Pattern Matching u Variables, Functions, Expressions, Constraints u Expert System Implementation u Salience, Rete Algorithm u Expert System Examples u Conclusions and Outlook

3. © 2002 Franz J. Kurfess Introduction 3 Overview Reasoning and Uncertainty u Motivation u Objectives u Sources of Uncertainty and Inexactness in Reasoning u Incorrect and Incomplete Knowledge u Ambiguities u Belief and Disbelief u Probability Theory u Bayesian Networks u Dempster-Shafer Theory u Certainty Factors u \Approximate Reasoning u Fuzzy Logic u Important Concepts and Terms u Chapter Summary

4. © 2002 Franz J. Kurfess Introduction 4 Logistics u Introductions u Course Materials u textbooks (see below) u lecture notes u PowerPoint Slides will be available on my Web page u handouts u Web page u http://www.csc.calpoly.edu/~fkurfess http://www.csc.calpoly.edu/~fkurfess u Term Project u Lab and Homework Assignments u Exams u Grading

5. © 2002 Franz J. Kurfess Introduction 5 Bridge-In

6. © 2002 Franz J. Kurfess Introduction 6 Pre-Test

7. © 2002 Franz J. Kurfess Introduction 7 Motivation

8. © 2002 Franz J. Kurfess Introduction 8 Objectives

9. © 2002 Franz J. Kurfess Introduction 9 Evaluation Criteria

10. © 2002 Franz J. Kurfess Introduction 10 Introductions  reasoning under uncertainty and with inexact knowledge  heuristics  ways to mimic heuristic knowledge processing  methods used by experts  empirical associations  experiential reasoning  based on limited observations  probabilities  objective (frequency counting)  subjective (human experience )  reproducibility  will observations deliver the same results when repeated

11. © 2002 Franz J. Kurfess Introduction 11 Dealing with Uncertainty  expressiveness  can concepts used by humans be represented adequately?  can the confidence of experts in their decisions be expressed?  comprehensibility  representation of uncertainty  utilization in reasoning methods  correctness  probabilities  relevance ranking  long inference chains  computational complexity  feasibility of calculations for practical purposes

12. © 2002 Franz J. Kurfess Introduction 12 Sources of Uncertainty  data  missing data, unreliable, ambiguous, imprecise representation, inconsistent, subjective, derived from defaults, …  expert knowledge  inconsistency between different experts  plausibility  “best guess” of experts  quality  causal knowledge  deep understanding  statistical associations  observations  scope  only current domain?

13. © 2002 Franz J. Kurfess Introduction 13 Sources of Uncertainty (cont.)  knowledge representation  restricted model of the real system  limited expressiveness of the representation mechanism  inference process  deductive  the derived result is formally correct, but wrong in the real system  inductive  new conclusions are not well-founded  unsound reasoning methods

14. © 2002 Franz J. Kurfess Introduction 14 Uncertainty in Individual Rules  individual rules  errors  domain errors  representation errors  inappropriate application of the rules  likelihood of evidence  for each premise  for the conclusion  combination of evidence from multiple premises

15. © 2002 Franz J. Kurfess Introduction 15 Uncertainty and Multiple Rules  conflict resolution  if multiple rules are applicable, which one is selected  explicit priorities, provided by domain experts  implicit priorities derived from rule properties »specificity of patterns, ordering of patterns creation time of rules, most recent usage, …  compatibility  contradictions between rules  subsumption  one rule is a more general version of another one  redundancy  missing rules  data fusion  integration of data from multiple sources

16. © 2002 Franz J. Kurfess Introduction 16 Basics of Probability Theory  mathematical approach for processing uncertain information  sample space set X = {x1, x2, …, xn}  collection of all possible events  can be discrete or continuous  probability number P(xi) likelihood of an event xi to occur  non-negative value in [0,1]  total probability of the sample space is 1  for mutually exclusive events, the probability for at least one of them is the sum of their individual probabilities  experimental probability  based on the frequency of events  subjective probability  based on expert assessment

17. © 2002 Franz J. Kurfess Introduction 17 Compound Probabilities  describes independent events  do not affect each other in any way  joint probability of two independent events A and B P(A  B) = n(A  B) / n(s) = P(A) * P (B) where n(S) is the number of elements in S  union probability of two independent events A and B P(A  B) = P(A) + P(B) - P(A  B) =P(A) + P(B) - P(A) * P (B)  where n(S) is the number of elements in S

18. © 2002 Franz J. Kurfess Introduction 18 Conditional Probabilities  describes dependent events  affect each other in some way  conditional probability of event a given that event B has already occurred P(A|B) = P(A  B) / P(B)

19. © 2002 Franz J. Kurfess Introduction 19 Advantages and Problems of Probabilities  advantages  formal foundation  reflection of reality (a posteriori)  problems  may be inappropriate  the future is not always similar to the past  inexact or incorrect  especially for subjective probabilities  knowledge may be represented implicitly

20. © 2002 Franz J. Kurfess Introduction 20 Bayesian Approaches  derive the probability of a cause given a symptom  has gained importance recently due to advances in efficiency  more computational power available  better methods  especially useful in diagnostic systems  medicine, computer help systems  inverse or a posteriori probability  inverse to conditional probability of an earlier event given that a later one occurred

21. © 2002 Franz J. Kurfess Introduction 21 Bayes’ Rule for Single Event  single hypothesis H, single event E P(H|E) = (P(E|H) * P(H)) / P(E) or  P(H|E) = (P(E|H) * P(H) / (P(E|H) * P(H) + P(E|  H) * P(  H) )

22. © 2002 Franz J. Kurfess Introduction 22 Bayes’ Rule for Multiple Events  multiple hypotheses Hi, multiple events E1, …, Ei, …, En P(Hi|E1, E2, …, En) = (P(E1, E2, …, En|Hi) * P(Hi)) / P(E1, E2, …, En) or  P(Hi|E1, E2, …, En) = (P(E1|Hi) * P(E2|Hi) * …* P(En|Hi) * P(Hi)) /  k P(E1|Hk) * P(E2|Hk) * … * P(En|Hk) * P(Hk) with independent pieces of evidence Ei

23. © 2002 Franz J. Kurfess Introduction 23 Advantages and Problems of Bayesian Reasoning  advantages  sound theoretical foundation  well-defined semantics for decision making  problems  requires large amounts of probability data  sufficient sample sizes  subjective evidence may not be reliable  independence of evidences assumption often not valid  relationship between hypothesis and evidence is reduced to a number  explanations for the user difficult  high computational overhead

24. © 2002 Franz J. Kurfess Introduction 24 Dempster-Shafer Theory  mathematical theory of evidence  notations  frame of discernment FD  power set of the set of possible conclusions  mass probability function m  assigns a value from [0,1] to every item in the frame of discernment  mass probability m(A)  portion of the total mass probability that is assigned to an element A of FD

25. © 2002 Franz J. Kurfess Introduction 25 Belief and Certainty  belief Bel(A) in a subset A  sum of the mass probabilities of all the proper subsets of A  likelihood that one of its members is the conclusion  plausibility Pl(A)  maximum belief of A  certainty Cer(A)  interval [Bel(A), Pl(A)]  expresses the range of belief

26. © 2002 Franz J. Kurfess Introduction 26 Combination of Mass Probabilities  m1  m2 (C) =  X  Y=C m1(X) * m2(Y) / 1-  X  Y=C m1(X) * m2(Y) where X, Y are hypothesis subsets and C is their intersection

27. © 2002 Franz J. Kurfess Introduction 27 Advantages and Problems of Dempster-Shafer  advantages  clear, rigorous foundation  ability oto express confidence through intervals  certainty about certainty  problems  non-intuitive determination of mass probability  very high computational overhead  may produce counterintuitive results due to normalization  usability somewhat unclear

28. © 2002 Franz J. Kurfess Introduction 28 Certainty Factors  shares some foundations with Dempster-Shafer theory, but more practical  denotes the belief in a hypothesis H given that some pieces of evidence are observed  no statements about the belief is no evidence is present  in contrast to Bayes’ method

29. © 2002 Franz J. Kurfess Introduction 29 Belief and Disbelief  measure of belief  degree to which hypothesis H is supported by evidence E  MB(H,E) = 1 IF P(H) =1 (P(H|E) - P(H)) / (1- P(H)) otherwise  measure of disbelief  degree to which doubt in hypothesis H is supported by evidence E  MB(H,E) = 1 IF P(H) =0 (P(H) - P(H|E)) / P(H)) otherwise

30. © 2002 Franz J. Kurfess Introduction 30 Certainty Factor  certainty factor CF  ranges between -1 (denial of the hypothesis H) and 1 (confirmation of H)  CF = (MB - MD) / (1 - min (MD, MB))  combining antecedent evidence  use of premises with less than absolute confidence  E1  E2 = min(CF(H, E1), CF(H, E2))  E1  E2 = max(CF(H, E1), CF(H, E2))   E =  CF(H, E)

31. © 2002 Franz J. Kurfess Introduction 31 Combining Certainty Factors  certainty factors that support the same conclusion  several rules can lead to the same conclusion  applied incrementally as new evidence becomes available  Cfrev(CFold, CFnew) =  CFold + CFnew(1 - CFold) if both > 0  CFold + CFnew(1 + CFold) if both < 0  CFold + CFnew / (1 - min(|CFold|, |CFnew|)) if one < 0

32. © 2002 Franz J. Kurfess Introduction 32 Advantages and Problems of Certainty Factors  Advantages  simple implementation  reasonable modeling of human experts’ belief  expression of belief and disbelief  successful applications for certain problem classes  evidence relatively easy to gather  no statistical base required  Problems  partially ad hoc approach  theoretical foundation through Dempster-Shafer theory was developed later  combination of non-independent evidence unsatisfactory  new knowledge may require changes in the certainty factors of existing knowledge  certainty factors can become the opposite of conditional probabilities for certain cases  not suitable for long inference chains

33. © 2002 Franz J. Kurfess Introduction 33 Fuzzy Logic  approach to a formal treatment of uncertainty  relies on quantifying and reasoning through natural language  uses linguistic variables to describe concepts with vague values  tall, large, small, heavy,...

34. © 2002 Franz J. Kurfess Introduction 34 Get Fuzzy

35. © 2002 Franz J. Kurfess Introduction 35 Fuzzy Set  categorization of elements xi into a set S  described through a membership function m(s)  associates each element xi with a degree of membership in S  possibility measure Poss{x  S}  degree to which an individual element x is a potential member in the fuzzy set S  possibility refers to allowed values  probability expresses expected occurrences of events  combination of multiple premises  Poss(A  B) = min(Poss(A),Poss(B))  Poss(A  B) = max(Poss(A),Poss(B))

36. © 2002 Franz J. Kurfess Introduction 36 Fuzzy Set Example membership height (cm) 0 0 50100150200250 0.5 1 shortmedium tall

37. © 2002 Franz J. Kurfess Introduction 37 Fuzzy vs. Crisp Set membership height (cm) 0 0 50100150200250 0.5 1 shortmedium tall

38. © 2002 Franz J. Kurfess Introduction 38 Fuzzy Inference Methods  how to combine evidence across rules  Poss(B|A) = min(1, (1 - Poss(A)+ Poss(B)))  implication according to Max-Min inference  also Max-Product inference and other rules  formal foundation through Lukasiewicz logic  extension of binary logic to infinite-valued logic

39. © 2002 Franz J. Kurfess Introduction 39 Example Fuzzy Reasoning

40. © 2002 Franz J. Kurfess Introduction 40 Advantages and Problems of Fuzzy Logic  advantages  general theory of uncertainty  wide applicability, many practical applications  natural use of vague and imprecise concepts  helpful for commonsense reasoning, explanation  problems  membership functions can be difficult to find  multiple ways for combining evidence  problems with long inference chains

41. © 2002 Franz J. Kurfess Introduction 41 Post-Test

42. © 2002 Franz J. Kurfess Introduction 42 Evaluation u Criteria

43. © 2002 Franz J. Kurfess Introduction 43 Use of References  [Giarratano & Riley 1998] [Giarratano & Riley 1998]  [Russell & Norvig 1995] [Russell & Norvig 1995]  [Jackson 1999] [Jackson 1999]  [Durkin 1994] [Durkin 1994] [Giarratano & Riley 1998]

44. © 2002 Franz J. Kurfess Introduction 44 Important Concepts and Terms  natural language processing  neural network  predicate logic  propositional logic  rational agent  rationality  Turing test  agent  automated reasoning  belief network  cognitive science  computer science  hidden Markov model  intelligence  knowledge representation  linguistics  Lisp  logic  machine learning  microworlds

45. © 2002 Franz J. Kurfess Introduction 45 Summary Chapter-Topic

46. © 2002 Franz J. Kurfess Introduction 46