1. Lecture 21, Bayesian Networks Wrap Up Intro to Decision Theory Slide 1

2. Announcements Practice Exercises on Bnets– 6a, 6b and 6c –Reminder: they are helpful for staying on top of the material, and for studying for the exam 2

3. Lecture Overview Recap Bnes Wrap up: simplified structures Intro to Decision theory Utility and expected utility Single-stage decision problems Decision networks 3

4. XYZval ttt0.1 ttf0.9 tft0.2 tff0.8 ftt0.4 ftf0.6 fft0.3 fff0.7 Factors A factor f(X 1,…,X j ) is a function from a tuple of random variables X 1,…,X j to the real numbers R A factor denotes one or more (possibly partial) distributions over the given tuple of variables, e.g., P(X 1, X 2 ) is a factor f(X 1, X 2 ) P(Z | X,Y) is a factor f(Z,X,Y) P(Z=f|X,Y) is a factor f(X,Y) Note: Factors do not have to sum to one Distribution Set of Distributions One for each combination of values for X and Y Set of partial Distributions f(X, Y ) Z = f 4

5. Recap If we assign variable A=a in factor f (A,B), what is the correct form for the resulting factor? –f(B). When we assign variable A we remove it from the factor’s domain If we marginalize variable A out from factor f (A,B), what is the correct form for the resulting factor? –f(B). When we marginalize out variable A we remove it from the factor’s domain If we multiply factors f 4 (X,Y) and f 6 (Z,Y), what is the correct form for the resulting factor? –f(X,Y,Z) –When multiplying factors, the resulting factor’s domain is the union of the multiplicands’ domains 5

6. The variable elimination algorithm, 1.Construct a factor for each conditional probability. 2.For each factor, assign the observed variables E to their observed values. 3.Given an elimination ordering, decompose sum of products 4.Sum out all variables Z i not involved in the query (one a time) Multiply factors containing Z i Then marginalize out Z i from the product 5.Multiply the remaining factors (which only involve Y ) 6. Normalize by dividing the resulting factor f(Y) by To compute P(Y=y i | E 1 =e 1, …, E j =e j ) = The JPD of a Bayesian network is Given: P(Y, E 1 …, E j, Z 1 …,Z k ) observed Other variables not involved in the query

7. The variable elimination algorithm, 1.Construct a factor for each conditional probability. 2.For each factor, assign the observed variables E to their observed values. 3.Given an elimination ordering, decompose sum of products 4.Sum out all variables Z i not involved in the query (one a time) Multiply factors containing Z i Then marginalize out Z i from the product 5.Multiply the remaining factors (which only involve Y ) 6. Normalize by dividing the resulting factor f(Y) by To compute P(Y=y i | E 1 =e 1, …, E j =e j ) = The JPD of a Bayesian network is Given: P(Y, E 1 …, E j, Z 1 …,Z k ) observed Other variables not involved in the query

8. Step 1: Construct a factor for each cond. probability P(G,H) =  A,B,C,D,E,F,I P(A)P(B|A)P(C)P(D|B,C)P(E|C)P(F|D)P(G|F,E)P(H|G)P(I|G) P(G,H) =  A,B,C,D,E,F,I f 0 (A) f 1 (B,A) f 2 (C) f 3 (D,B,C) f 4 (E,C) f 5 (F, D) f 6 (G,F,E) f 7 (H,G) f 8 (I,G) Compute P(G | H=h 1 ). 8

9. Previous state: P(G,H) =  A,B,C,D,E,F,I f 0 (A) f 1 (B,A) f 2 (C) f 3 (D,B,C) f 4 (E,C) f 5 (F, D) f 6 (G,F,E) f 7 (H,G) f 8 (I,G) Observe H : Step 2: assign to observed variables their observed values. P(G,H=h 1 )=  A,B,C,D,E,F,I f 0 (A) f 1 (B,A) f 2 (C) f 3 (D,B,C) f 4 (E,C) f 5 (F, D) f 6 (G,F,E) f 9 (G) f 8 (I,G) Compute P(G | H=h 1 ). 9

10. Step 3: Decompose sum of products Previous state: P(G,H=h 1 ) =  A,B,C,D,E,F,I f 0 (A) f 1 (B,A) f 2 (C) f 3 (D,B,C) f 4 (E,C) f 5 (F, D) f 6 (G,F,E) f 9 (G) f 8 (I,G) Elimination ordering A, C, E, I, B, D, F : P(G,H=h 1 ) = f 9 (G)  F  D f 5 (F, D)  B  I f 8 (I,G)  E f 6 (G,F,E)  C f 2 (C) f 3 (D,B,C) f 4 (E,C)  A f 0 (A) f 1 (B,A) Compute P(G | H=h 1 ). 10

11. Step 4: sum out non query variables (one at a time) Previous state: P(G,H=h 1 ) = f 9 (G)  F  D f 5 (F, D)  B  I f 8 (I,G)  E f 6 (G,F,E)  C f 2 (C) f 3 (D,B,C) f 4 (E,C)  A f 0 (A) f 1 (B,A) Eliminate A: perform product and sum out A in P(G,H=h 1 ) = f 9 (G)  F  D f 5 (F, D)  B f 10 (B)  I f 8 (I,G)  E f 6 (G,F,E)  C f 2 (C) f 3 (D,B,C) f 4 (E,C) Elimination order: A,C,E,I,B,D,F Compute P(G | H=h 1 ). f 10 (B) does not depend on C, E, or I, so we can push it outside of those sums. 11

12. Step 4: sum out non query variables (one at a time) Previous state: P(G,H=h 1 ) = f 9 (G)  F  D f 5 (F, D)  B f 10 (B)  I f 8 (I,G)  E f 6 (G,F,E)  C f 2 (C) f 3 (D,B,C) f 4 (E,C) Eliminate C: perform product and sum out C in P(G,H=h 1 ) = f 9 (G)  F  D f 5 (F, D)  B f 10 (B)  I f 8 (I,G)  E f 6 (G,F,E) f 11 (B,D,E) Compute P(G | H=h 1 ). Elimination order: A,C,E,I,B,D,F

13. Step 4: sum out non query variables (one at a time) Previous state: P(G,H=h 1 ) = P(G,H=h 1 ) = f 9 (G)  F  D f 5 (F, D)  B f 10 (B)  I f 8 (I,G)  E f 6 (G,F,E) f 11 (B,D,E) Eliminate E: perform product and sum out E in P(G,H=h 1 ) = P(G,H=h 1 ) = f 9 (G)  F  D f 5 (F, D)  B f 10 (B) f 12 (B,D,F,G)  I f 8 (I,G) Compute P(G | H=h 1 ). Elimination order: A,C,E,I,B,D,F 13

14. Previous state: P(G,H=h 1 ) = P(G,H=h 1 ) = f 9 (G)  F  D f 5 (F, D)  B f 10 (B) f 12 (B,D,F,G)  I f 8 (I,G) Eliminate I: perform product and sum out I in P(G,H=h 1 ) = P(G,H=h 1 ) = f 9 (G) f 13 (G)  F  D f 5 (F, D)  B f 10 (B) f 12 (B,D,F,G) Elimination order: A,C,E,I,B,D,F Step 4: sum out non query variables (one at a time) Compute P(G | H=h 1 ). 14

15. Previous state: P(G,H=h 1 ) = P(G,H=h 1 ) = f 9 (G) f 13 (G)  F  D f 5 (F, D)  B f 10 (B) f 12 (B,D,F,G) Eliminate B: perform product and sum out B in P(G,H=h 1 ) = P(G,H=h 1 ) = f 9 (G) f 13 (G)  F  D f 5 (F, D) f 14 (D,F,G) Elimination order: A,C,E,I,B,D,F Step 4: sum out non query variables (one at a time) Compute P(G | H=h 1 ). 15

16. Previous state: P(G,H=h 1 ) = P(G,H=h 1 ) = f 9 (G) f 13 (G)  F  D f 5 (F, D) f 14 (D,F,G) Eliminate D: perform product and sum out D in P(G,H=h 1 ) = P(G,H=h 1 ) = f 9 (G) f 13 (G)  F f 15 (F,G) Elimination order: A,C,E,I,B,D,F Step 4: sum out non query variables (one at a time) Compute P(G | H=h 1 ). 16 Multiply remaining factors (all in G): P(G,H=h 1 ) = f 17 (G)

17. Previous state: P(G,H=h 1 ) = P(G,H=h 1 ) = f 9 (G) f 13 (G)  F f 15 (F,G) Eliminate F: perform product and sum out F in P(G,H=h 1 ) = f 9 (G) f 13 (G)f 16 (G) Elimination order: A,C,E,I,B,D,F Step 4: sum out non query variables (one at a time) Compute P(G | H=h 1 ). 17 Multiply remaining factors (all in G): P(G,H=h 1 ) = f 17 (G) Normalize

18. Variable elimination: pruning Slide 18  Thus, if the query is P(G=g| C=c 1, F=f 1, H=h 1 ) we only need to consider this subnetwork We can also prune unobserved leaf nodes Since they are unobserved and not predecessors of the query nodes, they cannot influence the posterior probability of the query nodes 18 Before running VE, we can prune all variables Z that are conditionally independent of the query Y given evidence E: Z ╨ Y | E –They cannot change the belief over Y given E!

19. VE in AISpace To see how variable elimination works in the Aispace Applet Select “Network options -> Query Models > verbose” Compare what happens when you select “Prune Irrelevant variables” or not in the VE window that pops up when you query a node Try different heuristics for elimination ordering 19

20. 20 Query P(A given L=F,S=T)

21. Complexity of Variable Elimination (VE) A factor over n binary variables has to store 2 n numbers –The initial factors are typically quite small (variables typically only have few parents in Bayesian networks) –But variable elimination constructs larger factors by multiplying factors together The complexity of VE is exponential in the maximum number of variables in any factor during its execution –This number is called the treewidth of a graph (along an ordering) –Elimination ordering influences treewidth Finding the best ordering is NP complete –I.e., the ordering that generates the minimum treewidth –Heuristics work well in practice (e.g. least connected variables first) –Even with best ordering, inference is sometimes infeasible In those cases, we need approximate inference. See CS422 & CS540 Simplified network structures can help 21

22. Lecture Overview Recap Bnes Wrap up: simplified structures Intro to Decision theory Utility and expected utility Single-stage decision problems Decision networks 22

23. Very Popular Application  Naïve Bayes models: very simple Bayesian networks for classification Class variable (to be predicted) is the root node Attribute variables X i (observations) are the leaves  Naïve because it assumes that the attributes are conditionally independent of each other given the class  Deterministic prediction can be obtained by picking the most likely class  Scales up really well: with n boolean attributes we just need……. C X1X1 XiXi X2X2

24. Very Popular Application  Naïve Bayes models: very simple Bayesian networks for classification Class variable (to be predicted) is the root node Attribute variables X i (observations) are the leaves  Naïve because it assumes that the attributes are conditionally independent of each other given the class  Deterministic prediction can be obtained by picking the most likely class  Scales up really well: with n boolean attributes we just need 2n+1 parameters C X1X1 XiXi X2X2

25. Example  Naïve Classifier agent that needs to predict whether to recommend a newsgroup post for reading to its user

26. Example  Naïve Classifier for agent that needs to predict the user’s emotion during interaction

27. Canonical Probability Distributions  CPT grows exponentially with number of parents  Possible solution: canonical distributions that are defined compactly  Example: Noisy-OR distribution Models multiple non-interacting causes Logic OR with a probabilistic twist. In Propositional logic, we can define the following rule: Fever is TRUE if and only if Malaria, Cold or Flue are true The Noisy-OR model allows for uncertainty in the ability of each cause to generate the effect (i.e. one may have a cold without a fever) Malaria Fever Cold  Two assumptions 1.All possible causes a listed 2.For each of the causes, whatever inhibits it to generate the target effect is independent from the inhibitors of the other causes Flu

28. Noisy-OR  Parent nodes U 1,…,U k include all causes but I can always add a “dummy” cause, or leak to cover for left-out causes  For each of the causes, whatever inhibits it to generate the target effect is independent from the inhibitors of the other causes Independent probability of failure q i for each cause alone: P(⌐Effect| u i ) = q i P(⌐Effect| u 1,.. u j, ⌐ u j+1,., ⌐ u k ) = ∏ j i=1 P(⌐Effect| u i ) = ∏ j i=1 q i P(Effect| u 1,.. u j, ⌐ u j+1,., ⌐ u k ) = 1 - ∏ j i=1 q i U1U1 Effect UkUk LEAK

29. Example  P(⌐fever| cold, ⌐ flu, ⌐ malaria ) = 0.6  P(⌐fever| ⌐ cold, flu, ⌐ malaria ) = 0.2  P(⌐fever| ⌐ cold, ⌐ flu, malaria ) = 0.1 Malaria

30. Example  Note that we did not have a Leak node in this example, for simplicity, but it would have been useful since Fever can definitely be caused by reasons other than the three we had  If we include it, how does the CPT change? ColdFluMalariaLeakP(Fever)P(⌐Fever) FFFF FFFT FFTF FFTT FTFF FTFT FTTF FTTT TFFF TFFT TFTF T ………… F ………… T ………… T …………

31. Hidden Variables  Variables that can’t or won’t be observed, but are added to ensure sparse structure and reduce parameters  Let’s consider a diagnostic problem (e.g. “why my car does not start?”) Possible causes (orange nodes below) of observations of interest (e.g., “car won’t start”) Other “observable nodes” that I can test to assess causes (green nodes below) “Hidden variables” (grey nodes)

32. More on hidden variables  Hidden variables that can‘t be observed may affect the direction of the dependencies that need to be represented in the network structure  Example: representing a domain for an intelligent system that acts as a tutor (aka Intelligent Tutoring System, or ITS) Topics divided in sub-topics Student knowledge of a topic depends on student knowledge of its sub- topics We can never observe student knowledge directly, we can only observe it indirectly via student test answers

33. Two Ways of Representing Knowledge Overall Proficiency Topic 1 Sub-topic 1.1 Answer 3 Answer 4 Sub-topic 1.2 Answer 2Answer 1 Answer 3 Answer 4 Answer 2Answer 1 Sub-topic 1.1Sub-topic 1.2 Overall Proficiency Topic 1 Which one should I pick?

34. Two Ways of Representing Knowledge Change in probability for a given node always propagates to its siblings, because we never get direct evidence on knowledge Change in probability for a given node does not propagate to its siblings, because we never get direct evidence on knowledge Overall Proficiency Topic 1 Sub-topic 1.1 Answer 3 Answer 4 Sub-topic 1.2 Answer 2Answer 1 Answer 3 Answer 4 Answer 2Answer 1 Sub-topic 1.1 Sub-topic 1.2 Overall Proficiency Topic 1 Answer 1 Which one you want to chose depends on the domain you want to represent

35. Build a Bayesian Network for a given domain Identify the necessary CPTs Compare different network structures Understand dependencies and independencies Variable elimination –Understating factors and their operations –Carry out variable elimination by using factors and the related operations –Use techniques to simplify variable elimination Explain what is a Noisy-Or configuration, the assumptions behind it, and why it is useful Explain what is a Naïve-Bayes configuration and the assumptions behind it, and why it is useful Learning Goals For Bnets 35

36. Bioinformatics Big picture: Reasoning Under Uncertainty Dynamic Bayesian Networks Hidden Markov Models & Filtering Probability Theory Bayesian Networks & Variable Elimination Natural Language Processing Email spam filters Motion Tracking, Missile Tracking, etc Monitoring (e.g. credit card fraud detection) Diagnostic systems (e.g. medicine, ITS) 36

37. Where are we? Environment Problem Type Query Planning Deterministic Stochastic Constraint Satisfaction Search Arc Consistency Search Logics STRIPS Vars + Constraints Variable Elimination Belief Nets Decision Nets Static Sequential Representation Reasoning Technique Variable Elimination This concludes the module on answering queries in stochastic environments

38. What’s Next? Environment Problem Type Query Planning Deterministic Stochastic Constraint Satisfaction Search Arc Consistency Search Logics STRIPS Vars + Constraints Variable Elimination Belief Nets Decision Nets Static Sequential Representation Reasoning Technique Variable Elimination Now we will look at acting in stochastic environments

39. Lecture Overview Recap Bnes Wrap up: simplified structures Intro to Decision theory Utility and expected utility Single-stage decision problems Decision networks 39

40. Decisions Under Uncertainty: Intro Earlier in the course, we focused on decision making in deterministic domains –Planning Now we face stochastic domains –so far we've considered how to represent and update beliefs –what if an agent has to make decisions (act) under uncertainty? Making decisions under uncertainty is important –We represent the world probabilistically so we can use our beliefs as the basis for making decisions 40

41. Decisions Under Uncertainty: Intro An agent's decision will depend on –What actions are available –What beliefs the agent has –Which goals the agent has Differences between deterministic and stochastic setting –Obvious difference in representation: need to represent our uncertain beliefs –Actions will be pretty straightforward: represented as decision variables –Goals will be interesting: we'll move from all-or-nothing goals to a richer notion: rating how happy the agent is in different situations. –Putting these together, we'll extend Bayesian Networks to make a new representation called Decision Networks 41

42. Delivery Robot Example Robot needs to reach a certain room Robot can go the short way - faster but with more obstacles, thus more prone to accidents that can damage the robot and prevent it from reaching the room the long way - slower but less prone to accident Which way to go? Is it more important for the robot to arrive fast, or to minimize the risk of damage? The Robot can choose to wear pads to protect itself in case of accident, or not to wear them. Pads make it heavier, increasing energy consumption Again, there is a tradeoff between reducing risk of damage, saving resources and arriving fast Possible outcomes No pad, no accident Pad, no accident Pad, Accident No pad, accident

43. Next We’ll see how to represent and reason about situations of this nature using Decision Trees, as well as Probability to measure the uncertainty in action outcome Utility to measure agent’s preferences over the various outcomes Combined in a measure of expected utility that can be used to identify the action with the best expected outcome Best that an intelligent agent can do when it needs to act in a stochastic environment

44. Decision Tree for the Delivery Robot Example Decision variable 1: the robot can choose to wear pads –Yes: protection against accidents, but extra weight –No: fast, but no protection Decision variable 2: the robot can choose the way –Short way: quick, but higher chance of accident –Long way: safe, but slow Random variable: is there an accident? Agent decides Chance decides 44

45. Delivery Robot Example Decision variable 1: the robot can choose to wear pads –Yes: protection against accidents, but extra weight –No: fast, but no protection Decision variable 2: the robot can choose the way –Short way: quick, but higher chance of accident –Long way: safe, but slow Random variable: is there an accident? Agent decides Chance decides 45

46. Possible worlds and decision variables A possible world specifies a value for each random variable and each decision variable For each assignment of values to all decision variables –the probabilities of the worlds satisfying that assignment sum to 1. 0.2 0.8 46

47. Possible worlds and decision variables 0.01 0.99 0.2 0.8 47 A possible world specifies a value for each random variable and each decision variable For each assignment of values to all decision variables –the probabilities of the worlds satisfying that assignment sum to 1.

48. Possible worlds and decision variables 0.01 0.99 0.2 0.8 0.2 0.8 48 A possible world specifies a value for each random variable and each decision variable For each assignment of values to all decision variables –the probabilities of the worlds satisfying that assignment sum to 1.

49. Possible worlds and decision variables 0.01 0.99 0.2 0.8 0.01 0.99 0.2 0.8 49 A possible world specifies a value for each random variable and each decision variable For each assignment of values to all decision variables –the probabilities of the worlds satisfying that assignment sum to 1.

50. Lecture Overview Recap Intro to Decision theory Utility and expected utility Single-stage decision problems Decision networks 50

51. Utility Utility: a measure of desirability of possible worlds to an agent –Let U be a real-valued function such that U(w) represents an agent's degree of preference for world w –Expressed by a number in [0,100] 51

52. Utility for the Robot Example Which would be a reasonable utility function for our robot? Which are the best and worst scenarios? 0.01 0.99 0.2 0.8 0.01 0.99 0.2 0.8 52 Utility probability

53. Utility for the Robot Example Which would be a reasonable utility function for our robot? 0.01 0.99 0.2 0.8 0.01 0.99 0.2 0.8 53 Utility probability 35 95 30 75 3 100 0 80

54. Utility: Simple Goals –How can the simple (boolean) goal “reach the room” be specified? Which way Accident Wear PadsUtility long true true long true false long false true long false false short true true short true false short false true short false false 0 100 90 B. A. C. D. Not possible Which way Accident Wear PadsUtility long true true long true false long false true long false false short true true short true false short false true short false false 0 100 0 Which way Accident Wear PadsUtility long true true long true false long false true long false false short true true short true false short false true short false false 0 100 0 100

55. Utility Utility: a measure of desirability of possible worlds to an agent –Let U be a real-valued function such that U(w) represents an agent's degree of preference for world w –Expressed by a number in [0,100] Simple goals can still be specified –Worlds that satisfy the goal have utility 100 –Other worlds have utility 0 55 Which way Accident Wear PadsUtility long true true long true false long false true long false false short true true short true false short false true short false false 0 100 0 100 e.g., goal “reach the room”

56. Optimal decisions: combining Utility and Probability Each set of decisions defines a probability distribution over possible outcomes Each outcome has a utility For each set of decisions, we need to know their expected utility –the value for the agent of achieving a certain probability distribution over outcomes (possible worlds) The expected utility of a set of decisions is obtained by weighting the utility of the relevant possible worlds by their probability. 35 95 0.2 0.8 We want to find the decision with maximum expected utility value of this scenario?

57. Expected utility of a decision 0.01 0.99 0.2 0.8 0.01 0.99 0.2 0.8 57 Utility 35 95 probability E[U|D] 3530 75 353 100 350 80 The expected utility of decision D = d i is What is the expected utility of Wearpads=yes, Way=short ? E (U | D = d i ) =  w╞ (D = d i ) P( w ) U( w ) A. 7 B. 83 C. 76 D. 157.55

58. Expected utility of a decision 0.01 0.99 0.2 0.8 0.01 0.99 0.2 0.8 58 Utility 35 95 probability E[U|D] 3530 75 353 100 350 80 The expected utility of decision D = d i is What is the expected utility of Wearpads=yes, Way=short ? E (U | D = d i ) =  w╞ (D = d i ) P( w ) U( w ) 0.2 * 35 + 0.8 * 95 = 83

59. Expected utility of a decision 0.01 0.99 0.2 0.8 0.01 0.99 0.2 0.8 59 Utility 35 95 probability E[U|D] 83 3530 75 353 100 350 80 74.55 80.6 79.2 The expected utility of decision D = d i is E (U | D = d i ) =  w╞ (D = d i ) P( w ) U( w )

60. Lecture Overview Recap Intro to Decision theory Utility and expected utility Single-stage decision problems – NEXT TIME Decision networks 60