2. CPSC 322 Introduction to Artificial Intelligence December 3, 2004

3. Things... Slides for the last couple of weeks will be up this weekend. Did I mention that you have a final exam at noon on Friday, December 10, in MCML 166? Check the announcements part of the web page occasionally between now and then in case anything important comes up.

4. This should be fun Artificial Intelligence and Interactive Digital Entertainment First Annual Conference June 1-3, 2005 in Marina Del Rey, California www.aiide.org

5. Learning Definition: learning is the adaptive changes that occur in a system which enable that system to perform the same task or similar tasks more efficiently or more effectively over time. This could mean: The range of behaviors is expanded: the agent can do more The accuracy on tasks is improved: the agent can do things better The speed is improved: the agent can do things faster

6. Learning is about choosing the best representation That’s certainly true in a logic-based AI world. Our arch learner started with some internal representation of an arch. As examples were presented, the arch learner modified its internal representation to either make the representation accommodate positive examples (generalization) or exclude negative examples (specialization). There’s really nothing else the learner could modify... the reasoning system is what it is. So any learning problem can be mapped onto one of choosing the best representation...

7. Learning is about search...but wait, there’s more! By now, you’ve figured out that the arch learner was doing nothing more than searching the space of possible representations, right? So learning, like everything else, boils down to search. If that wasn’t obvious, you probably will want to do a little extra preparation for the final exam....

8. Same problem - different representation The arch learner could have represented the arch concept as a decision tree if we wanted

9. Same problem - different representation The arch learner could have represented the arch concept as a decision tree if we wanted arch

10. Same problem - different representation The arch learner could have represented the arch concept as a decision tree if we wanted do upright blocks support sideways block? no yes not arch arch

11. Same problem - different representation The arch learner could have represented the arch concept as a decision tree if we wanted do upright blocks support sideways block? no yes not arch do upright blocks touch each other? arch not arch no yes

12. Same problem - different representation The arch learner could have represented the arch concept as a decision tree if we wanted do upright blocks support sideways block? no yes not arch do upright blocks touch each other? is the not arch top block either a rectangle or a wedge? no yes not arch arch

13. Other issues with learning by example The learning process requires that there is someone to say which examples are positive and which are negative. This approach must start with a positive example to specialize or generalize from. Learning by example is sensitive to the order in which examples are presented. Learning by example doesn’t work well with noisy, randomly erroneous data.

14. Reinforcement learning Learning operator sequences based on reward or punishment Let’s say you want your robot to learn how to vacuum the living room iRobot Roomba 4210 Discovery Floorvac Robotic Vacuum $249.99 It’s the ideal Christmas gift!

15. Reinforcement learning Learning operator sequences based on reward or punishment Let’s say you want your robot to learn how to vacuum the living room goto livingroom vacuum floor goto trashcan empty bag Good Robot!

16. Reinforcement learning Learning operator sequences based on reward or punishment Let’s say you want your robot to learn how to vacuum the living room goto livingroomgoto trashcanvacuum floor goto trashcangoto livingroomempty bag Good Robot! Bad Robot!

17. Reinforcement learning Learning operator sequences based on reward or punishment Let’s say you want your robot to learn how to vacuum the living room goto livingroomgoto trashcanvacuum floor goto trashcangoto livingroomempty bag Good Robot! Bad Robot! (actually, there are no bad robots, there is only bad behavior...and Roomba can’t really empty its own bag)

18. Reinforcement learning Should the robot learn from success? How does it figure out which part of the sequence of actions is right (credit assignment problem)? goto livingroomgoto trashcanvacuum floor goto trashcangoto livingroomempty bag Good Robot! Bad Robot!

19. Reinforcement learning Should the robot learn from failure? How does it figure out which part of the sequence of actions is wrong (blame assignment problem)? goto livingroomgoto trashcanvacuum floor goto trashcangoto livingroomempty bag Good Robot! Bad Robot!

20. Reinforcement learning However you answer those questions, the learning task again boils down to the search for the best representation. Here’s another type of learning that searches for the best representation, but the representation is very different from what you’ve seen so far.

21. Learning in neural networks The perceptron is one of the earliest neural network models, dating to the early 1960s. x1x1 x2x2 x3x3 xnxn......  w1w1 w2w2 w3w3 wnwn inputs weights sum threshold

22. Learning in neural networks The perceptron can’t compute everything, but what it can compute it can learn to compute. Here’s how it works. Inputs are 1 or 0. Weights are reals (-n to +n). Each input is multiplied by its corresponding weight. If the sum of the products is greater than the threshold, then the perceptron outputs 1, otherwise the output is 0. x1x1 x2x2 x3x3 xnxn......  w1w1 w2w2 w3w3 wnwn inputs weights sum threshold

23. Learning in neural networks The perceptron can’t compute everything, but what it can compute it can learn to compute. Here’s how it works. The output, 1 or 0, is a guess or prediction about the input: does it fall into the desired classification (output = 1) or not (output = 0)? x1x1 x2x2 x3x3 xnxn......  w1w1 w2w2 w3w3 wnwn inputs weights sum threshold

24. Learning in neural networks That’s it? Big deal. No, there’s more to it.... x1x1 x2x2 x3x3 xnxn......  w1w1 w2w2 w3w3 wnwn inputs weights sum threshold

25. Learning in neural networks That’s it? Big deal. No, there’s more to it.... Say you wanted your perceptron to classify arches. x1x1 x2x2 x3x3 xnxn......  w1w1 w2w2 w3w3 wnwn inputs weights sum threshold

26. Learning in neural networks That’s it? Big deal. No, there’s more to it.... Say you wanted your perceptron to classify arches. That is, you present inputs representing an arch, and the output should be 1. x1x1 x2x2 x3x3 xnxn......  w1w1 w2w2 w3w3 wnwn inputs weights sum threshold

27. Learning in neural networks That’s it? Big deal. No, there’s more to it.... Say you wanted your perceptron to classify arches. That is, you present inputs representing an arch, and the output should be 1. You present inputs not representing an arch, and the output should be 0. x1x1 x2x2 x3x3 xnxn......  w1w1 w2w2 w3w3 wnwn inputs weights sum threshold

28. Learning in neural networks That’s it? Big deal. No, there’s more to it.... Say you wanted your perceptron to classify arches. That is, you present inputs representing an arch, and the output should be 1. You present inputs not representing an arch, and the output should be 0. If your perceptron does that correctly for all inputs, it knows the concept of arch. x1x1 x2x2 x3x3 xnxn......  w1w1 w2w2 w3w3 wnwn inputs weights sum threshold

29. Learning in neural networks But what if you present inputs for an arch, and your perceptron outputs a 0??? x1x1 x2x2 x3x3 xnxn......  w1w1 w2w2 w3w3 wnwn inputs weights sum threshold

30. Learning in neural networks But what if you present inputs for an arch, and your perceptron outputs a 0??? What could be done to make it more likely that the output will be 1 the next time the ‘tron sees those same inputs? x1x1 x2x2 x3x3 xnxn......  w1w1 w2w2 w3w3 wnwn inputs weights sum threshold

31. Learning in neural networks But what if you present inputs for an arch, and your perceptron outputs a 0??? What could be done to make it more likely that the output will be 1 the next time the ‘tron sees those same inputs? You increase the weights. Which ones? How much? x1x1 x2x2 x3x3 xnxn......  w1w1 w2w2 w3w3 wnwn inputs weights sum threshold

32. Learning in neural networks But what if you present inputs for not an arch, and your perceptron outputs a 1? x1x1 x2x2 x3x3 xnxn......  w1w1 w2w2 w3w3 wnwn inputs weights sum threshold

33. Learning in neural networks But what if you present inputs for not an arch, and your perceptron outputs a 1? What could be done to make it more likely that the output will be 0 the next time the ‘tron sees those same inputs? x1x1 x2x2 x3x3 xnxn......  w1w1 w2w2 w3w3 wnwn inputs weights sum threshold

34. Learning in neural networks But what if you present inputs for not an arch, and your perceptron outputs a 1? What could be done to make it more likely that the output will be 0 the next time the ‘tron sees those same inputs? You decrease the weights. Which ones? How much? x1x1 x2x2 x3x3 xnxn......  w1w1 w2w2 w3w3 wnwn inputs weights sum threshold

35. Let’s make one... First we need to come up with a representation language. We’ll abstract away most everything to make it simple.

36. Let’s make one... First we need to come up with a representation language. We’ll abstract away most everything to make it simple. All training examples have three blocks. A and B are upright blocks. A is always left of B. C is a sideways block. Our language will assume those things always to be true. The only things our language will represent are the answers to these five questions...

37. Let’s make one... yes = 1, no = 0 Does A support C? Does B support C? Does A touch C? Does B touch C? Does A touch B?

38. Let’s make one... yes = 1, no = 0 Does A support C?1 Does B support C?1 Does A touch C?1 Does B touch C?1 Does A touch B?0 AB C arch

39. Let’s make one... yes = 1, no = 0 Does A support C?1 Does B support C?1 Does A touch C?1 Does B touch C?1 Does A touch B?1 AB C not arch

40. Let’s make one... yes = 1, no = 0 Does A support C?0 Does B support C?0 Does A touch C?1 Does B touch C?1 Does A touch B?0 AB C not arch

41. Let’s make one... yes = 1, no = 0 Does A support C?0 Does B support C?0 Does A touch C?1 Does B touch C?1 Does A touch B?0 AB C not arch and so on.....

42. Our very simple arch learner x1x1 x2x2 x3x3 x5x5  x4x4

43. x1x1 x2x2 x3x3 x5x5  x4x4 - 0.5 0 0.5 0 -0.5 0.5

44. Our very simple arch learner 1 1 1 0  1 - 0.5 0 0.5 0 -0.5 0.5 AB C arch

45. Our very simple arch learner 1 1 1 0  1 - 0.5 0 0.5 0 -0.5 0.5 AB C arch sum = 1*-0.5 + 1*0 + 1*0.5 + 1*0 + 0*0.5

46. Our very simple arch learner 1 1 1 0  1 - 0.5 0 0.5 0 -0.5 0.5 AB C arch sum = -0.5 + 0 + 0.5 + 0 + 0 = 0 which is not > threshold so output is 0 0

47. Our very simple arch learner 1 1 1 0  1 - 0.5 0 0.5 0 -0.5 0.5 AB C arch sum = -0.5 + 0 + 0.5 + 0 + 0 = 0 which is not > threshold so output is 0 ‘tron said no when it should say yes so increase weights where input = 1 0

48. Our very simple arch learner 1 1 1 0  1 - 0.4 0.1 0.6 0.1 -0.5 0.5 AB C arch sum = -0.5 + 0 + 0.5 + 0 + 0 = 0 which is not > threshold so output is 0 so we increase the weights where the input is 1 0

49. Our very simple arch learner 1 1 1 1  1 - 0.4 0.1 0.6 0.1 -0.5 0.5 AB C not arch now we look at the next example

50. Our very simple arch learner 1 1 1 1  1 - 0.4 0.1 0.6 0.1 -0.5 0.5 AB C not arch sum = -0.4 + 0.1 + 0.6 + 0.1 - 0.5 = -0.1 which is not > 0.5 so output is 0 0

51. Our very simple arch learner 1 1 1 1  1 - 0.4 0.1 0.6 0.1 -0.5 0.5 AB C not arch sum = -0.4 + 0.1 + 0.6 + 0.1 - 0.5 = -0.1 which is not > 0.5 so output is 0 that’s the right output for this input, so we don’t touch the weights 0

52. Our very simple arch learner 0 0 1 0  1 - 0.4 0.1 0.6 0.1 -0.5 0.5 AB C not arch now we look at the next example

53. Our very simple arch learner 0 0 1 0  1 - 0.4 0.1 0.6 0.1 -0.5 0.5 AB C not arch sum = 0 + 0 + 0.6 + 0.1 + 0 = 0.7 which is > 0.5 so output = 1 1

54. Our very simple arch learner 0 0 1 0  1 - 0.4 0.1 0.6 0.1 -0.5 0.5 AB C not arch the ‘tron said yes when it should have said no, so we decrease the weights where the inputs = 1 1

55. Our very simple arch learner 0 0 1 0  1 - 0.4 0.1 0.5 0 -0.5 0.5 AB C not arch the ‘tron said yes when it should have said no, so we decrease the weights where the inputs = 1 1

56. We could do this for days......but let’s have the computer do it all for us. First, take a look at the training examples we’ve constructed...

57. Training Set a b a b a in class? supports supports touches touches touches c c c c b yes 1 1 1 1 0 no 1 1 1 1 1 no 0 0 0 0 0 no 0 0 1 1 0 no 1 0 1 0 1

58. Training Set a b a b a in class? supports supports touches touches touches c c c c b no 1 0 1 0 0 no 0 1 0 1 1 no 0 1 0 1 0 no 0 0 1 0 0 no 0 0 0 1 0

59. Now let’s look at the program The program isn’t in CILOG!

60. What’s going on? The perceptron goes through the training set, making a guess for each example and comparing it to the actual answer. Based on that comparison, the perceptron adjusts weights up, down, or not at all. For some concepts, the process converges on a set of weights such that the perceptron guesses correctly for every example in the training set -- that’s when the weights stop changing.

61. What’s going on? Another way of looking at it: Each of the possible inputs (2 5 in our case) maps onto a 1 or a 0. The perceptron is trying to find a set of weights such that it can draw a line through the set of all inputs and say “these inputs belong on one side of the line (output = 1) and those belong on the other side (output = 0)”.

62. What’s going on? one set of weights

63. What’s going on? another set of weights

64. What’s going on? still another set of weights

65. What’s going on? still another set of weights The perceptron looks for linear separability. That is, in the n-space defined by the inputs, it’s looking for a line or plane that divides the inputs.

66. Observations This perceptron can learn concepts involving “and” and “or” This perceptron can’t learn “exclusive or” (try ex3 in the Scheme code). Not linearly separble... it won’t converge Even a network of perceptrons arranged in a single layer can’t compute XOR (as well as other things)

67. Observations The representation for the learned concept (e.g., the arch concept) is just five numbers. What does that say about the physical symbol system hypothesis? However, if you know which questions or relations each individual weight is associated with, you still have a sense of what the numbers/weights mean.

68. Observations This is another example of intelligence as search for the best representation. The final set of weights that was the solution to the arch-learning problem is not the only solution. In fact, there are infinitely many solutions, corresponding to the infinitely many ways to draw the line or plane.

69. Beyond perceptrons Perceptrons were popular in the 1960s, and some folks thought they were the key to intelligent machines. But you needed multiple-layer perceptron networks to compute some things, but to make those work you had to build them by hand. Multiple-layer perceptron nets don’t necessarily converge, and when they don’t converge, they don’t learn. Building big nets by hand was too time consuming, so interest in perceptrons died off in the 1970s.

70. The return of the perceptron In the 1980s, people figured out that with some changes, multiple layers of perceptron-like units could compute anything. First, you get rid of the threshold -- replace the step function that generated output with a continuous function. Then you put hidden layers between the inputs and the output(s).

71. The return of the perceptron input layer hidden layer output layer

72. The return of the perceptron Then for each input example you let the activation feed forward to the outputs, compare outputs to desired outputs, and then backpropagate the information about the differences to inform decisions about how to adjust the weights in the multiple layers of network. Changes to weights give rise to immediate changes in output, because there’s no threshold.

73. The return of the perceptron This generation of networks is called neural networks, or backpropagation nets, or connectionist networks (don’t use perceptron now) They’re capable of computing anything computable They learn well They degrade gracefully They handle noisy input well They’re really good at modelling low-level perceptual processing and simple learning but...

74. The return of the perceptron This generation of networks is called neural networks, or backpropagation nets, or connectionist networks (don’t use perceptron now) They can’t explain their reasoning Neither can we, easily...what the pattern of weights in the hidden layers correspond to is opaque Not a lot of success yet in modelling higher levels of cognitive behavior (planning, story understanding, etc.)

75. The return of the perceptron But this discussion of perceptrons should put you in a position to read Chapter 11.2 and see how those backprop networks came about. Which leads us to the end of this term...

76. Things... Good bye! Thanks for being nice to the new guy. We’ll see you at noon on Friday, December 10, in MCML 166 for the final exam Have a great holiday break, and come visit next term.

77. Questions?