1. Decision Tree Learning Brought to you by Chris Creswell

2. Why learn about decision trees? A practical way to get AI to adapt to player – a simple form of user modeling A practical way to get AI to adapt to player – a simple form of user modeling –Enhances replayability –Player’s bot allies can be more effective –Opponent bots can learn player’s tactics, player can’t repeat the same strategy over and over

3. What we’ll learn What is a decision tree What is a decision tree How do we build a decision tree How do we build a decision tree What has been done with decision trees in games What has been done with decision trees in games –What else can we do with them

4. What is a decision tree Decision Tree Learning (DTL) is a form of inductive learning task, meaning it has the following objective: use a training set of examples to create a hypothesis that makes general conclusions Decision Tree Learning (DTL) is a form of inductive learning task, meaning it has the following objective: use a training set of examples to create a hypothesis that makes general conclusions

5. What is a decision tree – terms/concepts Attribute: a variable that we take into account in making a decision Attribute: a variable that we take into account in making a decision Target attribute: the attribute that we want to take on a certain value, we’ll decide based on it Target attribute: the attribute that we want to take on a certain value, we’ll decide based on it

6. What is a decision tree – an example ExampleHourWeatherAccidentStall Target -- Commute D18AMSunnyNoNoLong D28AMCloudyNoYesLong D310AMSunnyNoNoShort D49AMRainyYesNoLong D59AMSunnyYesYesLong D610AMSunnyNoNoShort D710AMCloudyNoNoShort D89AMRainyNoNoMedium D99AMSunnyYesNoLong D1010AMCloudyYesYesLong D1110AMRainyNoNoShort D128AMCloudyYesNoLong D139AMSunnyNoNoMedium

7. What is a decision tree – an example Hour 10AM Stall No Short Yes Long 8AM Long 9AM Accident No Medium Yes Long

8. What is a decision tree – how to use it Given a set of circumstances (values of attributes), use it to traverse the tree from root to leaf Given a set of circumstances (values of attributes), use it to traverse the tree from root to leaf The leaf node is a decision The leaf node is a decision

9. Why is this useful The hypothesis formed from the training set can be used to draw conclusions about sets of circumstances not present in the training set – it will generalize The hypothesis formed from the training set can be used to draw conclusions about sets of circumstances not present in the training set – it will generalize

10. How do we construct a decision tree? Guiding principle of inductive learning: Guiding principle of inductive learning: –Occam’s razor – choose the simplest possible hypothesis that is consistent with the provided examples General idea: recursively classify the examples based on one of the attributes until all examples have been used General idea: recursively classify the examples based on one of the attributes until all examples have been used Here’s the algorithm: Here’s the algorithm:

11. node LearnTree(examples, targetAttribute, attributes) examples is the training set targetAttribute is what to learn attributes is the set of available attributes returns a tree node begin if all the examples have the same targetAttribute value, return a leaf with that value else if the set of attributes is empty return a leaf with the most common targetAttribute value among examples else begin A = the “best” attribute among attributes having a range of values v1, v2, …, vk Partition examples according to their value for A into sets S1, S2, …, Sk Create a decision node N with attribute A for i = 1 to k begin Attach a branch B to node N with test V i if S i has elements (is non-empty) Attach B to LearnTree(S i, targetAttribute, attributes – {A}); else Attach B to a leaf node with most common targetAttribute end return decision node N endend

12. This is how we construct a decision tree This very simple pseudo-code basically implements the construction of a decision tree, except for one key thing that is abstracted away, this is … This very simple pseudo-code basically implements the construction of a decision tree, except for one key thing that is abstracted away, this is … Key step in the algorithm: choosing the “best” attribute to classify on Key step in the algorithm: choosing the “best” attribute to classify on One algorithm for doing this is ID3 (used in Black and White) One algorithm for doing this is ID3 (used in Black and White) –We’ll get to the algorithm in a bit

13. This is how we construct a decision tree – pseudo-code walkthrough First, LearnTree is called with all examples, the targetAttribute, and all attributes to classify on First, LearnTree is called with all examples, the targetAttribute, and all attributes to classify on It chooses the “best” (we’ll get to that) attribute to split on, creates a decision node for it, then recursively calls LearnTree for each partition of the examples It chooses the “best” (we’ll get to that) attribute to split on, creates a decision node for it, then recursively calls LearnTree for each partition of the examples

14. This is how we construct a decision tree – pseudo-code walkthrough Recursion stops when: Recursion stops when: –All examples have the same value –There are no more attributes –There are no more examples The first two need some explanation, the third one is trivial – all examples have been classified The first two need some explanation, the third one is trivial – all examples have been classified

15. This is how we construct a decision tree – pseudo-code walkthrough Recursion stops when all examples have the same value, when does this happen? Recursion stops when all examples have the same value, when does this happen? –When ancestor attributes and corresponding branch values, as well as the target attribute and value, are the same across examples

16. This is how we construct a decision tree – pseudo-code walkthrough Recursion stops when there are no more attributes Recursion stops when there are no more attributes –This happens when training set is inconsistent, e. g. there are 2 or more examples having the same values for all but the target attribute –The way our pseudo-code is written, it guesses when this happens, it picks the most popular target attribute value –This is a decision left up to the implementer –This is a weakness of the algorithm It doesn’t handle “noise” in its training set well It doesn’t handle “noise” in its training set well

17. This is how we construct a decision tree – pseudo-code walkthrough Let’s watch the algorithm in action … Let’s watch the algorithm in action … http://www.cs.ualberta.ca/~aixplore/learni ng/DecisionTrees/InterArticle/2- DecisionTree.html http://www.cs.ualberta.ca/~aixplore/learni ng/DecisionTrees/InterArticle/2- DecisionTree.html http://www.cs.ualberta.ca/~aixplore/learni ng/DecisionTrees/InterArticle/2- DecisionTree.html http://www.cs.ualberta.ca/~aixplore/learni ng/DecisionTrees/InterArticle/2- DecisionTree.html

18. ID3 algorithm Picks the best attribute to classify on in a call of LearnTree, does so by quantifying how useful an attribute will be w/respect to the remaining examples Picks the best attribute to classify on in a call of LearnTree, does so by quantifying how useful an attribute will be w/respect to the remaining examples How? Using Shannon’s Information theory, pick the attribute that favors the best reduction in entropy How? Using Shannon’s Information theory, pick the attribute that favors the best reduction in entropy

19. ID3 algorithm – Shannon’s Information Theory Choose an attribute that favors the best reduction in entropy Choose an attribute that favors the best reduction in entropy Entropy quantifies the variation in a set of examples with respect to the target attribute values Entropy quantifies the variation in a set of examples with respect to the target attribute values A set of ex’s with mostly the same targetAttr value has very low entropy (that’s good) A set of ex’s with mostly the same targetAttr value has very low entropy (that’s good) A set of ex’s with many varying targetAttr values will have high entropy (bad) A set of ex’s with many varying targetAttr values will have high entropy (bad) Ready? Here come some equations … Ready? Here come some equations …

20. ID3: Shannon’s Information Theory In the following, S is the set of examples, Si is a subset of S with value Vi under the target Attribute In the following, S is the set of examples, Si is a subset of S with value Vi under the target Attribute

21. ID3: Shannon’s Information Theory Expected entropy of candidate attribute A is weighted sum of subset Expected entropy of candidate attribute A is weighted sum of subset In the following, k is the size of range of attribute A: In the following, k is the size of range of attribute A:

22. ID3: Shannon’s Information Theory What we really want is to maximize information gain, defined: What we really want is to maximize information gain, defined:

23. ID3: Shannon’s Information Theory Entropy of the commute time example: Entropy of the commute time example: The thirteens are because there are thirteen examples. The fours, twos, and sevens come from how many short, medium, and long commutes there are, respectively.

24. ID3: Shannon’s Information Theory Attribute Expected Entropy Info Gain Hour0.651100.768449 Weather1.288840.130719 Accident0.923070.496479 Stall1.170710.248842

25. ID3: Drawbacks Does not guarantee the smallest possible decision tree Does not guarantee the smallest possible decision tree –Selects classifying attribute based on best expected information gain, not always right Not very good with continuous values, best with symbolic data Not very good with continuous values, best with symbolic data –When given lots of distinct continuous values, ID3 will create very “bushy” trees – 1 or 2 levels deep, lots and lots of leaves –We can make this less serious, but it’s still a drawback

26. Decision Trees in games First successful use of a decision tree was in “Black and White” (Lionhead studios, 2001) First successful use of a decision tree was in “Black and White” (Lionhead studios, 2001) http://www.gameai.com/blackandwhite.html http://www.gameai.com/blackandwhite.html “In Black & White you can be the god you want to be. Will you rule with a fair hand, making life better for your people? Or will you be evil and scare them into prayer and submission? No one can tell you which way to be. You, as a god, can play the game any way you choose.” “In Black & White you can be the god you want to be. Will you rule with a fair hand, making life better for your people? Or will you be evil and scare them into prayer and submission? No one can tell you which way to be. You, as a god, can play the game any way you choose.”

27. Decision Trees in games “And as a god, you get to own a Creature. Chosen by you from magical, special animals, your Creature will copy you, you will teach him and he will learn by himself. He will grow, ultimately to 30 metres, and can do anything you can do in the game. Your Creature can help the people or can kill and eat them. He can cast Miracles to bring rain to their crops or he can drown them in thesea. Your Creature is your physical manifestation in the world of Eden, He is “And as a god, you get to own a Creature. Chosen by you from magical, special animals, your Creature will copy you, you will teach him and he will learn by himself. He will grow, ultimately to 30 metres, and can do anything you can do in the game. Your Creature can help the people or can kill and eat them. He can cast Miracles to bring rain to their crops or he can drown them in thesea. Your Creature is your physical manifestation in the world of Eden, He is whatever you want him to be.... And the game also boasts a new level of artificial intelligence. Your Creature is almost a living, breathing thing. He learns, remembers and makes connections. His huge range of abilities and decisions is born of a ground- breakingly powerful and complex AI system.”

28. Decision Trees in games So you teach your creature by giving it feedback – it learns to perform actions that get it the highest feedback So you teach your creature by giving it feedback – it learns to perform actions that get it the highest feedback Problem: feedback is a continuous variable Problem: feedback is a continuous variable We have to make it discrete We have to make it discrete We do so using K-means clustering We do so using K-means clustering

29. Decision Trees in games In K-means clustering, we find out how many clusters we want to create, then use an algorithm to successively associate or dissociate instances with clusters until associations stabilize around k clusters In K-means clustering, we find out how many clusters we want to create, then use an algorithm to successively associate or dissociate instances with clusters until associations stabilize around k clusters The author’s reference for this is from a computer vision textbook The author’s reference for this is from a computer vision textbook –I wasn’t about to go buy it Not important to know clustering algorithm Not important to know clustering algorithm

30. Decision Trees in games Example from B&W: should your creature attack a town Example from B&W: should your creature attack a town Examples : Examples : ExampleAllegianceDefenseTribeFeedback D1FriendlyWeakCeltic D2EnemyWeakCeltic0.4 D3FriendlyStrongNorse D4EnemyStrongNorse-0.2 D5FriendlyWeakGreek D6EnemyMediumGreek0.2 D7EnemyStrongGreek-0.4 D8EnemyMediumAztec0.0 D9FriendlyWeakAztec

31. Decision Trees in games If we ask for 4 clusters, K-means clustering will create clusters around -1, 0.4, 0.1, -0.3. The memberships in these clusters will be {D1, D3, D5, D9}, {D2}, {D6, D8}, {D4, D7} respectively. If we ask for 4 clusters, K-means clustering will create clusters around -1, 0.4, 0.1, -0.3. The memberships in these clusters will be {D1, D3, D5, D9}, {D2}, {D6, D8}, {D4, D7} respectively. The tree ID3 will create using these examples and clusters: The tree ID3 will create using these examples and clusters:

32. Decision Trees in games Allegiance Friendly Enemy Defense Weak 0.4 Medium 0.1 Strong -0.3

33. Decision Trees in games So in this case, the tree the creature learned can be reduced to a nice compact logical expression: So in this case, the tree the creature learned can be reduced to a nice compact logical expression: ((Allegiance = Enemy) AND (Defense = weak)) OR ((Allegiance = Enemy) AND (Defense = Medium)) ((Allegiance = Enemy) AND (Defense = weak)) OR ((Allegiance = Enemy) AND (Defense = Medium)) This happens sometimes This happens sometimes Makes it easier and more efficient to apply Makes it easier and more efficient to apply

34. An Extension to ID3 to better handle continuous values Seems simple, use an inequality, right? Seems simple, use an inequality, right? Not that simple – need to pick cut points Not that simple – need to pick cut points Cut points are the boundaries we create for our inequalities, where do they go? Cut points are the boundaries we create for our inequalities, where do they go? Key insight: optimal cut points must always reside at boundary points Key insight: optimal cut points must always reside at boundary points Okay, so what are boundary points? Okay, so what are boundary points?

35. An Extension to ID3 to better handle continuous values If we sort the list of examples according to their values of the candidate attribute, a boundary point is a value in this list between 2 adjacent instances having different attribute values of the target attribute. If we sort the list of examples according to their values of the candidate attribute, a boundary point is a value in this list between 2 adjacent instances having different attribute values of the target attribute. In the worst case, the number of boundary points is about equal to the number of instances In the worst case, the number of boundary points is about equal to the number of instances –This happens if the target attribute oscillates back and forth between good and bad

36. Example software on CD Show an example made using the software on the CD Show an example made using the software on the CD

37. Conclusions Decision Trees are an elegant way of learning – it is easy to expose their logic and understand what it has learned Decision Trees are an elegant way of learning – it is easy to expose their logic and understand what it has learned Decision Trees are not always the best way to learn – they have some weaknesses Decision Trees are not always the best way to learn – they have some weaknesses But it also has its own set of strengths But it also has its own set of strengths

38. Conclusions Decision Trees work best for symbolic, discrete values Decision Trees work best for symbolic, discrete values Can be extended to work with continuous values Can be extended to work with continuous values B&W had to do some clustering of feedback values to use decision trees B&W had to do some clustering of feedback values to use decision trees

39. Conclusions Up to now, the only use of Decision Trees in games has been in B&W Up to now, the only use of Decision Trees in games has been in B&W What are they good for? What are they good for? –User modeling -- teaching the computer how to react to the player, enhances replayability –Can be used to make bots that are the player’s allies more effective as in B&W –Could also make enemies more intelligent – the player would be forced to come up with new strategies How else can they be used? How else can they be used? –This is relatively unexplored territory people – if you think you have a great idea, go for it