2. Machine Learning A Quick look Sources: Artificial Intelligence – Russell & Norvig Artifical Intelligence - Luger By: Héctor Muñoz-Avila

3. What Is Machine Learning? “Logic is not the end of wisdom, it is just the beginning” --- Spock System Knowledge Action 1 time Game changed Game Action 2 Knowledge System

4. Learning: The Big Picture Two forms of learning:  Supervised: the input and output of the learning component can be perceived (for example: experienced player giving friendly teacher)  Unsupervised: there is no hint about the correct answers of the learning component (for example to find clusters of data)

5. 4 Online – during gameplay –Adapt to player tactics –Avoid repetition of mistakes –Requirements: computationally cheap, effective, robust, fast learning (Spronck 2004) Offline – Between the end of a game and the next –Devise new tactics –Discover exploits Offline vs. Online Learning

6. Classification (According to the language representation) Symbolic  Version Spaces  Decision Trees  Explanation-Based Learning  … Sub-symbolic  Reinforcement Learning  Connectionist  Evolutionary

7. Classification (According to the language representation) Symbolic  Version Space  Decision Trees  Explanation-Based Learning  … Sub-symbolic  Reinforcement Learning  Connectionist  Evolutionary

8. Version Space Idea: Learn a concept from a group of instances, some positive and some negative Example: target: obj(Size,Color,Shape) Size = {large, small} Color = {red, white, blue} Shape = {ball, brick, cube} Instances: +: obj(large,white,ball) obj(small,blue,ball) −: obj(small,red,brick) obj(large,blue,cube) Two extremes (temptative) solutions: obj(X,Y,Z) obj(large,white,ball) obj(small,blue,ball) … too general too specific obj(large,Y,ball)obj(small,Y,ball) obj(X,Y,ball) … concept space

9. How Version Space Works + + + + + + + + − − If we consider only positives If we consider positive and negatives + + + + + + + + − − What is the role of the negative instances? to help prevent over-generalizations

10. Classification (According to the language representation) Symbolic  Version Space  Decision Trees  Explanation-Based Learning  … Sub-symbolic  Reinforcement Learning  Connectionist  Evolutionary

11. Explanation-Based learning A C B ABC AC B C B A B A C B A C BC A C A B A C B B C A AB C A B C A B C Can we avoid making this error again? ? ? ?

12. Explanation-Based learning (2) A C B ABC C B A A C B AB C ? ? ? Possible rule: If the initial state is this and the final state is this, don’t do that More sensible rule: don’t stack anything above a block, if the block has to be free in the final state

13. Classification (According to the language representation) Symbolic  Version Space  Decision Trees  Explanation-Based Learning  … Sub-symbolic  Reinforcement Learning  Connectionist  Evolutionary

14. Motivation # 1: Analysis Tool Suppose that a gaming company have a data base of runs with a beta version of the game, lots of data How can that company’s developers use this data to figure out an good strategies for their AI

15. Motivation # 1: Analysis Tool (cont’d) Ex’ple Bar Fri Hun Pat TypeReswai t x1 no yes some french yes x4 no yes full thai no yes x5 no yes no full french yes no x6 x7 x8 x9 x10 x11 Games data “if built center hall & has built 4 workers then build defense tower” induction Decision Tree

16. The Knowledge Base in Expert Systems A knowledge base consists of a collection of IF-THEN rules: if built center hall & has built 4 workers then build defense tower if built center hall & mine then upgrade center hall Knowledge bases of expert systems contain hundreds and sometimes even thousands such rules. Frequently rules are contradictory and/or overlap

17. Sample Expert System in Games: Age of Empires (defrule � (current-age == dark-age � (building-type-count-total mining-camp > 0) � (not (research-available feudal-age)) � => � (set-strategic-number sn-food-gatherer-percentage 52) � (set-strategic-number sn-wood-gatherer-percentage 35) � (set-strategic-number sn-gold-gatherer-percentage 13) � (set-strategic-number sn-stone-gatherer-percentage 0) � (disable-self) ) http://www.youtube.com/watch?v=GEbnqc82lew

18. Main Drawback of Expert Systems: The Knowledge Acquisition Bottle-Neck The main problem of expert systems is acquiring knowledge from human specialist is a difficult, cumbersome and long activity. NameKB #Rules Const. time (man/years) Maint. time (man/years) MYCIN KA 500 10 N/A XCONKA 2500 18 3 KB = Knowledge Base KA = Knowledge Acquisition

19. Motivation # 2: Avoid Knowledge Acquisition Bottle-Neck GASOIL is an expert system for designing gas/oil separation systems stationed of-shore The design depends on multiple factors including: proportions of gas, oil and water, flow rate, pressure, density, viscosity, temperature and others To build that system by hand would had taken 10 person years It took only 3 person-months by using inductive learning! GASOIL saved BP millions of dollars

20. Motivation # 2 : Avoid Knowledge Acquisition Bottle-Neck NameKB #Rules Const. time (man/years) Maint. time (man/months) MYCIN KA 500 10 N/A XCONKA 2500 18 3 GASOILIDT 2800 1 0.1 BMTKA (IDT) 30000+ 9 (0.3) 2 (0.1) KB = Knowledge Base KA = Knowledge Acquisition IDT = Induced Decision Trees

21. Example of a Decision Tree Patrons? noyes none some waitEstimate? no yes 0-10 >60 Full Alternate? Reservation? Yes 30-60 no yes No no Bar? Yes no yes Fri/Sat? NoYes yes no yes Hungry? yes No 10-30 Alternate? yes Yes no Raining? no yes no yes

22. Definition of A Decision Tree A decision tree is a tree where: The leaves are labeled with classifications (if the classification is “yes” or “no”. The tree is called a boolean tree) The non-leaves nodes are labeled with attributes The arcs out of a node labeled with an attribute A are labeled with the possible values of the attribute A

23. Induction Ex’ple Bar Fri Hun Pat TypeReswai t x1 no yes some french yes x4 no yes full thai no yes x5 no yes no full french yes no x6 x7 x8 x9 x10 x11 Data pattern Databases: what are the data that matches this pattern? database Induction: what is the pattern that matches these data? induction

24. Induction of Decision Trees Objective: find a concise decision tree that agrees with the examples The guiding principle we are going to use is the Ockham’s razor principle: the most likely hypothesis is the simplest one that is consistent with the examples Problem: finding the smallest decision tree is NP-complete However, with simple heuristics we can find a small decision tree (approximations)

25. Induction of Decision Trees: Algorithm Algorithm: 1.Initially all examples are in the same group 2.Select the attribute that makes the most difference (i.e., for each of the values of the attribute most of the examples are either positive or negative) 3.Group the examples according to each value for the selected attribute 4.Repeat 1 within each group (recursive call)

26. Example Ex’ple Bar Fri Hun PatAltTypewait x1 no yes some yesFrench yes x4 no yes full yes Thai yes x5 no yes no full yesFrench no x6 yes no yes some noItalianyes x7 yes no none noBurgerno x8 no yes some no Thaiyes x9 yes no full noBurgerno x10 yes full yesItalianno x11 noNo no none no Thaino

27. IDT: Example Lets compare two candidate attributes: Patrons and Type. Which is a better attribute? Patrons? none X7(-),x11(-) some X1(+),x3(+),x6(+),x8(+) full X4(+),x12(+), x2(-),x5(-),x9(-),x10(-) Type? french X1(+), x5(-) italian X6(+), x10(-) burger X3(+),x12(+), x7(-),x9(-) X4(+),x12(+) x2(-),x11(-) thai

28. Example of a Decision Tree Patrons? noyes none some waitEstimate? no yes 0-10 >60 Full Alternate? Reservation? Yes 30-60 no yes No no Bar? Yes no yes Fri/Sat? NoYes yes no yes Hungry? yes No 10-30 Alternate? yes Yes no Raining? no yes no yes

29. Decision Trees in Gaming http://www.youtube.com/watch?v=HMdOyUp5Rvk Black & White, developed by Lionhead Studios, and released in 2001Black & White Used to predict a player’s reaction to a certain creature’s action In this model, a greater feedback value means the creature should attack –This is done by inducing a decision tree

30. Decision Trees in Black & White ExampleAttributes Target AllegianceDefenseTribeFeedback D1FriendlyWeakCeltic D2EnemyWeakCeltic0.4 D3FriendlyStrongNorse D4EnemyStrongNorse-0.2 D5FriendlyWeakGreek D6EnemyMediumGreek0.2 D7EnemyStrongGreek-0.4 D8EnemyMediumAztec0.0 D9FriendlyWeakAztec should your creature attack a town?

31. Decision Trees in Black & White Allegiance Defense Friendly Enemy 0.4-0.3 WeakStrong 0.1 Medium Note that this decision tree does not even use the tribe attribute

32. Decision Trees in Black & White Now suppose we don’t want the entire decision tree, but we just want the 2 highest feedback values We can create a Boolean expressions, such as ((Allegiance = Enemy) ^ (Defense = Weak)) v ((Allegiance = Enemy) ^ (Defense = Medium))

33. Classification (According to the language representation) Symbolic  Version Space  Decision Trees  Explanation-Based Learning  … Sub-symbolic  Reinforcement Learning  Connectionist  Evolutionary Next class