1. Learning from Observations Copyright, 1996 © Dale Carnegie & Associates, Inc. Chapter 18

2. CS 471/598 by H. Liu2 Learning agents zImprove their behavior through diligent study of their own experiences. zActing -> Experience -> Better Acting zWe’ll study how to make a learning agent to learn; what is needed for learning; and some representative methods of learning from observations

3. CS 471/598 by H. Liu3 A general model zWhat are the components of a learning agent? yLearning element - learn and improve (Fig 18.1) yPerformance element - an agent itself to perceive & act yProblem generator - suggest some exploratory actions yCritic - provide feedback how the agent is doing zThe design of an LA is affected by four issues: prior info, feedback, representation, performance

4. CS 471/598 by H. Liu4 What do we need zComponents of the performance element (p 527) yEach component should be learnable given feedback zRepresentation of the components yPropositional Logic, FOL, or others zAvailable feedback ySupervised, Reinforcement, Unsupervised zPrior knowledge yNil, some, (Why not all?) zPut it all together as learning some functions

5. CS 471/598 by H. Liu5 Inductive Learning zData described by examples yan example is a pair (x, f(x)) zInduction - given a collection of examples of f, return a function h that approximates f. yFig 18.2 xHypothesis xBias zLearning incrementally or in batch

6. CS 471/598 by H. Liu6 Some questions about inductive learning zAre there many forms of inductive learning? yWe’ll learn some zCan we achieve both expressiveness and efficiency? zHow can one possibly know that one’s learning algorithm has produced a theory that will correctly predict the future? zIf one does not, how can one say that the algorithm is any good?

7. CS 471/598 by H. Liu7 Learning decision trees zA decision tree takes as input an object described by a set of properties and outputs yes/no “decision”. zOne of the simplest and yet most successful forms of learning zTo make a decision “wait” or “not wait”, we need information such as … (page 532 for 10 attributes, a data set in Fig 18.5) Patrons(Full)^WaitEstimate(0-10)^Hungry(N)=>WillWait

8. CS 471/598 by H. Liu8 Let’s make a decision zWhere to start?

9. CS 471/598 by H. Liu9 Expressiveness of a DT zContinued from page 7 - A possible DT ( Fig 18.4 ) zThe decision tree language is essentially propositional, with each attribute test being a proposition. zAny Boolean functions can be written as a decision tree (truth tables DTs) zDTs can represent many functions with much smaller trees, but not for all Boolean functions (parity, majority)

10. CS 471/598 by H. Liu10 zHow many different functions are in the set of all Boolean functions on n attributes? zHow to find consistent hypotheses in the space of all possible ones? zAnd which one is most likely the best?

11. CS 471/598 by H. Liu11 Inducing DTs from examples zExtracting a pattern (DTs) means being able to describe a large number of cases in a concise way - a consistent & concise tree. zApplying Occam’s razor: the most likely hypothesis is the simplest one that is consistent with all observations. zHow to find the smallest DT? yExamine the most important attribute first (Fig 18.6) yAlgorithm (Fig 18.7, page 537) yA DT (Fig 18.8)

12. CS 471/598 by H. Liu12 Choosing the best attribute zA computational method - information theory yInformation - informally, the more surprise you have, the more information you have; mathematically, I(P(v1),…,P(vn)) = sum[-P(vi)logP(vi)] xI(1/2,1/2) = 1 xI(0,1) = (1,0) = 0 yInformation alone can’t help much to answer “what is the correct classification?”.

13. CS 471/598 by H. Liu13 zInformation gain - the difference between the original and the new info requirement: yRemainder(A) = Sum[p1*I(B1)+…+pn*I(Bn)] where p1+…+pn = 1 yGain(A) = I(A) - Remainder(A)

14. CS 471/598 by H. Liu14 Which attribute? zRevisit the example of “Wait” or “Not Wait” using your favorite 2 attributes.

15. CS 471/598 by H. Liu15 Assessing the performance zA fair assessment: the one the learner has not seen. zErrors zTraining and test sets: yDivide the data into two sets yLearn on the training set yTest on the test set yIf necessary, shuffle the data and repeat zLearning curve - “happy graph” (Fig 18.9)

16. CS 471/598 by H. Liu16 Practical use of DT learning zBP’s use of GASOIL zLearning to fly on a flight simulator zAn industrial strength system - Quinlan’s C4.5 zWho’s the next hero?

17. CS 471/598 by H. Liu17 Some issues of DT applications zMissing values zMultivalued attributes zContinuous-valued attributes

18. CS 471/598 by H. Liu18 Learning general logical descriptions zInductive learning is a process of searching for a good hypothesis in the hypothesis space defined by the representation language. zThere are logical connections among examples, hypotheses, and the goal. zGo beyond decision tree induction.

19. CS 471/598 by H. Liu19 What’re Goal and Hypotheses zGoal predicate Q - WillWait zLearning is to find an equivalent logical expression we can classify examples zEach hypothesis proposes such an expression - a candidate definition of Q. zE.g., Fig 18.8 expresses the following (Hr):

20. CS 471/598 by H. Liu20 zHypothesis space is the set of all hypotheses the learning algorithm is designed to entertain. zOne of the hypotheses is correct: H 1 V H 2 V…V H n zEach H i predicts a certain set of examples - the extension of the goal predicate. zTwo hypotheses with different extensions are logically inconsistent with each other, otherwise, they are logically equivalent.

21. CS 471/598 by H. Liu21 What are Examples zAn example is an object of some logical description to which the goal concept may or may not apply. yOne instance/tuple is an example zIdeally, we want to find a hypothesis that agrees with all the examples. zThe relation between f and h are: ++, --, +- (false negative), -+ (false positive). If the last two occur, example I and h are logically inconsistent.

22. CS 471/598 by H. Liu22 Current-best hypothesis search zMaintain a single hypothesis zAdjust it as new examples arrive to maintain consistency (Fig 18.10) yGeneralization for positive examples ySpecialization for negative examples zAlgorithm (Fig 18.11, page 547) yNeed to check for consistency with all existing examples each time taking a new example

23. CS 471/598 by H. Liu23 Example of WillWait zProblems: nondeterministic, no guarantee for simplest and correct h, need backtrack

24. CS 471/598 by H. Liu24 Least-commitment search zKeeping one h as its best guess is the problem -> Can we keep as many as possible? zVersion space (candidate elimination) Algo yincremental yleast-commitment zFrom intervals to boundary sets yG-set and S-set yEverything between is guaranteed to be consistent wit examples.

25. CS 471/598 by H. Liu25 Version space zGeneralization and specialization (Fig 18.13) yFalse positive for Si, too general, discard it yFalse negative for Si, too specific, generalize it minimally yFalse positive for Gi, too general, specialize it minimally yFalse negative for Gi, too specific, discard it zWhen to stop yOne concept left (Si = Gi) yThe version space collapses yRun out of examples zOne major problem: can’t handle noise

26. CS 471/598 by H. Liu26 Why learning works? zHow can one possibly know that his/her learning algorithm will correctly predict the future? zHow do we know that h is close enough to f without knowing f? zComputational learning theory has provided some answers. The basic idea is that because any wrong h will make an incorrect prediction, it will be found out with high probability after a small number of examples. So, if h is consistent with a sufficient number of examples, it is unlikely to to seriously wrong - probably approximately correct.

27. CS 471/598 by H. Liu27 Summary zLearning is essential for intelligent agents ydealing with the unknowns yimproving its capability over time zAll types of learning can be considered as learning an accurate representation h of f. zInductive learning - f from data to h zDecision trees - deterministic Boolean functions zLearning logical theories - Current Best and Least Commitment