Data Mining CSCI 307, Spring 2019 Lecture 6 Output: Tables, Linear Models, Trees

Output: Representing Structural Patterns Many different ways of representing patterns Decision trees, rules, instance-based, … Also called “knowledge” representation Representation determines inference method Understanding the output is the key to understanding the underlying learning methods Different types of output for different learning problems (e.g. classification, regression, …)

Tables Simplest way of representing output: Use the same format as input! Decision table for the weather problem: Simply find the row with the appropriate conditions and assign the class, in this case, play or not. Outlook Temperature Humidity Windy Play Sunny Hot High False No Sunny Hot High True No Overcast Hot High False Yes Rainy Mild High False Yes Rainy Cool Normal False Yes Rainy Cool Normal True No Overcast Cool Normal True Yes ...... If it is numeric prediction, the concept is the same, except instead of calling it a decision table, it's called a regression table.

Tables Sometimes some of the attributes aren't necessary for the decision. What if we don't need temperature and windy attributes? A smaller, condensed table might be better: Main problem: selecting the right attributes so as to make the right decision. Outlook Humidity Play Sunny High No Sunny Normal Yes Overcast High Yes Overcast Normal Yes Rainy High No Rainy Normal No

Another Simple Representation: Linear Models Regression model Used when all the inputs (attribute values) and the output are numeric Output is the sum of weighted attribute values The trick is to find good values for the weights

A Linear Regression Function for the CPU Performance Data Only the cache attribute here is used to predict the CPU performance. (It is easier to see in two dimensions.) PRP = 37.06 + 2.47CACH The 37.06 is the "bias" term and is a weight as is the cache weight of 2.47. The least squares linear regression method was used to come up with the weights. (We'll see how in Chapter 4.) The training data is used to come up with the weights. Given a test instance, plug the value of the cache attribute into the expression and the value of performance (i.e. the output/class) will be on the line. 6

Linear Models for Binary Classification The line separates the two classes Decision boundary - defines where the decision changes from one class value to the other Prediction is made by plugging in observed values of the attributes into the expression Predict one class if output >= 0, and the other class if output < 0 Boundary becomes a high-dimensional plane (hyperplane) when there are multiple attributes

A Linear Decision Boundary Separating Iris Setosas from Iris Versicolors setosa if result is >= 0 versicolor if result is < 0 2.0 – 0.5PetalLength – 0.8PetalWidth = 0

Trees “Divide-and-conquer” approach produces tree Nodes involve testing a particular attribute Usually, attribute value is compared to constant Other possibilities: Comparing values of two attributes Using a function of one or more attributes Option nodes (choose more than one branch), i.e. an instance leads to two (or more) leaves, then the alternative predictions must be combined somehow (majority voting) Leaves assign classification, set of classifications, or probability distribution to instances Unknown instance is routed down the tree

Nominal and Numeric Attributes number of children usually equal to number values ==> attribute won’t get tested more than once Other possibility: division into two subsets, so may get tested more than once

Nominal and Numeric Attributes test whether value is greater or less than constant ==> attribute may get tested several times Other possibility: three-way split (or multi-way split) Integer: less than, equal to, greater than Real: below, within (i.e. close enough to be equal), above

Missing Values Does absence of value have some significance? Yes ==> “missing” is a separate value No ==> “missing” must be treated in a special way Solution A: assign instance to most popular branch Solution B: split instance into pieces Pieces receive weights according to fraction of training instances that go down each branch Classifications from leaf nodes are combined using the weights that have percolated to them