Dr. Chen, Data Mining  A/W & Dr. Chen, Data Mining Chapter 3 Basic Data Mining Techniques Jason C. H. Chen, Ph.D. Professor of MIS School of Business Administration Gonzaga University Spokane, WA

Dr. Chen, Data Mining  A/W & Dr. Chen, Data Mining Objectives The chapter introduces several common data mining techniques. In Section 3.1, it focus on supervised learning by presenting a standard algorithm for creating decision trees. In Section 3.2, an efficient technique for generating association rules is presented. In Section 3.3, unsupervised clustering and the K- Means algorithm are illustrated. Section 3.4 show you how genetic algorithms can perform supervised learning and unsupervised clustering. (to be skipped)

Dr. Chen, Data Mining  A/W & Dr. Chen, Data Mining 3.1 Decision Trees Decision trees are constructed using only those attributes best able to differentiate the concepts to be learned. A decision tree is built by initially selecting a subset of instances from a training set. The subset is then used the algorithm to construct a decision tree. The remaining training set instances test the accuracy of the constructed tree. –If the decision tree classifies the instances correctly, the procedure terminates. –If an instance is incorrectly classified, the instance is added to the selected subset of training instances and a new tree is constructed.

Dr. Chen, Data Mining  A/W & Dr. Chen, Data Mining An Algorithm for Building Decision Trees 1. Let T be the set of training instances. 2. Choose an attribute that best differentiates the instances in T. 3. Create a tree node whose value is the chosen attribute. -Create child links from this node where each link represents a unique value for the chosen attribute. -Use the child link values to further subdivide the instances into subclasses. 4. For each subclass created in step 3: a. If the instances in the subclass satisfy predefined criteria or if the set of remaining attribute choices for this path is null, specify the classification for new instances following this decision path. b. If the subclass does not satisfy the criteria and there is at least one attribute to further subdivide the path of the tree, let T be the current set of subclass instances and return to step 2.

Dr. Chen, Data Mining  A/W & Dr. Chen, Data Mining Table 3.1 The Credit Card Promotion Database (same as Table2.3)

Dr. Chen, Data Mining  A/W & Dr. Chen, Data Mining Figure 3.1 A partial decision tree with root node = income range

Dr. Chen, Data Mining  A/W & Dr. Chen, Data Mining Figure 3.2 A partial decision tree with root node = credit card insurance

Dr. Chen, Data Mining  A/W & Dr. Chen, Data Mining Figure 3.3 A partial decision tree with root node = age

Dr. Chen, Data Mining  A/W & Dr. Chen, Data Mining Decision Trees for the Credit Card Promotion Database

Dr. Chen, Data Mining  A/W & Dr. Chen, Data Mining Figure 3.4 A three-node decision tree for the credit card database

Dr. Chen, Data Mining  A/W & Dr. Chen, Data Mining Figure 3.5 A two-node decision treee for the credit card database

Dr. Chen, Data Mining  A/W & Dr. Chen, Data Mining

Decision Tree Rules

Dr. Chen, Data Mining  A/W & Dr. Chen, Data Mining A Rule for the Tree in Figure 3.4 IF Age <=43 & Sex = Male & Credit Card Insurance = No THEN Life Insurance Promotion = No

Dr. Chen, Data Mining  A/W & Dr. Chen, Data Mining A Simplified Rule Obtained by Removing Attribute Age IF Sex = Male & Credit Card Insurance = No THEN Life Insurance Promotion = No

Dr. Chen, Data Mining  A/W & Dr. Chen, Data Mining Other Methods for Building Decision Trees CART CHAID

Dr. Chen, Data Mining  A/W & Dr. Chen, Data Mining Advantages of Decision Trees Easy to understand. Map nicely to a set of production rules. Applied to real problems. Make no prior assumptions about the data. Able to process both numerical and categorical data.

Dr. Chen, Data Mining  A/W & Dr. Chen, Data Mining Disadvantages of Decision Trees Output attribute must be categorical. Limited to one output attribute. Decision tree algorithms are unstable. Trees created from numeric datasets can be complex.

Dr. Chen, Data Mining  A/W & Dr. Chen, Data Mining 3.2 Generating Association Rules

Dr. Chen, Data Mining  A/W & Dr. Chen, Data Mining Confidence and Support

Dr. Chen, Data Mining  A/W & Dr. Chen, Data Mining Rule Confidence Given a rule of the form “If A then B”, rule confidence is the conditional probability that B is true when A is known to be true.

Dr. Chen, Data Mining  A/W & Dr. Chen, Data Mining Rule Support The minimum percentage of instances in the database that contain all items listed in a given association rule.

Dr. Chen, Data Mining  A/W & Dr. Chen, Data Mining Mining Association Rules: An Example

Dr. Chen, Data Mining  A/W & Dr. Chen, Data Mining

General Considerations We are interested in association rules that show a lift in product sales where the lift is the result of the product’s association with one or more other products. We are also interested in association rules that show a lower than expected confidence for a particular association.

Dr. Chen, Data Mining  A/W & Dr. Chen, Data Mining Up here for now!

Dr. Chen, Data Mining  A/W & Dr. Chen, Data Mining 3.3 The K-Means Algorithm 1.Choose a value for K, the total number of clusters. 2.Randomly choose K points as cluster centers. 3.Assign the remaining instances to their closest cluster center. 4.Calculate a new cluster center for each cluster. 5.Repeat steps 3-5 until the cluster centers do not change.

Dr. Chen, Data Mining  A/W & Dr. Chen, Data Mining An Example Using K-Means

Dr. Chen, Data Mining  A/W & Dr. Chen, Data Mining

Figure 3.6 A coordinate mapping of the data in Table 3.6

Dr. Chen, Data Mining  A/W & Dr. Chen, Data Mining

Figure 3.7 A K-Means clustering of the data in Table 3.6 (K = 2)

Dr. Chen, Data Mining  A/W & Dr. Chen, Data Mining General Considerations Requires real-valued data. We must select the number of clusters present in the data. Works best when the clusters in the data are of approximately equal size. Attribute significance cannot be determined. Lacks explanation capabilities.

Dr. Chen, Data Mining  A/W & Dr. Chen, Data Mining 3.4 Genetic Learning

Dr. Chen, Data Mining  A/W & Dr. Chen, Data Mining Genetic Learning Operators Crossover Mutation Selection

Dr. Chen, Data Mining  A/W & Dr. Chen, Data Mining Genetic Algorithms and Supervised Learning

Dr. Chen, Data Mining  A/W & Dr. Chen, Data Mining Figure 3.8 Supervised genetic learning

Dr. Chen, Data Mining  A/W & Dr. Chen, Data Mining

Figure 3.9 A crossover operation

Dr. Chen, Data Mining  A/W & Dr. Chen, Data Mining

Genetic Algorithms and Unsupervised Clustering

Dr. Chen, Data Mining  A/W & Dr. Chen, Data Mining Figure 3.10 Unsupervised genetic clustering

Dr. Chen, Data Mining  A/W & Dr. Chen, Data Mining

General Considerations Global optimization is not a guarantee. The fitness function determines the complexity of the algorithm. Explain their results provided the fitness function is understandable. Transforming the data to a form suitable for genetic learning can be a challenge.

Dr. Chen, Data Mining  A/W & Dr. Chen, Data Mining 3.5 Choosing a Data Mining Technique

Dr. Chen, Data Mining  A/W & Dr. Chen, Data Mining Initial Considerations Is learning supervised or unsupervised? Is explanation required? What is the interaction between input and output attributes? What are the data types of the input and output attributes?

Dr. Chen, Data Mining  A/W & Dr. Chen, Data Mining Further Considerations Do We Know the Distribution of the Data? Do We Know Which Attributes Best Define the Data? Does the Data Contain Missing Values? Is Time an Issue? Which Technique Is Most Likely to Give a Best Test Set Accuracy?

Dr. Chen, Data Mining  A/W & Dr. Chen, Data Mining