Rule Generation from Decision Tree Decision tree classifiers are popular method of classification due to it is easy understanding However, decision tree can become large and difficult to interpret In comparison with decision tree, the IF-THEN rules may be easier for humans to understand, particularly if the decision tree is very large

Rule Generation from Decision Tree Rules are easier to understand than large trees One rule is created for each path from the root to a leaf Each attribute-value pair along a path forms a conjunction: the leaf holds the class prediction

Rule Generation from Decision Tree Example: Rule extraction from our buys_computer decision-tree IF age = young AND student = no THEN buys_computer = no IF age = young AND student = yes THEN buys_computer = yes IF age = mid-age THEN buys_computer = yes IF age = old AND credit_rating = excellent THEN buys_computer = yes IF age = young AND credit_rating = fair THEN buys_computer = no

Rule Generation from Decision Tree Rules are expected to be mutually exclusive and exhaustive Mutually exclusive: we can not have rules conflict because no two rules will triggered for the same tuple Exhaustive: there is one rule for each possible attribute-value combination, so that the set of rules does not require a default rule

Association Classification Association rules show strong associations between items that occur frequently in a given data set The discovery of association rules is based on frequent itemset mining The general idea of association classification is that we can search for strong associations between frequent patterns and class labels

Association Classification All association rules must satisfy certain criteria regarding their: Support – the proportion of the data set that they actually represent Confidence – their accuracy

Association Classification Association rules can have any number of items in the rule antecedent (left- hand side) and any number of items in the rule consequent (right –hand side) However, in association classification, we are only interested in association rules of the form p1 ^ p2^ … => A class

Association Classification Age=young ^ credit=ok => buys_computer=yes [support=20%, confidence=93%] The percentage of tuples in D satisfying the rules antecedent and having class label C is called the support of R A support of 20% for association rule means that 20% of the customer in D are young, have an OK credit rating, and belong to the class buys_ciomputer=yes The confidence is the accuracy

Association Classification Regard each row as on transaction

Association Classification A1B1C1D1N A1B1C1D2N A2B1C1D1Y A3B2C1D1Y A3B3C2D1Y A3B3C2D2N A2B3C2D2Y A1B2C1D1N A1B3C2D1Y A3B2C2D1Y A1B2C2D2Y A2B2C1D2Y A2B1C2D1Y A3B2C1D2N A1: age<=30 A2:age between 31~40 A3: Age >40 B1: high income B2: medium income B3: low income C1: not student C2: student D1: fair credit D2: excellent credit Y: buy computer N: don’t buy computer

Association Classification Let support become 20% 14*20%=2.8 therefore minimum support count=3

Association Classification Find 1-itemset on attributes: A1: 5 A2:4 A3: 5 B1:4 B2:6 B3:4 C1:7 C2: 7 D1:8 D2:6

Association Classification Generate All 2-item combination: A lot of combination!!! Use Apriori and some observation (A1 and A2 is not going to be frequent item set)

Association Classification A1 B1 A1 B2 A1 B3 A1 C1 A1 C2 A1 D1 A1 D2 A2 B1 A2 B2 A2 B3 A2 C1 A2 C2 A2 D1 A2 D2

Association Classification A3 B1 A3 B2 A3 B3 A3 C1 A3 C2 A3 D1 A3 D2 B1 C1 B1 C2 B1 D1 B1 D2 B2 C1 B2 C2 B2 D1 B2 D2

Association Classification B3 C1 B3 C2 B3 D1 C1 D1 C1 D2

Association Classification A2 Y Support: 4/14 Confidence(A2=>Y): 4/4 A1 C1 N Support: 3/14 Confidence(A1, C1=>N): 3/3 A1 C2 Y Support: 2/14