Notes from 02_CAINE conference Some data classification techniques are: Decision Tree Induction Bayesian Neural Networks K-Nearest Neighbor Case Based Reasoning Genetic Algorithm rough sets fuzzy logic techniques When using P-trees, are they all essentially the same?

Bayesian Classifier Based on Bayes Theorem: Pr ( X | C ) * Pr ( C ) i = i Pr ( X ) Pr(Ci | X) is the posterior probability Pr(Ci) is the prior probability Can find conditional probabilities, Pr(X|Ci). Classify X with Max Pr(Ci | X) Pr(X) is independent of class, therefore, maximize Pr(X|Ci) * Pr(Ci). Use naïve assumption? Pr(X | Ci ) = Pr( X1 | Ci ) × Pr( X2 | Ci )… × Pr( Xn | Ci )

Calculating Probabilities Pr(X|Ci) Traditional Bayesian classification calculates Pr(X|Ci) and Pr(Ci) for each X, by doing a training database (TDB) scan. If (X,Ci) exists in the training DB, then Pr(X | Ci ) is estimated = cnt(X,Ci)/cnt(Ci). But this requires a TDB scan to determine. If cnt(X,Ci)=0 for all I, then one can choose the default, else use the naïve assumption: cnt(X1___,Ci)/cnt(Ci) × cnt(_X2__,Ci)/cnt(Ci) × … × cnt(___Xn,Ci)/cnt(Ci). The use of Bayesian belief networks simply replaces the the naïve assumption with other methods of estimating Pr(X | Ci ) (e.g., domain knowledge). All probabilities above involve counts. They can all be computed using P-trees: Pr(X|Ci)*Pr(Ci) = {RC[P1(X1)^P2(X2)^…^Pn(Xn)^PC(Ci)]/ RC[PC(Ci)]}*RC[PC(Ci)] / cntTDB Pr(X|Ci)*Pr(Ci) = RC[P1(X1)^P2(X2)^…^Pn(Xn)^PC(Ci)]/ cntTDB Problem ? : if RC[ P1(X1) ^ P2(X2) ^ … ^Pn(Xn) ^ PC(Ci) ] = 0 for all i i.e unclassified pattern does not exist in the training set.

Band-based-P-tree Approach When all RC = 0 for given pattern Reduce the restrictiveness of the pattern Removing the attribute with least information gain Calculate (assume attribute 2 has the least IG) Pr( X | Ci )*Pr( Ci ) = RC[ P1X1 ^ P3X3 ^ … ^ PnXn ^ PCCi ] / cnt[TDB] Can calculation of information gains Using P-trees 1 time calculation for the entire training data

Bit-based Approach Search for similar patterns by removing the least significant bits in the attribute space. The order of the bits to be removed is selected by calculating the info gain (IG). E.g., Calculate the Bayesian conditional probability value for the pattern [G,R] = [10,01] in 2-attribute space. Assume IG for 1st significant bit of R < that of G. Assume IG for 2nd significant bit of G < that of R. Initially, search for the pattern, [10,01] (a). If not found, search for [1_,01] considering IG for the 2nd significant bit. Search space will increase (b). If not found, search for [1_,0_] considering IG for the 2nd significant bit. Search space will increase (c). If not found, search for [1_,_ _] considering IG for the 1st significant bit. Search space will increase (d). This is almost identical to KNN using HOBbit nbrhds! Seems to be very similar to DTI also! Is there really just 1 P-tree classifier? R 00 01 10 11 G R 00 01 10 11 G (a) (b) R R 11 11 10 10 01 01 00 00 00 01 10 11 G 00 01 10 11 G (c) (d) Idea: Use domain knowledge to weight feature attributes. Decide as above using weighted IG. Use GA’s to improve weights! Note: would want to consider all patterns, not just the four above

Bit-based Approach For [G,R] = [10,01] the nine ways of ignoring bits (9 hobbit neighborhoods) Of course Hobbit neighborhoods can be replaced by Lp-neighborhoods at some cost in complexity but some benefit wrt accuracy (using the OR to get Lp-nbhds). R 00 01 10 11 G R 00 01 10 11 G R R R 11 11 11 10 10 10 01 01 01 00 00 00 00 01 10 11 G 00 01 10 11 G 00 01 10 11 [10,01] [1_,01] [10,0_] [_ _,01] [10,_ _] R R R R 11 11 11 11 10 10 10 10 01 01 01 01 00 00 00 00 00 01 10 11 G 00 01 10 11 G 00 01 10 11 G 00 01 10 11 G [1_,0_ ] [1_,_ _] [_ _,0_] [_ _,_ _]

Rank order for moving windows The element of rank r in a sequence is the rth smallest element. Rank order filtering widely used in signal and image processing (and voice) These are called order-statistic filters Good for removing shot noise from images while maintaining sharp edges. Excellent noise elimination and reduction properties Good for image enhancement and restoration Research project: r-rank order in 2-D using 2nx2n-trees or using 3nx3n-trees 1 1 1 1 1 1 0 0 1 1 1 1 1 1 0 0 0 0 1 1 1 1 1 1 1 0 0 1 1 1 1 1 1 1 1 0 1 1 1 1 1 1 1 1 1 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 1 1 1 1 1 0 0 0 1 1 1 1 1 1 1 0 1 1 1 1 1 1 1 1 0 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 1 1 1 1 1 1 0 0 0 0 1 1 1 1 1 1 1 0 0 1 1 1 1 1 1 1 1 0 1 1 1 1 1 1 1 1 1 0 1 1 1 1 1 1 1 1