Action Rules Discovery /Lecture I/ by Zbigniew W. Ras UNC-Charlotte, USA

: two conditions occur together, with some confidence Rule: two conditions occur together, with some confidence PresumptiveObjective E = [Cond 1 => Cond 2 ] Interestingness measure Data Mining Task: For a given dataset D, interestingness measure I D and threshold c, find association E such that I D (E) > c. Knowledge Engineer defines c

Interestingness Function Two types of [Silberschatz and Tuzhilin, 1995]: Two types of Interestingness Measure [Silberschatz and Tuzhilin, 1995]: subjective and objective. Subjective measure: user-driven, domain-dependent. Silberschatz and Tuzhilin, 1995] Include unexpectedness [Silberschatz and Tuzhilin, 1995], novelty, actionability [Piatesky-Shapiro & Matheus, 1994]. : data-driven and domain-independent. Objective measure: data-driven and domain-independent. They evaluate rules based on They evaluate rules based on statistics and structures of patterns, e.g., support, confidence, etc.

Objective Interestingness Basic Measures for    : Domain: card[  ] Support or Strength: card[  ] Confidence or Certainty Factor: card[  ]/card[  ] Coverage Factor: card[  ]/card[  ] Leverage: card[  ]/n – [card[  ]/n]*[card[  ]/n] Lift: n  card[  ]/[card[  ]*card[  ]]

Subjective Interestingness Rule is interesting if it is: Rule is interesting if it is:  unexpected, if it contradicts the user belief about the domain and therefore surprises the user  novel, if to some extent contributes to new knowledge  actionable, if the user can take an action to his/her advantage based on this rule Unexpectedness [Suzuki, 1997] /does not depend on domain knowledge/ If r = [A  B 1 ] has a high confidence and r 1 = [A*C  B 2 ] has a high confidence, then r 1 is unexpected. [Padmanabhan & Tuzhilin] A  B is unexpected with respect to the belief    on the dataset D if the following conditions hold: B    = False [ B and  logically contradict each other] A   holds on a large subset of D  A*   B holds which means A*   

Action rules: suggest a way to re-classify objects (for instance customers) to a desired state. Action rules: suggest a way to re-classify objects (for instance customers) to a desired state. Action rules can be constructed from classification rules. Action rules can be constructed from classification rules. To discover action rules it is required that the set of conditions (attributes) is partitioned into stable and flexible. To discover action rules it is required that the set of conditions (attributes) is partitioned into stable and flexible. For example, date of birth is a stable attribute, and interest rate on any customer account is a flexible attribute (dependable on bank). For example, date of birth is a stable attribute, and interest rate on any customer account is a flexible attribute (dependable on bank). Actionable rules The notion of action rules was proposed by [Ras & Wieczorkowska, PKDD’00]. Slowinski at al [JETAI, 2004] introduced similar notion called intervention.

Decision table Any information system of the form Any information system of the form S = (U, A Fl  A St  {d}), where S = (U, A Fl  A St  {d}), where  d  A Fl  A St is a distinguished attribute called decision.  A St - stable attributes, A Fl  {d} - flexible Action rule [Ras & Wieczorkowska]: [t(A St )  (b 1, v 1  w 1 )  (b 2, v 2  w 2 )  …  (b p, v p  w p )](x) [t(A St )  (b 1, v 1  w 1 )  (b 2, v 2  w 2 )  …  (b p, v p  w p )](x)  [(d, k 1  k 2 )](x), where  [(d, k 1  k 2 )](x), where (  i)[(1  i  p)  (b i  A Fl )] Action Rules E-Action rule [Ras & Tsay]: [t(A St )  (b 1,  w 1 )  (b 2, v 2  w 2 )  …  (b p,  w p )](x)  [(d, k 1  k 2 )](x), where  [(d, k 1  k 2 )](x), where (  i)[(1  i  p)  (b i  A Fl )]

Table: Set of rules R with supporting objects Figure of (d, H)-tree T1 Figure of (d, L)-tree T2 Objectsabcd x1, x2, x3, x4 0L x1, x3 0L x2, x4 2L 1L x5, x6 3L x7, x8 21H 12H Objectsabc x1, x2, x3, x4 0 x1, x3 0 x2, x4 2 1 x5, x6 3 Objectsbc x1, x3 0 x2, x4 2 1 x5, x6 3 Objectsb x2, x4 2 x5, x6 3 c = 1c = ? c = 0Objectsbc x1, x2, x3, x4 Objectsb x1, x3 a = 0Objectsb x2, x4 a = ?Objectsabc x7, x Objectsbc 1 a = 2Objectsbc x7, x8 12 a = ? Stable Attribute: {a, c} Flexible Attribute: b Decision Attribute: d T1 T2 T3 T4 (T3, T1) : (a = 2)  (b, 2  1)  ( d, L  H) (a = 2)  (b, 3  1)  ( d, L  H)Objectsb x7, x8 1 c = ? c = 2Objectsb x7, x8 1 c = ?Objectsb x1, x2, x3, x4 T5 T6 Action Rules Discovery (Tsay & Ras)

Application domain: Customer Attrition  On average, most US corporations lose half of their customers every five years (Rombel, 2001).  Longer a customer stays with the organization, the more profitable he or she becomes (Pauline, 2000; Hanseman, 2004).  The cost of attracting new customers is five to ten times more than retaining existing ones.  About 14% to 17% of the accounts are closed for reasons that can be controlled like price or service (Lunt, 1993). Action: Reducing the outflow of the customers by 5% can double a typical company’s profit (Rombel, 2001). Facts:

Action Rules Discovery Decision table S = (U, A Fl  A St  {d}). Assumption: {a 1,a 2,...,a p }  A St, {b 1,b 2,...,b q }  A Fl, a i,1  Dom(a i ), b i,1  Dom(b i ). Rule: r = [a 1,1  a 2,1 ...  a p,1 ]  [b 1,1  b 2,1 ...  b q,1 ]  d 1 stable part flexible part Question: Do we have to consider pairs of classification rules in order to construct action rules?

Action Rules Discovery Decision table S = (U, A Fl  A St  {d}). Assumption: {a 1,a 2,...,a p }  A St, {b 1,b 2,...,b q }  A Fl, a i,1  Dom(a i ), b i,1  Dom(b i ). Rule: r = [a 1,1  a 2,1 ...  a p,1 ]  [b 1,1  b 2,1 ...  b q,1 ]  d 1 stable part flexible part Action rule r [d2  d1] associated with r and re-classification task (d, d 2  d 1 ): [a 1,1  a 2,1 ...  a p,1 ]  [(b 1,  b 1,1 )  (b 2,  b 2,1 ) ...  (b q,  b q,1 )]  (d, d 2  d 1 )

Action Rules Discovery Action rule r [d2  d1] : [a 1,1  a 2,1 ...  a p,1 ]  [(b 1,  b 1,1 )  (b 2,  b 2,1 ) ...  (b q,  b q,1 )]  (d, d 2  d 1 ) Support Sup(r [d2  d1] ) = {x  U: (a 1 (x)=a 1,1 )  (a 2 (x)=a 2,1 ) ...  (a p (x)=a p,1 )  (d(x)=d 2 )}. /d 2 -objects which potentially can be reclassified by r [d2  d1] to d 1 / Sup(R [d2  d1] ) =  {Sup(r [d2  d1] ): r  R}, where R- classification rules extracted from S. /d 2 -objects which potentially can be reclassified by r [d2  d1] to d 1 /

Action Rules Discovery Action rule r [d2  d1] : [a 1,1  a 2,1 ...  a p,1 ]  [(b 1, b’ 1,1  b 1,1 )  (b 2, b’ 2,1  b 2,1 ) ...  (b q,  b q,1 )]  (d, d 2  d 1 ) Support Sup(r [d2  d1] ) = {x  U: (b 1 (x)=b’ 1,1 )  (b 2 (x)=b’ 2,1 )  (a 1 (x)=a 1,1 )  (a 2 (x)=a 2,1 ) ...  (a p (x)=a p,1 )  (d(x)=d 2 )}. /d 2 -objects which potentially can be reclassified by r [d2  d1] to d 1 /

Let U d2 = {x  U: d(x)=d 2 }. Then B d2  d1 = U d2 - Sup(R [d2  d1] ) is a set of d 2 -objects in S which are d 1 -resistant. Action Rules Discovery Let Sup(R [  d1] ) =  {Sup(R [d2  d1] ) : d 2  d 1 }. Then B  d1 = U - Sup(R [  d1] ) is a set of objects in S which are d 1 -resistant (can not be re-classified to class d 1 ).

Action Rules Discovery Action rules r [d2  d1], r‘ [d2  d3] are p-equivalent (  ), if r/b i = r'/b i always holds when r/b i, r'/b i are both defined, for every b i  A St  A Fl. Let x  Sup(r [d2  d1] ). We say that x positively supports r [d2  d1] if there is no action rule r‘ [d2  d3] extracted from S, d 3  d 1, which is p-equivalent to r [d2  d1] and x  Sup( r‘ [d2  d3] ).

Action Rules Discovery Let Sup + (R [d2  d1] ) = {x  Sup(r [d2  d1] ): x positively supports r [d2  d1] }. Confidence Conf(r [d2  d1] ) = {card[Sup + (r [d2  d1] )]/card[Sup(r [d2  d1] )]}  Conf(r). Conf(r [  d1] ) = {card[Sup + (r [  d1] )]/card[Sup(r [  d1] )]}  Conf(r).

Assumption: S = (X, A, V) is information system, Y  X. Attribute b  A is flexible in S and b 1, b 2  V b. By  S (Y, b 1, b 2 ) we mean a number from (0, +  ] which describes the average predicted cost of approved action associated with a possible re- classification of qualifying objects in Y from class b 1 to b 2. Object x  Y qualifies for re-classification from b 1 to b 2, if b(x) = b 1.  S (Y, b 1, b 2 ) = + , if there is no action approved which is required for a possible re-classification of qualifying objects in Y from class b 1 to b 2 Cost of Action Rule [Tzacheva & Ras]  S (Y, b 1, b 2 ). If Y is uniquely defined, we often write  S (b 1, b 2 ) instead of  S (Y, b 1, b 2 ).

Cost of Action Rule Action rule : Action rule r: [(b 1, v 1 → w 1 )  (b 2, v 2 → w 2 )  …  ( b p, v p → w p )](x)  (d, k 1 → k 2 )(x) : The cost of r in S: cost S (r) =  {  S (v i, w i ) : 1  i  p} Action rule r is feasible in S, if cost S (r) <  S (k 1, k 2 ). For any feasible action rule r, the cost of the conditional part of r is lower than the cost of its decision part.

Assumption: Cost of r is too high! r = [(b 1, v 1 → w 1 )  …  (b j, v j → w j )  …  ( b p, v p → w p )](x)  (d, k 1 → k 2 )(x) r 1 = [(b j1, v j1 → w j1 )  (b j2, v j2 → w j2 )  …  ( b jq, v jq → w jq )](x)  (b j, v j → w j )(x) Then, we can compose r with r 1 and the same replace term (b j, v j → w j ) by term from the left hand side of r 1 : [(b 1, v 1 → w 1 )  …  [(b j1, v j1 → w j1 )  (b j2, v j2 → w j2 )  …  ( b jq, v jq → w jq )]  …  ( b p, v p → w p )](x)  (d, k 1 → k 2 )(x) Cost of Action Rule

Class movability-index F S - decision attribute ranking – positive integer associated with a decision value /objects of higher decision attribute ranking are seen as objects more preferably movable between decision classes than objects of lower rank/. N j + = {i  N: F S (d j ) – F S (d i )  0}. Class movability-index assigned to N j, ind(N j ) =  {F S (d j )– F S (d i ): i  N j + }Xabd FSFSFSFSx1a1b1d31 x2a2b1d22 x3a1b2d22 x4a3b2d13

Class movability-index Let P j (i) = Sup + (r [dj  di] ) /P j (i) – all objects in U which can be reclassified from the decision class d j to the decision class d i P j (N) =  {P j (i): i  N, i  j}, for any N  {1,2,…,k} where {d 1,d 2,…,d k } are all decision classes. Class movability-index (m-index) assigned to d j -object x: ind S (x) = max{ind(N j ): N j  {1,2,…,k}  x  P j (N)}

Questions? Thank You