1. Action Rules Discovery without pre-existing classification rules B y Zbigniew W. Ras 1,2 and Agnieszka Dardzinska 1,3 1)University of North Carolina, Charlotte, NC, USA 2)Polish Academy of Science, Warsaw, Poland 3)Bialystok Technical University, Bialystok, Poland

2. 1. Introduction : Action Rule An action rule is a rule extracted from a datatable that describes a possible transition of objects from one state to another with respect to a distinguished attribute called a decision attribute Information System

3. 1. Introduction : Action Rule An action rule is a rule extracted from a database that describes a possible transition of objects from one state to another with respect to a distinguished attribute called a decision attribute Information System Assumption: - attributes are partitioned into stable and flexible

4. 1. Introduction : Action Rule An action rule is a rule extracted from a database that describes a possible transition of objects from one state to another with respect to a distinguished attribute called a decision attribute Information System Assumption: - attributes are partitioned into stable and flexible Values of attributes can be changed

5. 1. Introduction : Action Rule Action rule is defined as a term Information System conjunction of fixed condition features shared by both groups proposed changes in values of flexible features desired effect of the action [(ω) ∧ (α → β)] →(ϕ→ψ)

6. 2. Background and Objectives Xabcd x1x1x1x1 a1a1a1a1 b1b1b1b1 c1c1c1c1 d1d1d1d1 x2x2x2x2 a2a2a2a2 b1b1b1b1 c2c2c2c2 d1d1d1d1 x3x3x3x3 a2a2a2a2 b2b2b2b2 c2c2c2c2 d1d1d1d1 x4x4x4x4 a2a2a2a2 b1b1b1b1 c1c1c1c1 d1d1d1d1 x5x5x5x5 a2a2a2a2 b3b3b3b3 c2c2c2c2 d1d1d1d1 x6x6x6x6 a1a1a1a1 b1b1b1b1 c2c2c2c2 d2d2d2d2 x7x7x7x7 a1a1a1a1 b2b2b2b2 c2c2c2c2 d1d1d1d1 x8x8x8x8 a1a1a1a1 b2b2b2b2 c1c1c1c1 d3d3d3d3 Information System S = (X,A,V) X = {x 1, x 2, x 3, x 4, x 5, x 6, x 7, x 8 } A = {a, b, c, d} X is a set of objects A is a set of attributes (partial functions: X  V) V a is a set of values of attribute a, where a  A, and V =  {V a : a  A}, Information System S

7. Decision Attribute : {d} Stable Attributes : A st = {a} Flexible Attributes : A fl = {b,c} Constraints: Minimum Support 1 Minimum Confidence 2 3. ARED Algorithm (Stable and Flexible Att) Xabcd x1x1x1x1 a1a1a1a1 b1b1b1b1 c1c1c1c1 d1d1d1d1 x2x2x2x2 a2a2a2a2 b1b1b1b1 c2c2c2c2 d1d1d1d1 x3x3x3x3 a2a2a2a2 c2c2c2c2 d1d1d1d1 x4x4x4x4 a2a2a2a2 b1b1b1b1 c1c1c1c1 d1d1d1d1 x5x5x5x5 a2a2a2a2 b3b3b3b3 c2c2c2c2 d1d1d1d1 x6x6x6x6 a1a1a1a1 b1b1b1b1 d2d2d2d2 x7x7x7x7 a1a1a1a1 b2b2b2b2 c2c2c2c2 d1d1d1d1 x8x8x8x8 a1a1a1a1 b2b2b2b2 c1c1c1c1 d3d3d3d3 Decision System

8. 2. Background and Objectives Action rule generation in earlier work used existing classification rules (generated, for instance, by rule discovery algorithms such as LERS or ERID) Proposed Algorithm 1. Extract action rules directly from an information system without using pre-existing classification rules. 2. Extract action rules that have minimal attribute involvement.

9. Set of rules R with supporting objects Objectsabcd x1, x2, x3, x40L x1, x30L x2, x42L 1L x5, x63L x7, x821H 12H Objectsabc x1, x2, x3, x40 x1, x30 x2, x42 1 x5, x63 Objectsbc x1, x30 x2, x42 1 x5, x63 Objectsb x2, x42 x5, x63 c = ? c = 0 Objectsbc x1, x2, x3, x4 Objectsb x1, x3 a = 0 Objectsb x2, x4 a = ? Objectsabc x7, x821 12 Objectsbc x7, x81 a = 2 Objectsbc x7, x812 a = ? Stable Attributes: {a, c} Flexible Attribute: b Decision Attribute: d T1 T2 T3 T4 (T3, T1) : (a = 2)  (b, 2  1)  ( d, L  H) (a = 2)  (b, 3  1)  ( d, L  H) Objectsb x7, x81 c = ? c = 2 Objectsb x7, x81 c = ? Objectsb x1, x2, x3, x4 T5 T6 c = 1 Action Rules Discovery System DEAR

10. 3. Action Rules Atomic Action Term:(a, a 1 → a 2 ) attribute a 1, a 2 ∈ V a If a 1 = a 2 then a – stable on a 1

11. 3. Action Rules Atomic Action Term:(a, a 1 → a 2 ) attribute a 1, a 2 ∈ V a If a 1 = a 2 then a – stable on a 1 Set of Action Terms: 1.If t – atomic action term, then t – an action term 2.If t 1, t 2 – action terms, then t 1 *t 2 – an action term 3.If t – action term containing sub-terms (a, a 1 → a 2 ), (b, b 1 → b 2 ), then a ≠ b Domain of Action Term t:Dom(t) – set of all attribute names listed in t

12. 3. Action Rules Atomic Action Term:(a, a 1 → a 2 ) attribute a 1, a 2 ∈ V a If a 1 = a 2 then a – stable on a 1 Action Rule: r = [t 1 → t 2 ] Dom(t 2 )={d} and Dom(t 1 ) in A action termatomic action term

13. 3. Example Xabcd x1x1x1x1 a1a1a1a1 b1b1b1b1 c1c1c1c1 d1d1d1d1 x2x2x2x2 a2a2a2a2 b1b1b1b1 c2c2c2c2 d1d1d1d1 x3x3x3x3 a2a2a2a2 b2b2b2b2 c2c2c2c2 d1d1d1d1 x4x4x4x4 a2a2a2a2 b1b1b1b1 c1c1c1c1 d1d1d1d1 x5x5x5x5 a2a2a2a2 b3b3b3b3 c2c2c2c2 d1d1d1d1 x6x6x6x6 a1a1a1a1 b1b1b1b1 c2c2c2c2 d2d2d2d2 x7x7x7x7 a1a1a1a1 b2b2b2b2 c2c2c2c2 d1d1d1d1 x8x8x8x8 a1a1a1a1 b2b2b2b2 c1c1c1c1 d3d3d3d3 (a, a2 → a2) (b, b1 → b3) (c, c2 → c2) (d, d1 → d2) Decision System S atomic action terms r=[[(a, a2 → a2)*(b, b1 → b3)] → (d, d1 → d2) action rule Dom (r) ={a, b, d}

14. 4. Interpretation N S of action terms 1. If (a, a1 → a2) – atomic action term then: N S ((a, a1 → a2 ))=[{x ∈ X:a(x)=a1}, { x ∈ X:a(x)=a2 }] 2. If t1=(a, a1 → a2) * t and N S (t)=[Y1, Y2], then: N S (t1)=[ Y1 ∩ {x ∈ X:a(x)=a1}, Y2 ∩ {x ∈ X:a(x)=a2} ] Information System S=(X, A, V) Assume that [Y1, Y2] ∩[Z1, Z2] = [Y1 ∩ Z1, Y2 ∩ Z2], and N S (t1)=[Y1, Y2],N S (t2)=[Z1, Z2], then:N S (t1*t2)=N S (t1) ∩N S (t2)

15. 4. Interpretation N S of action terms Information System S=(X, A, V) N S (t1)=[Y1, Y2] N S (t2)=[Z1, Z2] Action Rule: r = [t 1 → t 2 ] Support: Confidence:

16. Xabcd x1x1x1x1 a1a1a1a1 b1b1b1b1 c1c1c1c1 d1d1d1d1 x2x2x2x2 a2a2a2a2 b1b1b1b1 c2c2c2c2 d1d1d1d1 x3x3x3x3 a2a2a2a2 b2b2b2b2 c2c2c2c2 d1d1d1d1 x4x4x4x4 a2a2a2a2 b1b1b1b1 c1c1c1c1 d1d1d1d1 x5x5x5x5 a2a2a2a2 b3b3b3b3 c2c2c2c2 d1d1d1d1 x6x6x6x6 a1a1a1a1 b1b1b1b1 c2c2c2c2 d2d2d2d2 x7x7x7x7 a1a1a1a1 b2b2b2b2 c2c2c2c2 d1d1d1d1 x8x8x8x8 a1a1a1a1 b2b2b2b2 c1c1c1c1 d3d3d3d3 Information System S r = [[(a, a2 → a2)*(b, b1 → b2)] → (d, d1 → d2) N S ((a, a2 → a2))=[{x 2, x 3, x 4, x 5 }, {x 2, x 3, x 4, x 5 }] N S ((b, b1 → b2))=[{x 1, x 2, x 4, x 6 }, {x 3, x 7, x 8 }] N S ((d, d1 → d2))=[{x 1, x 2, x 3, x 4, x 5, x 7 }, {x 6 }]

17. Xabcd x1x1x1x1 a1a1a1a1 b1b1b1b1 c1c1c1c1 d1d1d1d1 x2x2x2x2 a2a2a2a2 b1b1b1b1 c2c2c2c2 d1d1d1d1 x3x3x3x3 a2a2a2a2 b2b2b2b2 c2c2c2c2 d1d1d1d1 x4x4x4x4 a2a2a2a2 b1b1b1b1 c1c1c1c1 d1d1d1d1 x5x5x5x5 a2a2a2a2 b3b3b3b3 c2c2c2c2 d1d1d1d1 x6x6x6x6 a1a1a1a1 b1b1b1b1 c2c2c2c2 d2d2d2d2 x7x7x7x7 a1a1a1a1 b2b2b2b2 c2c2c2c2 d1d1d1d1 x8x8x8x8 a1a1a1a1 b2b2b2b2 c1c1c1c1 d3d3d3d3 Information System S r = [[(a, a2 → a2)*(b, b1 → b2)] → (d, d1 → d2) N S ((a, a2 → a2))=[{x 2, x 3, x 4, x 5 }, {x 2, x 3, x 4, x 5 }] N S ((b, b1 → b2))=[{x 1, x 2, x 4, x 6 }, {x 3, x 7, x 8 }] N S ((d, d1 → d2))=[{x 1, x 2, x 3, x 4, x 5, x 7 }, {x 6 }] N S ((a, a2 → a2)*(b, b1 → b2))=[{x 2, x 4 }, {x 3 }] sup (r) = 2, conf (r) = 0

18. 5. Algorithm ARED -Similar to ERID and LERS -Only positive marks yield action rules -Action terms of length k are built from unmarked action terms of length k – 1 and unmarked atomic action terms of length 1 S=(X, A ∪ {d}, V ) – decision system λ 1, λ 2 – minimum support and confidence C S (a)={N S (t a ): t a – an atomic action term built from elements in V a } a ∈ A

19. 5. ARED Algorithm For decision attribute in S: N S (d,d 1  d 2 )=[{x 1,x 2, x 3, x 4, x 5, x 7 }, {x 6 }] Xabcd x1x1x1x1 a1a1a1a1 b1b1b1b1 c1c1c1c1 d1d1d1d1 x2x2x2x2 a2a2a2a2 b1b1b1b1 c2c2c2c2 d1d1d1d1 x3x3x3x3 a2a2a2a2 b2b2b2b2 c2c2c2c2 d1d1d1d1 x4x4x4x4 a2a2a2a2 b1b1b1b1 c1c1c1c1 d1d1d1d1 x5x5x5x5 a2a2a2a2 b3b3b3b3 c2c2c2c2 d1d1d1d1 x6x6x6x6 a1a1a1a1 b1b1b1b1 c2c2c2c2 d2d2d2d2 x7x7x7x7 a1a1a1a1 b2b2b2b2 c2c2c2c2 d1d1d1d1 x8x8x8x8 a1a1a1a1 b2b2b2b2 c1c1c1c1 d3d3d3d3 stable attribute flexible attributes Object reclassification from class d 1 to d 2 λ1=2, λ2=1/4 Notation: t 1 =(b,b 1  b 1 ), t 2 =(b,b 2  b 2 ), t 3 =(b,b 3  b 3 ), t 4 =(a,a 1  a 2 ), t 5 =(a,a 1  a 1 ), t 6 =(a,a 2  a 2 ), t 7 =(a,a 2  a 1 ), t 8 =(c,c 1  c 2 ), t 9 =(c,c 2  c 1 ), t 10 =(c,c 1  c 1 ), t 11 =(c,c 2  c 2 ), t 12 = (d,d 1  d 2 ),

20. 5. ARED Algorithm For decision attribute in S: N S (d,d1  d2)=[{x 1,x 2, x 3, x 4, x 5, x 7 }, {x 6 }] Object reclassification from class d1 to d2λ1=2, λ2=1/4 For classification attribute in S: N S (t 1 )=[{x 1,x 2, x 4, x 6 }, {x 1,x 2, x 4, x 6 }] Not marked λ1=3 N S (t 2 )=[{x 3,x 7, x 8 }, {x 3,x 7, x 8 }] Mark “-” λ1=1 N S (t 3 )=[{x 5 }, {x 5 }] Mark “-” λ1=1 N S (t 4 )=[{x 1,x 6, x 7, x 8 }, {x 2,x 3, x 4, x 5 }] Mark “-” λ2=0 N S (t 5 )=[{x 1,x 6, x 7, x 8 }, {x 1,x 6, x 7, x 8 }] Not marked λ1=4 N S (t 6 )=[{x 2,x 3, x 4, x 5 }, {x 2,x 3, x 4, x 5 }] Mark “-” λ2=0 Mark “+” λ1=4,λ2=1/4 N S (t 7 )=[{x 2,x 3, x 4, x 5 }, {x 1,x 6, x 7, x 8 }]

21. 5. ARED Algorithm For decision attribute in S: N S (t 12 )=[{x 1,x 2, x 3, x 4, x 5, x 7 }, {x 6 }] Object reclassification from class d1 to d2λ1=2, λ2=1/4 For classification attribute in S: N S (t 1 )=[{x 1,x 2, x 4, x 6 }, {x 1,x 2, x 4, x 6 }] Not marked λ1=3 N S (t 2 )=[{x 3,x 7, x 8 }, {x 3,x 7, x 8 }] Marked “-” λ2=0 N S (t 3 )=[{x 5 }, {x 5 }] Marked “-” λ1=1 N S (t 4 )=[{x 1,x 6, x 7, x 8 }, {x 2,x 3, x 4, x 5 }] Marked “-” λ2=0 N S (t 5 )=[{x 1,x 6, x 7, x 8 }, {x 1,x 6, x 7, x 8 }] Not marked λ2=2 N S (t 6 )=[{x 2,x 3, x 4, x 5 }, {x 2,x 3, x 4, x 5 }] Marked “-” λ2=0 Mark “+” λ1=4, λ2=1/4 N S (t 7 )=[{x 2,x 3, x 4, x 5 }, {x 1,x 6, x 7, x 8 }] r = [t 7  t 1 ]

22. 5. ARED Algorithm For decision attribute in S: N S (t 12 )=[{x 1,x 2, x 3, x 4, x 5, x 7 }, {x 6 }] Object reclassification from class d1 to d2λ1=2, λ2=1/4 For classification attribute in S: N S (t 8 )=[{x 1,x 4, x 8 }, {x 2, x 3, x 5, x 6, x 7 }] Not marked Marked “-” N S (t 10 )=[{x 1, x 4, x 8 }, {x 1, x 4, x 8 }] Marked “-” N S (t 11 )=[{x 2, x 3, x 5, x 6, x 7 }, {x 2, x 3, x 5, x 6, x 7 }] Not marked Rule r = [t 8 →t 12 ], conf = 2/3 *1/5 <λ 2, sup=2 ≥ λ 1 N S (t 9 )=[{x 2, x 3, x 5, x 6, x 7 }, {x 1, x 4, x 8 }]

23. 5. ARED Algorithm For decision attribute in S: N S (t 12 )=[{x 1,x 2, x 3, x 4, x 5, x 7 }, {x 6 }] Object reclassification from class d1 to d2λ1=2, λ2=1/4 For classification attribute in S: Marked “+” N S (t 1 *t 11 )=[{x 2, x 6 }, {x 2, x 6 }] Not marked N S (t 5 *t 8 )=[{x 1, x 8 }, {x 6, x 7 }] Marked “-” Rule r = [t 1 *t 8 →t 12 ], conf = 1/2 ≥ λ 2, sup=2 ≥ λ 1 Now action terms of length 2 from unmarked action terms of length 1 N S (t 1 *t 5 )=[{x 1, x 6 }, {x 1, x 6 }] Not marked N S (t 1 *t 8 )=[{x 1, x 4 }, {x 2, x 6 }] Rule r = [t 5 *t 8 →t 12 ], conf = 1/2 ≥ λ 2, sup= 1 < λ 1 N S (t 5 *t 11 )=[{x 6, x 7 }, {x 6, x 7 }] Not marked N S (t 8 *t 11 )=[ Ø, {x 2, x 3, x 5, x 6, x 7 }] Marked “-”

24. 5. ARED Algorithm For decision attribute in S: N S (t 12 )=[{x 1,x 2, x 3, x 4, x 5, x 7 }, {x 6 }] Object reclassification from class d1 to d2λ1=2, λ2=1/4 For classification attribute in S: Now action terms of length 3 from unmarked action terms of length 2 (t 1 *t 5 *t 8 ) –extension of (t 5 *t 8 ) Marked “-” Algorithm STOPS!

25. 5. ARED Algorithm For decision attribute in S: N S (t 12 )=[{x 1,x 2, x 3, x 4, x 5, x 7 }, {x 6 }] Object reclassification from class d1 to d2λ1=2, λ2=1/4 For classification attribute in S: Action rules: [[(b,b1 →b1 )*(c,c1 → c2)] → (d, d1→d2) ] [[(a,a2 →a1 ] → (d, d1→d2) ] Xabcd x1x1x1x1 a1a1a1a1 b1b1b1b1 c1c1c1c1 d1d1d1d1 x2x2x2x2 a2a2a2a2 b1b1b1b1 c2c2c2c2 d1d1d1d1 x3x3x3x3 a2a2a2a2 b2b2b2b2 c2c2c2c2 d1d1d1d1 x4x4x4x4 a2a2a2a2 b1b1b1b1 c1c1c1c1 d1d1d1d1 x5x5x5x5 a2a2a2a2 b3b3b3b3 c2c2c2c2 d1d1d1d1 x6x6x6x6 a1a1a1a1 b1b1b1b1 c2c2c2c2 d2d2d2d2 x7x7x7x7 a1a1a1a1 b2b2b2b2 c2c2c2c2 d1d1d1d1 x8x8x8x8 a1a1a1a1 b2b2b2b2 c1c1c1c1 d3d3d3d3

26. 6. Application Database HEPAR (Medical Center of Postgraduate Education, Warsaw, Poland) prepared by Dr. med. Hanna Wasyluk  758 patients described by 106 attributes (including 31 laboratory tests with values discretized to: “below normal”, “normal”, “above normal”). 14 attributes are stable. Two tests are invasive tests:  758 patients described by 106 attributes (including 31 laboratory tests with values discretized to: “below normal”, “normal”, “above normal”). 14 attributes are stable. Two tests are invasive tests: HBsAg [in tissue], HBcAg [in tissue]. There are 7 decision values: I. Acute hepatitis IIa. IIa. Subacute hepatitis (types BC) IIb. IIb. Subacute hepatitis (alcohol-abuse) IIIa. IIIa.Chronic hepatitis (curable) IIIb. IIIb.Chronic hepatitis (non-curable) IV. IV.Cirrhosis hepatitis V. V.Liver cancer We can ask for re-classifications: IIb  I, IIIa  I

27. 6. Application Database HEPAR (heavily incomplete) Handling data incompleteness  Attributes with more than 90% of null values have been removed  Subjective attributes have been removed  [example:  [example: History of alcohol abuse]  Removal of absolute medical tests  RSES null value imputation module was used to make the resulting dataset complete.

28. 6. Application Chosen reduct has no invasive tests, 11% incomplete data {m, n, q, u, y, aa, ah, ai, am, an, aw, bb, bg, by, cj, cm} m Bleeding nSubjaundice symptoms qEructation uObstruction yWeight loss aaSmoking ahHistory of viral hepatitis (stable) aiSurgeries in the past (stable) amHistory of hospitalization (stable) anJaundice in pregnancy awErythematous dermatitis bbCysts bgSharp liver edge (stable) bmBlood cell plaque byAlkaline phosphatase cjProthrombin index cmTotal cholesterol ddDecision attribute

29. 6. Application Action Rules discovery in Database HEPAR but with history of s, s, no s, t, e w  No history of viral hepatitis but with history of surgery and hospitalization, sharp liver edge normal, no subjaundice symptoms, total cholesterol normal, erythematous dermatitis normal, weight normal, no cysts, patient does not smoke  [(ah=1)  (ai=2)  (am=2)  (bg=1)]  (cm=1)  (aw=1)  (u,  1)  (bb=1)  (aa=1)   (q,  1)  (m,  1)  (n=1)  (bm,  down)  (y=1)  (by, norm  up)   (dd, IIIa  I )  [(ah=1)  (ai=2)  (am=2)  (bg=1)]  (cm=1)  (aw=1)  (u,  1)  (bb=1)  (aa=1)   (q,  1)  (m,  1)  (n=1)  (bm,  down)  (y=1)  (by, norm  down)  (dd, IIIa  I )  Two quite similar action rules have been constructed: By getting rid of obstruction, eructation, bleeding, by decreasing the b and by changing the level of a we should be able reclassify the patient from IIIa group to I. By getting rid of obstruction, eructation, bleeding, by decreasing the blood cell plaque and by changing the level of alkaline phosphatase we should be able reclassify the patient from IIIa group to I. Medical doctor should decide for a given patient if level needs to be decreased or increased. Medical doctor should decide for a given patient if the alkaline phosphatase level needs to be decreased or increased.  Attribute values of twhave to stay unchanged.  Attribute values of total cholesterol, weight, and smoking have to stay unchanged.

30. Presented algorithm discovers action rules directly from a decision table. The proposed algorithm generates the shortest action rules without using pre-existing classification rules Future work includes - Action rule generation from incomplete information system - Action rule generation when attributes are hierarchical - Considering different flexibilities of attributes (e.g. the social condition is most likely less flexible than the health condition ) 7. Conclusion and Future Work

31. Thank You Questions?