1. © Jesse Davis 2006 View Learning Extended: Learning New Tables Jesse Davis 1, Elizabeth Burnside 1, David Page 1, Vítor Santos Costa 2 1 University of Wisconsin-Madison USA 2 Federal University of Rio de Janeiro Brasil

2. © Jesse Davis 2006 1 P1 5/02 No 0.03 RU4 B 2 P1 5/04 Yes 0.05 RU4 M 3 P1 5/04 No 0.04 LL3 B 4 P2 6/00 No 0.02 RL2 B … … … … … … … Abnormality Patient Date Calcification … Mass Loc Benign/ Fine/Linear Size Malignant View Learning Framework [Davis et al. IJCAI05] Learn fields predictive of target concept

3. © Jesse Davis 2006 1 P1 5/02 No 0.03 No RU4 B 2 P1 5/04 Yes 0.05 Yes RU4 M 3 P1 5/04 No 0.04 No LL3 B 4 P2 6/00 No 0.02 No RL2 B … … … … … … … … Abnormality Patient Date Calcification … Mass Increase Loc Benign/ Fine/Linear Size in size Malignant Extend Schema Increase In Size No Yes No …

4. © Jesse Davis 2006 Integrated Search for New Fields [Landwehr et al. AAAI 2005, Davis et al. ECML 2005] Old approach: Old approach: Step 1 use ILP to learn new fields Step 1 use ILP to learn new fields Step 2 learn statistical model Step 2 learn statistical model Score As You Use (SAYU): Score As You Use (SAYU): Combine steps 1 and 2 Combine steps 1 and 2 Score new field by how much it helps statistical model Score new field by how much it helps statistical model Parallel development: nFOIL Parallel development: nFOIL

5. © Jesse Davis 2006 Relevant Intermediate Concepts Advisedby(Student,Professor) ta_for(Student,Professor) ta(Student,Class)teach(Professor,Class) coauthor(Person,Person)paper(Person,Ref) Automatically generate and incorporate intermediate concepts Goal: Automatically generate and incorporate intermediate concepts

6. © Jesse Davis 2006 Limitations to Our Old Work Previously View Learning adds new fields Previously View Learning adds new fields More expressive to learn predicates More expressive to learn predicates not approximations to target concept not approximations to target concept represent new tables represent new tables Solution: Extend SAYU Solution: Extend SAYU

7. © Jesse Davis 2006 VISTA Algorithm View Invention through Scoring Tables with Aggregation

8. © Jesse Davis 2006 Distinguished types [id, patient, visit] Algorithm Illustration p1(id,id) p1/2 Rule 14Rule N Class Value … Score = 0.020.120.100.150.35 Rule 1Rule 2Rule 3 p2/1 p2(patient) :-sameStudy(Id1,Id2):-historyOfBC(Patient):-hadBiopsy(_,Patient) Background Knowledge

9. © Jesse Davis 2006 Algorithm Details Learn predicates with Learn predicates with Target predicate arity Target predicate arity Target predicate arity + 1 Target predicate arity + 1 Moded language Moded language Breadth first search over clause bodies Breadth first search over clause bodies

10. © Jesse Davis 2006 Count Aggregation 1 2 3 4 5 6 … Id Count density_increase(A,B) :- density(A,D1), prior_mammogram_same_loc(A,B), density(B,D2), D1 > D2. prior_mammogram_same_loc(A,B), density(B,D2), D1 > D2. 0 1 0 1 2 … Count 1 P1 5/02 low RU4 B 2 P1 5/04 high RU4 M 3 P1 5/04 none LL3 B 4 P2 6/00 none RL2 B 5 P2 6/02 low RL2 B 6 P2 9/03 high RL2 M … … … … … … Id Patient Date … Mass Loc Benign/ Density Malignant

11. © Jesse Davis 2006 Linkage Distinguished variable may not correspond to example key Distinguished variable may not correspond to example key p1(Patient) :- historyOfBC(Patient), hadBiopsy(Patient). historyOfBC(Patient), hadBiopsy(Patient). Above rule adds a field to Patient table Above rule adds a field to Patient table Q: How do we score p1?

12. © Jesse Davis 2006 Linkage Example P1 No P2 Yes P3 No P4 No P5 Yes P6 No … Patient Family History p1(Patient) :- historyOfBC(Patient), hadBiopsy(_,Patient). 1 P1 5/02 low RU4 B 2 P1 5/04 high RU4 M 3 P1 5/04 none LL3 B 4 P2 6/00 none RL2 B 5 P2 6/02 low RL2 B 6 P2 9/03 high RL2 M … … … … … … Id Patient Date … Mass Loc Benign/ Density Malignant

13. © Jesse Davis 2006 1 P1 5/02 low RU4 B 2 P1 5/04 high RU4 M 3 P1 5/04 none LL3 B 4 P2 6/00 none RL2 B 5 P2 6/02 low RL2 B 6 P2 9/03 high RL2 M … … … … … … Id Patient Date … Mass Loc Benign/ Density Malignant P1 No No P2 Yes Yes P3 No No P4 No No P5 Yes No P6 No No … … … Patient Family p1 History Linkage Example p1(Patient) :- historyOfBC(Patient), hadBiopsy(_,Patient).

14. © Jesse Davis 2006 Linkage Example p1(Patient) :- historyOfBC(Patient), hadBiopsy(_,Patient). 1 P1 5/02 low RU4 B No 2 P1 5/04 high RU4 M No 3 P1 5/04 none LL3 B No 4 P2 6/00 none RL2 B Yes 5 P3 6/02 low RL2 B Yes 6 P4 9/03 high RL2 M Yes … … … … … … … Id Patient Date … Mass Loc Benign/ p1 Density Malignant

15. © Jesse Davis 2006 New Features in VISTA User declares set of distinguished types that appear in clause head User declares set of distinguished types that appear in clause head Allow reuse of learned predicate Allow reuse of learned predicate Count aggregation Count aggregation Linkage permits learning predicates with: Linkage permits learning predicates with: Higher arity than target (new tables) Higher arity than target (new tables) Different types than target Different types than target

16. © Jesse Davis 2006 Experiment Q: Does VISTA or SAYU perform better?

17. © Jesse Davis 2006 Datasets Cora (5 x 2 fold cross validation) [McCallum et al. 00, Kok & Domingos 05] Cora (5 x 2 fold cross validation) [McCallum et al. 00, Kok & Domingos 05] UW-CSE (5 fold cross validation) [Richardson & Domingos 04] UW-CSE (5 fold cross validation) [Richardson & Domingos 04] Mammography (10 fold cross validation) [Davis et al. 05] Mammography (10 fold cross validation) [Davis et al. 05]

18. © Jesse Davis 2006 Area Under Precision-Recall Curve Generate whole PR Curve Generate whole PR Curve Area Under PR for Recall > 0.5 Area Under PR for Recall > 0.5 Precision Recall

19. © Jesse Davis 2006

22. MLN data from Singla & Domingos AAAI 2005

23. © Jesse Davis 2006 Related Topic: Predicate Invention Cigol: Muggleton & Buntine (1988) Cigol: Muggleton & Buntine (1988) CHILLIN: Zelle & Mooney (1994) CHILLIN: Zelle & Mooney (1994) FOIL-PILFS: Craven & Slattery (2001) FOIL-PILFS: Craven & Slattery (2001) SLR: Popescul & Ungar (2004) SLR: Popescul & Ungar (2004)

24. © Jesse Davis 2006 Related Work: Feature Construction Pompe & Kononenko, ILP’95 Pompe & Kononenko, ILP’95 Srinivasan & King, ILP’97 Srinivasan & King, ILP’97 Perlich & Provost, KDD’03 Perlich & Provost, KDD’03 Knobbe, de Haas & Siebes, PKDD’01 Knobbe, de Haas & Siebes, PKDD’01

25. © Jesse Davis 2006 Future Work Further investigate benefits of VISTA Further investigate benefits of VISTA Linkage as jumping deeper into search space Linkage as jumping deeper into search space Reuse of predicates Reuse of predicates Extensions to VISTA Extensions to VISTA Negation Negation Disjunction Disjunction Stochastic search Stochastic search Comparisons to other SRL systems Comparisons to other SRL systems

26. © Jesse Davis 2006 Conclusions VISTA adds capabilities VISTA adds capabilities Add fields to tables other than target relation Add fields to tables other than target relation Learn new relations Learn new relations VISTA empirically VISTA empirically Better Cora (p-value < 0.001) Better Cora (p-value < 0.001) Almost better on UW-CSE (p-value < 0.06) Almost better on UW-CSE (p-value < 0.06) No worse on Mammography (p-value < 0.94) No worse on Mammography (p-value < 0.94)

27. © Jesse Davis 2006 Acknowledgements Mark Craven Mark Craven Jude Shavlik Jude Shavlik Inês Dutra Inês Dutra Mark Goadrich Mark Goadrich Irene Ong Irene Ong Trevor Walker Trevor Walker Raghu Ramakrishnan Raghu Ramakrishnan Rich Maclin Rich Maclin Lisa Torrey Lisa Torrey Jan Struyf Jan Struyf Allison Holloway Allison Holloway This work was partially supported by Air Force grant F30602-01-2-0571

28. © Jesse Davis 2006 Thank You! Questions? Questions?