ICML-Tutorial, Banff, Canada, 2004 Kristian Kersting University of Freiburg Germany „Application of Probabilistic ILP II“, FP James Cussens University of York UK Probabilistic Logic Learning Probability LogicLearning al and Relational

ICML-Tutorial, Banff, Canada, 2004 Special thanks for discussions, materials, and collaborations to Alexandru Cocura, Luc De Raedt, Uwe Dick, Pedro Domingos, Thomas Gaertner, Lise Getoor, Martin Guetlein, Bernd Gutmann, Manfred Jaeger, Stephen Muggleton,Tapani Raiko, Reimund Renner, Richard Schmidt, Ingo Thon

ICML-Tutorial, Banff, Canada, 2004 Tutorial´s Aims Introductory survey Identification of important probabilistic, relational/logical and learning concepts

ICML-Tutorial, Banff, Canada, 2004 The integration of probabilistic reasoning with Objectives One of the key open questions of AI concerns Probabilistic Logic Learning: machine learning. first order / relational logic representations and Probabilitiy Learning Logic

ICML-Tutorial, Banff, Canada, 2004 Diagnosis Prediction Classification Decision-making Description Medicine Computational Biology Robotics Web Mining PLMs Economic Text Classification Computer troubleshooting Why do we need PLL? Let‘s look at an example

ICML-Tutorial, Banff, Canada, 2004 Web Mining / Linked Bibliographic Data / Recommendation Systems / … B1 B2 B3 B4 P1 books authors publishers Real World P2 A2 A1 [illustration inspired by Lise Getoor]

ICML-Tutorial, Banff, Canada, 2004 Web Mining / Linked Bibliographic Data / Recommendation Systems / … B1 B2 B3 B4 P1 books authors publishers series author-of publisher-of Real World Fantasy Science Fiction P2 A2 A1

ICML-Tutorial, Banff, Canada, 2004 Why do we need PLL? Real World Applications Let‘s look at some more examples Structured Domains Not flat but rich representations: Multi-relational, heterogeneous and semi-structured Uncertainty Dealing with noisy data, missing data and hidden variables Machine Learning Knowledge Acquisition Bottleneck, Data cheap

ICML-Tutorial, Banff, Canada, 2004 Blood Type / Genetics/ Breeding CEPH Genotype DB, AA Aa AA Aa aa Aa aa Aa AA Aa aa AA aa Aa Prior

ICML-Tutorial, Banff, Canada, 2004 Bongard´s Problems Noise? Opaque? (partially observable)

ICML-Tutorial, Banff, Canada, 2004 Bongard´s Problems Noise? Clustering? Opaque? (partially observable)

ICML-Tutorial, Banff, Canada, 2004 Others Protein Secondary Structure Metabolic Pathways Phylogenetic Trees Scene interpretation Social Networks Data Cleaning ?

ICML-Tutorial, Banff, Canada, 2004 Why do we need PLL ? Real World Applications Uncertainty Machine Learning Structured Domains Statistical Learning (SL) Probabilistic Logics Inductive Logic Programming (ILP) Multi-Relational Data Mining (MRDM) - attribute-value representations: some learning problems cannot (elegantly) be described using attribute value representations + soft reasoning, learning - no learning: to expensive to handcraft models + soft reasoning, expressivity - crisp reasoning: some learning problems cannot (elegantly) be described wihtout explicit handling of uncertainty + expressivity, learning

ICML-Tutorial, Banff, Canada, 2004 Why do we need PLL? Rich Probabilistic Models Comprehensibility Generalization (similar situations/individuals) Knowledge sharing Parameter Reduction / Compression Learning –Reuse of experience (training one RV might improve prediction at other RV) –More robust –Speed-up

ICML-Tutorial, Banff, Canada, 2004 Why Learning ? Knowledge acquisition bottleneck / data is cheap General purpose systems Combining domain expert knowledge with data Logical structure provides insight into domain Handling missing data bt(luc)=? Database Model Learning Algorithm

ICML-Tutorial, Banff, Canada, 2004 When to apply PLL ? When it is impossible to elegantly represent your problem in attribute value form –variable number of ‘objects’ in examples –relations among objects are important Background knowledge can be defined intensionally : –define ‘benzene rings’ as view predicates

ICML-Tutorial, Banff, Canada, 2004 LAPD: Bruynooghe Vennekens,Verbaeten Markov Logic: Domingos, Richardson CLP(BN): Cussens,Page, Qazi,Santos Costa (Incomplete) Historical Sketch 2003 Present PRMs: Friedman,Getoor,Koller, Pfeffer,Segal,Taskar ´03 ´96 SLPs: Cussens,Muggleton ´90´95 First KBMC approaches: Bresse, Bacchus, Charniak, Glesner, Goldman, Koller, Poole, Wellmann ´00 BLPs: Kersting, De Raedt RMMs: Anderson,Domingos, Weld LOHMMs: De Raedt, Kersting, Raiko [names in alphabetical order] many more... Future Prob. CLP: Eisele, Riezler ´02 PRISM: Kameya, Sato ´94 PLP: Haddawy, Ngo ´97 ´93 Prob. Horn Abduction: Poole ´99 1BC(2): Flach, Lachiche Logical Bayesian Networks: Blockeel,Bruynooghe, Fierens,Ramon,

ICML-Tutorial, Banff, Canada, 2004 Overview 1.Introduction to PLL 2.Foundations of PLL –Logic Programming, Bayesian Networks, Hidden Markov Models, Stochastic Grammars 3.Frameworks of PLL –Independent Choice Logic,Stochastic Logic Programs, PRISM,Probabilistic Logic –Programs,Probabilistic Relational Models, Bayesian Logic Programs –Relational Hidden Markov Models 4.Applications