1. APRIL, Application of Probabilistic Inductive Logic Programming, IST-2001-33053 Albert-Ludwigs-University, Freiburg, Germany & Imperial College of Science, Technology and Medicine, London, Great Britain. APRIL Albert-Ludwigs University Freiburg, Germany Applications of Probabilistic Inductive Logic Programming Imperial College of Science, Technology and Medicine, London, Great Britain

2. APRIL, Application of Probabilistic Inductive Logic Programming, IST-2001-33053 Albert-Ludwigs-University, Freiburg, Germany & Imperial College of Science, Technology and Medicine, London, Great Britain. Abstract The project adresses „probabilistic logic learning“ i.e. the integration of probabilistic reasoning with first order logic representations and machine learning. The objective is to critically assess the promise of this approach using application of functional genomics.

3. APRIL, Application of Probabilistic Inductive Logic Programming, IST-2001-33053 Albert-Ludwigs-University, Freiburg, Germany & Imperial College of Science, Technology and Medicine, London, Great Britain. Context Real-world applications Uncertainty Complex, structured domains Logic: objects, relations, functors Probability theory Hidden Markov Models, Stochastic Context-Free Grammars, Bayesian networks,.... Logic Programs (Prolog) „Probabilistic Logic“ Learning „Probabilistic Logic Learning“ ?

4. APRIL, Application of Probabilistic Inductive Logic Programming, IST-2001-33053 Albert-Ludwigs-University, Freiburg, Germany & Imperial College of Science, Technology and Medicine, London, Great Britain. Objectives (1) One of the key open questions of artificial intelligence concerns "probabilisitic logic learning", Logic Learning i.e. the integration of probabilistic reasoning with first order logic representations and machine learning. Probabilitstic

5. APRIL, Application of Probabilistic Inductive Logic Programming, IST-2001-33053 Albert-Ludwigs-University, Freiburg, Germany & Imperial College of Science, Technology and Medicine, London, Great Britain. Learning What is „Probabilistic Logic Learning? Representations and reasoning mechanisms grounded in probability theory, e.g. HMMs, Bayesian networs, stochastics grammars... Successfully used in a wide range of applications such as computational biology, speech recognition,... Robust models Logic Probabilistic - Probabilistic

6. APRIL, Application of Probabilistic Inductive Logic Programming, IST-2001-33053 Albert-Ludwigs-University, Freiburg, Germany & Imperial College of Science, Technology and Medicine, London, Great Britain. First-order logic Elegant representation of complex situations involving a variety of objects as well as reltions among these objects: Learning Probabilistic What is „Probabilistic Logic Learning? - Logic Logic bloodtype(X,a) <- mother(M,X),bloodtype(M,a), father(F,X),bloodtype(F,a).

7. APRIL, Application of Probabilistic Inductive Logic Programming, IST-2001-33053 Albert-Ludwigs-University, Freiburg, Germany & Imperial College of Science, Technology and Medicine, London, Great Britain. Computing various aspects of a „probabilistic logic“ on the basis of data Logic Probabilistic Learning What is „Probabilistic Logic Learning? - Learning Often it is easier to obtain data and to learn a model than using traditional knowledge engineering techniques Parameter estimation Learning the „logical“ structure Fully vs. unobservable random variables

8. APRIL, Application of Probabilistic Inductive Logic Programming, IST-2001-33053 Albert-Ludwigs-University, Freiburg, Germany & Imperial College of Science, Technology and Medicine, London, Great Britain. Learning Research Context:„Probabilistic Logics“ Studies of probabilistic logics: Nielson, Halpern, Poole,... Recently: Koller, Sato, Jaeger,... Logic Probabilistic deterministic probabilistic propositional objects/relations Probabilistic Logics HMMs, Bayesian networks,... Logics

9. APRIL, Application of Probabilistic Inductive Logic Programming, IST-2001-33053 Albert-Ludwigs-University, Freiburg, Germany & Imperial College of Science, Technology and Medicine, London, Great Britain. Research Context: „Probabilistic Learning“ Logic Learning Probabilistic (ML) Parameter estimation Gradient-based algorithms Family of EM algorithm Different scores LogLikelihood, BIC, MDL, MML,... Structural learning Score-based hill-climbing Independency tests Monte Carlo Markov Chain (MCMC) techniques

10. APRIL, Application of Probabilistic Inductive Logic Programming, IST-2001-33053 Albert-Ludwigs-University, Freiburg, Germany & Imperial College of Science, Technology and Medicine, London, Great Britain. Inductive Logic Learning Machine Learning and Data Mining within first-order representations Broadened the application domain of Data Mining, especially in bio- and chemoinformatics Some of the best-known examples of Scientific Discovery by AI systems Research Context: „Logic Learning“ Probabilistic Logic Learning

11. APRIL, Application of Probabilistic Inductive Logic Programming, IST-2001-33053 Albert-Ludwigs-University, Freiburg, Germany & Imperial College of Science, Technology and Medicine, London, Great Britain. Objectives (2) 1.Are there significant applications for which "first order probabilistic logic" is better than state-of- the-art representations? 2.Can models be learned within such a "first order probabilistic logic"? Overall goal: To critically investigate "probabilistic logic learning" methods by answering the following questions:

12. APRIL, Application of Probabilistic Inductive Logic Programming, IST-2001-33053 Albert-Ludwigs-University, Freiburg, Germany & Imperial College of Science, Technology and Medicine, London, Great Britain. Description of Work (1) In order to answer the questions, we plan 1.to investigate and evaluate various alternative first order probabilistic representations and reasoning mechanisms such as: oStochastic Logic Programs [Muggleton 95, Cussens 99], oBayesian Logic Programs" [Kersting, De Raedt 00].

13. APRIL, Application of Probabilistic Inductive Logic Programming, IST-2001-33053 Albert-Ludwigs-University, Freiburg, Germany & Imperial College of Science, Technology and Medicine, London, Great Britain. Description of Work (2) 2.to employ the most promising such representations to model a functional genomics application; 3.to develop and to employ simple learning techniques to enable the learning of parts of the probabilistic logic representation; 4.to identify the main goals, directions and questions to be addressed in probabilistic logic learning.

14. APRIL, Application of Probabilistic Inductive Logic Programming, IST-2001-33053 Albert-Ludwigs-University, Freiburg, Germany & Imperial College of Science, Technology and Medicine, London, Great Britain. Application: Functional Genomics (1) Hieter P. and Boguski M. (1997) Functional Genomics: It's all how you read it. Science 278, 601-02: „Development and application of global (genome-wide or system-wide) experimental approaches to assess gene function by making use of the information and reagents provided by structural genomics.“ Functional Genomics

15. APRIL, Application of Probabilistic Inductive Logic Programming, IST-2001-33053 Albert-Ludwigs-University, Freiburg, Germany & Imperial College of Science, Technology and Medicine, London, Great Britain. Application: Functional Genomics (2) Hieter P. and Boguski M.: "[Functional genomics] is characterized by high throughput or large-scale experimental methodologies combine with statistical and computational analysis of the results. The fundamental strategy in a functional genomics approach is to expand the scope of biological investigation from studying single genes or proteins to studying all genes or proteins at once in a systematic fashion." Probabilistic Logic Learning

16. APRIL, Application of Probabilistic Inductive Logic Programming, IST-2001-33053 Albert-Ludwigs-University, Freiburg, Germany & Imperial College of Science, Technology and Medicine, London, Great Britain. Application: Functional Genomics (3) Inductive Logic Learning supports the use (as background knowledge) of generally known metabolic pathways and encoded enzyme chemistry. Probabilistic approaches make probabilistic descriptions of outcomes of experiments, i.e. they account for uncertainty due to e.g. noise, hidden variables, etc. deterministic propositional Probabilistic Inductive Logic Learning should enable probabilistic descriptions of experiments, as well as supporting the general, i.e. first-order description of probabilistic behaviour in e.g. the enzyme chemistry.

17. APRIL, Application of Probabilistic Inductive Logic Programming, IST-2001-33053 Albert-Ludwigs-University, Freiburg, Germany & Imperial College of Science, Technology and Medicine, London, Great Britain. Stochastic Logic Programs / Bayesian Logic Programs Bayesian Logic Programs [Kersting, De Raedt 00] Ground atoms = random variables Conditional probability distribution associated to Horn clause Dependency graph resticted to least Herbrand universum= (possibly infinite) Bayesian network Stochastic Logic Programs [Muggleton 95, Cussens 99] Probabilities defined over proofs Single probability values associated to Horn clause Log-linear models

18. APRIL, Application of Probabilistic Inductive Logic Programming, IST-2001-33053 Albert-Ludwigs-University, Freiburg, Germany & Imperial College of Science, Technology and Medicine, London, Great Britain. Related Projects DARPA project: Evidence Extraction and Link Discovery (EELD) http://www.darpa.mil/iso/EELD/