1. Stefan Schoenmackers, Oren Etzioni, and Daniel S. Weld Presented by Kristine Monteith CS 652 - 5/8/09 1

2. The Problem Lots of information on the web, but answers to questions aren’t always stated explicitly Query: “What vegetables help prevent osteoporosis?” Not going to find “Kale prevents osteoporosis” Need to infer this from: kale is a vegetable kale contains calcium calcium helps prevent osteoporosis 2

3. Overview HOLMES Architecture (performs textual inference) Scaling Inference to the Web Experimental Results Related Work 3

4. The HOLMES Architecture Information from Knowledge Bases e.g. IsHighIn(kale, calcium), Prevents(calcium, osteoporosis) Inference Rules e.g. Prevents(X,Z) :- IsHighIn(X,Z) ^ Prevents(Y,Z) Queries e.g. query(X) :- IS-A(X,vegetable) ^ Prevents(X,osteoporosis) 4

5. Partial proof tree (DAG) for the query “What vegetables help prevent osteoporosis?” 5

6. Incremental Expansion Exact probabilistic inference is NP-complete To deal with this, HOLMES Uses approximate methods (loopy belief propagation) Focused queries help keep probabilistic inference manageable Creates networks incrementally (searches for additional proof trees and updates the network if there is more time) Exploits standard Datalog optimization (e.g. only expands proofs of recently added nodes) 6

7. Markov Logic Inference Rules 1. Observed relations are likely to be true: R(X,Y) :- ObservedInCorpus(X, R, Y) 2. Synonym substitution preserves meaning: RTR(X’,Y) :- RTR(X,Y) ^ Synonym(X, X’) RTR(X,Y’) :- RTR(X,Y) ^ Synonym(Y, Y’) 3. Generalizations preserve meaning: RTR(X’,Y) :- RTR(X,Y) ^ IS-A(X, X’) RTR(X,Y’) :- RTR(X,Y) ^ IS-A(Y, Y’) 4. Transitivity of Part Meronyms: RTR(X,Y’) :- RTR(X,Y) ^ Part-Of(Y, Y’) where RTR matches ‘* in’ (e.g., ‘born in’). 7

8. Scaling Inference to the Web In order to scale Textual Inference to the web, it has to scale linearly Assumptions: Number of ground assertions |A| grows linearly with size of corpus (True for assertions extracted by TextRunner) Size of every proof tree is bounded by some constant m (Seems to be true in practice, could be enforced by terminating search for proof trees at a certain depth) Need to show that constructing proof trees takes O(|A|) time 8

9. Constructing proof trees in O(|A|) time Using function free horn clauses means that logical inference can be done in polynomial time Still not good enough to scale to the Web Need to ensure two more things: Number of different types of proofs doesn’t grow too quickly (e.g. Fixed number of rules results in a constant number of first-order search trees) Number of tuples participating in each relation doesn’t grow too quickly 9

10. Approximately Pseudo-Functional 10

11. Experimental Results Uses two knowledge bases: TextRunner (183 million ground assertions from 117 million web pages) WordNet (159 thousand manually created IS-A, Part-Of, and Synonym assertions) Twenty queries in three domains Geography Business Nutrition 11

12. Geography Queries “Who was born in one of the following countries?” Q(X) :- BornIn(X,{country}) Possible countries: France, Germany, China, Thailand, Kenya, Morocco, Peru, Columbia, Guatemala Example: Ground assertion: BornIn(Alberto Fujimori, Lima) Background knowledge: LocatedIn(Lima, Peru) New conclusion: BornIn(Alberto Fujimori, Peru) 12

13. Business Queries Which companies are acquiring software companies? Q(X) :- Acquired(X, Y)^ Develops(Y, ‘software’) This query tests HOLMES’s ability to scalably join a large number of assertions from multiple pages. Which companies are headquartered in the USA? Q(X) :- HeadquarteredIn(X, ‘USA’) ^ IS-A(X, ‘company’) Join on HeadquarteredIn and IS-A Transitive inference: Seattle is PartOf Washington which is PartOf the USA Microsoft IS-A software company which IS-A company 13

14. Nutrition Queries “What foods prevent disease?” Q(X, {disease}) :- Prevents(X, {disease}) ^ IS-A(X, {food}) Possible foods: fruit, vegetable, grain Possible diseases: anemia, scurvy, or osteoporosis. 14

15. Effect of Inference on Recall Baseline: Number of query answers derived from information explicitly stated in the Knowledge Bases (TextRunner and WordNet) Inference increases the number of query answers by 102% for the Geography domain, and considerable more for the other two domains 15

16. 16

17. Prevalence of APF Relations Examined 500 binary relations selected randomly from TextRunners assertions Largest two relations had over 1.25 million unique instances 52% of the relations had more than 10,000 instances Found most of the smallest value Kmin such that the relation was APF with degree Kmin 80% of relations were APF with degree less than 496 17

18. 18

19. Related Work Van Durme and Schubert (2008) Use highly expressive representations (e.g. negation, temporal information) HOLMES is less expressive but more scalable Open-domain Question-Answering Systems Attempt to find individual documents or sentences containing the answer HOLMES can infer from multiple texts, but is not well suited to answering more abstract or open-ended questions Statistical Relational Learning Techniques for combining logical and probabilistic inference HOLMES uses more restrictive inference rules, but again is more scalable 19

20. Conclusions 1. We introduce and evaluate the HOLMES system, which leverages KBMC methods in order to scale a class of TI methods to the Web. 2. We define the notion of Approximately Pseudo-Functional (APF) relations and prove that, for a APF relations, HOLMES’s inference time increases linearly with the size of the input corpus. We show empirically that APF relations appear to be prevalent in our Web corpus and that HOLMES’s runtime does scale linearly with the size of its input taking only a few CPU minutes when run over 183 million distinct ground assertions. 3. We present experiments demonstrating that, for a set of queries in the domains of geography, business, and nutrition, HOLMES substantially improves the quality of answers (measured by AuC) relative to a “no inference” baseline. 20

21. Questions??? 21