1. A Review of Relational Machine Learning for Knowledge Graphs CVML Reading Group Xiao Lin

2. M. Nickel, K. Murphy, V. Tresp, E. Gabrilovich. “A Review of Relational Machine Learning for Knowledge Graphs. ” arXiv 2015. 2

3. VQA Knowledge base MRF vs embeddings 3

4. The relational learning problem Neural embedding models 4

5. “Leonard Nimoy was an actor who played the character Spock in the science-fiction movie Star Trek” 5

6. subjectpredicateobject Leonard NimoyprofessionActor Leonard Nimoystarred inStar Trek Leonard NimoyplayedSpock character inStar Trek genreScience-fiction “SPO triples” 6

7. “Leonard Nimoy was an actor who played the character Spock in the science-fiction movie Star Trek” Leonard Nimoy Star Trek Science-fiction profession starred in played character in genre Leonard Nimoy Spock Star Trek Actor Spock 7 Knowledge Graphs

8. “Leonard Nimoy was an actor who played the character Spock in the science-fiction movie Star Trek” Spock Leonard Nimoy Actor Star Trek Science-fiction profession starred in played character in genre 8

9. Knowledge Graphs a.k.a. Relational data a.k.a. Linked data Spock Leonard Nimoy Actor Star Trek Science-fiction profession starred in played character in genre 9

10. Knowledge Graphs a.k.a. Semantic network a.k.a. Knowledge base a.k.a. Semantic web a.k.a. Big data a.k.a. Knowledge graph Spock Leonard Nimoy Actor Star Trek Science-fiction profession starred in played character in genre (1960s) (1970s) (2000s) (2010s) (2012) a.k.a. Artificial Intelligence 10

11. Query Knowledge Graphs Spock Leonard Nimoy Actor Star Trek Science-fiction profession starred in played character in genre Was Leonard Nimoy in Star Trek? Leonard Nimoy starred in Star Trek 11

12. Query Knowledge Graphs Spock Leonard Nimoy Actor Star Trek Science-fiction profession starred in played character in genre Was Leonard Nimoy in Star Trek? Leonard Nimoy starred in Star Trek True 12

13. Query Knowledge Graphs Spock Leonard Nimoy Actor Star Trek Science-fiction profession starred in played character in genre Which character did Leonard Nimoy play in Star Trek? Leonard Nimoy played (_X_) (_X_) character in Star Trek 13

14. Query Knowledge Graphs Spock Leonard Nimoy Actor Star Trek Science-fiction profession starred in played character in genre Which character did Leonard Nimoy play in Star Trek? Leonard Nimoy played (Spock) (Spock) character in Star Trek True 14

15. Query Knowledge Graphs Spock Leonard Nimoy Actor Star Trek Science-fiction profession starred in played character in genre Which character did Leonard Nimoy play in Star Trek? Leonard Nimoy played (Spock) (Spock) character in Star Trek True 15

16. Query Knowledge Graphs Spock Leonard Nimoy Actor Star Trek Science-fiction profession played character in genre Was Leonard Nimoy in Star Trek? Leonard Nimoy starred in Star Trek 16

17. Relational Machine Learning Goal: Infer relationships between objects and answer queries Database of (partially) annotated relations 17

18. Relational Machine Learning Goal: Infer relationships between objects and answer queries Database of (partially) annotated relations Learn rules or models * * 18

19. Relational Machine Learning Goal: Infer relationships between objects and answer queries Database of (partially) annotated relations Answer queries on unseen relations Learn rules or models 19 * *

20. Relational Machine Learning Goal: Infer relationships between objects and answer queries Database of (partially) annotated relations Answer queries on unseen relations Learn rules or models WordNet Wikipedia Freebase YAGO NELL ReVerb Knowledge Vault Web …. ILP MLN Random walk Word2Vec Tensor factorization Neural embeddings … 20

21. Relational Machine Learning Applications Bioinformatics Given that: Some DNA produces some protein Some protein has some function Some protein interacts some other protein What does this protein do? 21

22. Relational Machine Learning Applications Bioinformatics Question Answering (QA) “Find a table for 4 tonight in Chicago.” “How’s the weather tomorrow?” “Read my latest emails” “Why did the chickens cross the road?” 22

23. Relational Machine Learning Applications Bioinformatics Question Answering (QA) Search 23

24. Relational Machine Learning Applications Bioinformatics Question Answering (QA) Search 24

25. Relational Machine Learning Applications Bioinformatics Question Answering (QA) Search Visualization [Wang et al. 2012] Reviews for La Baguette Bakery, Stanford, CA 25

26. Relational Machine Learning Applications Bioinformatics Question Answering (QA) Search Visualization Social networks Recommendation Controversial stuff http://arstechnica.co.uk/security/2016/02/the-nsas-skynet- program-may-be-killing-thousands-of-innocent-people/ 26

27. Relational Machine Learning What I really do 27

28. Relational Machine Learning Goal: Infer relationships between objects and answer queries Database of (partially) annotated relations Answer queries on unseen relations Learn rules or models 28 * *

29. Relational Machine Learning Goal: Infer relationships between objects and answer queries Database of (partially) annotated relations Answer queries on unseen relations Learn rules or models WordNet Wikipedia Freebase YAGO NELL ReVerb Knowledge Vault Web …. ILP MLN Random walk Word2Vec Tensor factorization Neural embeddings … 29

30. Relational Machine Learning Challenges Database of (partially) annotated relations Quantity Collaborative Automatic IE Negative examples Locally closed world Noise Grounding Barrack Obama /m/03gh4 Bias 30

31. Relational Machine Learning Challenges Database of (partially) annotated relations Learn rules or models Quantity Collaborative Automatic IE Negative examples Locally closed world Noise Grounding Barrack Obama /m/03gh4 Bias Accuracy Efficiency Scalability Interpretability 31

32. Relational Machine Learning Challenges Database of (partially) annotated relations Answer queries on unseen relations Learn rules or models Quantity Collaborative Automatic IE Negative examples Locally closed world Noise Grounding Barrack Obama /m/03gh4 Bias Accuracy Efficiency Scalability Interpretability Fast inference Scale to complex queries Conjunctions  32

33. Relational Machine Learning Models Symbolic vs. Connectionist 33

34. Relational Machine Learning Models Symbolic vs. Connectionist MRF vs. Embedding MRF (Markov Logic Nets) Spock Leonard Nimoy Actor Star Trek Science-fiction profession played character in genre  Unobserved relations are jointly inferred  Learn a set of rules (factors) and weights (MCMC)  Infer all unobserved relations jointly by maximizing probability -------------------This is not an MRF---------------------- 34

35. Relational Machine Learning Models Symbolic vs. Connectionist MRF vs. Embedding MRF Embedding (RESCAL) Spock Leonard Nimoy Actor Star Trek Science-fiction profession played character in genre  Unobserved relations are independent given SPO embeddings  Learn SPO embeddings (matrix/tensor factorization, neural nets)  Predict relations SPO Embeddings 35

36. 1 Cat Dog 1 Cat Dog 1 36

37. Relational Machine Learning Models Symbolic vs. Connectionist MRF vs. Embedding (S,P,O)=0/1 classification performance [Nickel et al. 2015] 37

38. Relational Machine Learning Models Symbolic vs. Connectionist MRF vs. Embedding Graph mining  Predict relations using graph features, usually independently given graph features  In degree, out degree  Common ancestor  Random walk  S->O paths  … Spock Leonard Nimoy Actor Star Trek Science-fiction profession played character in genre 38

39. Neural Embedding Models Predicting a score based on Subject, Object, Predicate embeddings 39

40. Neural Embedding Models Predicting a score based on Subject, Object, Predicate embeddings S O P LookupTable 40

41. Neural Embedding Models Predicting a score based on Subject, Object, Predicate embeddings Something that allows backprop S O P LookupTable 41

42. Neural Embedding Models Predicting a score based on Subject, Object, Predicate embeddings Something that allows backprop S O LookupTable P 42

43. Neural Embedding Models Predicting a score based on Subject, Object, Predicate embeddings ER-MLP [Dong et al. 2014] Linear E-MLP Linear 43

44. Neural Embedding Models Predicting a score based on Subject, Object, Predicate embeddings Linear -L1 distance Linear Structured Embedding (SE) [Bordes et al. 2011] CAddTable -L2 distance TransE [Bordes et al. 2013] CAddTable -L2 distance TransR [Lin et al. 2015] Linear 44

45. Neural Embedding Models Predicting a score based on Subject, Object, Predicate embeddings RESCAL [Nickel et al. 2011] NTN [Socher et al. 2013] Linear HolE [Nickel et al. 2015] 45

46. Neural Embedding Models WordNet (WN18) Words 40943 S&O (words) 18 Predicates (relations) 151442 triples Search Engine GoogleYahooAsk Jeeves Has_instance Trademark Direct hypernym 46

47. Neural Embedding Models WordNet (WN18) Words 40943 S&O (words) 18 Predicates (relations) 151442 triples Freebase (FB15k) General facts 14951 S&O (words) 1345 Predicates (relations) 592213 triples Top_equivalent_webpage Composition type Piano sonata Pathétique Also_known_as Opus 13 http://imslp.org/wiki/index.html?curid=1410 Notable_for /type/object/key Pianosonates 47

48. Neural Embedding Models WordNet (WN18) Words 40943 S&O (words) 18 Predicates (relations) 151442 triples Freebase (FB15k) General facts 14951 S&O (words) 1345 Predicates (relations) 592213 triples (?,P,O) => S from all possible {S} 48

49. Neural Embedding Models WordNet (WN18) Words 40943 S&O (words) 18 Predicates (relations) 151442 triples Freebase (FB15k) General facts 14951 S&O (words) 1345 Predicates (relations) 592213 triples (?,P,O) => S from all possible {S} Locally Closed World Assumption (LCWA) Multiple S Raw Filter: remove other true S’ 49

50. Neural Embedding Models WordNet (WN18) Words 40943 S&O (words) 18 Predicates (relations) 151442 triples Freebase (FB15k) General facts 14951 S&O (words) 1345 Predicates (relations) 592213 triples Accuracy @1,3,10 Mean Reciprocal Rank (MRR) 50

51. Neural Embedding Models [Nickel et al. 2015] WordNet MRRAccuracy @x (filtered?) MethodFilteredRaw1310 ER-MLP0.7120.52862.677.586.3 TransE0.4950.35111.388.894.3 TransR0.6050.42733.587.694.3 RESCAL0.8900.60384.290.492.8 HolE0.9380.61693.094.594.9 51

52. Neural Embedding Models [Nickel et al. 2015] Freebase MRRAccuracy @x (filtered?) MethodFilteredRaw1310 ER-MLP0.2880.15517.331.750.1 TransE0.4630.22229.757.874.9 TransR0.3460.19821.840.458.2 RESCAL0.3540.18923.540.958.7 HolE0.5240.23240.261.373.9 52

53. Neural Embedding Models [Nickel et al. 2015] Freebase MRRAccuracy @x (filtered?) MethodFilteredRaw1310 ER-MLP0.2880.15517.331.750.1 TransE0.4630.22229.757.874.9 TransR0.3460.19821.840.458.2 RESCAL0.3540.18923.540.958.7 HolE0.5240.23240.261.373.9 53

54. Neural Embedding Models Qualitative: Word2Vec visualizations (again) [Mikolov et al. 2013] 54

55. Big Knowledge Graphs Confident predictions [Dong et al. 2014] Prob(SPO)>0.9 55

56. Big Knowledge Graphs Confident predictions Google Knowledge Vault [Dong et al. 2014] Web information extraction Neural embedding Path ranking 56

57. Knowledge base and VQA Incorporating entity and relation embeddings Improving VQA model When entities and relations become compositional 57