A Review of Relational Machine Learning for Knowledge Graphs CVML Reading Group Xiao Lin.

Slides:



Advertisements
Similar presentations
Haystack: Per-User Information Environment 1999 Conference on Information and Knowledge Management Eytan Adar et al Presented by Xiao Hu CS491CXZ.
Advertisements

Knowledge Base Completion via Search-Based Question Answering
Probabilistic Latent-Factor Database Models Denis Krompaß 1, Xueyan Jiang 1,Maximilian Nickel 2 and Volker Tresp 1,3 1 Department of Computer Science.
Prof. Carolina Ruiz Computer Science Department Bioinformatics and Computational Biology Program WPI WELCOME TO BCB4003/CS4803 BCB503/CS583 BIOLOGICAL.
Semantic Web and Web Mining: Networking with Industry and Academia İsmail Hakkı Toroslu IST EVENT 2006.
CSE 574 – Artificial Intelligence II Statistical Relational Learning Instructor: Pedro Domingos.
Semantics For the Semantic Web: The Implicit, the Formal and The Powerful Amit Sheth, Cartic Ramakrishnan, Christopher Thomas CS751 Spring 2005 Presenter:
Ranking by Odds Ratio A Probability Model Approach let be a Boolean random variable: document d is relevant to query q otherwise Consider document d as.
Representation learning for Knowledge Bases LivesIn BornIn LocateIn Friendship Nationality Nicole Kidman PerformIn Nationality Sydney Hugh Jackman Australia.
Information Retrieval in Practice
Classifiers, Part 3 Week 1, Video 5 Classification  There is something you want to predict (“the label”)  The thing you want to predict is categorical.
Latent Semantic Analysis Hongning Wang VS model in practice Document and query are represented by term vectors – Terms are not necessarily orthogonal.
Tables to Linked Data Zareen Syed, Tim Finin, Varish Mulwad and Anupam Joshi University of Maryland, Baltimore County
Ensemble Solutions for Link-Prediction in Knowledge Graphs
Ontology-Based Information Extraction: Current Approaches.
A Probabilistic Graphical Model for Joint Answer Ranking in Question Answering Jeongwoo Ko, Luo Si, Eric Nyberg (SIGIR ’ 07) Speaker: Cho, Chin Wei Advisor:
Latent Semantic Analysis Hongning Wang Recap: vector space model Represent both doc and query by concept vectors – Each concept defines one dimension.
Joint Models of Disagreement and Stance in Online Debate Dhanya Sridhar, James Foulds, Bert Huang, Lise Getoor, Marilyn Walker University of California,
1/52 Overlapping Community Search Graph Data Management Lab, School of Computer Science
Question Answering over Implicitly Structured Web Content
Data Mining In contrast to the traditional (reactive) DSS tools, the data mining premise is proactive. Data mining tools automatically search the data.
WEEK INTRODUCTION IT440 ARTIFICIAL INTELLIGENCE.
MICROSOFT SEMANTIC ENGINE Unified Search, Discovery and Insight.
Exploiting Relevance Feedback in Knowledge Graph Search
DeepDive Model Dongfang Xu Ph.D student, School of Information, University of Arizona Dec 13, 2015.
Tutorial: Knowledge Bases for Web Content Analytics
The Unreasonable Effectiveness of Data
Unsupervised Relation Detection using Automatic Alignment of Query Patterns extracted from Knowledge Graphs and Query Click Logs Panupong PasupatDilek.
WHAT IS DATA MINING?  The process of automatically extracting useful information from large amounts of data.  Uses traditional data analysis techniques.
WHAT IS DATA MINING?  The process of automatically extracting useful information from large amounts of data.  Uses traditional data analysis techniques.
Scalable Statistical Relational Learning for NLP
Look, Ma, No Neurons! Knowledge Base Completion Using Explicit Inference Rules William W Cohen Machine Learning Department Carnegie Mellon University joint.
Artificial Intelligence DNA Hypernetworks Biointelligence Lab School of Computer Sci. & Eng. Seoul National University.
Multi-Modal Bayesian Embeddings for Learning Social Knowledge Graphs Zhilin Yang 12, Jie Tang 1, William W. Cohen 2 1 Tsinghua University 2 Carnegie Mellon.
Learning Bayesian Networks for Complex Relational Data
Brief Intro to Machine Learning CS539
Ensembling Diverse Approaches to Question Answering
An Introduction to Markov Logic Networks in Knowledge Bases
A Brief Introduction to Distant Supervision
KDD CUP 2001 Task 1: Thrombin Jie Cheng (
School of Computer Science & Engineering
Wenhan Xiong, Thien Hoang, William Wang Department of Computer Science
General Graphical Model Learning Schema
Machine Learning With Python Sreejith.S Jaganadh.G.
Giuseppe Attardi Dipartimento di Informatica Università di Pisa
Summarizing Entities: A Survey Report
Mining and Analyzing Data from Open Source Software Repository
What is Pattern Recognition?
Web IR: Recent Trends; Future of Web Search
Distributed Representation of Words, Sentences and Paragraphs
Variational Knowledge Graph Reasoning
Web-Mining Agents Multi-Relational Latent Semantic Analysis
Semantic Network & Knowledge Graph
Knowledge Base Completion
Knowledge Graph Embedding
Navi 下一步工作的设想 郑 亮 6.6.
Giuseppe Attardi Dipartimento di Informatica Università di Pisa
Seminar Topics and Projects
Graph and Tensor Mining for fun and profit
Graph and Tensor Mining for fun and profit
Improving Knowledge Graph Embedding Using Simple Constraints
Enriching Taxonomies With Functional Domain Knowledge
Giuseppe Attardi Dipartimento di Informatica Università di Pisa
Neural Modular Networks
Rachit Saluja 03/20/2019 Relation Extraction with Matrix Factorization and Universal Schemas Sebastian Riedel, Limin Yao, Andrew.
DATA-Intensive systems Department of computer science
NAÏVE BAYES CLASSIFICATION
Topic: Semantic Text Mining
Heterogeneous Graph Convolutional Network
A framework for ontology Learning FROM Big Data
Presentation transcript:

A Review of Relational Machine Learning for Knowledge Graphs CVML Reading Group Xiao Lin

M. Nickel, K. Murphy, V. Tresp, E. Gabrilovich. “A Review of Relational Machine Learning for Knowledge Graphs. ” arXiv

VQA Knowledge base MRF vs embeddings 3

The relational learning problem Neural embedding models 4

“Leonard Nimoy was an actor who played the character Spock in the science-fiction movie Star Trek” 5

subjectpredicateobject Leonard NimoyprofessionActor Leonard Nimoystarred inStar Trek Leonard NimoyplayedSpock character inStar Trek genreScience-fiction “SPO triples” 6

“Leonard Nimoy was an actor who played the character Spock in the science-fiction movie Star Trek” Leonard Nimoy Star Trek Science-fiction profession starred in played character in genre Leonard Nimoy Spock Star Trek Actor Spock 7 Knowledge Graphs

“Leonard Nimoy was an actor who played the character Spock in the science-fiction movie Star Trek” Spock Leonard Nimoy Actor Star Trek Science-fiction profession starred in played character in genre 8

Knowledge Graphs a.k.a. Relational data a.k.a. Linked data Spock Leonard Nimoy Actor Star Trek Science-fiction profession starred in played character in genre 9

Knowledge Graphs a.k.a. Semantic network a.k.a. Knowledge base a.k.a. Semantic web a.k.a. Big data a.k.a. Knowledge graph Spock Leonard Nimoy Actor Star Trek Science-fiction profession starred in played character in genre (1960s) (1970s) (2000s) (2010s) (2012) a.k.a. Artificial Intelligence 10

Query Knowledge Graphs Spock Leonard Nimoy Actor Star Trek Science-fiction profession starred in played character in genre Was Leonard Nimoy in Star Trek? Leonard Nimoy starred in Star Trek 11

Query Knowledge Graphs Spock Leonard Nimoy Actor Star Trek Science-fiction profession starred in played character in genre Was Leonard Nimoy in Star Trek? Leonard Nimoy starred in Star Trek True 12

Query Knowledge Graphs Spock Leonard Nimoy Actor Star Trek Science-fiction profession starred in played character in genre Which character did Leonard Nimoy play in Star Trek? Leonard Nimoy played (_X_) (_X_) character in Star Trek 13

Query Knowledge Graphs Spock Leonard Nimoy Actor Star Trek Science-fiction profession starred in played character in genre Which character did Leonard Nimoy play in Star Trek? Leonard Nimoy played (Spock) (Spock) character in Star Trek True 14

Query Knowledge Graphs Spock Leonard Nimoy Actor Star Trek Science-fiction profession starred in played character in genre Which character did Leonard Nimoy play in Star Trek? Leonard Nimoy played (Spock) (Spock) character in Star Trek True 15

Query Knowledge Graphs Spock Leonard Nimoy Actor Star Trek Science-fiction profession played character in genre Was Leonard Nimoy in Star Trek? Leonard Nimoy starred in Star Trek 16

Relational Machine Learning Goal: Infer relationships between objects and answer queries Database of (partially) annotated relations 17

Relational Machine Learning Goal: Infer relationships between objects and answer queries Database of (partially) annotated relations Learn rules or models * * 18

Relational Machine Learning Goal: Infer relationships between objects and answer queries Database of (partially) annotated relations Answer queries on unseen relations Learn rules or models 19 * *

Relational Machine Learning Goal: Infer relationships between objects and answer queries Database of (partially) annotated relations Answer queries on unseen relations Learn rules or models WordNet Wikipedia Freebase YAGO NELL ReVerb Knowledge Vault Web …. ILP MLN Random walk Word2Vec Tensor factorization Neural embeddings … 20

Relational Machine Learning Applications Bioinformatics Given that: Some DNA produces some protein Some protein has some function Some protein interacts some other protein What does this protein do? 21

Relational Machine Learning Applications Bioinformatics Question Answering (QA) “Find a table for 4 tonight in Chicago.” “How’s the weather tomorrow?” “Read my latest s” “Why did the chickens cross the road?” 22

Relational Machine Learning Applications Bioinformatics Question Answering (QA) Search 23

Relational Machine Learning Applications Bioinformatics Question Answering (QA) Search 24

Relational Machine Learning Applications Bioinformatics Question Answering (QA) Search Visualization [Wang et al. 2012] Reviews for La Baguette Bakery, Stanford, CA 25

Relational Machine Learning Applications Bioinformatics Question Answering (QA) Search Visualization Social networks Recommendation Controversial stuff program-may-be-killing-thousands-of-innocent-people/ 26

Relational Machine Learning What I really do 27

Relational Machine Learning Goal: Infer relationships between objects and answer queries Database of (partially) annotated relations Answer queries on unseen relations Learn rules or models 28 * *

Relational Machine Learning Goal: Infer relationships between objects and answer queries Database of (partially) annotated relations Answer queries on unseen relations Learn rules or models WordNet Wikipedia Freebase YAGO NELL ReVerb Knowledge Vault Web …. ILP MLN Random walk Word2Vec Tensor factorization Neural embeddings … 29

Relational Machine Learning Challenges Database of (partially) annotated relations Quantity Collaborative Automatic IE Negative examples Locally closed world Noise Grounding Barrack Obama /m/03gh4 Bias 30

Relational Machine Learning Challenges Database of (partially) annotated relations Learn rules or models Quantity Collaborative Automatic IE Negative examples Locally closed world Noise Grounding Barrack Obama /m/03gh4 Bias Accuracy Efficiency Scalability Interpretability 31

Relational Machine Learning Challenges Database of (partially) annotated relations Answer queries on unseen relations Learn rules or models Quantity Collaborative Automatic IE Negative examples Locally closed world Noise Grounding Barrack Obama /m/03gh4 Bias Accuracy Efficiency Scalability Interpretability Fast inference Scale to complex queries Conjunctions  32

Relational Machine Learning Models Symbolic vs. Connectionist 33

Relational Machine Learning Models Symbolic vs. Connectionist MRF vs. Embedding MRF (Markov Logic Nets) Spock Leonard Nimoy Actor Star Trek Science-fiction profession played character in genre  Unobserved relations are jointly inferred  Learn a set of rules (factors) and weights (MCMC)  Infer all unobserved relations jointly by maximizing probability This is not an MRF

Relational Machine Learning Models Symbolic vs. Connectionist MRF vs. Embedding MRF Embedding (RESCAL) Spock Leonard Nimoy Actor Star Trek Science-fiction profession played character in genre  Unobserved relations are independent given SPO embeddings  Learn SPO embeddings (matrix/tensor factorization, neural nets)  Predict relations SPO Embeddings 35

1 Cat Dog 1 Cat Dog 1 36

Relational Machine Learning Models Symbolic vs. Connectionist MRF vs. Embedding (S,P,O)=0/1 classification performance [Nickel et al. 2015] 37

Relational Machine Learning Models Symbolic vs. Connectionist MRF vs. Embedding Graph mining  Predict relations using graph features, usually independently given graph features  In degree, out degree  Common ancestor  Random walk  S->O paths  … Spock Leonard Nimoy Actor Star Trek Science-fiction profession played character in genre 38

Neural Embedding Models Predicting a score based on Subject, Object, Predicate embeddings 39

Neural Embedding Models Predicting a score based on Subject, Object, Predicate embeddings S O P LookupTable 40

Neural Embedding Models Predicting a score based on Subject, Object, Predicate embeddings Something that allows backprop S O P LookupTable 41

Neural Embedding Models Predicting a score based on Subject, Object, Predicate embeddings Something that allows backprop S O LookupTable P 42

Neural Embedding Models Predicting a score based on Subject, Object, Predicate embeddings ER-MLP [Dong et al. 2014] Linear E-MLP Linear 43

Neural Embedding Models Predicting a score based on Subject, Object, Predicate embeddings Linear -L1 distance Linear Structured Embedding (SE) [Bordes et al. 2011] CAddTable -L2 distance TransE [Bordes et al. 2013] CAddTable -L2 distance TransR [Lin et al. 2015] Linear 44

Neural Embedding Models Predicting a score based on Subject, Object, Predicate embeddings RESCAL [Nickel et al. 2011] NTN [Socher et al. 2013] Linear HolE [Nickel et al. 2015] 45

Neural Embedding Models WordNet (WN18) Words S&O (words) 18 Predicates (relations) triples Search Engine GoogleYahooAsk Jeeves Has_instance Trademark Direct hypernym 46

Neural Embedding Models WordNet (WN18) Words S&O (words) 18 Predicates (relations) triples Freebase (FB15k) General facts S&O (words) 1345 Predicates (relations) triples Top_equivalent_webpage Composition type Piano sonata Pathétique Also_known_as Opus 13 Notable_for /type/object/key Pianosonates 47

Neural Embedding Models WordNet (WN18) Words S&O (words) 18 Predicates (relations) triples Freebase (FB15k) General facts S&O (words) 1345 Predicates (relations) triples (?,P,O) => S from all possible {S} 48

Neural Embedding Models WordNet (WN18) Words S&O (words) 18 Predicates (relations) triples Freebase (FB15k) General facts S&O (words) 1345 Predicates (relations) triples (?,P,O) => S from all possible {S} Locally Closed World Assumption (LCWA) Multiple S Raw Filter: remove other true S’ 49

Neural Embedding Models WordNet (WN18) Words S&O (words) 18 Predicates (relations) triples Freebase (FB15k) General facts S&O (words) 1345 Predicates (relations) triples Mean Reciprocal Rank (MRR) 50

Neural Embedding Models [Nickel et al. 2015] WordNet (filtered?) MethodFilteredRaw1310 ER-MLP TransE TransR RESCAL HolE

Neural Embedding Models [Nickel et al. 2015] Freebase (filtered?) MethodFilteredRaw1310 ER-MLP TransE TransR RESCAL HolE

Neural Embedding Models [Nickel et al. 2015] Freebase (filtered?) MethodFilteredRaw1310 ER-MLP TransE TransR RESCAL HolE

Neural Embedding Models Qualitative: Word2Vec visualizations (again) [Mikolov et al. 2013] 54

Big Knowledge Graphs Confident predictions [Dong et al. 2014] Prob(SPO)>0.9 55

Big Knowledge Graphs Confident predictions Google Knowledge Vault [Dong et al. 2014] Web information extraction Neural embedding Path ranking 56

Knowledge base and VQA Incorporating entity and relation embeddings Improving VQA model When entities and relations become compositional 57