Representation learning for Knowledge Bases LivesIn BornIn LocateIn Friendship Nationality Nicole Kidman PerformIn Nationality Sydney Hugh Jackman Australia.

Slides:



Advertisements
Similar presentations
Document Summarization using Conditional Random Fields Dou Shen, Jian-Tao Sun, Hua Li, Qiang Yang, Zheng Chen IJCAI 2007 Hao-Chin Chang Department of Computer.
Advertisements

Image classification Given the bag-of-features representations of images from different classes, how do we learn a model for distinguishing them?
Knowledge Base Completion via Search-Based Question Answering
Linked data: P redicting missing properties Klemen Simonic, Jan Rupnik, Primoz Skraba {klemen.simonic, jan.rupnik,
A CTION R ECOGNITION FROM V IDEO U SING F EATURE C OVARIANCE M ATRICES Kai Guo, Prakash Ishwar, Senior Member, IEEE, and Janusz Konrad, Fellow, IEEE.
Hongliang Li, Senior Member, IEEE, Linfeng Xu, Member, IEEE, and Guanghui Liu Face Hallucination via Similarity Constraints.
Jean-Eudes Ranvier 17/05/2015Planet Data - Madrid Trustworthiness assessment (on web pages) Task 3.3.
Relational Learning with Gaussian Processes By Wei Chu, Vikas Sindhwani, Zoubin Ghahramani, S.Sathiya Keerthi (Columbia, Chicago, Cambridge, Yahoo!) Presented.
Discriminative and generative methods for bags of features
Image classification Given the bag-of-features representations of images from different classes, how do we learn a model for distinguishing them?
Chen Cheng1, Haiqin Yang1, Irwin King1,2 and Michael R. Lyu1
Natural Language Processing Lab, Tsinghua University
Distributed Representations of Sentences and Documents
General Mining Issues a.j.m.m. (ton) weijters Overfitting Noise and Overfitting Quality of mined models (some figures are based on the ML-introduction.
Adapting Deep RankNet for Personalized Search
Learning Table Extraction from Examples Ashwin Tengli, Yiming Yang and Nian Li Ma School of Computer Science Carnegie Mellon University Coling 04.
Longbiao Kang, Baotian Hu, Xiangping Wu, Qingcai Chen, and Yan He Intelligent Computing Research Center, School of Computer Science and Technology, Harbin.
Jinhui Tang †, Shuicheng Yan †, Richang Hong †, Guo-Jun Qi ‡, Tat-Seng Chua † † National University of Singapore ‡ University of Illinois at Urbana-Champaign.
Attention Deficit Hyperactivity Disorder (ADHD) Student Classification Using Genetic Algorithm and Artificial Neural Network S. Yenaeng 1, S. Saelee 2.
Cao et al. ICML 2010 Presented by Danushka Bollegala.
Object Bank Presenter : Liu Changyu Advisor : Prof. Alex Hauptmann Interest : Multimedia Analysis April 4 th, 2013.
Reyyan Yeniterzi Weakly-Supervised Discovery of Named Entities Using Web Search Queries Marius Pasca Google CIKM 2007.
Ensemble Solutions for Link-Prediction in Knowledge Graphs
BING: Binarized Normed Gradients for Objectness Estimation at 300fps
Exploit of Online Social Networks with Community-Based Graph Semi-Supervised Learning Mingzhen Mo and Irwin King Department of Computer Science and Engineering.
Online Multiple Kernel Classification Steven C.H. Hoi, Rong Jin, Peilin Zhao, Tianbao Yang Machine Learning (2013) Presented by Audrey Cheong Electrical.
Xutao Li1, Gao Cong1, Xiao-Li Li2
CoNMF: Exploiting User Comments for Clustering Web2.0 Items Presenter: He Xiangnan 28 June School of Computing National.
Mining Dependency Relations for Query Expansion in Passage Retrieval Renxu Sun, Chai-Huat Ong, Tat-Seng Chua National University of Singapore SIGIR2006.
Wenyuan Dai, Ou Jin, Gui-Rong Xue, Qiang Yang and Yong Yu Shanghai Jiao Tong University & Hong Kong University of Science and Technology.
ASSOCIATIVE BROWSING Evaluating 1 Jinyoung Kim / W. Bruce Croft / David Smith for Personal Information.
11 A Classification-based Approach to Question Routing in Community Question Answering Tom Chao Zhou 1, Michael R. Lyu 1, Irwin King 1,2 1 The Chinese.
KAIST TS & IS Lab. CS710 Know your Neighbors: Web Spam Detection using the Web Topology SIGIR 2007, Carlos Castillo et al., Yahoo! 이 승 민.
Collaborative Filtering via Euclidean Embedding M. Khoshneshin and W. Street Proc. of ACM RecSys, pp , 2010.
From Paraphrase Database to Compositional Paraphrase Model and Back John Wieting University of Illinois Joint work with Mohit Bansal, Kevin Gimpel, Karen.
Identifying “Best Bet” Web Search Results by Mining Past User Behavior Author: Eugene Agichtein, Zijian Zheng (Microsoft Research) Source: KDD2006 Reporter:
1 Using Graph Theory to Analyze Gene Network Coherence José A. Lagares Jesús S. Aguilar Norberto Díaz-Díaz Francisco A. Gómez-Vela
Rich feature hierarchies for accurate object detection and semantic segmentation 2014 IEEE Conference on Computer Vision and Pattern Recognition Ross Girshick,
Parsing Natural Scenes and Natural Language with Recursive Neural Networks INTERNATIONAL CONFERENCE ON MACHINE LEARNING (ICML 2011) RICHARD SOCHER CLIFF.
Spectral Algorithms for Learning HMMs and Tree HMMs for Epigenetics Data Kevin C. Chen Rutgers University joint work with Jimin Song (Rutgers/Palentir),
Learning to Answer Questions from Image Using Convolutional Neural Network Lin Ma, Zhengdong Lu, and Hang Li Huawei Noah’s Ark Lab, Hong Kong
A Review of Relational Machine Learning for Knowledge Graphs CVML Reading Group Xiao Lin.
Distributed Representations for Natural Language Processing
Neural Collaborative Filtering
Big data classification using neural network
CNN-RNN: A Unified Framework for Multi-label Image Classification
Compact Bilinear Pooling
A Three-way Model for Collective Learning on Multi-Relational Data
Wenhan Xiong, Thien Hoang, William Wang Department of Computer Science
Cross-lingual Knowledge Linking Across Wiki Knowledge Bases
Neural networks (3) Regularization Autoencoder
Variational Knowledge Graph Reasoning
Web-Mining Agents Multi-Relational Latent Semantic Analysis
Knowledge Base Completion
Vessel Extraction in X-Ray Angiograms Using Deep Learning
Knowledge Graph Embedding
iSRD Spam Review Detection with Imbalanced Data Distributions
Graph and Tensor Mining for fun and profit
Graph and Tensor Mining for fun and profit
Improving Knowledge Graph Embedding Using Simple Constraints
Asymmetric Transitivity Preserving Graph Embedding
GANG: Detecting Fraudulent Users in OSNs
Heterogeneous convolutional neural networks for visual recognition
Topological Signatures For Fast Mobility Analysis
NAACL‘18 Dai Quoc Nguyen, Tu Dinh Nguyen, Dat Quoc Nguyen, Dinh Phung
Keshav Balasubramanian
Motivation State-of-the-art two-stage instance segmentation methods depend heavily on feature localization to produce masks.
Presenter: Yu Chen Computer Science Department
Heterogeneous Graph Convolutional Network
Neural Machine Translation by Jointly Learning to Align and Translate
Presentation transcript:

Representation learning for Knowledge Bases LivesIn BornIn LocateIn Friendship Nationality Nicole Kidman PerformIn Nationality Sydney Hugh Jackman Australia (Nation) Australia (Movie) U.S.A Embedding Entities and Relations for Learning and Inference in Knowledge Bases Bishan Yang 1, Wen-tau Yih 2, Xiaodong He 2, Jianfeng Gao 2, Li Deng 2 1 Cornell University, 2 Microsoft Research  Large-scale knowledge bases (KBs) such as Freebase and YAGO store knowledge about real-world entities in the form of RDF triples (i.e., (subject, predicate, object)). How to represent entities and relations? How to learn from existing knowledge? How to infer new knowledge?  Related Work Matrix/Tensor Factorization  RESCAL [Nickel et al., 2011; 2012]  [Jenatton et. al., 2012]  TRESCAL [Chang et al., 2014] Neural-Embedding models  TransE [Bordes et al., 2013]  NTN [Socher et. al., 2013]  TransH [Wang et al., 2014]  Tatec [García-Durán et. al., 2014] Contributions  A neural network framework that unifies several popular neural-embedding models, including TransE [Bordes et al., 2013] and NTN [Socher et. al., 2013]  A simple bilinear-based model that achieves the state-of-the-art performance on link prediction on Freebase and WordNet  Propose the modeling of relation composition using matrix multiplication of relation embeddings  Propose an embedding-based rule extraction method that outperforms AMIE [Galárraga et al., 2013], a state-of-the-art rule mining approach for large KBs, on extracting closed-path Horn-clause rules on Freebase Representation Learning Framework Experimental Setup Inference Task I: Link Prediction Inference Task II: Rule Extraction FB15k (Freebase)FB15k-401WN (WordNet) Entities14,95114,54140,943 Relations1, Train483,142456,974141,442 Test50,07155,8765,000 Valid50,00047,3595,000 Table 1: Data statistics  Training specifics: Mini-batch SGD with AdaGrad Randomly sample negative examples (corrupting both subject and object) L2 regularization Entity vector dim = 100 ModelsBilinear ParamLinear ParamScoring Function NTN Bilinear+Linear TransE (DistAdd)- Bilinear - Bilinear-diag (DistMult) - Table 2: Compared models Models FB15kFB15k-401WN NTN Bilinear+Linear TransE (DistAdd) Bilinear Bilinear-diag (DistMult) Result breakdown on FB15k-401: multiplicative distance > additive distance Models Predicting subject entities Predicting object entities 1-to-11-to-nn-to-1n-to-n1-to-11-to-nn-to-1n-to-n DistAdd DistMult Table 4: Results by different relation categories: one-to-one, one-to-many, many-to-one and many- to-many. Main Results: bilinear > linear, diagonal matrix > full matrix > tensor Table 3: Link prediction results. MRR denotes the mean reciprocal rank and denotes top-10 accuracy, both the higher the better. (w/ type checking) DistMult DistMult-tanh DistMult-tanh-WV-init DistMult-tanh-EV-init Table 5: Variants of DistMult: (1) adding non-linearity (2) using pre-trained word vectors (3) using pre-trained entity vectors. MAP with type checking applies entity type information to filter predicted entities. Entity Representation: nonlinearity> linearity, pre-trained entity vectors > pre-trained word vectors  Can relation embeddings capture relation composition? For example, in Horn clauses like  Embedding-based Horn-clause rule extraction For each relation r KNN search on possible relation combinations (paths) by computing Figure 4: Aggregated precision of top length-2 rules. AMIE [Galárraga et al., 2013] is an association-rule-mining- based approach for large-scale KBs. EmbedRule denotes our embedding-based approach, where DistAdd uses additive composition while Bilinear, DistMult and DistMult-tanh-EV-init uses multiplicative composition. Precision is the ratio of predictions that are in the test data to all the generated unseen predictions. Examples of top extracted rules (based on DistMult-tanh-EV-init) FilmInCountry Figure 2: Knowledge graph (Nicole Kidman, Nationality, Australia) (Hugh Jackman, Nationality, Australia) (Hugh Jackman, Friendship, Nicole Kidman) (Nicole Kidman, PerformIn, Cold Mountain) (Cold Mountain, FilmInCountry, U.S.A.) … Figure 1: RDF triples in KBs Results on FB15k-401: matrix multiplication better captures relation composition! t-SNE visualization of relation embeddings Figure 5: Relation embeddings of DistAdd Figure 6: Relation embeddings of DistMult celebrity_frienship location_division influenced celebrity_friendship celebrity_dated persion_spouse Location_division Capital_of hub_county Additional results Fast and Accurate! Horn-clause Rule Mining using Knowledge Base Embedding. Nicole Kidman Nationality Australia Figure 3: A neural network framework for multi-relational learning Ranking loss: