Knowledge Graph Embedding 龚赛赛 saisaigong@gmail.com 2015-01-26

Reference Zhen Wang, Jianwen Zhang, Jianlin Feng, Zheng Chen. Knowledge Graph Embedding by Translating on Hyperplanes. AAAI14 Antoine Bordes, Nicolas Usunier, Alberto Garcia-Dur´an. Translating Embeddings for Modeling Multi-relational Data. NIPS13 Antoine Bordes, JasonWeston, Ronan Collobert, Yoshua Bengio. Learning Structured Embeddings of Knowledge Bases. AAAI11

Contents Introduction Overview Algorithm framework Applications

Introduction Great achievements in building large scale Knowledge graphs Widely usage of several KGs such as WordNet, Freebase, Dbpedia NLP, collaborative filtering, computer vision problem, etc However, the general paradigm to support computing is still not clear Two major difficulties KG: symbolic and logical vs Application: numerical computing in continuous spaces difficult to aggregate global knowledge over a graph E.g. reasoning neither tractable nor robust

Introduction Embed a knowledge graph into a continuous vector space while preserving certain properties of the original graph Also apply to multi-relational data

Overview KG: a set of triplets (h,r,t ) h: head (subject) r: relation (predicate) t: tail (object) Each entity is represented by a k-dimensional vector h (t) A scoring function f r(h,t): measure the plausibility of a triplet (h,r,t) in the embedding space Imply a transformation r on the pair of the entities which characterizes the relation r

Overview Scoring functions Simple difference Translation Affine General linear Bilinear Nonlinear transformations

Overview Scoring functions Simple difference Translation Affine General linear Bilinear Nonlinear transformations V(马德里)-v(西班牙)+v(法国)=v(巴黎)

Overview Translation Linear Translation

Overview TransE (nips13) TransH(aaai14)

Overview Distant(aaai11) More complex General linear Bilinear (tensor) NTN (neural network) …

Algorithm framwork Optimization object: minimize the scores of the true triplets and maximize the scores of the negative triplets Training Algorithm: stochastic gradient descent (minibatch) Negative triplets (∆’(h,r,t)): corrupt true triplets, e.g. (h,r,t)->(h,r,t’) or (h’,r,t) Reduce false negative triplets TransH

Algorithm framework 1. 初始化vectors 2. 每次随机挑选出一组true triplets

Applications Link prediction Triplets classification Complete a triple (h,r,t) with h or t missing e.g. predict the RDF classes of an entity (rdf:type) Triplets classification Confirm whether (h,r,t) is correct or not Relational fact extraction from text Learn on riplets from text-> induce new facts Entity Relatedness e.g. Learning taxonomies

Performance Dataset Link prediction Rank candidate objects/subjects Evaluation metric: averaged rank, Hits@10

Performance

Performance sparse dense

Thank you for you attention !