1. Knowledge Graph Embedding
龚赛赛

2. Reference
Zhen Wang, Jianwen Zhang, Jianlin Feng, Zheng Chen. Knowledge Graph Embedding by Translating on Hyperplanes. AAAI14
Antoine Bordes, Nicolas Usunier, Alberto Garcia-Dur´an. Translating Embeddings for Modeling Multi-relational Data. NIPS13
Antoine Bordes, JasonWeston, Ronan Collobert, Yoshua Bengio. Learning Structured Embeddings of Knowledge Bases. AAAI11

3. Contents
Introduction
Overview
Algorithm framework
Applications

4. Introduction
Great achievements in building large scale Knowledge graphs
Widely usage of several KGs such as WordNet, Freebase, Dbpedia
NLP, collaborative filtering, computer vision problem, etc
However, the general paradigm to support computing is still not clear
Two major difficulties
KG: symbolic and logical vs Application: numerical computing in continuous spaces
difficult to aggregate global knowledge over a graph
E.g. reasoning neither tractable nor robust

5. Introduction
Embed a knowledge graph into a continuous vector space while preserving certain properties of the original graph
Also apply to multi-relational data

6. Overview KG: a set of triplets (h,r,t )
h: head (subject)
r: relation (predicate)
t: tail (object)
Each entity is represented by a k-dimensional vector h (t)
A scoring function f r(h,t): measure the plausibility of a triplet (h,r,t) in the embedding space
Imply a transformation r on the pair of the entities which characterizes the relation r

7. Overview Scoring functions Simple difference Translation Affine
General linear
Bilinear
Nonlinear transformations

8. Overview Scoring functions Simple difference Translation Affine
General linear
Bilinear
Nonlinear transformations
V(马德里)-v(西班牙)+v(法国)=v(巴黎)

9. Overview
Translation
Linear
Translation

10. Overview
TransE (nips13)
TransH(aaai14)

11. Overview Distant(aaai11) More complex General linear Bilinear (tensor)
NTN (neural network) …

12. Algorithm framwork
Optimization object: minimize the scores of the true triplets and maximize the scores of the negative triplets
Training Algorithm: stochastic gradient descent (minibatch)
Negative triplets (∆’(h,r,t)): corrupt true triplets, e.g. (h,r,t)->(h,r,t’) or (h’,r,t)
Reduce false negative triplets
TransH

13. Algorithm framework 1. 初始化vectors 2. 每次随机挑选出一组true triplets

14. Applications Link prediction Triplets classification
Complete a triple (h,r,t) with h or t missing
e.g. predict the RDF classes of an entity (rdf:type)
Triplets classification
Confirm whether (h,r,t) is correct or not
Relational fact extraction from text
Learn on riplets from text-> induce new facts
Entity Relatedness
e.g. Learning taxonomies

15. Performance Dataset Link prediction Rank candidate objects/subjects
Evaluation metric: averaged rank,

16. Performance

17. Performance
sparse
dense

18. Thank you for you attention !