Improving Cross-lingual Entity Alignment via Optimal Transport

Slides:



Advertisements
Similar presentations
1 Manifold Alignment for Multitemporal Hyperspectral Image Classification H. Lexie Yang 1, Melba M. Crawford 2 School of Civil Engineering, Purdue University.
Advertisements

Principal Component Analysis Based on L1-Norm Maximization Nojun Kwak IEEE Transactions on Pattern Analysis and Machine Intelligence, 2008.
Carolina Galleguillos, Brian McFee, Serge Belongie, Gert Lanckriet Computer Science and Engineering Department Electrical and Computer Engineering Department.
Linear Model Incorporating Feature Ranking for Chinese Documents Readability Gang Sun, Zhiwei Jiang, Qing Gu and Daoxu Chen State Key Laboratory for Novel.
Maurice Hermans.  Ontologies  Ontology Mapping  Research Question  String Similarities  Winkler Extension  Proposed Extension  Evaluation  Results.
Proportion Priors for Image Sequence Segmentation Claudia Nieuwenhuis, etc. ICCV 2013 Oral.
Funding Networks Abdullah Sevincer University of Nevada, Reno Department of Computer Science & Engineering.
One-Shot Multi-Set Non-rigid Feature-Spatial Matching
Chen Cheng1, Haiqin Yang1, Irwin King1,2 and Michael R. Lyu1
Prénom Nom Document Analysis: Data Analysis and Clustering Prof. Rolf Ingold, University of Fribourg Master course, spring semester 2008.
Margin Based Sample Weighting for Stable Feature Selection Yue Han, Lei Yu State University of New York at Binghamton.
Learning Table Extraction from Examples Ashwin Tengli, Yiming Yang and Nian Li Ma School of Computer Science Carnegie Mellon University Coling 04.
Machine Learning in Simulation-Based Analysis 1 Li-C. Wang, Malgorzata Marek-Sadowska University of California, Santa Barbara.
Joint Image Clustering and Labeling by Matrix Factorization
Longbiao Kang, Baotian Hu, Xiangping Wu, Qingcai Chen, and Yan He Intelligent Computing Research Center, School of Computer Science and Technology, Harbin.
Jinhui Tang †, Shuicheng Yan †, Richang Hong †, Guo-Jun Qi ‡, Tat-Seng Chua † † National University of Singapore ‡ University of Illinois at Urbana-Champaign.
OMAP: An Implemented Framework for Automatically Aligning OWL Ontologies SWAP, December, 2005 Raphaël Troncy, Umberto Straccia ISTI-CNR
Transfer Learning From Multiple Source Domains via Consensus Regularization Ping Luo, Fuzhen Zhuang, Hui Xiong, Yuhong Xiong, Qing He.
1 A Discriminative Approach to Topic- Based Citation Recommendation Jie Tang and Jing Zhang Presented by Pei Li Knowledge Engineering Group, Dept. of Computer.
Introduction to variable selection I Qi Yu. 2 Problems due to poor variable selection: Input dimension is too large; the curse of dimensionality problem.
Mining Discriminative Components With Low-Rank and Sparsity Constraints for Face Recognition Qiang Zhang, Baoxin Li Computer Science and Engineering Arizona.
1 Formal Models for Expert Finding on DBLP Bibliography Data Presented by: Hongbo Deng Co-worked with: Irwin King and Michael R. Lyu Department of Computer.
Xiaoxiao Shi, Qi Liu, Wei Fan, Philip S. Yu, and Ruixin Zhu
Mining High Utility Itemset in Big Data
Data Mining and Machine Learning Lab Unsupervised Feature Selection for Linked Social Media Data Jiliang Tang and Huan Liu Computer Science and Engineering.
Online Kinect Handwritten Digit Recognition Based on Dynamic Time Warping and Support Vector Machine Journal of Information & Computational Science, 2015.
A Novel Local Patch Framework for Fixing Supervised Learning Models Yilei Wang 1, Bingzheng Wei 2, Jun Yan 2, Yang Hu 2, Zhi-Hong Deng 1, Zheng Chen 2.
Mingyang Zhu, Huaijiang Sun, Zhigang Deng Quaternion Space Sparse Decomposition for Motion Compression and Retrieval SCA 2012.
Andreas Papadopoulos - [DEXA 2015] Clustering Attributed Multi-graphs with Information Ranking 26th International.
Paired Sampling in Density-Sensitive Active Learning Pinar Donmez joint work with Jaime G. Carbonell Language Technologies Institute School of Computer.
2005/12/021 Fast Image Retrieval Using Low Frequency DCT Coefficients Dept. of Computer Engineering Tatung University Presenter: Yo-Ping Huang ( 黃有評 )
Dual Transfer Learning Mingsheng Long 1,2, Jianmin Wang 2, Guiguang Ding 2 Wei Cheng, Xiang Zhang, and Wei Wang 1 Department of Computer Science and Technology.
H. Lexie Yang1, Dr. Melba M. Crawford2
Optimal Dimensionality of Metric Space for kNN Classification Wei Zhang, Xiangyang Xue, Zichen Sun Yuefei Guo, and Hong Lu Dept. of Computer Science &
WSP: A Network Coordinate based Web Service Positioning Framework for Response Time Prediction Jieming Zhu, Yu Kang, Zibin Zheng and Michael R. Lyu The.
Detecting New a Priori Probabilities of Data Using Supervised Learning Karpov Nikolay Associate professor NRU Higher School of Economics.
CoNMF: Exploiting User Comments for Clustering Web2.0 Items Presenter: He Xiangnan 28 June School of Computing National.
Iterative similarity based adaptation technique for Cross Domain text classification Under: Prof. Amitabha Mukherjee By: Narendra Roy Roll no: Group:
Ultra-high dimensional feature selection Yun Li
2016/9/301 Exploiting Wikipedia as External Knowledge for Document Clustering Xiaohua Hu, Xiaodan Zhang, Caimei Lu, E. K. Park, and Xiaohua Zhou Proceeding.
Experience Report: System Log Analysis for Anomaly Detection
Bag-of-Visual-Words Based Feature Extraction
CCNT Lab of Zhejiang University
Deep Compositional Cross-modal Learning to Rank via Local-Global Alignment Xinyang Jiang, Fei Wu, Xi Li, Zhou Zhao, Weiming Lu, Siliang Tang, Yueting.
An Empirical Study of Learning to Rank for Entity Search
Chinese Academy of Sciences, Beijing, China
کاربرد نگاشت با حفظ تنکی در شناسایی چهره
Joint Training for Pivot-based Neural Machine Translation
Cold-Start Heterogeneous-Device Wireless Localization
Fast Preprocessing for Robust Face Sketch Synthesis
Asymmetric Correlation Regularized Matrix Factorization for Web Service Recommendation Qi Xie1, Shenglin Zhao2, Zibin Zheng3, Jieming Zhu2 and Michael.
Presenter: Hajar Emami
Community Distribution Outliers in Heterogeneous Information Networks
outline Two region based shape analysis approach
MEgo2Vec: Embedding Matched Ego Networks for User Alignment Across Social Networks Jing Zhang+, Bo Chen+, Xianming Wang+, Fengmei Jin+, Hong Chen+, Cuiping.
Deep Cross-media Knowledge Transfer
Y2Seq2Seq: Cross-Modal Representation Learning for 3D Shape and Text by Joint Reconstruction and Prediction of View and Word Sequences 1, Zhizhong.
Binghui Wang, Le Zhang, Neil Zhenqiang Gong
Asymmetric Transitivity Preserving Graph Embedding
How To Extend the Training Data
Graph-based Security and Privacy Analytics via Collective Classification with Joint Weight Learning and Propagation Binghui Wang, Jinyuan Jia, and Neil.
View Inter-Prediction GAN: Unsupervised Representation Learning for 3D Shapes by Learning Global Shape Memories to Support Local View Predictions 1,2 1.
Automated Analysis and Code Generation for Domain-Specific Models
Actively Learning Ontology Matching via User Interaction
FOCUS PRIOR ESTIMATION FOR SALIENT OBJECT DETECTION
Learning Incoherent Sparse and Low-Rank Patterns from Multiple Tasks
Shichao Pei, Lu Yu, Robert Hoehndorf, Xiangliang Zhang
Learning to Detect Human-Object Interactions with Knowledge
Learning Dual Retrieval Module for Semi-supervised Relation Extraction
Low-Rank Sparse Feature Selection for Patient Similarity Learning
Presentation transcript:

Improving Cross-lingual Entity Alignment via Optimal Transport Shichao Pei, Lu Yu, Xiangliang Zhang CEMSE King Abdullah University of Science and Technology (KAUST), SA 10/14/2019 The 28th International Joint Conference on Artificial Intelligence. August 10-16, 2019, Macao, China

Outline The Task Background and Related Work Proposed Model Experiment Conclusion 10/14/2019 The 28th International Joint Conference on Artificial Intelligence. August 10-16, 2019, Macao, China

The Task Problem KGs differ in language and content – Similar domain, e.g., geography. In each KG which includes a set of triples, each of which includes a head entity (e.g., Mexico), a relation (e.g., neighbor of) and a tail entity (e.g., USA). Entity alignment is to find pairs of entities with the same meaning (one in English KG and the other in French KG), so called aligned entities, e.g., Mexico with Mexique, USA with Etats-Unis. 10/14/2019 The 28th International Joint Conference on Artificial Intelligence. August 10-16, 2019, Macao, China

The Task Motivation Various methods, sources, and languages have been explored to construct KGs, and most existing KGs are developed separately. These KGs are inevitably heterogeneous in surface forms and typically supplementary in contents. It is thus essential to align entities in multiple KGs and join them into a unified KG for knowledge-driven applications. 10/14/2019 The 28th International Joint Conference on Artificial Intelligence. August 10-16, 2019, Macao, China

Background Related Work Feature Engineering Methods The semantics of OWL properties [Hu et al., 2011] Compatible neighbors and attribute values of entities [Suchanek et al., 2012] Structural information of relations [Lacoste-Julien et al., 2013] Well-designed hand-crafted features [Mahdisoltani et al., 2014] Time-consuming, labor-expensive and suffers from extension inflexibility. Embedding-based Methods Encoding the KGs in separated embedding space or unified embedding space. MTransE [Chen et al., 2017], ITransE [Zhu et al., 2017]. Jointly modeling the KGs and attributes. JAPE [Sun et al., 2017], KDCoE [Chen et al., 2018]. Iteratively enlarging the labeled entity pairs based on the bootstrapping strategy. BootEA [Sun et al., 2018] 10/14/2019 The 28th International Joint Conference on Artificial Intelligence. August 10-16, 2019, Macao, China

Background Limitations of Current Methods Limited gain due to the shortage of labeled entity pairs Ignorance of duality Failure on matching the whole distribution 10/14/2019 The 28th International Joint Conference on Artificial Intelligence. August 10-16, 2019, Macao, China

Our Objective Objective Challenges Learning the translation matrix by dually minimizing both entity-level and group-level loss. The group-level loss describes the discrepancy between distributions of different embeddings. Challenges The group-level loss is difficult to measure using a statistical distance. GAN still suffers from an unstably weak learning signal. Inspired by the progress of optimal transport, how to use the theory to match distributions is still not explored. 10/14/2019 The 28th International Joint Conference on Artificial Intelligence. August 10-16, 2019, Macao, China

Contribution Proposed to solve entity alignment by dually minimizing both the entity-level loss and the group-level loss via optimal transport theory. Imposed L2,1 norm on the dual translation matrices, which can enforce the translation matrix to be close to orthogonal. Conducted extensive experiments on six real-world datasets and show the superior performance of our proposed model over the state- of-the-art methods. 10/14/2019 The 28th International Joint Conference on Artificial Intelligence. August 10-16, 2019, Macao, China

Outline The Task Background and Related Work Proposed Model Experiment Conclusion 10/14/2019 The 28th International Joint Conference on Artificial Intelligence. August 10-16, 2019, Macao, China

Proposed Model Knowledge Graph Embedding Entity-level loss TransE Margin-based ranking Entity-level loss 1 3 2 In our paper, we consider two KGs, our method is suitable for multiple KGs. First, We build our model based on the basic TransE which is based on the margin based ranking loss. The formulas are same as the standard TransE loss. Then, we define the entity-level loss. After obtaining entity embeddings of graph Gi and Gj from TransE, we make labeled entities aligned by dually minimizing two distances. That is to say, M1 is learned to transfer the embeddings of Gi into the embedding space of Gj, and M2 is to transfer the embeddings of Gj into the embedding space of Gi. Embeddings of head, tail entity, and relation. M1 transfer embedding of Gi into the embedding space of Gj 10/14/2019 The 28th International Joint Conference on Artificial Intelligence. August 10-16, 2019, Macao, China

Proposed Model Group-level Loss – Optimal Transport based. WGAN Transferred embedding entity embedding Then, we define the group-level loss. We define the group-level loss by measuring the difference between p and q with optimal transport distance, 􏰠(p,q) denotes the set of all joint distributions γ(p,q) with marginals p(x) and q(y); c(x,y):G×G→R indicates the transportation cost function for moving x to y. The group-level loss measured by Eq. (8) is hard to calculate directly due to its high computational complexity. Then we can convert it to the next one according to some math theorems. Hence, solving the optimal transport problem has been tran-formed to optimize Wasserstein GAN. Transport matrix WGAN 10/14/2019 The 28th International Joint Conference on Artificial Intelligence. August 10-16, 2019, Macao, China

Proposed Model Regularizer The translation matrix is desired to be orthogonal. Employing L2,1 norm as the regularizer. Preventing the matrix to be dense, and mitigating the error induced by dense matrix. 10/14/2019 The 28th International Joint Conference on Artificial Intelligence. August 10-16, 2019, Macao, China

Outline The Task Background and Related Work Proposed Model Experiment Conclusion 10/14/2019 The 28th International Joint Conference on Artificial Intelligence. August 10-16, 2019, Macao, China

Experiment Datasets Baselines: three categories Evaluation Metric Category 1: Encoding the KGs in separated embedding space or unified embedding space. MTransE, ITransE. Category 2: Jointly modeling the KGs and attributes. JAPE, GCN based method. Category 3: Iteratively enlarging the labeled entity pairs based on the bootstrapping strategy. BootEA, ITransE Evaluation Metric We adopt popular metrics Hits@k and MRR 10/14/2019 The 28th International Joint Conference on Artificial Intelligence. August 10-16, 2019, Macao, China

Experiment 10/14/2019 The 28th International Joint Conference on Artificial Intelligence. August 10-16, 2019, Macao, China

Experiment Sensitivity to the Proportion of Prior Aligned Entities. All methods have better performance with the growth of the proportion of aligned entities. OTEA and BootEA have much better performance than other baselines, due to the employment of unlabeled data and selection of labeled data. 10/14/2019 The 28th International Joint Conference on Artificial Intelligence. August 10-16, 2019, Macao, China

Experiment Sensitivity to the Dimension of KG Embeddings Time Complexity Comparison OTEA method is consistently better than all other baselines. And its performance is quite stable when varying d. OTEA is faster than BootEA, because the bootstrapping based method need to propose the new aligned entities by calculating the similarity with all unaligned entities. 10/14/2019 The 28th International Joint Conference on Artificial Intelligence. August 10-16, 2019, Macao, China

Outline The Task Background and Related Work Proposed Model Experiment Conclusion 10/14/2019 The 28th International Joint Conference on Artificial Intelligence. August 10-16, 2019, Macao, China

Conclusion Introduced a novel framework for cross-lingual entity alignment. Solved the entity alignment by dually minimizing both the entity-level loss and group-level loss via optimal transport theory. Imposed regularizer on the dual translation matrices to mitigate the effect of noise during transformation. Achieved superior results comparing with other SOTA methods. In future work, how to combine the model with attribute and relation information. 10/14/2019 The 28th International Joint Conference on Artificial Intelligence. August 10-16, 2019, Macao, China

Thank you for your attention Thank you for your attention! Q&A Lab of Machine Intelligence and kNowledge Engineering (MINE): http://mine.kaust.edu.sa/ 10/14/2019 The 28th International Joint Conference on Artificial Intelligence. August 10-16, 2019, Macao, China