BiG-Align: Fast Bipartite Graph Alignment

Slides:

Advertisements

Similar presentations

Location Recognition Given: A query image A database of images with known locations Two types of approaches: Direct matching: directly match image features.

Advertisements

Social network partition Presenter: Xiaofei Cao Partick Berg.

CMU SCS I2.2 Large Scale Information Network Processing INARC 1 Overview Goal: scalable algorithms to find patterns and anomalies on graphs 1. Mining Large.

School of Computer Science Carnegie Mellon University Duke University DeltaCon: A Principled Massive- Graph Similarity Function Danai Koutra Joshua T.

School of Computer Science Carnegie Mellon University National Taiwan University of Science & Technology Unifying Guilt-by-Association Approaches: Theorems.

LEARNING INFLUENCE PROBABILITIES IN SOCIAL NETWORKS Amit Goyal Francesco Bonchi Laks V. S. Lakshmanan University of British Columbia Yahoo! Research University.

Dept. of Computer Science Rutgers Node and Graph Similarity: Theory and Applications Danai Koutra (CMU) Tina Eliassi-Rad (Rutgers) Christos Faloutsos (CMU)

Multi-label Relational Neighbor Classification using Social Context Features Xi Wang and Gita Sukthankar Department of EECS University of Central Florida.

Siddharth Choudhary.  Refines a visual reconstruction to produce jointly optimal 3D structure and viewing parameters  ‘bundle’ refers to the bundle.

Wavelength Assignment in Optical Network Design Team 6: Lisa Zhang (Mentor) Brendan Farrell, Yi Huang, Mark Iwen, Ting Wang, Jintong Zheng Progress Report.

CS 678 –Boltzmann Machines1 Boltzmann Machine Relaxation net with visible and hidden units Learning algorithm Avoids local minima (and speeds up learning)

Online Social Networks and Media. Graph partitioning The general problem – Input: a graph G=(V,E) edge (u,v) denotes similarity between u and v weighted.

Fast Iterative Alignment of Pose Graphs with Poor Initial Estimates Edwin Olson John Leonard, Seth Teller

Endend endend Carnegie Mellon University Korea Advanced Institute of Science and Technology VoG: Summarizing and Understanding Large Graphs Danai Koutra.

1 Social Influence Analysis in Large-scale Networks Jie Tang 1, Jimeng Sun 2, Chi Wang 1, and Zi Yang 1 1 Dept. of Computer Science and Technology Tsinghua.

Communities in Heterogeneous Networks Chapter 4 1 Chapter 4, Community Detection and Mining in Social Media. Lei Tang and Huan Liu, Morgan & Claypool,

Using Structure Indices for Efficient Approximation of Network Properties Matthew J. Rattigan, Marc Maier, and David Jensen University of Massachusetts.

A General Model for Relational Clustering Bo Long and Zhongfei (Mark) Zhang Computer Science Dept./Watson School SUNY Binghamton Xiaoyun Wu Yahoo! Inc.

© 2011 IBM Corporation IBM Research SIAM-DM 2011, Mesa AZ, USA, Non-Negative Residual Matrix Factorization w/ Application to Graph Anomaly Detection Hanghang.

SCS CMU Proximity Tracking on Time- Evolving Bipartite Graphs Speaker: Hanghang Tong Joint Work with Spiros Papadimitriou, Philip S. Yu, Christos Faloutsos.

Design Patterns for Efficient Graph Algorithms in MapReduce Jimmy Lin and Michael Schatz University of Maryland Tuesday, June 29, 2010 This work is licensed.

Web Projections Learning from Contextual Subgraphs of the Web Jure Leskovec, CMU Susan Dumais, MSR Eric Horvitz, MSR.

Research at Intel Distributed Localization of Modular Robot Ensembles Robotics: Science and Systems 25 June 2008 Stanislav Funiak, Michael Ashley-Rollman.

Fast Random Walk with Restart and Its Applications

The Role of Specialization in LDPC Codes Jeremy Thorpe Pizza Meeting Talk 2/12/03.

Fast Spectrum Allocation in Coordinated Dynamic Spectrum Access Based Cellular Networks Anand Prabhu Subramanian*, Himanshu Gupta*,

SGD ON HADOOP FOR BIG DATA & HUGE MODELS Alex Beutel Based on work done with Abhimanu Kumar, Vagelis Papalexakis, Partha Talukdar, Qirong Ho, Christos.

Ahsanul Haque *, Swarup Chandra *, Latifur Khan * and Charu Aggarwal + * Department of Computer Science, University of Texas at Dallas + IBM T. J. Watson.

School of Computer Science Carnegie Mellon University National Taiwan University of Science & Technology Unifying Guilt-by-Association Approaches: Theorems.

MapReduce and Graph Data Chapter 5 Based on slides from Jimmy Lin’s lecture slides ( (licensed.

Adaptive CSMA under the SINR Model: Fast convergence using the Bethe Approximation Krishna Jagannathan IIT Madras (Joint work with) Peruru Subrahmanya.

Ahsanul Haque *, Swarup Chandra *, Latifur Khan * and Michael Baron + * Department of Computer Science, University of Texas at Dallas + Department of Mathematical.

EMIS 8381 – Spring Netflix and Your Next Movie Night Nonlinear Programming Ron Andrews EMIS 8381.

1 1 Stanford University 2 MPI for Biological Cybernetics 3 California Institute of Technology Inferring Networks of Diffusion and Influence Manuel Gomez.

PREDIcT: Towards Predicting the Runtime of Iterative Analytics Adrian Popescu 1, Andrey Balmin 2, Vuk Ercegovac 3, Anastasia Ailamaki

Fast Random Walk with Restart and Its Applications Hanghang Tong, Christos Faloutsos and Jia-Yu (Tim) Pan ICDM 2006 Dec , HongKong.

Finding Top-k Shortest Path Distance Changes in an Evolutionary Network SSTD th August 2011 Manish Gupta UIUC Charu Aggarwal IBM Jiawei Han UIUC.

A Passive Approach to Sensor Network Localization Rahul Biswas and Sebastian Thrun International Conference on Intelligent Robots and Systems 2004 Presented.

Random Graph Generator University of CS 8910 – Final Research Project Presentation Professor: Dr. Zhu Presented: December 8, 2010 By: Hanh Tran.

Rank Minimization for Subspace Tracking from Incomplete Data

Data Structures and Algorithms in Parallel Computing Lecture 3.

Data Structures and Algorithms in Parallel Computing Lecture 7.

Panther: Fast Top-k Similarity Search in Large Networks JING ZHANG, JIE TANG, CONG MA, HANGHANG TONG, YU JING, AND JUANZI LI Presented by Moumita Chanda.

Single-Pass Belief Propagation

Kijung Shin Jinhong Jung Lee Sael U Kang

Speaker : Yu-Hui Chen Authors : Dinuka A. Soysa, Denis Guangyin Chen, Oscar C. Au, and Amine Bermak From : 2013 IEEE Symposium on Computational Intelligence.

Sporadic model building for efficiency enhancement of the hierarchical BOA Genetic Programming and Evolvable Machines (2008) 9: Martin Pelikan, Kumara.

Center-Piece Subgraphs: Problem definition and Fast Solutions Hanghang Tong Christos Faloutsos Carnegie Mellon University.

Learning to Rank: From Pairwise Approach to Listwise Approach Authors: Zhe Cao, Tao Qin, Tie-Yan Liu, Ming-Feng Tsai, and Hang Li Presenter: Davidson Date:

Example Apply hierarchical clustering with d min to below data where c=3. Nearest neighbor clustering d min d max will form elongated clusters!

Bo Zong, Yinghui Wu, Ambuj K. Singh, Xifeng Yan 1 Inferring the Underlying Structure of Information Cascades

Optimal Relay Placement for Indoor Sensor Networks Cuiyao Xue †, Yanmin Zhu †, Lei Ni †, Minglu Li †, Bo Li ‡ † Shanghai Jiao Tong University ‡ HK University.

1 1 Stanford University 2 MPI for Biological Cybernetics 3 California Institute of Technology Inferring Networks of Diffusion and Influence Manuel Gomez.

Biao Wang 1, Ge Chen 1, Luoyi Fu 1, Li Song 1, Xinbing Wang 1, Xue Liu 2 1 Shanghai Jiao Tong University 2 McGill University

Accelerating K-Means Clustering with Parallel Implementations and GPU Computing Janki Bhimani Miriam Leeser Ningfang Mi

Inferring Networks of Diffusion and Influence

Outline Introduction State-of-the-art solutions

Data Driven Resource Allocation for Distributed Learning

A Tensor-Based Algorithm for High-Order Graph Matching

RankClus: Integrating Clustering with Ranking for Heterogeneous Information Network Analysis Yizhou Sun, Jiawei Han, Peixiang Zhao, Zhijun Yin, Hong Cheng,

Latent Space Model for Road Networks to Predict Time-Varying Traffic

Community Distribution Outliers in Heterogeneous Information Networks

Collaborative Filtering Matrix Factorization Approach

Neural Networks for Vertex Covering

Binghui Wang, Le Zhang, Neil Zhenqiang Gong

GANG: Detecting Fraudulent Users in OSNs

Tahsin Reza Matei Ripeanu Nicolas Tripoul

Markov Networks.

Analysis of Large Graphs: Overlapping Communities

Presentation transcript:

BiG-Align: Fast Bipartite Graph Alignment Danai Koutra Hanghang Tong David Lubensky IEEE ICDM, 7-10 December 2013, Dallas, Texas, USA

Can we identify users across social networks? Same or “similar” users? Task: link prediction (friend link), recommendations Danai Koutra (CMU)

protein-protein alignment More applications? chemical compound comparison protein-protein alignment Optical character recognition link prediction & viral marketing IR: Information Retrieval link prediction and viral marketing in social networks 2) protein-protein interaction networks 3) optical character recognition 4) chemical compounds comparison 5) structure matching in databases 6) synonyms extraction in information retrieval 7) Wiki translation (graphs of documents from different language – learn automatic translation) IR: synonym extraction wiki translation Structure matching in DB Danai Koutra (CMU)

RoadMap Problem Definition What’s different? BiG-Align Uni-Align Conclusions Danai Koutra (CMU)

Problem Definition INPUT: A, B A B u users s e r groups 1 1 0 0 0 0 0 1 0 1 0 1 0 1 1 0 1 users 1 1 0 0 0 0 1 0 1 0 0 0 1 1 1 0 0 0 1 0 A B Combinatorial problem Relax the problem + approximate the solution Danai Koutra (CMU)

Problem Definition INPUT: A, B OUTPUT: P and … A B P (users) B A (permutation matrices) u s e r groups 1 1 0 0 0 0 1 0 1 0 1 0 1 1 0 1 users 1 1 0 0 0 0 1 0 1 0 0 0 1 1 1 0 0 0 1 0 A B P (users) A B Combinatorial problem Relax the problem + approximate the solution Output: mappings at two granularities; users and groups/communities Danai Koutra (CMU)

Problem Definition INPUT: A, B OUTPUT: P and Q (permutation matrices) s.t. min || PAQ - B|| F 2 u s e r groups 1 1 0 0 0 0 1 0 1 0 1 0 1 1 0 1 users 1 1 0 0 0 0 1 0 1 0 0 0 1 1 1 0 0 0 1 0 A B constraints / relaxations Graph isomorphism: HARD (P or NP complete?) Subgraph isomorphism: NP-complete And now what? P (users) A B Q (groups) Output: mappings at two granularities; users and groups/communities users/groups permutation of A permutation of users/groups in A Danai Koutra (CMU)

Problem Definition: constraints u g 1 1 0 0 … … 1 1 0 1 1 1 0 0 0 … … … 0 1 0 1 0 A B INPUT: A, B OUTPUT: P, Q correspondence matrices s.t. min || PAQ - B|| F 2 P (users) A B Q (groups) Danai Koutra (CMU)

Problem Definition: constraints u g 1 1 0 0 … … 1 1 0 1 1 1 0 0 0 … … … 0 1 0 1 0 A B INPUT: A, B OUTPUT: P, Q correspondence matrices s.t. min || PAQ - B|| F 2 CONSTRAINTS: Pij, Qij = probabilities (not 1-1 mapping) sparse matrices P and Q (more efficient for large scale graphs) P (users) A B Q (groups) Danai Koutra (CMU)

RoadMap Problem Definition What’s different? BiG-Align Uni-Align Conclusions Danai Koutra (CMU)

What’s different? Focus on bipartite graphs BiG-Align other approaches vs. Focus on bipartite graphs Danai Koutra (CMU)

What’s different? Focus on bipartite graphs BiG-Align other approaches vs. Focus on bipartite graphs New optimization problem/constraints Danai Koutra (CMU)

What’s different? Focus on bipartite graphs BiG-Align other approaches vs. Focus on bipartite graphs New optimization problem/constraints The hope is: the specific graph structure will lead to more accurate graph alignment Danai Koutra (CMU)

Why bipartite graphs? ubiquitous – e.g., users-files, authors-papers, customers-products, users-msg/groups user-movie rating graphs Danai Koutra (CMU)

Why bipartite graphs? nodes communities ubiquitous – e.g., users-files, authors-papers, customers-products, users-msg/groups user-movie rating graphs coupled alignment: individual & community-level nodes communities Danai Koutra (CMU)

Why bipartite graphs? nodes communities ubiquitous – e.g., users-files, authors-papers, customers-products, users-msg/groups user-movie rating graphs coupled alignment: individual & community-level conversion of uni-partite graph to bi-partite --> clustering + (2) nodes communities Danai Koutra (CMU)

Why bipartite graphs? nodes communities users time email ubiquitous – e.g., users-files, authors-papers, customers-products, users-msg/groups user-movie rating graphs coupled alignment: individual & community-level conversion of unipartite graph to bipartite --> clustering + (2) general formulation: match clouds of points (point-feature graph) tensors (e.g. time-evolving, or other 3rd dimension) nodes communities users email time Danai Koutra (CMU)

RoadMap Problem Definition What’s different? BiG-Align Uni-Align Conclusions Danai Koutra (CMU)

alternating, projected BiG-Align: algorithm DeTaIlS INPUT: A, B initialize(P) initialize(Q) Pk+1 = Pk – ηP df(Pk, Qk)/dP valid_projection(Pk+1) Qk+1 = Qk – ηQ df(Pk+1, Qk)/dQ valid_projection(Qk+1) update(ηP, ηQ) OUTPUT: P, Q alternating, projected gradient descent until convergence Danai Koutra (CMU)

BiG-Align: algorithm DeTaIlS INPUT: A, B initialize(P) initialize(Q) Pk+1 = Pk – ηP df(Pk, Qk)/dP valid_projection(Pk+1) Qk+1 = Qk – ηQ df(Pk+1, Qk)/dQ valid_projection(Qk+1) update(ηP, ηQ) OUTPUT: P, Q Probabilistic Constraint until convergence Danai Koutra (CMU)

BiG-Align: algorithm DeTaIlS INPUT: A, B initialize(P) initialize(Q) Pk+1 = Pk – ηP df(Pk, Qk)/dP valid_projection(Pk+1) Qk+1 = Qk – ηQ df(Pk+1, Qk)/dQ valid_projection(Qk+1) update(ηP, ηQ) OUTPUT: P, Q Sparsity Constraint until convergence Danai Koutra (CMU)

min f = min||| PAQ – B||F 2 + λΣPij + μΣQij BiG-Align: algorithm DeTaIlS INPUT: A, B initialize(P) initialize(Q) Pk+1 = Pk – ηP df(Pk, Qk)/dP valid_projection(Pk+1) Qk+1 = Qk – ηQ df(Pk+1, Qk)/dQ valid_projection(Qk+1) update(ηP, ηQ) OUTPUT: P, Q Sparsity Constraint until convergence min f = min||| PAQ – B||F 2 + λΣPij + μΣQij Danai Koutra (CMU)

RoadMap Problem Definition What’s different? BiG-Align Uni-Align Optimizations Uni-Align Conclusions Danai Koutra (CMU)

BiG-Align: Optimizations Details INPUT: A, B initialize(P) initialize(Q) Pk+1 = Pk – ηP df(Pk, Qk)/dP valid_projection(Pk+1) Qk+1 = Qk – ηQ df(Pk+1, Qk)/dQ valid_projection(Qk+1) update(ηP, ηQ) OUTPUT: P, Q alternating, projected gradient descent until convergence Danai Koutra (CMU)

BiG-Align: Optimizations Details INPUT: A, B initialize(P) initialize(Q) Pk+1 = Pk – ηP df(Pk, Qk)/dP valid_projection(Pk+1) Qk+1 = Qk – ηQ df(Pk+1, Qk)/dQ valid_projection(Qk+1) update(ηP, ηQ) OUTPUT: P, Q alternating, projected gradient descent alternating, projected gradient descent until convergence Danai Koutra (CMU)

Optimization 1: Structurally equivalent nodes Details Aggregation to super-nodes Graph A Danai Koutra (CMU)

BiG-Align: Optimizations Details INPUT: A, B initialize(P) initialize(Q) Pk+1 = Pk – ηP df(Pk, Qk)/dP valid_projection(Pk+1) Qk+1 = Qk – ηQ df(Pk+1, Qk)/dQ valid_projection(Qk+1) update(ηP, ηQ) OUTPUT: P, Q alternating, projected gradient descent alternating, projected gradient descent until convergence Danai Koutra (CMU)

Optimization 2: Initialization of P and Q Details Why is the initialization important? global minimum local minima … Danai Koutra (CMU)

Optimization 2: Initialization of P and Q Details Social networks are structured: the degree distribution is power-law like. ranked nodes log(degree) Social networks are structured: the degree distribution of the nodes is power-law like. Danai Koutra (CMU)

Optimization 2: Initialization of P and Q Details 1-1 matching of clusters of degrees Network-inspired initialization degree rank of node knee k 1-1 matching of top k nodes user degrees of GA user degrees in GB … . 2000 1500 1000 945 940 800 799 750 740 735 730 3 2 1 500 450 449 445 P k cluster 1 cluster 2 cluster n Top k nodes with significantly higher degree -> handle them separately from the low degree nodes cluster 1 cluster 2 cluster n Danai Koutra (CMU)

BiG-Align: Optimizations Details INPUT: A, B initialize(P) initialize(Q) Pk+1 = Pk – ηP df(Pk, Qk)/dP valid_projection(Pk+1) Qk+1 = Qk – ηQ df(Pk+1, Qk)/dQ valid_projection(Qk+1) update(ηP, ηQ) OUTPUT: P, Q alternating, projected gradient descent alternating, projected gradient descent until convergence Danai Koutra (CMU)

Optimization 3: Steps of gradient descent Details Constant step: thrashing or slow convergence Danai Koutra (CMU)

Optimization 3: Steps of gradient descent Details Variable step with line search: strategy for local optimum ηP = argmin f(ηP) = g1(P,Q,A,B) ηQ = argmin f(ηQ) = g2(P,Q,A,B) closed formulas Danai Koutra (CMU)

Optimization 3: Steps of gradient descent Details Variable step with line search: strategy for local optimum BiG-Align-Exact: computes the steps at every iteration ηP = argmin f(ηP) = g1(P,Q,A,B) ηQ = argmin f(ηQ) = g2(P,Q,A,B) closed formulas Danai Koutra (CMU)

Optimization 3: Steps of gradient descent Details But 3.104 104 2.104 10-4 3. 10-4 5. 10-4 step size (η) iterations Slow change in the steps Danai Koutra (CMU)

Optimization 3: Steps of gradient descent Details But BiG-Align-Skip: compute η’s every m (=500) iterations 3.104 104 2.104 10-4 3. 10-4 5. 10-4 step size (η) iterations Slow change in the steps Danai Koutra (CMU)

RoadMap Problem Definition What’s different? BiG-Align Uni-Align Experiments Uni-Align Conclusions Danai Koutra (CMU)

Experimental Setup Implementation: Matlab Dataset: IMDB movie-genre graph and subgraphs (1027 movies x 27 genres) Setup: random permutations noise level: 0 - 20 % Ground truth Simulate real-world applications Danai Koutra (CMU)

State-of-the-art Background Umeyama’s algorithm [Umeyama88]: SVD-based NMF-based approach [Ding+08]: Builds on top of Umeyama’s approach Net-Align [Bayati+09] Belief Propagation Danai Koutra (CMU)

State-of-the-art Background Umeyama’s algorithm [Umeyama88]: SVD-based NMF-based approach [Ding+08]: Builds on top of Umeyama’s approach Net-Align [Bayati+09] Belief Propagation Bi-partite  Uni-partite Danai Koutra (CMU)

Big-Align: Accuracy vs. Runtime Umeyama NetAlign NMF-based BiG-Align exact BiG-Align skip Small marker size: 300 nodes Big marker size: 700 nodes marker size related to graph size Danai Koutra (CMU)

Big-Align: Accuracy vs. Runtime Umeyama NetAlign NMF-based BiG-Align exact BiG-Align skip Big-Align improves both speed and accuracy. Danai Koutra (CMU)

Big-Align: Accuracy w.r.t. noise BiG-Align-exact BiG-Align-skip NMF-based NetAlign-deg NetAlign-full Umeyama Danai Koutra (CMU)

Big-Align: Accuracy w.r.t. noise BiG-Align-exact BiG-Align-skip NMF-based NetAlign-deg NetAlign-full Umeyama BiG-Align improves the accuracy for almost all levels of noise. Danai Koutra (CMU)

RoadMap Problem Definition What’s different? BiG-Align Uni-Align Conclusions Danai Koutra (CMU)

Algorithm: Uni-Align Details n nodes d features min || PAQ - B||F 2 P fixed node degree clustering coeff … … Danai Koutra (CMU)

Algorithm: Uni-Align Details n nodes d features O(n.d2) min || PAQ - B||F 2 P P = g*(A,B,S,U)= = closed-form solution SVD A = USVT O(n.d2) Explain it somehow Danai Koutra (CMU)

RoadMap Problem Definition What’s different? BiG-Align Uni-Align Experiments Conclusions Danai Koutra (CMU)

Uni-Align Dataset: Facebook friendship graph (64K users) Setup: uni-partite  bi-partite graph Feature extraction node degree egonet degree edges in egonet mean degree of node’s neighbors egonet Danai Koutra (CMU)

Uni-Align: Accuracy vs. Runtime NMF-based NetAlign Umeyama Uni-Align, followed by Net-Align, is more accurate and faster than other approaches. Danai Koutra (CMU)

Uni-Align: Runtime Uni-Align Umeyama Uni-Align NMF-based NetAlign-deg NetAlign-full Uni-Align is 2x - 31,700x faster depending on graph size. Danai Koutra (CMU)

RoadMap Problem Definition What’s different? BiG-Align Uni-Align Conclusions Danai Koutra (CMU)

Conclusions Formulation: new problem / constraints Danai Koutra (CMU)

Conclusions Formulation: new problem / constraints Algorithms: BiG-Align: optimized alternating projected gradient descent Uni-Align: alignment for uni-partite graphs Danai Koutra (CMU)

Conclusions Formulation: new problem / constraints Algorithms: BiG-Align: optimized alternating projected gradient descent Uni-Align: alignment for uni-partite graphs Evaluations: more accurate and efficient Danai Koutra (CMU)

Beyond BiG-Align: Multi-way Linkage ~ S1: Dynamic Graph Linkage S2: Community-level Linkage ~ ~ beyond this paper, we found that big-align forms a core building block for several other more complicated matching task, such as ..... we will not go into the details, but the bottom line is that all of them can be built upon big-align. and this also led to 7 patents S3: Hetero. Graph Linkage S4: Multi-relational DB Linkage All build upon BiG-Align (2) Led to 7 patents Danai Koutra (CMU)

Thank you! http://www.cs.cmu.edu/~dkoutra danai@cs.cmu.edu Danai Koutra (CMU)