Presentation is loading. Please wait.

Presentation is loading. Please wait.

BiG-Align: Fast Bipartite Graph Alignment

Published byMadalyn Liff Modified over 9 years ago

Similar presentations

Presentation on theme: "BiG-Align: Fast Bipartite Graph Alignment"— Presentation transcript:

1 BiG-Align: Fast Bipartite Graph Alignment
Danai Koutra Hanghang Tong David Lubensky IEEE ICDM, 7-10 December 2013, Dallas, Texas, USA

2 Can we identify users across social networks?
Same or “similar” users? Task: link prediction (friend link), recommendations Danai Koutra (CMU)

3 protein-protein alignment
More applications? chemical compound comparison protein-protein alignment Optical character recognition link prediction & viral marketing IR: Information Retrieval link prediction and viral marketing in social networks 2) protein-protein interaction networks 3) optical character recognition 4) chemical compounds comparison 5) structure matching in databases 6) synonyms extraction in information retrieval 7) Wiki translation (graphs of documents from different language – learn automatic translation) IR: synonym extraction wiki translation Structure matching in DB Danai Koutra (CMU)

4 RoadMap Problem Definition What’s different? BiG-Align Uni-Align
Conclusions Danai Koutra (CMU)

5 Problem Definition INPUT: A, B A B u users s e r groups 1 1 0 0 0
users A B Combinatorial problem Relax the problem + approximate the solution Danai Koutra (CMU)

6 Problem Definition INPUT: A, B OUTPUT: P and … A B P (users) B A
(permutation matrices) u s e r groups users A B P (users) A B Combinatorial problem Relax the problem + approximate the solution Output: mappings at two granularities; users and groups/communities Danai Koutra (CMU)

7 Problem Definition INPUT: A, B OUTPUT: P and Q
(permutation matrices) s.t. min || PAQ - B|| F 2 u s e r groups users A B constraints / relaxations Graph isomorphism: HARD (P or NP complete?) Subgraph isomorphism: NP-complete And now what? P (users) A B Q (groups) Output: mappings at two granularities; users and groups/communities users/groups permutation of A permutation of users/groups in A Danai Koutra (CMU)

8 Problem Definition: constraints
u g … … … … … A B INPUT: A, B OUTPUT: P, Q correspondence matrices s.t. min || PAQ - B|| F 2 P (users) A B Q (groups) Danai Koutra (CMU)

9 Problem Definition: constraints
u g … … … … … A B INPUT: A, B OUTPUT: P, Q correspondence matrices s.t. min || PAQ - B|| F 2 CONSTRAINTS: Pij, Qij = probabilities (not 1-1 mapping) sparse matrices P and Q (more efficient for large scale graphs) P (users) A B Q (groups) Danai Koutra (CMU)

10 RoadMap Problem Definition What’s different? BiG-Align Uni-Align
Conclusions Danai Koutra (CMU)

11 What’s different? Focus on bipartite graphs BiG-Align other approaches
vs. Focus on bipartite graphs Danai Koutra (CMU)

12 What’s different? Focus on bipartite graphs
BiG-Align other approaches vs. Focus on bipartite graphs New optimization problem/constraints Danai Koutra (CMU)

13 What’s different? Focus on bipartite graphs
BiG-Align other approaches vs. Focus on bipartite graphs New optimization problem/constraints The hope is: the specific graph structure will lead to more accurate graph alignment Danai Koutra (CMU)

14 Why bipartite graphs? ubiquitous – e.g., users-files, authors-papers, customers-products, users-msg/groups user-movie rating graphs Danai Koutra (CMU)

15 Why bipartite graphs? nodes communities
ubiquitous – e.g., users-files, authors-papers, customers-products, users-msg/groups user-movie rating graphs coupled alignment: individual & community-level nodes communities Danai Koutra (CMU)

16 Why bipartite graphs? nodes communities
ubiquitous – e.g., users-files, authors-papers, customers-products, users-msg/groups user-movie rating graphs coupled alignment: individual & community-level conversion of uni-partite graph to bi-partite --> clustering + (2) nodes communities Danai Koutra (CMU)

17 Why bipartite graphs? nodes communities users time email
ubiquitous – e.g., users-files, authors-papers, customers-products, users-msg/groups user-movie rating graphs coupled alignment: individual & community-level conversion of unipartite graph to bipartite --> clustering + (2) general formulation: match clouds of points (point-feature graph) tensors (e.g. time-evolving, or other 3rd dimension) nodes communities users time Danai Koutra (CMU)

18 RoadMap Problem Definition What’s different? BiG-Align Uni-Align
Conclusions Danai Koutra (CMU)

19 alternating, projected
BiG-Align: algorithm DeTaIlS INPUT: A, B initialize(P) initialize(Q) Pk+1 = Pk – ηP df(Pk, Qk)/dP valid_projection(Pk+1) Qk+1 = Qk – ηQ df(Pk+1, Qk)/dQ valid_projection(Qk+1) update(ηP, ηQ) OUTPUT: P, Q alternating, projected gradient descent until convergence Danai Koutra (CMU)

20 BiG-Align: algorithm DeTaIlS INPUT: A, B initialize(P) initialize(Q)
Pk+1 = Pk – ηP df(Pk, Qk)/dP valid_projection(Pk+1) Qk+1 = Qk – ηQ df(Pk+1, Qk)/dQ valid_projection(Qk+1) update(ηP, ηQ) OUTPUT: P, Q Probabilistic Constraint until convergence Danai Koutra (CMU)

21 BiG-Align: algorithm DeTaIlS INPUT: A, B initialize(P) initialize(Q)
Pk+1 = Pk – ηP df(Pk, Qk)/dP valid_projection(Pk+1) Qk+1 = Qk – ηQ df(Pk+1, Qk)/dQ valid_projection(Qk+1) update(ηP, ηQ) OUTPUT: P, Q Sparsity Constraint until convergence Danai Koutra (CMU)

22 min f = min||| PAQ – B||F 2 + λΣPij + μΣQij
BiG-Align: algorithm DeTaIlS INPUT: A, B initialize(P) initialize(Q) Pk+1 = Pk – ηP df(Pk, Qk)/dP valid_projection(Pk+1) Qk+1 = Qk – ηQ df(Pk+1, Qk)/dQ valid_projection(Qk+1) update(ηP, ηQ) OUTPUT: P, Q Sparsity Constraint until convergence min f = min||| PAQ – B||F λΣPij + μΣQij Danai Koutra (CMU)

23 RoadMap Problem Definition What’s different? BiG-Align Uni-Align
Optimizations Uni-Align Conclusions Danai Koutra (CMU)

24 BiG-Align: Optimizations
Details INPUT: A, B initialize(P) initialize(Q) Pk+1 = Pk – ηP df(Pk, Qk)/dP valid_projection(Pk+1) Qk+1 = Qk – ηQ df(Pk+1, Qk)/dQ valid_projection(Qk+1) update(ηP, ηQ) OUTPUT: P, Q alternating, projected gradient descent until convergence Danai Koutra (CMU)

25 BiG-Align: Optimizations
Details INPUT: A, B initialize(P) initialize(Q) Pk+1 = Pk – ηP df(Pk, Qk)/dP valid_projection(Pk+1) Qk+1 = Qk – ηQ df(Pk+1, Qk)/dQ valid_projection(Qk+1) update(ηP, ηQ) OUTPUT: P, Q alternating, projected gradient descent alternating, projected gradient descent until convergence Danai Koutra (CMU)

26 Optimization 1: Structurally equivalent nodes
Details Aggregation to super-nodes Graph A Danai Koutra (CMU)

27 BiG-Align: Optimizations
Details INPUT: A, B initialize(P) initialize(Q) Pk+1 = Pk – ηP df(Pk, Qk)/dP valid_projection(Pk+1) Qk+1 = Qk – ηQ df(Pk+1, Qk)/dQ valid_projection(Qk+1) update(ηP, ηQ) OUTPUT: P, Q alternating, projected gradient descent alternating, projected gradient descent until convergence Danai Koutra (CMU)

28 Optimization 2: Initialization of P and Q
Details Why is the initialization important? global minimum local minima … Danai Koutra (CMU)

29 Optimization 2: Initialization of P and Q
Details Social networks are structured: the degree distribution is power-law like. ranked nodes log(degree) Social networks are structured: the degree distribution of the nodes is power-law like. Danai Koutra (CMU)

30 Optimization 2: Initialization of P and Q
Details 1-1 matching of clusters of degrees Network-inspired initialization degree rank of node knee k 1-1 matching of top k nodes user degrees of GA user degrees in GB … . 2000 1500 1000 945 940 800 799 750 740 735 730 3 2 1 500 450 449 445 P k cluster 1 cluster 2 cluster n Top k nodes with significantly higher degree -> handle them separately from the low degree nodes cluster 1 cluster 2 cluster n Danai Koutra (CMU)

31 BiG-Align: Optimizations
Details INPUT: A, B initialize(P) initialize(Q) Pk+1 = Pk – ηP df(Pk, Qk)/dP valid_projection(Pk+1) Qk+1 = Qk – ηQ df(Pk+1, Qk)/dQ valid_projection(Qk+1) update(ηP, ηQ) OUTPUT: P, Q alternating, projected gradient descent alternating, projected gradient descent until convergence Danai Koutra (CMU)

32 Optimization 3: Steps of gradient descent
Details Constant step: thrashing or slow convergence Danai Koutra (CMU)

33 Optimization 3: Steps of gradient descent
Details Variable step with line search: strategy for local optimum ηP = argmin f(ηP) = g1(P,Q,A,B) ηQ = argmin f(ηQ) = g2(P,Q,A,B) closed formulas Danai Koutra (CMU)

34 Optimization 3: Steps of gradient descent
Details Variable step with line search: strategy for local optimum BiG-Align-Exact: computes the steps at every iteration ηP = argmin f(ηP) = g1(P,Q,A,B) ηQ = argmin f(ηQ) = g2(P,Q,A,B) closed formulas Danai Koutra (CMU)

35 Optimization 3: Steps of gradient descent
Details But 3.104 104 2.104 10-4 step size (η) iterations Slow change in the steps Danai Koutra (CMU)

36 Optimization 3: Steps of gradient descent
Details But BiG-Align-Skip: compute η’s every m (=500) iterations 3.104 104 2.104 10-4 step size (η) iterations Slow change in the steps Danai Koutra (CMU)

37 RoadMap Problem Definition What’s different? BiG-Align Uni-Align
Experiments Uni-Align Conclusions Danai Koutra (CMU)

38 Experimental Setup Implementation: Matlab
Dataset: IMDB movie-genre graph and subgraphs (1027 movies x 27 genres) Setup: random permutations noise level: % Ground truth Simulate real-world applications Danai Koutra (CMU)

39 State-of-the-art Background Umeyama’s algorithm [Umeyama88]:
SVD-based NMF-based approach [Ding+08]: Builds on top of Umeyama’s approach Net-Align [Bayati+09] Belief Propagation Danai Koutra (CMU)

40 State-of-the-art Background Umeyama’s algorithm [Umeyama88]:
SVD-based NMF-based approach [Ding+08]: Builds on top of Umeyama’s approach Net-Align [Bayati+09] Belief Propagation Bi-partite  Uni-partite Danai Koutra (CMU)

41 Big-Align: Accuracy vs. Runtime
Umeyama NetAlign NMF-based BiG-Align exact BiG-Align skip Small marker size: 300 nodes Big marker size: 700 nodes marker size related to graph size Danai Koutra (CMU)

42 Big-Align: Accuracy vs. Runtime
Umeyama NetAlign NMF-based BiG-Align exact BiG-Align skip Big-Align improves both speed and accuracy. Danai Koutra (CMU)

43 Big-Align: Accuracy w.r.t. noise
BiG-Align-exact BiG-Align-skip NMF-based NetAlign-deg NetAlign-full Umeyama Danai Koutra (CMU)

44 Big-Align: Accuracy w.r.t. noise
BiG-Align-exact BiG-Align-skip NMF-based NetAlign-deg NetAlign-full Umeyama BiG-Align improves the accuracy for almost all levels of noise. Danai Koutra (CMU)

45 RoadMap Problem Definition What’s different? BiG-Align Uni-Align
Conclusions Danai Koutra (CMU)

46 Algorithm: Uni-Align Details n nodes d features min || PAQ - B||F 2 P
fixed node degree clustering coeff … … Danai Koutra (CMU)

47 Algorithm: Uni-Align Details n nodes d features O(n.d2)
min || PAQ - B||F 2 P P = g*(A,B,S,U)= = closed-form solution SVD A = USVT O(n.d2) Explain it somehow Danai Koutra (CMU)

48 RoadMap Problem Definition What’s different? BiG-Align Uni-Align
Experiments Conclusions Danai Koutra (CMU)

49 Uni-Align Dataset: Facebook friendship graph (64K users)
Setup: uni-partite  bi-partite graph Feature extraction node degree egonet degree edges in egonet mean degree of node’s neighbors egonet Danai Koutra (CMU)

50 Uni-Align: Accuracy vs. Runtime
NMF-based NetAlign Umeyama Uni-Align, followed by Net-Align, is more accurate and faster than other approaches. Danai Koutra (CMU)

51 Uni-Align: Runtime Uni-Align
Umeyama Uni-Align NMF-based NetAlign-deg NetAlign-full Uni-Align is 2x - 31,700x faster depending on graph size. Danai Koutra (CMU)

52 RoadMap Problem Definition What’s different? BiG-Align Uni-Align
Conclusions Danai Koutra (CMU)

53 Conclusions Formulation: new problem / constraints Danai Koutra (CMU)

54 Conclusions Formulation: new problem / constraints Algorithms:
BiG-Align: optimized alternating projected gradient descent Uni-Align: alignment for uni-partite graphs Danai Koutra (CMU)

55 Conclusions Formulation: new problem / constraints Algorithms:
BiG-Align: optimized alternating projected gradient descent Uni-Align: alignment for uni-partite graphs Evaluations: more accurate and efficient Danai Koutra (CMU)

56 Beyond BiG-Align: Multi-way Linkage
~ S1: Dynamic Graph Linkage S2: Community-level Linkage ~ ~ beyond this paper, we found that big-align forms a core building block for several other more complicated matching task, such as we will not go into the details, but the bottom line is that all of them can be built upon big-align. and this also led to 7 patents S3: Hetero. Graph Linkage S4: Multi-relational DB Linkage All build upon BiG-Align (2) Led to 7 patents Danai Koutra (CMU)

57 Thank you! http://www.cs.cmu.edu/~dkoutra danai@cs.cmu.edu
Danai Koutra (CMU)

Download ppt "BiG-Align: Fast Bipartite Graph Alignment"

Similar presentations

Location Recognition Given: A query image A database of images with known locations Two types of approaches: Direct matching: directly match image features.

Location Recognition Given: A query image A database of images with known locations Two types of approaches: Direct matching: directly match image features.
Social network partition Presenter: Xiaofei Cao Partick Berg.

Social network partition Presenter: Xiaofei Cao Partick Berg.
CMU SCS I2.2 Large Scale Information Network Processing INARC 1 Overview Goal: scalable algorithms to find patterns and anomalies on graphs 1. Mining Large.

CMU SCS I2.2 Large Scale Information Network Processing INARC 1 Overview Goal: scalable algorithms to find patterns and anomalies on graphs 1. Mining Large.
School of Computer Science Carnegie Mellon University Duke University DeltaCon: A Principled Massive- Graph Similarity Function Danai Koutra Joshua T.

School of Computer Science Carnegie Mellon University Duke University DeltaCon: A Principled Massive- Graph Similarity Function Danai Koutra Joshua T.
School of Computer Science Carnegie Mellon University National Taiwan University of Science & Technology Unifying Guilt-by-Association Approaches: Theorems.

School of Computer Science Carnegie Mellon University National Taiwan University of Science & Technology Unifying Guilt-by-Association Approaches: Theorems.
LEARNING INFLUENCE PROBABILITIES IN SOCIAL NETWORKS Amit Goyal Francesco Bonchi Laks V. S. Lakshmanan University of British Columbia Yahoo! Research University.

LEARNING INFLUENCE PROBABILITIES IN SOCIAL NETWORKS Amit Goyal Francesco Bonchi Laks V. S. Lakshmanan University of British Columbia Yahoo! Research University.
Dept. of Computer Science Rutgers Node and Graph Similarity: Theory and Applications Danai Koutra (CMU) Tina Eliassi-Rad (Rutgers) Christos Faloutsos (CMU)

Dept. of Computer Science Rutgers Node and Graph Similarity: Theory and Applications Danai Koutra (CMU) Tina Eliassi-Rad (Rutgers) Christos Faloutsos (CMU)
Multi-label Relational Neighbor Classification using Social Context Features Xi Wang and Gita Sukthankar Department of EECS University of Central Florida.

Multi-label Relational Neighbor Classification using Social Context Features Xi Wang and Gita Sukthankar Department of EECS University of Central Florida.
Siddharth Choudhary.  Refines a visual reconstruction to produce jointly optimal 3D structure and viewing parameters  ‘bundle’ refers to the bundle.

Siddharth Choudhary.  Refines a visual reconstruction to produce jointly optimal 3D structure and viewing parameters  ‘bundle’ refers to the bundle.
Wavelength Assignment in Optical Network Design Team 6: Lisa Zhang (Mentor) Brendan Farrell, Yi Huang, Mark Iwen, Ting Wang, Jintong Zheng Progress Report.

Wavelength Assignment in Optical Network Design Team 6: Lisa Zhang (Mentor) Brendan Farrell, Yi Huang, Mark Iwen, Ting Wang, Jintong Zheng Progress Report.
CS 678 –Boltzmann Machines1 Boltzmann Machine Relaxation net with visible and hidden units Learning algorithm Avoids local minima (and speeds up learning)

CS 678 –Boltzmann Machines1 Boltzmann Machine Relaxation net with visible and hidden units Learning algorithm Avoids local minima (and speeds up learning)
Online Social Networks and Media. Graph partitioning The general problem – Input: a graph G=(V,E) edge (u,v) denotes similarity between u and v weighted.

Online Social Networks and Media. Graph partitioning The general problem – Input: a graph G=(V,E) edge (u,v) denotes similarity between u and v weighted.
Fast Iterative Alignment of Pose Graphs with Poor Initial Estimates Edwin Olson John Leonard, Seth Teller

Fast Iterative Alignment of Pose Graphs with Poor Initial Estimates Edwin Olson John Leonard, Seth Teller
Endend endend Carnegie Mellon University Korea Advanced Institute of Science and Technology VoG: Summarizing and Understanding Large Graphs Danai Koutra.

Endend endend Carnegie Mellon University Korea Advanced Institute of Science and Technology VoG: Summarizing and Understanding Large Graphs Danai Koutra.
1 Social Influence Analysis in Large-scale Networks Jie Tang 1, Jimeng Sun 2, Chi Wang 1, and Zi Yang 1 1 Dept. of Computer Science and Technology Tsinghua.

1 Social Influence Analysis in Large-scale Networks Jie Tang 1, Jimeng Sun 2, Chi Wang 1, and Zi Yang 1 1 Dept. of Computer Science and Technology Tsinghua.
Communities in Heterogeneous Networks Chapter 4 1 Chapter 4, Community Detection and Mining in Social Media. Lei Tang and Huan Liu, Morgan & Claypool,

Communities in Heterogeneous Networks Chapter 4 1 Chapter 4, Community Detection and Mining in Social Media. Lei Tang and Huan Liu, Morgan & Claypool,
Using Structure Indices for Efficient Approximation of Network Properties Matthew J. Rattigan, Marc Maier, and David Jensen University of Massachusetts.

Using Structure Indices for Efficient Approximation of Network Properties Matthew J. Rattigan, Marc Maier, and David Jensen University of Massachusetts.
A General Model for Relational Clustering Bo Long and Zhongfei (Mark) Zhang Computer Science Dept./Watson School SUNY Binghamton Xiaoyun Wu Yahoo! Inc.

A General Model for Relational Clustering Bo Long and Zhongfei (Mark) Zhang Computer Science Dept./Watson School SUNY Binghamton Xiaoyun Wu Yahoo! Inc.
© 2011 IBM Corporation IBM Research SIAM-DM 2011, Mesa AZ, USA, Non-Negative Residual Matrix Factorization w/ Application to Graph Anomaly Detection Hanghang.

© 2011 IBM Corporation IBM Research SIAM-DM 2011, Mesa AZ, USA, Non-Negative Residual Matrix Factorization w/ Application to Graph Anomaly Detection Hanghang.
SCS CMU Proximity Tracking on Time- Evolving Bipartite Graphs Speaker: Hanghang Tong Joint Work with Spiros Papadimitriou, Philip S. Yu, Christos Faloutsos.

SCS CMU Proximity Tracking on Time- Evolving Bipartite Graphs Speaker: Hanghang Tong Joint Work with Spiros Papadimitriou, Philip S. Yu, Christos Faloutsos.

Similar presentations

Ads by Google