1. 1 Modularity and Community Structure in Networks* Final project *Based on a paper by M.E.J Newman in PNAS 2006

2. 2 Introduction

3. 3 Networks A network: presented by a graph G(V,E): V = nodes, E = edges (link node pairs) Examples of real-life networks: –social networks (V = people) –World Wide Web (V= webpages) –protein-protein interaction networks (V = proteins)

4. 4 Communities (clusters) in a network A community (cluster) is a densely connected group of vertices, with only sparser connections to other groups.

5. 5 Protein-protein Interaction Networks Nodes – proteins (6K), edges – interactions (15K). Reflect the cell’s machinery and signaling pathways.

6. 6 Distilling Modules from Networks Motivation: identifying protein complexes responsible for certain functions in the cell

7. 7 Newman's network division algorithm

8. 8 Modularity of a division (Q) Q = #(edges within groups) - E(#(edges within groups in a RANDOM graph with same node degrees)) Trivial division: all vertices in one group ==> Q(trivial division) = 0 Edges within groups k i = degree of node i M =  k i = 2|E| Aij = 1 if (i,j)  E, 0 otherwise Eij = expected number of edges between i and j in a random graph with same node degrees. Lemma: Eij  k i *k j / M Q =  (Aij - ki*kj/M | i,j in the same group)

9. 9 Algorithm 1: Division into two groups (1) Suppose we have n vertices {1,...,n} s - {  1} vector of size n. Represent a 2-division: –si == sj iff i and j are in the same group –½ (si*sj+1) = 1 if si==sj, 0 otherwise ==> Q =  (Aij - ki*kj/M | i,j in the same group)

10. 10 Algorithm 1: Division into two groups (2) Since where B = the modularity matrix - symmetric - row sum = 0 0 is an eigvenvalue of B

11. 11 Modularity matrix: example

12. 12 Algorithm 1: Division into two groups (3) Which vector s maximizes Q? –clearly s ~ u1 maximizes Q, but u1 may not be {  1} vector –Greedy heuristic: choose s ~ u1: si= +1 if ui>0, si=-1 otherwise B's eigen values B's corresponding eigen vectors B is symmetric  B is diagonalizable (real eigenvalues)

13. 13

14. 14 Example: a 2-division of a social network A network showing relationships between people in a karate club which eventually split into 2. The division algorithm predicts exactly the two groups after the split known group leader known group leaders Color matches the entries of the eigen vector u1: light = positive entry (si=1) dark: negative (si=-1)

15. 15 Dividing into more than 2 (1) How to compute into more than 2? Idea: apply the algorithm recursively on every group. g - a group of n g vertices s - a {  1} vector of size n g Compute  Q for a 2-division of g New: elements of g are split into two subgroups (corresponding to s) Old: all the elements of g are within one group (g) Bij0|1

16. 16 Dividing into more than 2 (2) where B[g] = the submatrix of B defined by g f i (g) = sum of ith row B[g] f i ({1,...,n}) = 0 generalized modularity matrix

17. 17 Generalized modularity matrix: example g = {1, 4, 5} (1 is the minimal index)

18. 18 A "generalized" 2-division algorithm (divides a group in a network)

19. 19

20. 20 Further techniques for modularity maximization (Combined with Neman's "generalized' 2-division algorithm)

21. 21 A heuristic for 2-division 1.{g1, g2} - an initial 2-division of g 2.While there is an unmoved node: 1.Let v be an unmoved node, whose moving between g1 and g2 maximizes  Q 2.Move v between g1 and g2 3.From the n g 2-divisions generated in the previous step - let {g1, g2} be the one with maximum  Q 4.If  Q>0 ==> go to 1 The last iteration produces a 2-division which equals the initial 2-division

22. 22 Choosing j' with maximum  Q 2.While there is an unmoved node: 1. Let v be an unmoved node, whose moving between g1 and g2 maximizes  Q 2. Move v between g1 and g2 Computing  Q for each node moving j' and storing its  Q

23. 23 Algorithm 4 -cont. 3. From the n g 2-divisions generated in the previous step - let {g1, g2} be the one with maximum  Q 4. If  Q>0 ==> go to 1

24. 24 Finding the leading eigen-pair The power method

25. 25 The Power Method (1) A - a diagonalizable matrix Let ( 1,V 1 ),..., ( n,V n ) be n eigenpairs of A where | 1 | > | 2 |  | 3 | ...  | n | The power method finds the dominant eigenpair of A, i.e. (V 1, 1 ) (Note that 1 is not necessarily the leading eigenvalue) X = any vector.  X = c 1 V 1 +... +c n V n, where c i = X  V i

26. 26 The Power Method (2) AX = A (c 1 V 1 +... +c n V n ) = c 1 AV 1 +... +c n AV n = c 1 1 V 1 +....+ c n n V n A 2 X = A (AX) = A (c 1 1 V 1 +....+ c n n V n ) = c 1 1 2 V 1 +....+ c n n 2 V n... A m X = A (A m-1 X) = A (c 1 1 m-1 V 1 +....+ c n n m-1 V n ) = c 1 1 m V 1 +....+ c n n m V n If m is large enough 

27. 27 Power Method (3) Suppose V 1  Y  0  For simplicity, Y=A m X

28. 28 Power method - Example Example:  We perform only matrix- vector multiplications!

29. 29 Power method – convergence condition To avoid numerical problems due to large numbers – normalize X before each iteration, i.e: X 0 = X / ||X| X 1 = AX 0 / ||AX 0 || X 2 = AX 1 / || AX 1 ||.... The desired precision

30. 30 Finding the leading eigenpair using matrix shifting Let be the eigenvalues of A, and U1,...,Un their corresponding eigenvectors Let ||A|| 1 =  max | i | (exercise) Q: What is the dominant eigenpair of A+||A|| 1 I? A: ( 1+ ||A|| 1, U1)

31. 31 Implementation Robustness and Efficiency

32. 32 Checking "positiveness" #define IS_POSITIVE(X) ((X) > 0.00001) Instead "x>0" ==> use IS_POSITIVE(X)

33. 33 Efficient multiplications in the (extended) modularity matrix: O(n) instead O(n 2 ) multiplication in a sparse matrix inner product  f (g) i x i ("matrix shifting") "matrix shifting"

34. 34 Fast score computations Computing  Q for each node ==>O(n 2 ) Computing  Q for each node in O(n) before moving 1st node Updating the score AFTER a move of a node k (s is already updated)

35. 35 sparse_matrix_lst Holds connected lists of non-zero elements – for rows and columns (2n lists) For each non-zero element: –value + column index + row index

36. 36 sparse_matrix_lst (2) typedef struct cell_t { struct cell_t *row_next; /* next element in the row. */ struct cell_t *col_next; /* next element in the column. */ int rowind; /* index in row */ int colind; /* index in column */ type value; /* value of the element*/ } cell_t; /* matrix cell data type */ typedef struct { int n; /* matrix size */ cell_t **rows; /* array of row lists.*/ cell_t **cols; /* array of columns lists.*/ } sparse_matrix_lst;

37. 37 Project specifications

38. 38 programs 1.sparse_mlpl < matrix_vec.in 2.modularity_mat 3.spectral_div 4.improve_div 5.cluster for the power method computing a 2-division The complete clustering algorithm (including the improvement)

39. 39 Analyzing clusters in yeast and fly protein-protein interaction networks Input: true PPI network + 2 random networks Task 1: infer the true network Solution: the true network is more modular Task 2: compute associated functions (using cytoscape + BiNGO) Saccharomyces cerevisiae drosophila melanogaster

40. 40 Cytoscape, BiNGO www.cytoscape.com –A framework for analyzing networks –provides visualization of networks and clusters http://www.psb.ugent.be/cbd/papers/BiNGO/ –Finding functions associated with gene cluster –runs from cytoscape

41. 41 BiNGO output (GO = Gene Ontology)

42. 42 Visualization with cytoscape

43. 43 How is the project checked? Most checks (points): "BLACK BOX" –The common checks in "real world" –Running with fixed input files, comparing to fixed output files –Score = #(successful checks) / #(total checks) "WHITE BOX" checks: code review (10 points maximum) –code simplicity / efficiency

44. 44 A simple data structure for maintaining a division Complexity: –Finding a group containing element i: O(n) –Splitting a group into 2: O(n) typedef struct Division_{ int n; int* group-ids; int numGroups; double Q; } Division; #nodes in the network for each node - its group id (initially 0 - all nodes within on group)

45. 45 Maintaining the generalized modularity matrix Should we maintain the modularity matrix? –No: 1) we do not use it explicitly 2) it is a dense matrix - consumes a large memory space –Yes: 1) simplifies L1-norm computation (though a single row-vector will suffice) 2) Can be used in validating the correctness of optimized multiplications (debug mode only!)

46. 46 Suggestion for modules Handling sparse matrices: - Data structure: sparse_matrix_lst -Reading a sparse matrix ( file / stdin) -Multiplication in a vector -Computing A[g] -Methods hiding the inner structure (allows a simple replacement of sparse_matrix_lst with another data structure for holding sparse matrices) Handling a division Handling a group The spectral algorithm: -2-division -full-division The improvement algorithm Handling the modularity matrix: - Data structure: A[g], k[g], M, f[g], L1-norm -Multiplication in a vector -Computing Q -printing the modularity matrix

47. 47 Good luck! (and have fun...)