Approximating the Community Structure of the Long Tail Akshay Java, Anupam Joshi, Tim Finin Problem Statement Community Structure of the Long Tail Original Adjacency Matrix Sorted (degree) Adjacency Matrix “Approximate the membership of the blogs in the long tail using only a small portion of the entire graph” Long Tail Head Motivation It is expensive to compute the community structure for large networks. Blog graphs are large, but extremely sparse and exhibit the long tail/ core-periphery structure. CORE Intuition Communities emerge around the core (A). The membership of the blogs in the long tail (B) can be approximated by the structure of the core and how the long tail links to the core. PERIPHERY Detecting Communities Approximation Methods A community is a set of nodes that has more connections within the set than outside it. Normalized Cut The graph can be partitioned using the eigenspectrum of the Laplacian. (Shi and Malik) The second smallest eigenvector of the graph Laplacian is the Fiedler vector. Singular Value Decomposition (reduced rank) Sampling-based Technique By Nyström Approximation (Fowlkes et al.) Thus eigenvectors of L can be easily approximated by sampling. Heuristic Method Use the head of the distribution to find the initial communities. Approximate membership of the tail by using number of links to each community in the head. The graph can be recursively partitioned using the sign of the values in its Fielder vector. Evaluation Conclusions and Ongoing Research Conclusion Community structure of the entire network can be well approximated by sampling and heuristic methods. Advantage Achieve fast, accurate approximation using much smaller sample of the entire graph. Ongoing Research Applying the approximation on temporal graphs Clustering graph, tag information simultaneously Modularity Score A measure of the quality of community detection Q = (fraction of intra-community edges)-(expected fraction, disregarding communities)