Visual Analysis of Large Graphs Using (X, Y)-clustering and Hybrid Visualizations V. Batagelj, W. Didimo, G. Liotta, P. Palladino, M. Patrignani (Univ. Ljubljana, Univ. Perugia, Univ. Roma Tre) In Proc. IEEE Pacific Visualization 2010

Outline The problem of visualizing large graphs State of the art Our contribution Conclusions and open problems

The problem of visualizing large graphs Some major issues in the visualization of large graphs: Readability: optimization of aesthetic criteria Scalability: fast computation Visual complexity: interaction tools that allow users to limit the amount of information displayed on the screen — overview of the graph — details on demand — user’s mental map preservation

State of the art Readability: there are many effective algorithms that are computationally fast for relatively small and sparse graphs (see the graph drawing book of Di Battista, Eades, Tamassia, Tollis, 1999)

State of the art Scalability: there are some fast graph drawing algorithms based on physical or algebraic models; the drawings have high visual complexity and do not allow detailed views (see the survey of Hacul and Jünger, 2007)

State of the art Visual complexity: draw the whole graph and then interact with it; ex. focus+context techniques, like fisheye view or hyperbolic layouts; conceived for tree-like graphs (see the survey of Herman, Melançon, Marshall, 2000)

State of the art Interactive approaches for visualizing and exploring large graphs: – graph visualized incrementally or at different levels of details – strong interaction between the user and the drawing

Interactive Approaches Bottom-up strategies: the graph is visualized a piece at a time —topological window moving through canvas (Eades et al.,1997) —Limits: no overview, the user’s mental map preservation is difficult

Interactive Approaches Bottom-up strategies: the graph is visualized a piece at a time —incremental enhancement of the drawing (ex. Carmignani et al., 2002) —Limits: no overview, the user’s mental map preservation is difficult without readability degradation

Interactive Approaches Top-down approaches

Interactive Approaches Top-down approaches – the graph is clustered (vertices are grouped together)

Interactive Approaches Top-down approaches – the graph is clustered (vertices are grouped together) – a simplified view is shown (overview)

Interactive Approaches Top-down approaches – the graph is clustered (vertices are grouped together) – a simplified view is shown (overview) – the user interactively explores the clusters (detailed views)

Interactive Approaches Top-down strategies – the graph is clustered (vertices are grouped together) – a simplified view is shown – the user interactively explores the clusters Limits – someone/something has to define clustering rules – existing clustering algorithms do not guarantee properties on the graph of clusters

Our contribution A top-down approach with these ingredients: – a new clustering framework – new clustering algorithm within the framework – hybrid visualizations A system: VHyXY Some case studies

Basic Terminology: Clustering G=(V, E): graph with vertex set V and edge set E A cluster of G=(V, E) is a subset of V A clustering C of G is a set of disjoint clusters of G

Basic Terminology: Clustering The graph of clusters H(G, C) is the graph obtained by collapsing each cluster of C into a single vertex and by replacing multiple edges with a single one

Basic Terminology: Clustering The graph of clusters H(G, C) is the graph obtained by collapsing each cluster of C into a single vertex and by replacing multiple edges with a single one

A new clustering framework Clustering algorithms usually detect groups of highly connected vertices without taking care of the graph of clusters We adopt a new framework for the design of automatic clustering algorithms that guarantee: – desired properties for the clusters – desired properties for the graph of clusters

The (X,Y)-clustering X and Y are two classes of graphs with certain properties G is called an (X,Y)-graph if there exists a clustering of G such that: – each cluster induces a subgraph that belongs to Y – the graph of clusters belongs to X

(X,Y)-graph example Let X be the class of cycles and let Y be the class of K 4

(X,Y)-graph example Let X be the class of cycles and let Y be the class of K 4

(X,Y)-graph example The graph is a (cycle,K 4 )-graph Let X be the class of cycles and let Y be the class of K 4

Interesting combinations X is some class of sparse graphs: – planar graphs, cycles, trees, paths, … Y is some class of highly connected graphs: – cliques, subgraphs with high-degree vertices, … One can think of using different visual paradigms and algorithms for drawing the graph of clusters and the subgraph induced by each cluster (hybrid visualization)

Remark on (X,Y)-clustering (X, Y)-clustering was previously defined by Brandenburg (GD 1997), but his model requires that every vertex belongs to some cluster Our model does not have this requirement, which poses severe practical limitations

The (X,Y)-clustering problem Problem: Given a graph G and two desired classes X and Y, is G an (X,Y)-graph? NP This problem is NP-hard in general NP Theorem: Deciding whether G is a (planar, k-clique)-graph for desired k ≥ 5 is NP-hard This result motivates us to look for some relaxation of cliques

K-core components The subgraph induced by a cluster is a k-core component if it is a maximal connected subgraph such that every vertex has degree at least k 5-core component 4-core component

(Planar, K-core component)-graphs We investigate (X,Y)-graphs G such that: – X is the class of planar graphs – Y is the class of k-core components of G In particular, for a given k > 0, one can ask whether G is a (planar, k-core component)-graph – this decision problem can be solved in polynomial time – we give a polynomial-time algorithm that finds the maximum k for which G is a (planar, k-core component)- graph, and that computes the corresponding clustering

Properties of (planar, k-core component)-graphs The union of all k-core components of G is called the k-core of G (the k-core of G, if it exists, is unique) Property. If G has the k-core G k (for some k ≥ 1), then G has the (k−1)-core G (k−1) and G k ⊆ G (k−1) Lemma. If G is a (planar, k-core component)- graph then it is a (planar, (k−1)-core component)- graph

Proof of the lemma

V1V1 V2V2

u(V 1 ) H(G, C) u(V 2 ) V1V1 V2V2

Proof of the lemma H(G, C) u(V 1 ) u(V 2 ) V1’V1’ V2’V2’

Proof of the lemma u(V 1 ’) H(G, C’) H(G, C) u(V 2 ’) u(V 1 ) u(V 2 ) V1’V1’ V2’V2’

Proof of the lemma H(G, C) u(V 1 ) u(V 2 ) V1’V1’ V2’V2’ u(V 1 ’) u(V 2 ’) H(G, C’)

Clustering Algorithm Theorem: Let G be a graph with n vertices and m edges. There exists an O((n+m)log n)-time algorithm that computes the maximum k for which G is a (planar, k-core component)-graph, and the corresponding clustering Steps of the algorithm: 1.Compute core-numbers for the vertices 2.Perform a binary search on core-numbers 3.For each graph of clusters, test its planarity

Algorithm animation Compute the core number of each vertex, i.e., the maximum k for which there exists a k-core that contains the vertex

Algorithm animation Compute the core number of each vertex, i.e., the maximum k for which there exists a k-core that contains the vertex

Algorithm animation

Hybrid Visualizations The (X, Y)-clustering technique can be used to design hybrid visualizations – combination of different drawing conventions for different parts of the graph – Example: node-link representation for sparse subgraphs matrix-based representation for dense subgraphs – Highly readable drawings for the graph of clusters (which is always planar)

Matrix based representation Matrix-based representation – vertices are rows and columns – edges are cells The ordering of vertices in rows/columns may strongly affect the number of crossings in the drawing

Crossings minimization heuristic vertex1 vertex2 vertex3 vertex4 vertex5 vertex6 vertex7 vertex8 vertex10 vertex11 vertex12 vertex13 vertex14 vertex15 vertex16 vertex17 vertex18 vertex19 vertex10 vertex20

Crossings minimization heuristic vertex1 vertex2 vertex3 vertex4 vertex5 vertex6 vertex7 vertex8 vertex10 vertex11 vertex12 vertex13 vertex14 vertex15 vertex16 vertex17 vertex18 vertex19 vertex10 vertex20

Crossings minimization heuristic vertex1 vertex2 vertex3 vertex4 vertex5 vertex6 vertex7 vertex8 vertex10 vertex11 vertex12 vertex13 vertex14 vertex15 vertex16 vertex17 vertex18 vertex19 vertex10 vertex20

vertex12 vertex13 vertex14 vertex15 Crossings minimization heuristic vertex1 vertex2 vertex3 vertex4 vertex5 vertex6 vertex7 vertex8 vertex10 vertex11 vertex16 vertex17 vertex18 vertex19 vertex10 vertex20

Remark about hybrid visualizations A hybrid visualization that combines node-link and matrix-based representations was previously used in the literature (Henry et al., NodeTrix) Clusters are manually defined – no automatic clustering – no automatic ordering for rows-columns

The System VHyXY VHyXY VHyXY integrates the clustering algorithm and hybrid visualizations – X-class chooser (e.g., planar, forest) – Y-class chooser (e.g., k-core component) – Filters on edge weights – Specific drawing algorithms for each component

User interface

Case Study: Co-authorship networks DBLP DBLP: on-line database of publications in Computer Science VHyXY VHyXY allows user to query DBLP on a specific topic – It retrieves data about all papers on that topic (looking at the title of the papers) – It builds a network where authors are vertices there is an edge between two authors if they share a paper (edge’s weight = number of papers)

Co-authorship network for “orthogonal drawing”

Hybrid visualizations: a matrix and a circular in an orthogonal layout

Hybrid visualizations: a matrix and a circular inside an orthogonal

Larger network for “graph drawing” 114 vertices and 494 edges

Same network with edge filtering (weight > 2)

Clustering algorithm performance Index nameValue (0-1) Graph clustering0.62 Coverage0.56 Clustering performance 0.94 Clustering error0.06 Graph clustering – Property of a graph: the higher the value the better can be the clustering Coverage – How the computed clusters covers edges of the whole graph Performance – Counts the number of “correctly interpreted pairs of nodes” in a graph Error – 1-performance [Brandes et al. “Engineering graph clustering: Models and experimental evaluation” ACM Journal of Experimental Algorithmics 2007] Index nameValue (0-1) Graph clustering0.64 Coverage0.37 Clustering performance Clustering error

Open problems Explore additional X-classes or Y-classes for which polynomial-time clustering algorithms exist – X: forest, path, outerplanar, … – Y: relaxations of cliques, … Extend our techniques to – multi-level clustering (hierarchical clustering) – overlapping clusters Experiment the system on a larger set of application domains – biological networks, criminal networks, …