NERCOMP Workshop, Dec. 2, 2008 Information Visualization: the Other Half of Data Analysis Dr. Matthew Ward Computer Science Department Worcester Polytechnic Institute

NERCOMP Workshop, Dec. 2, 2008 A Data Analysis Pipeline Raw Data Processed Data Hypotheses Models Results Cleaning Filtering Transforming Statistical Analysis Pattern Rec Knowledge Disc Validation ACB D

NERCOMP Workshop, Dec. 2, 2008 Where Does Visualization Come In?  All stages can benefit from visualization  A: identify bad data, select subsets, help choose transforms (exploratory)  B: help choose computational techniques, set parameters, use vision to recognize, isolate, classify patterns (exploratory)  C: Superimpose derived models on data (confirmatory)  D: Present results (presentation)

NERCOMP Workshop, Dec. 2, 2008 What do we need to know to do Information Visualization?  Characteristics of data Types, size, structure Semantics, completeness, accuracy  Characteristics of user Perceptual and cognitive abilities Knowledge of domain, data, tasks, tools  Characteristics of graphical mappings What are possibilities Which convey data effectively and efficiently  Characteristics of interactions Which support the tasks best Which are easy to learn, use, remember

NERCOMP Workshop, Dec. 2, 2008 Issues Regarding Data  Type may indicate which graphical mappings are appropriate Nominal vs. ordinal Discrete vs. continuous Ordered vs. unordered Univariate vs. multivariate Scalar vs. vector vs. tensor Static vs. dynamic Values vs. relations  Trade-offs between size and accuracy needs  Different orders/structures can reveal different features/patterns

NERCOMP Workshop, Dec. 2, 2008 Issues Regarding Users  What graphical attributes do we perceive accurately?  What graphical attributes do we perceive quickly?  Which combinations of attributes are separable?  Coping with change blindness  How can visuals support the development of accurate mental models of the data?  Relative vs. absolute judgements – impact on tasks

NERCOMP Workshop, Dec. 2, 2008 Issues Regarding Mappings  Variables include shape, size, orientation, color, texture, opacity, position, motion….  Some of these have an order, others don’t  Some use up significant screen space  Sensitivity to occlusion  Domain customs/expectations

NERCOMP Workshop, Dec. 2, 2008 www3.sympatico.ca/blevis/Image10.gif

NERCOMP Workshop, Dec. 2, 2008 Issues Regarding Interactions  Interaction critical component  Many categories of techniques Navigation, selection, filtering, reconfiguring, encoding, connecting, and combinations of above  Many “spaces” in which interactions can be applied Screen/pixels, data, data structures, graphical objects, graphical attributes, visualization structures

NERCOMP Workshop, Dec. 2, 2008 Importance of Evaluation  Easy to design bad visualizations  Many design rules exist – many conflict, many routinely violated  5 E’s of evaluation: effective, efficient, engaging, error tolerant, easy to learn  Many styles of evaluation (qualitative and quantitative): Use/case studies Usability testing User studies Longitudinal studies Expert evaluation Heuristic evaluation

NERCOMP Workshop, Dec. 2, 2008 Different Rules -> Different Views Courtesy of Aisee.com

NERCOMP Workshop, Dec. 2, 2008 Categories of Mappings  Based on data characteristics Numbers, text, graphs, software, ….  Logical groupings of techniques (Keim) Standard: bars, lines, pie charts, scatterplots Geometrically transformed: landscapes, parallel coordinates Icon-based: stick figures, faces, profiles Dense pixels: recursive segments, pixel bar charts Stacked: treemaps, dimensional stacking  Based on dimension management (Ward) Dimension subsetting: scatterplots, pixel-oriented methods Dimension reconfiguring: glyphs, parallel coordinates Dimension reduction: PCA, MDS, Self Organizing Maps Dimension embedding: dimensional stacking, worlds within worlds

NERCOMP Workshop, Dec. 2, 2008 Scatterplot Matrix  Each pair of dimensions generates a single scatterplot  All combinations arranged in a grid or matrix, each dimension controls a row or column  Look for clusters, outliers, partial correlations, trends

NERCOMP Workshop, Dec. 2, 2008 Parallel Coordinates  Each variable/dimension is a vertical line  Bottom of line is low value, top is high  Each record creates a polyline across all dimensions  Similar records cluster on the screen  Look for clusters, outliers, line angles, crossings

NERCOMP Workshop, Dec. 2, 2008 Star Glyph  Glyphs are shapes whose attributes are controlled by data values  Star glyph is a set of N rays spaced at equal angles  Length of each ray proportional to value for that dimension  Line connects all endpoints of shape  Lay glyphs out in rows and columns  Look for shape similarities and differences, trends

NERCOMP Workshop, Dec. 2, 2008 Other Types of Glyphs

NERCOMP Workshop, Dec. 2, 2008 Dimensional Stacking  Break each dimension range into bins  Break the screen into a grid using the number of bins for 2 dimensions  Repeat the process for 2 more dimensions within the subimages formed by first grid, recurse through all dimensions  Look for repeated patterns, outliers, trends, gaps

NERCOMP Workshop, Dec. 2, 2008 Pixel-Oriented Techniques  Each dimension creates an image  Each value controls color of a pixel  Many organizations of pixels possible (raster, spiral, circle segment, space-filling curves)  Reordering data can reveal interesting features, relations between dimensions

NERCOMP Workshop, Dec. 2, 2008 Methods to Cope with Scale  Many modern datasets contain large number of records (millions and billions) and/or dimensions (hundreds and thousands)  Several strategies to handle scale problems Sampling Filtering Clustering/aggregation  Techniques can be automated or user- controlled

NERCOMP Workshop, Dec. 2, 2008 Examples of Data Clustering

NERCOMP Workshop, Dec. 2, 2008 Example of Dimension Clustering

NERCOMP Workshop, Dec. 2, 2008 Example of Data Sampling

NERCOMP Workshop, Dec. 2, 2008 The Visual Data Analysis (VDA) Process  Overview  Filter/cluster/sample  Scan  Select “interesting”  Details on demand  Link between different views

NERCOMP Workshop, Dec. 2, 2008 Demonstration

NERCOMP Workshop, Dec. 2, 2008 Summary  Visualization a powerful component of the data analysis process  Each stage of analysis can be enhanced  Visualization can help guide computational analysis, and vice versa  Multiple linked views and a rich assortment of interactions key to success

NERCOMP Workshop, Dec. 2, 2008 For Further Info on XmdvTool   Contains source code, windows executable, data sets, documentation, copies of most Xmdv publications, case studies  We gratefully acknowledge support for the development of XmdvTool from the National Science Foundation (IIS , IRIS , IIS , IIS , CCF , and IIS ) and the National Security Agency

NERCOMP Workshop, Dec. 2, 2008 Questions?