Nonphotorealistic Visualization of Multidimensional Datasets SIGGRAPH 2001 Christopher G. Healey Department of Computer Science, North Carolina State University Supported by NSF-IIS , NSF-ACI

Goals of Multidimensional Visualization Effective visualization of large, multidimensional datasets size: number of elements n in dataset dimensionality: number of attributes m embedded in each element Display effectively multiple attributes at a single spatial location? Rapidly, accurately, and effortlessly explore large amounts of data?

Visualization Pipeline Dataset Management Visualization Assistant Perceptual Visualization Nonphotorealistic Visualization Assisted Navigation Multidimensional Dataset Perception

Formal Specification Dataset D = { e 1, …, e n } containing n elements e i D represents m data attributes A = { A 1, …, A m } Each e i encodes m attribute values e i = { a i,1, …, a i,m } Visual features V = { V 1, …, V m } used to represent A Function  j : A j  V j maps domain of A j to range of displayable values in V j Data-feature mapping M( V,  ), a visual representation of D Visualization: Selection of M and viewers interpretation of images produced by M

Separate Displays PrecipitationTemperatureWindspeedPressure n = 42,224 elements m = 4 A 1 = temperature A 2 = windspeed A 3 = precipitation A 4 = pressure V = colour  = dark blue  bright pink

Integrated Display n = 42,224 elements m = 4 A 1 = temperature A 2 = windspeed A 3 = precipitation A 4 = pressure V 1 = colour V 2 = size V 3 = orientation V 4 = density  1 = dark blue  bright pink  2 = 0.25  1.15  3 = 0º  90º  4 = 1x1  3x3

Cognitive Vision Psychological study of the human visual system Perceptual (preattentive) features used to perform simple tasks in < 200 milliseconds –features: hue, intensity, orientation, size, length, curvature, closure, motion, depth of field, 3D cues –tasks: target detection, boundary detection, region tracking, counting and estimation Perceptual (preattentive) tasks performed independent of display size Develop, extend, and apply results to visualization

Preattentive Processing Video

How can we choose effectively multiple hues? Suppose: { A, B }Suppose: { A, B, C, D, E, F } Rapidly and accurately identifiable colors? Equally distinguishable colors? Maximum number of colors? Three selection criteria: color distance, linear separation, color category Effective Hue Selection ABABCDEF

Colour Distance A B C CIE LUV isoluminant slice; AB = AC implies equal perceived colour difference

Linear Separation Without linear separation (T in A & B, harder) vs. with linear separation (T in A & C, easier) A B T C

Colour Category red purple blue green B A T Between named categories (T & B, harder) vs. within named categories (T & A, easier)

Distance / Linear Separation B GY Y R P l d d Constant linear separation l, constant distance d to two nearest neighbours

Example Experiment Displays Target: red square; 3-colour, 17 element displays and 7-colour, 49 element displays 3 colours 17 elements 7 colours 49 elements

3-Color w/LUV, Separation

7-Color w/LUV, Separation

7-Color w/LUV, Separation, Category

CT Volume Visualization

Perceptual Texture Elements Design perceptual texture elements (pexels) Pexels support variation of perceptual texture dimensions height, density, regularity Attach a pexel to each data element Element attributes control pexel appearance Psychophysical experiments used to measure: –perceptual salience of each texture dimension –visual interference between texture dimensions

Pexel Examples Regularity Density Height

Example “ Taller ” Display

Example “ Regular ” Display

Results Subject accuracy used to measure performance Taller pexels identified preattentively with no interference (93% accuracy) Shorter, denser, sparser identified preattentively Some height, density, regularity interference Irregular difficult to identify (76% accuracy); height, density interference Regular cannot be identified (50% accuracy)

Typhoon Visualization n = 572,474 m = 3 A 1 = windspeed; A 2 = pressure; A 3 = precipitation V 1 = height; V 2 = density; V 3 = color  1 = short  tall;  2 = dense  sparse;  3 = blue  purple Typhoon Amber approaches Taiwan, August 28, 1997

Typhoon Visualization n = 572,474 m = 3 A 1 = windspeed; A 2 = pressure; A 3 = precipitation V 1 = height; V 2 = density; V 3 = color  1 = short  tall;  2 = dense  sparse;  3 = blue  purple Typhoon Amber strikes Taiwan, August 29, 1997

Impressionism Underlying principles of impressionist art: –Object and environment interpenetrate –Colour acquires independence –Show a small section of nature –Minimize perspective –Solicit a viewer ’ s optics Hue, luminance, color explicitly studied and controlled Other stroke and style properties correspond closely to low- level visual features –path, length, energy, coarseness, weight Can we bind data attributes with stroke properties? Can we use perception to control painterly rendering?

Water Lilies (The Clouds) 1903; Oil on canvas, 74.6 x cm (29 3/8 x 41 7/16 in); Private collection

Rock Arch West of Etretat (The Manneport) 1883; Oil on canvas, 65.4 x 81.3 cm (25 3/4 x 32 in); Metropolitan Museum of Art, New York

Wheat Field 1889; Oil on canvas, 73.5 x 92.5 cm (29 x 36 1/2 in); Narodni Galerie, Prague

Gray Weather, Grande Jatte 1888; Oil on canvas, 27 3/4 x 34 in; Philadelphia Museum of Art. Walter H. Annenberg Collection

Stroke  Feature Correspondence Close correspondence between V j and S j –hue  color, luminance  lighting, contrast  density, orientation  path, area  size e i in D analogous to brush strokes in a painting To build a painterly visualization of D: –construct M( V,  ) –map V j in V to corresponding painterly styles S j in S M now maps e i to brush strokes b i a i,j in e i control painterly appearance of b i

Eastern US, January n = 69,884 m = 4 A 1 = temperature; A 2 = windspeed; A 3 = pressure; A 4 = precipitation V 1 = color; V 2 = density; V 3 = size; V 4 = orientation  1 = blue  pink;  2 = sparse  dense;  3 = small  large;  4 = upright  flat

Rocky Mountains, January n = 69,884 m = 4 A 1 = temperature; A 2 = windspeed; A 3 = pressure; A 4 = precipitation V 1 = color; V 2 = density; V 3 = size; V 4 = orientation  1 = blue  pink;  2 = sparse  dense;  3 = small  large;  4 = upright  flat

Pacific Northwest, February n = 69,884 m = 4 A 1 = temperature; A 2 = windspeed; A 3 = pressure; A 4 = precipitation V 1 = color; V 2 = density; V 3 = size; V 4 = orientation  1 = blue  pink;  2 = sparse  dense;  3 = small  large;  4 = upright  flat

Canyon Photo

Canyon NPR

Sloping Hills Photo

Sloping Hills NPR

Conclusions Formalisms identify a visual feature  painterly style correspondence Can exploit correspondence to construct perceptually salient painterly visualizations Recent and future work +psychophysical experiments confirm perceptual guidelines extend to painterly environment –subjective aesthetics experiments –improved computational models of painterly images –additional painterly styles –dynamic paintings (e.g., flicker, direction and velocity of motion)