1. Automatic Design of Graphical Presentations Michael Schiff UC Berkeley April 14, 1998

2. The Automatic Presentation Design Problem

3. Why? Provide guidance for non-experts Explore designs for complex datasets Forces precision in design criteria

4. Outline General issues Mackinlay’s approach –Composition of graphical languages Other approaches –Casner User goals and perceptual tasks –Me “First principles” design

5. Input Datasets of tuples/relations –E.g., {enrollment(Fall 1995, 100), enrollment(Spring 1996, 120)... } (also written {,,…} Data Characterization –Domain types (nominal/ordered/quantitative) –Functional dependencies (e.g., only one enrollment figure per semester)

6. Input Goal characterization –General What is the enrollment for a given year? What is the trend in enrollment? –Specific Find least expensive pair of flights between Pittsburgh and Mexico City with one layover of no more than 4 hours.

7. Output Static Two-dimensional Method of presentation as well as picture –Layout / rendering the presentation is a separate problem

8. Fundamental Issues What is space of possible designs? How to select appropriate design? –Must be precise

9. Mackinlay’s approach “Presentations are sentences of graphical languages” Graphical languages have precise definitions Languages can be characterized in terms of expressiveness & effectiveness Primitive languages can be composed to create new languages

10. Graphical Languages Syntax - what is a well-formed sentence of the language? –HorzPos(s)  s = h 4 m. c m  [o = plusobj. Ymax(h)  Ypos(l) = const. Xmin(h)  Xpos(l)  Xmax(h) ]

11. Graphical Languages Semantics - How do sentences encode facts? –Encodes(o, a, HorzPos)  b = scale  (Position(o,h) + offset). Encodes(Position(o,h), r(a,b), Horzpos).

12. Expressiveness Two conditions –Must be able to encode all facts in set –Must encode only those facts Mexico US Canada

13. Value of graphical languages Can prove theorems about both expressiveness conditions –Can’t use Horzpos language to encode one-to-many relation –Can’t use bar graphs to encode nominal to nominal relations E.g., Nationality(CarMaker, Country) More realistically, can state provable conditions for using different techniques

14. Expressiveness conditions If bar graphs encode order through size relationships...

15. Effectiveness Each language involves certain perceptual tasks Perceptual tasks can be ranked

16. Accuracy Ranking of Quantitative Perceptual Tasks (Mackinlay 88 from Cleveland & McGill) Position Length AngleSlope Area Volume ColorDensity More Accurate Less Accurate

17. QUANTITATIVEORDINALNOMINAL PositionPositionPosition LengthDensityColor Hue AngleColor SaturationTexture SlopeColor HueConnection AreaTextureContainment VolumeConnectionDensity DensityContainmentColor Saturation Color SaturationLengthShape Color HueAngleLength Ranking of Applicability of Properties for Different Data Types (Mackinlay 86, Not Empirically Verified)

18. Assumptions Rankings take interpretation into account Most accurate = best pattern perception

19. Design Space - Composition of Primitive Languages Primitive languages –Single-position - Horizontal, Vertical axis –Apposed position - Line chart, bar chart, plot chart –Retinal-list - Color, shape, size, saturation, texture –Map - Road map, topographic map –Connection - Tree, network –Misc (angle, contain) - Pie chart, Venn diagram Compose languages to create new languages

20. Composition example

21. APT Partition -> Selection -> Composition –Break tuples down until they match primitive languages –Choose best primitive language possible –Compose using composition operators –Backtrack if necessary

22. APT-like systems SAGE (Roth et al) –More primitive languages and composition operators –Interactive as well as automatic design Others...

23. Casner’s Approach Uses Mackinlay’s basic framework –Somewhat more primitive graphical languages Focus on user goals and tasks for selection –This is crucial for determining effectiveness of designed presentation –Example: table vs. bar chart

24. BOZ Start with complex task description stated in terms of logical operators Translate into sequence of perceptual operators and computations Choose graphic languages with most effective perceptual operators Make sure they can be composed

25. Example: Task Find a pair of connecting flights that travel from Pittsburgh to Mexico City, with a layover of no more than four hours. Both flights must be available, and the combined cost of the flights cannot exceed $500.

26. Example: Logical task description (while (findFlightWithOrigin FLIGHT ’pit) do (if (available? flight) then (findDestination flight LAYOVERCITY) (determineArrival flight ARRIVAL) (while (findFlightWithOrigin CONNECTING layovercity) do (if available? connecting) then (findDestination flight FINALDESTINATION) (if (landsInDestinationCity? finaldestination ’mex) then (determineDeparture connecting DEPARTURE) (computeLayover departure arrival LAYOVER)...

27. Example: Translating to perceptual operators findFlightWithOrigin = search available = lookup findDestination = lookup determineArrival = lookup determineDeparture = lookup computerLayover = subtraction

28. Example: Perceptual task description (while (search-object-with-label FLIGHT ’pit) do (if (shaded? flight) then (read-label flight LAYOVERCITY) (determine-horz-pos flight ARRIVAL) (while (search-object-with-label CONNECTING layovercity) do (if (shaded? connecting) then (read-label flight FINALDESTINATION) (if (same-labels? finaldestination ’mex) then (determine-horz-pos connecting DEPARTURE) (determine-horz-distance departure arrival LAYOVER)...

29. Advantages / Disadvantages Advantages –Chooses presentation appropriate for task, not just appropriate for data –Very fine-grain analysis of perceptual effectiveness Disadvantages –Requires very complex initial input –Possibly unrealistic to try to determine exact sequence of perceptual operators

30. My approach “First principles” instead of graphic languages –The space of possible representations –Logical principles of expressiveness –Interpretive principles –Perceptual principles Design space and selection –More complicated design space –Select by multiple criteria

31. What is a graphic language? Description of how information corresponds to graphics?? –If we relax our preconceptions, there are lots of arbitrary ways to represent data Systematic set of encoding conventions –People need systematic conventions Same types of objects/properties encode same types of information

32. What is the space of systematic graphic languages? Property values of objects encode data values One or more objects with certain perceptual relationships encode a tuple Possible extra constraints on objects –E.g., alignment

33. Example: Bar Graph -> Rectangle hpos and height (+ alignment)

34. Example: Dot Plot -> Circle hpos, vpos, color

35. Example: Network Graph -> Circle label, Circle label, Line color (+ line touches circles)

36. Generating potential graphic languages Partition tuples into groups Represent each group by some properties of some type of object Combine multiple objects into compound objects if necessary Use additional constraints to make presentation more effective This results in a very large design space!

37. Filtering: Logical principles Object properties must be able to represent object domains Can’t overconstrain objects –E.g., can’t use left edge position, right edge position, and width of rectangle to encode 3 values Other expressiveness concerns

38. Selection: Interpretive Principles Main idea: Will people understand encoding conventions quickly and correctly? Necessary if we’re not starting with known graphical languages Good/Bad vs. Better/Worse

39. Interpretive Principles Simplicity Consistency Compatibility Relevance Conventionality

40. Simplicity Use fewer objects

41. Consistency Use same properties to represent same domains, same objects to represent same sets of domains

42. Compatibility Don’t violate metaphors of encoding –E.g., CEO goes at top of tree, not bottom Use perceptually-supported mappings –Larger size = larger value, proportionality 200 210 220 230 240 250 0 10 Violation of proportionality principle

43. Relevance Avoid random variation of object properties that don’t encode information

44. Relevance The most perceptually salient properties should encode information

45. Conventionality Individual conventions for domain to property mappings Varying degrees of generality, cultural specificity Examples –Color in weather maps –Time goes from left to right, or top to bottom

46. Selection: Perceptual Principles Different principles for different goals –Basic goals - extract information from single tuple (e.g., profits for 1995) –Summary goals - extract information from sets of tuples (e.g., trend in 1st quarter profits over a series of years) Principles help filter bad presentations and rank remaining ones

47. Basic Goals and Perceptual Operations Minimize time / maximize accuracy of perceptual operations needed –Choose properties that allow for fast search & discrimination –Choose property values that maximize speed and accuracy Preattentive search issues (e.g., in choice of colors) Choose maximally discriminable values

48. Summary goals and configural properties Configural properties –Because of perceptual organization, groups of objects may have their own “properties” –Enable summary goals to be satisfied in a “single glance” –E.g, shape and density of point cloud in scatterplot

49. Configural Properties example

50. Configural Properties example 2

51. AUTOGRAPH Consider many possible choices –Partioning –What objects and properties to use –How to form compound objects Throw out bad graphic languages as soon as they’re determined to be bad Rank by principles and choose best one (Use composition for multiple datasets)

52. Advantages / Disadvantages Advantages –Wider variety of possible designs, possibly including novel ones –More uniform, so easier to extend ? Disadvantages –Exponential number of possible graphic languages, most of them bad –Not always easy to apply principles –Have to figure out how to draw new designs (not necessarily that hard)

53. Conclusions General issues for evaluation –What is the design space? –How are methods of representation chosen? General issues in design –Must be precise in dealing with logical limitations of different techniques –Must be precise in specifying criteria for evaluating designs