1. Color Compatibility From Large Datasets Peter O’Donovan University of Toronto Aseem Agarwala Adobe Systems, Inc. Aaron Hertzmann University of Toronto

2. Choosing colors is hard for many people

4. ?

5. How do designers choose colors?

6. Picasso How do designers choose colors?

7. You the Designer How do designers choose colors?

8. Krause [2002] How do designers choose colors?

9. Goethe [1810] Complementary Color Theory: colors opposite on the color wheel are compatible

10. Hue Templates: relative orientations producing compatible colors ComplementaryMonochromatic AnalogousTriad

11. Photo and Video Quality Evaluation: Focusing on the Subject Luo and Tang 2008 Aesthetic Visual Quality Assessment of Paintings Li and Chen 2009 Color Harmonization for Videos Sawant and Mitra 2008 Color Harmonization Cohen-Or et al. 2006

13. Adobe Kuler 527,935 themes Ratings: 1-5 stars

14. Adobe Kuler 527,935 themes Ratings: 1-5 stars

15. Adobe Kuler 527,935 themes Ratings: 1-5 stars

16. Adobe Kuler 527,935 themes Ratings: 1-5 stars

17. COLOURLovers 1,672,657 themes Views and “Likes”

18. COLOURLovers 1,672,657 themes Views and “Likes”

19. Goals 1. Analysis Test hypotheses and compatibility models 2. Learn Models Predict mean ratings for themes 3. New Applications Develop new tools for choosing colors

20. Goals 1. Analysis Test hypotheses and compatibility models 2. Learn Models Predict mean ratings for themes 3. New Applications Develop new tools for choosing colors

21. 104,426 themes Ratings: 1-5 stars 383,938 themes # Views and “Likes” Kuler Dataset COLOURLovers Dataset

22. Mechanical Turk dataset 10,743 themes from Kuler 40 ratings per theme 1,301 total participants

23. Overall preference for warmer hues and cyans Histogram of hue usage Hue % of all themes

24. Mean rating for themes containing a hue Overall preference for warmer hues and cyans Hue Mean Rating

25. Histogram of hue adjacency (Kuler)

27. is more likely than Histogram of hue adjacency (Kuler)

28. Significant structure Histogram of hue adjacency (Kuler)

29. Significant structure Warm hues pair well with each other Histogram of hue adjacency (Kuler)

30. Significant structure Warm hues pair well with each other Greens and purples more compatible with themselves Histogram of hue adjacency (Kuler)

31. Hue Template Analysis

32. Hue Templates: relative orientations producing compatible colors

33. Templates are rotationally invariant Hue Templates: relative orientations producing compatible colors

34. Different templates equally compatible ComplementaryMonochromatic AnalogousTriad Hue Templates: relative orientations producing compatible colors

35. Diagonal lines are hue templates (Kuler interface bias) Hue adjacency in a theme (Kuler)

36. Complementary template Hue adjacency in a theme (Kuler) Diagonal lines are hue templates (Kuler interface bias)

37. Hue adjacency in a theme (Kuler) Complementary:

38. Data: Hue adjacency in a theme (Kuler)

39. In template theory, diagonals should be uniform Hue adjacency in a theme (Kuler)

40. In template theory, diagonals should be uniform Large dark bands indicates no rotational invariance Hue adjacency in a theme (Kuler)

41. Kuler CL Hue adjacency in a theme COLOURLovers’ has less interface bias Templates are not present

42. Distance to template Rating

43. Distance to template Themes near a template score worse Rating

44. Themes near a template score worse - “Newbie” factor - “Too simple” factor Distance to template Rating

45. MTurk has no interface bias: much flatter Distance to template Rating

46. Template Conclusions 1)Templates do not model color preferences 2)Themes near a template do not score better than those farther away 3)Not all templates are equally popular -Simple templates preferred (see paper)

47. Hue Entropy: entropy of hues along the hue wheel

48. Low Entropy Few Distinct Colors

49. Hue Entropy: entropy of hues along the hue wheel Low EntropyHigh Entropy Few Distinct ColorsMany Distinct Colors

50. Hue Entropy: entropy of hues along the hue wheel Hue Entropy Rating

51. Hue Entropy: entropy of hues along the hue wheel Hue Entropy Rating

52. Hue Entropy Rating Hue Entropy: entropy of hues along the hue wheel

53. Hue Entropy Rating

54. Main Analysis Results 1. Overall preference for warmer hues and cyans

55. Main Analysis Results 1. Overall preference for warmer hues and cyans 2. Strong preferences for certain adjacent colors

56. Main Analysis Results 1. Overall preference for warmer hues and cyans 2. Strong preferences for certain adjacent colors 3. Hue templates a poor model for compatibility

57. Main Analysis Results 1. Overall preference for warmer hues and cyans 2. Strong preferences for certain adjacent colors 3. Hue templates a poor model for compatibility 4. People prefer simpler themes (but not too simple)

58. Main Analysis Results 1. Overall preference for warmer hues and cyans 2. Strong preferences for certain adjacent colors 3. Hue templates a poor model for compatibility 4. People prefer simpler themes (but not too simple) See paper for other tests

59. Goals 1. Analysis Test hypotheses and compatibility models 2. Learn Models Predict mean ratings for themes 3. New Applications Develop new tools for choosing colors

60. 3.63

62. Mean rating over all users 3.63

63. Features (326 total) -Colors, sorted colors, differences, min/max, max-in, mean/std dev, PCA features, hue probability, hue entropy -RGB, HSV, CIELab, Kuler color wheel -“Kitchen Sink” 3.63

64. Models -Constant baseline: mean of training targets -SVM-R, KNN -Lasso -Linear regression model with L1 norm on weights -Solutions have many zero weights: feature selection 3.63

65. Dataset MAEConstant Baseline KNNSVM-RLassoLasso over Baseline Kuler0.5720.5330.5310.5219% COLORLovers0.7030.6740.6500.6448% MTurk0.2670.2050.1820.17933%

66. Dataset MAEConstant Baseline KNNSVM-RLassoLasso over Baseline Kuler0.5720.5330.5310.5219% COLORLovers0.7030.6740.6500.6448% MTurk0.2670.2050.1820.17933%

67. Dataset MAEConstant Baseline KNNSVM-RLassoLasso over Baseline Kuler0.5720.5330.5310.5219% COLORLovers0.7030.6740.6500.6448% MTurk0.2670.2050.1820.17933% Many more ratings per theme in MTurk

68. Dataset MAEConstant Baseline KNNSVM-RLassoLasso over Baseline Kuler0.5720.5330.5310.5219% COLORLovers0.7030.6740.6500.6448% MTurk0.2670.2050.1820.17933% MTurk has an average std dev of 0.33 Kuler has an average std dev of 0.72

69. MTurk Test Set Human Rating Lasso Rating

70. High-rated

71. Low-rated

72. High-rated Low-rated High prediction error

73. Model Analysis

74. Important Lasso Features Positive: high lightness mean & max, mean hue probability

75. Important Lasso Features Positive: high lightness mean & max, mean hue probability Negative: high lightness variance, min hue probability

76. Goals 1. Analysis Test hypotheses and compatibility models 2. Learn Models Predict mean ratings for theme 3. New Applications Develop new tools for choosing colors

77. 1. Improve a Theme

79. Select order which maximizes score

80. Optimize colors with CMA [Hansen 1995]

81. OriginalBest OrderColor and Order

82. OriginalBest OrderColor and Order

83. MTurk A/B test with original and optimized themes Order and Color

84. 2. Choose 5 colors that best ‘represent’ an image

85. One approach: k-means clustering

86. This ignores color compatibility

87. Optimize 5 colors that 1)Match the image well 2)Maximize regression score

88. Optimize 5 colors that 1)Match the image well 2)Maximize regression score See paper for details

89. With Compatibility Model W/O Compatibility Model

90. MTurk A/B test with and without compatibility model

91. 3. Given 4 colors for foreground, suggest background

92. Given 4 colors, choose 5 th color to maximize score Want contrast with existing colors

93. ,, …

94. Model Suggestions Random Suggestions

95. MTurk tests selecting ‘Worst’ and ‘Best’ 4 model & 4 random

96. Model Limitations & Future Work

97. Hard to interpret Features Weights

98. Model has very few abstract colors, only 1-D spatial layout

99. VS Model does not understand how colors are used

100. VS

101. Conclusions Color preferences are subjective, but analysis reveals many overall trends Simple linear models can represent compatibility fairly well Models can be useful for color selection tasks Our datasets and learned models are available online