Color Compatibility From Large Datasets Peter O’Donovan University of Toronto Aseem Agarwala Adobe Systems, Inc. Aaron Hertzmann University of Toronto

Choosing colors is hard for many people

?

How do designers choose colors?

Picasso How do designers choose colors?

You the Designer How do designers choose colors?

Krause [2002] How do designers choose colors?

Goethe [1810] Complementary Color Theory: colors opposite on the color wheel are compatible

Hue Templates: relative orientations producing compatible colors ComplementaryMonochromatic AnalogousTriad

Photo and Video Quality Evaluation: Focusing on the Subject Luo and Tang 2008 Aesthetic Visual Quality Assessment of Paintings Li and Chen 2009 Color Harmonization for Videos Sawant and Mitra 2008 Color Harmonization Cohen-Or et al. 2006

Adobe Kuler 527,935 themes Ratings: 1-5 stars

Adobe Kuler 527,935 themes Ratings: 1-5 stars

Adobe Kuler 527,935 themes Ratings: 1-5 stars

Adobe Kuler 527,935 themes Ratings: 1-5 stars

COLOURLovers 1,672,657 themes Views and “Likes”

COLOURLovers 1,672,657 themes Views and “Likes”

Goals 1. Analysis Test hypotheses and compatibility models 2. Learn Models Predict mean ratings for themes 3. New Applications Develop new tools for choosing colors

Goals 1. Analysis Test hypotheses and compatibility models 2. Learn Models Predict mean ratings for themes 3. New Applications Develop new tools for choosing colors

104,426 themes Ratings: 1-5 stars 383,938 themes # Views and “Likes” Kuler Dataset COLOURLovers Dataset

Mechanical Turk dataset 10,743 themes from Kuler 40 ratings per theme 1,301 total participants

Overall preference for warmer hues and cyans Histogram of hue usage Hue % of all themes

Mean rating for themes containing a hue Overall preference for warmer hues and cyans Hue Mean Rating

Histogram of hue adjacency (Kuler)

is more likely than Histogram of hue adjacency (Kuler)

Significant structure Histogram of hue adjacency (Kuler)

Significant structure Warm hues pair well with each other Histogram of hue adjacency (Kuler)

Significant structure Warm hues pair well with each other Greens and purples more compatible with themselves Histogram of hue adjacency (Kuler)

Hue Template Analysis

Hue Templates: relative orientations producing compatible colors

Templates are rotationally invariant Hue Templates: relative orientations producing compatible colors

Different templates equally compatible ComplementaryMonochromatic AnalogousTriad Hue Templates: relative orientations producing compatible colors

Diagonal lines are hue templates (Kuler interface bias) Hue adjacency in a theme (Kuler)

Complementary template Hue adjacency in a theme (Kuler) Diagonal lines are hue templates (Kuler interface bias)

Hue adjacency in a theme (Kuler) Complementary:

Data: Hue adjacency in a theme (Kuler)

In template theory, diagonals should be uniform Hue adjacency in a theme (Kuler)

In template theory, diagonals should be uniform Large dark bands indicates no rotational invariance Hue adjacency in a theme (Kuler)

Kuler CL Hue adjacency in a theme COLOURLovers’ has less interface bias Templates are not present

Distance to template Rating

Distance to template Themes near a template score worse Rating

Themes near a template score worse - “Newbie” factor - “Too simple” factor Distance to template Rating

MTurk has no interface bias: much flatter Distance to template Rating

Template Conclusions 1)Templates do not model color preferences 2)Themes near a template do not score better than those farther away 3)Not all templates are equally popular -Simple templates preferred (see paper)

Hue Entropy: entropy of hues along the hue wheel

Low Entropy Few Distinct Colors

Hue Entropy: entropy of hues along the hue wheel Low EntropyHigh Entropy Few Distinct ColorsMany Distinct Colors

Hue Entropy: entropy of hues along the hue wheel Hue Entropy Rating

Hue Entropy: entropy of hues along the hue wheel Hue Entropy Rating

Hue Entropy Rating Hue Entropy: entropy of hues along the hue wheel

Hue Entropy Rating

Main Analysis Results 1. Overall preference for warmer hues and cyans

Main Analysis Results 1. Overall preference for warmer hues and cyans 2. Strong preferences for certain adjacent colors

Main Analysis Results 1. Overall preference for warmer hues and cyans 2. Strong preferences for certain adjacent colors 3. Hue templates a poor model for compatibility

Main Analysis Results 1. Overall preference for warmer hues and cyans 2. Strong preferences for certain adjacent colors 3. Hue templates a poor model for compatibility 4. People prefer simpler themes (but not too simple)

Main Analysis Results 1. Overall preference for warmer hues and cyans 2. Strong preferences for certain adjacent colors 3. Hue templates a poor model for compatibility 4. People prefer simpler themes (but not too simple) See paper for other tests

Goals 1. Analysis Test hypotheses and compatibility models 2. Learn Models Predict mean ratings for themes 3. New Applications Develop new tools for choosing colors

3.63

Mean rating over all users 3.63

Features (326 total) -Colors, sorted colors, differences, min/max, max-in, mean/std dev, PCA features, hue probability, hue entropy -RGB, HSV, CIELab, Kuler color wheel -“Kitchen Sink” 3.63

Models -Constant baseline: mean of training targets -SVM-R, KNN -Lasso -Linear regression model with L1 norm on weights -Solutions have many zero weights: feature selection 3.63

Dataset MAEConstant Baseline KNNSVM-RLassoLasso over Baseline Kuler % COLORLovers % MTurk %

Dataset MAEConstant Baseline KNNSVM-RLassoLasso over Baseline Kuler % COLORLovers % MTurk %

Dataset MAEConstant Baseline KNNSVM-RLassoLasso over Baseline Kuler % COLORLovers % MTurk % Many more ratings per theme in MTurk

Dataset MAEConstant Baseline KNNSVM-RLassoLasso over Baseline Kuler % COLORLovers % MTurk % MTurk has an average std dev of 0.33 Kuler has an average std dev of 0.72

MTurk Test Set Human Rating Lasso Rating

High-rated

Low-rated

High-rated Low-rated High prediction error

Model Analysis

Important Lasso Features Positive: high lightness mean & max, mean hue probability

Important Lasso Features Positive: high lightness mean & max, mean hue probability Negative: high lightness variance, min hue probability

Goals 1. Analysis Test hypotheses and compatibility models 2. Learn Models Predict mean ratings for theme 3. New Applications Develop new tools for choosing colors

1. Improve a Theme

Select order which maximizes score

Optimize colors with CMA [Hansen 1995]

OriginalBest OrderColor and Order

OriginalBest OrderColor and Order

MTurk A/B test with original and optimized themes Order and Color

2. Choose 5 colors that best ‘represent’ an image

One approach: k-means clustering

This ignores color compatibility

Optimize 5 colors that 1)Match the image well 2)Maximize regression score

Optimize 5 colors that 1)Match the image well 2)Maximize regression score See paper for details

With Compatibility Model W/O Compatibility Model

MTurk A/B test with and without compatibility model

3. Given 4 colors for foreground, suggest background

Given 4 colors, choose 5 th color to maximize score Want contrast with existing colors

,, …

Model Suggestions Random Suggestions

MTurk tests selecting ‘Worst’ and ‘Best’ 4 model & 4 random

Model Limitations & Future Work

Hard to interpret Features Weights

Model has very few abstract colors, only 1-D spatial layout

VS Model does not understand how colors are used

VS

Conclusions Color preferences are subjective, but analysis reveals many overall trends Simple linear models can represent compatibility fairly well Models can be useful for color selection tasks Our datasets and learned models are available online