Transcript slides
Color Compatibility From Large Datasets Peter O’Donovan Aseem Agarwala Aaron Hertzmann University of Toronto Adobe Systems, Inc. University of Toronto Choosing colors is hard for many people Choosing colors is hard for many people ? How do designers choose colors? How do designers choose colors? Picasso How do designers choose colors? You the Designer How do designers choose colors? Krause [2002] Goethe [1810] Complementary Color Theory: colors opposite on the color wheel are compatible Hue Templates: relative orientations producing compatible colors Complementary Monochromatic Analogous Triad Color Harmonization Cohen-Or et al. 2006 Photo and Video Quality Evaluation: Focusing on the Subject Luo and Tang 2008 Color Harmonization for Videos Sawant and Mitra 2008 Aesthetic Visual Quality Assessment of Paintings Li and Chen 2009 Adobe Kuler 527,935 themes Ratings: 1-5 stars Adobe Kuler 527,935 themes Ratings: 1-5 stars Adobe Kuler 527,935 themes Ratings: 1-5 stars Adobe Kuler 527,935 themes Ratings: 1-5 stars COLOURLovers 1,672,657 themes Views and “Likes” COLOURLovers 1,672,657 themes Views and “Likes” Goals 1. Analysis Test hypotheses and compatibility models 2. Learn Models Predict mean ratings for themes 3. New Applications Develop new tools for choosing colors Goals 1. Analysis Test hypotheses and compatibility models 2. Learn Models Predict mean ratings for themes 3. New Applications Develop new tools for choosing colors Kuler Dataset 104,426 themes Ratings: 1-5 stars COLOURLovers Dataset 383,938 themes # Views and “Likes” Mechanical Turk dataset 10,743 themes from Kuler 40 ratings per theme 1,301 total participants Histogram of hue usage % of all themes Hue Overall preference for warmer hues and cyans Mean rating for themes containing a hue Mean Rating Hue Overall preference for warmer hues and cyans Histogram of hue adjacency (Kuler) Histogram of hue adjacency (Kuler) Histogram of hue adjacency (Kuler) is more likely than Histogram of hue adjacency (Kuler) Significant structure Histogram of hue adjacency (Kuler) Significant structure Warm hues pair well with each other Histogram of hue adjacency (Kuler) Significant structure Warm hues pair well with each other Greens and purples more compatible with themselves Hue Template Analysis Hue Templates: relative orientations producing compatible colors Hue Templates: relative orientations producing compatible colors Templates are rotationally invariant Hue Templates: relative orientations producing compatible colors Complementary Monochromatic Analogous Different templates equally compatible Triad Hue adjacency in a theme (Kuler) Diagonal lines are hue templates (Kuler interface bias) Hue adjacency in a theme (Kuler) Complementary template Diagonal lines are hue templates (Kuler interface bias) Hue adjacency in a theme (Kuler) Complementary: Hue adjacency in a theme (Kuler) Complementary: Data: Hue adjacency in a theme (Kuler) In template theory, diagonals should be uniform Hue adjacency in a theme (Kuler) In template theory, diagonals should be uniform Large dark bands indicates no rotational invariance Hue adjacency in a theme CL Kuler COLOURLovers’ has less interface bias Templates are not present Rating Distance to template Rating Distance to template Themes near a template score worse Rating Distance to template Themes near a template score worse - “Newbie” factor - “Too simple” factor Rating Distance to template MTurk has no interface bias: much flatter Template Conclusions 1) Templates do not model color preferences 2) Themes near a template do not score better than those farther away 3) Not all templates are equally popular - Simple templates preferred (see paper) Hue Entropy: entropy of hues along the hue wheel Hue Entropy: entropy of hues along the hue wheel Low Entropy Few Distinct Colors Hue Entropy: entropy of hues along the hue wheel Low Entropy High Entropy Few Distinct Colors Many Distinct Colors Hue Entropy: entropy of hues along the hue wheel Rating Hue Entropy Hue Entropy: entropy of hues along the hue wheel Rating Hue Entropy Hue Entropy: entropy of hues along the hue wheel Rating Hue Entropy Hue Entropy: entropy of hues along the hue wheel Rating Hue Entropy Main Analysis Results 1. Overall preference for warmer hues and cyans Main Analysis Results 1. Overall preference for warmer hues and cyans 2. Strong preferences for certain adjacent colors Main Analysis Results 1. Overall preference for warmer hues and cyans 2. Strong preferences for certain adjacent colors 3. Hue templates a poor model for compatibility Main Analysis Results 1. Overall preference for warmer hues and cyans 2. Strong preferences for certain adjacent colors 3. Hue templates a poor model for compatibility 4. People prefer simpler themes (but not too simple) Main Analysis Results 1. Overall preference for warmer hues and cyans 2. Strong preferences for certain adjacent colors 3. Hue templates a poor model for compatibility 4. People prefer simpler themes (but not too simple) See paper for other tests Goals 1. Analysis Test hypotheses and compatibility models 2. Learn Models Predict mean ratings for themes 3. New Applications Develop new tools for choosing colors 3.63 3.63 𝑓 𝒙 =𝑦 3.63 𝑓 𝒙 =𝑦 Mean rating over all users 3.63 𝑓 𝒙 =𝑦 Features (326 total) - Colors, sorted colors, differences, min/max, max-in, mean/std dev, PCA features, hue probability, hue entropy - RGB, HSV, CIELab, Kuler color wheel - “Kitchen Sink” 3.63 𝑓 𝒙 =𝑦 Models - Constant baseline: mean of training targets - SVM-R, KNN - Lasso - Linear regression model with L1 norm on weights - Solutions have many zero weights: feature selection Dataset MAE Constant Baseline KNN SVM-R Lasso Lasso over Baseline Kuler 0.572 0.533 0.531 0.521 9% COLORLovers 0.703 0.674 0.650 0.644 8% MTurk 0.267 0.205 0.182 0.179 33% Dataset MAE Constant Baseline KNN SVM-R Lasso Lasso over Baseline Kuler 0.572 0.533 0.531 0.521 9% COLORLovers 0.703 0.674 0.650 0.644 8% MTurk 0.267 0.205 0.182 0.179 33% Dataset MAE Constant Baseline KNN SVM-R Lasso Lasso over Baseline Kuler 0.572 0.533 0.531 0.521 9% COLORLovers 0.703 0.674 0.650 0.644 8% MTurk 0.267 0.205 0.182 0.179 33% Many more ratings per theme in MTurk Dataset MAE Constant Baseline KNN SVM-R Lasso Lasso over Baseline Kuler 0.572 0.533 0.531 0.521 9% COLORLovers 0.703 0.674 0.650 0.644 8% MTurk 0.267 0.205 0.182 0.179 33% MTurk has an average std dev of 0.33 Kuler has an average std dev of 0.72 MTurk Test Set Lasso Rating Human Rating High-rated 𝑦 = 3.90, f 𝐱 = 3.41 𝑦 = 3.79, f 𝐱 = 3.50 High-rated 𝑦 = 3.90, f 𝐱 = 3.41 𝑦 = 3.79, f 𝐱 = 3.50 Low-rated 𝑦 = 1.71, f 𝐱 = 2.27 𝑦 = 1.78, f 𝐱 = 2.25 High-rated 𝑦 = 3.90, f 𝐱 = 3.41 𝑦 = 3.79, f 𝐱 = 3.50 Low-rated 𝑦 = 1.71, f 𝐱 = 2.27 𝑦 = 1.78, f 𝐱 = 2.25 High prediction error 𝑦 = 2.74, f 𝐱 = 1.78 𝑦 = 2.22, f 𝐱 = 3.16 Model Analysis Important Lasso Features Positive: high lightness mean & max, mean hue probability Important Lasso Features Positive: high lightness mean & max, mean hue probability Negative: high lightness variance, min hue probability Goals 1. Analysis Test hypotheses and compatibility models 2. Learn Models Predict mean ratings for theme 3. New Applications Develop new tools for choosing colors 1. Improve a Theme Maximize regression score Stay within a distance 𝑑𝑚𝑖𝑛 of original (L2 in CIELab) Select order which maximizes score Optimize colors with CMA [Hansen 1995] Original Best Order Color and Order f 𝐱 = 2.92 f 𝐱 = 3.04 f 𝐱 = 3.35 f 𝐱 = 3.00 f 𝐱 = 3.11 f 𝐱 = 3.37 f 𝐱 = 3.50 f 𝐱 = 3.50 f 𝐱 = 3.70 Original Best Order Color and Order f 𝐱 = 2.92, 𝑦 = 3.04 f 𝐱 = 3.04, 𝑦 = 2.99 f 𝐱 = 3.35, 𝑦 = 3.40 f 𝐱 = 3.00, 𝑦 = 2.96 f 𝐱 = 3.11, 𝑦 = 3.21 f 𝐱 = 3.37, 𝑦 = 3.35 f 𝐱 = 3.50, 𝑦 = 3.72 f 𝐱 = 3.50, 𝑦 = 3.69 f 𝐱 = 3.70, 𝑦 = 3.82 Order and Color MTurk A/B test with original and optimized themes 2. Choose 5 colors that best ‘represent’ an image One approach: k-means clustering One approach: k-means clustering This ignores color compatibility Optimize 5 colors that 1) Match the image well 2) Maximize regression score Optimize 5 colors that 1) Match the image well 2) Maximize regression score See paper for details W/O Compatibility With Compatibility Model Model MTurk A/B test with and without compatibility model 3. Given 4 colors for foreground, suggest background Given 4 colors, choose 5th color to maximize score Want contrast with existing colors , ,… Find next best color, 𝑑𝑚𝑖𝑛 away from previous choices Model Suggestions Random Suggestions MTurk tests selecting ‘Worst’ and ‘Best’ 4 model & 4 random Model Limitations & Future Work Features Weights Hard to interpret Model has very few abstract colors, only 1-D spatial layout VS Model does not understand how colors are used VS Conclusions Color preferences are subjective, but analysis reveals many overall trends Simple linear models can represent compatibility fairly well Models can be useful for color selection tasks Our datasets and learned models are available online