Transcript Presentation Slides - The Chinese University of Hong Kong

Contrast Preserving Decolorization
Cewu Lu, Li Xu, Jiaya Jia,
The Chinese University of Hong Kong
Mono printers are still the majority
• Fast
• Economic
• Environmental friendly
Documents generally have color figures
The printing problem
The printing problem
The printing problem
The printing problem
The printing problem
HP printer
The printing problem
Our Result
Decolorization
Mapping
Single Channel
Applications
Color Blindness
Applications
Color Blindness
Decolorization could lose contrast
Mapping(
) =
Mapping(
) =
Decolorization could lose contrast
Mapping
Pervious Work (Local methods)
• Bala and Eschbach 2004
• Neumann et al. 2007
• Smith et al. 2008
Pervious Work (Local methods)
Naive Mapping
Result
Color Contrast
Pervious Work (Global methods)
• Gooch et al. 2004
• Rasche et al. 2005
• Kim et al. 2009
Pervious Work (Global methods)
Color feature
preserving
optimization
g  f (c )
mapping function
Pervious Work (Global methods)
g  f (c )
In most global methods, color order is strictly
satisfied
Color order could be ambiguous
Can you tell the order?
Color order could be ambiguous
brightness(
) < brightness (
) YUV space
Lightness(
) > Lightness (
) LAB space
Color order could be ambiguous
The order of different colors cannot be defined uniquely by people
B. Wong et al., Nature Methods, 2010
People with different culture and language background have
different senses of brightness with respect to color.
E. Ozgen et al., Current Directions in Psychological Science, 2004
K. Zhou et al., National Academy of Sciences, 2010
Color order could be ambiguous
If we enforce the color order constraint, contrast
loss could happen
Input
[Rasche et al. 2005] [Kim et al. 2009]
Ours
Our Contribution
Relax the color
order constraint
Unambiguous color pairs
Global Mapping
Bimodal Contrast-Preserving
Weak Color Order
Polynomial Mapping
The Framework
• Objective Function
 Bimodal Contrast-Preserving
 Weak Color Order
• Finite Multivariate Polynomial Mapping Function
• Numerical Solution
Bimodal Contrast-Preserving
• Color pixel { x , y } , grayscale contrast
color contrast  (CIELab distance)
g xy  g x  g y
,
xy
•
g xy
follows a Gaussian distribution with mean 

G  xy , 
2

 g 
xy
xy
 exp  
2

2

2




xy
Bimodal Contrast-Preserving
• Color pixel { x , y } , grayscale contrast
color contrast  (CIELab distance)
g xy  g x  g y ,
xy
•
a Gaussian distribution with mean  .
g xy follows
xy
 xy

G  xy , 
2

g xy
 g 
xy
xy
 exp  
2

2

2




Bimodal Contrast-Preserving
• Tradition methods (order preserving):
m ax
g
N


G sign ( x , y )  xy , 
2

( x , y ) N
: neighborhood pixel set
• Our bimodal contrast-preserving for ambiguous color
pairs:
m ax
g

( x , y ) N
 
G  xy , 
2


 G   xy , 
2

Bimodal Contrast-Preserving
m ax
g


G sign ( x , y )  xy , 
2

( x , y ) N
sign(x,y) = 1
 xy
m ax
g
 G  
( x , y ) N
,
xy
2
  G  
,
xy
2
g xy

  xy
 xy
g xy
Bimodal Contrast-Preserving
m ax
g


G sign ( x , y )  xy , 
2

( x , y ) N
sign(x,y) =  1
g xy
  xy
m ax
g
 G  
( x , y ) N
,
xy
2
  G  
,
xy
2

  xy
 xy
g xy
Our Work
• Objective Function
 Bimodal Contrast-Preserving
 Weak Color Order
• Finite Multivariate Polynomial Mapping Function
• Numerical Solution
Weak Color Order
• Unambiguous color pairs:
 rx  ry & g x  g y & b x  b y  or  rx
 ry & g x  g y & b x  b y 
Weak Color Order
• Unambiguous color pairs:
 rx  ry & g x  g y & b x  b y  or  rx
 x,y
1.0


 0.5

 ry & g x  g y & b x  b y 
unam biguous color pair
am biguous color pair
• Our model thus becomes
m ax
g
 
( x , y ) N

G  xy , 
x,y
2
  1    G   
x,y
,
xy
2

Our Work
• Objective Function
 Bimodal Contrast-Preserving
 Weak Color Order
• Finite Multivariate Polynomial Mapping Function
• Numerical Solution
Multivariate Polynomial Mapping Function
m ax
g

( x , y ) N

 x , y G   xy , 
2

 1   x , y  G   xy , 

2

Solve for grayscale image: g
Variables (pixels): 400x250 = 100,000
Example
Too many (easily produce unnatural
structures)
Multivariate Polynomial Mapping Function
m ax
g

( x , y ) N

 x , y G   xy , 
2

 1   x , y  G   xy , 

2

• Parametric global color-to-grayscale mapping
Small Scale
grayscale value  f (color vector,  )
Multivariate Polynomial Mapping Function
• Parametric color-to-grayscale
f ( c , ) 
 m
i
i
i
m i is the i monomial basis of  n , c  { r , g , b } .
th
 n  span{ r 1 g 2 b 3 : d i = 0, 1, 2, ... d 1  d 2  d 3  n}
d
d
d
When n = 2, a grayscale is a linear combination of elements
2
2
2
{ r , g , b , rg , gb , rb , r , g , b }
Multivariate Polynomial Mapping Function
• Parametric color-to-grayscale
f ( c , ) 
 m
i
i
i
Multivariate Polynomial Mapping Function
• Parametric color-to-grayscale
f ( c , ) 
 m
i
i
i
r
g
b
rg
rb
gb
2
2
2
r
g
b
Multivariate Polynomial Mapping Function
• Parametric color-to-grayscale
f ( c , ) 
 m
i
i
i
g
b
rg
rb
gb
2
2
2
r
g
b
Multivariate Polynomial Mapping Function
• Parametric color-to-grayscale
f ( c , ) 
 m
i
i
i
b
rg
rb
gb
2
2
2
r
g
b
Multivariate Polynomial Mapping Function
• Parametric color-to-grayscale
f ( c , ) 
 m
i
i
i
rg
rb
gb
2
2
2
r
g
b
Multivariate Polynomial Mapping Function
• Parametric color-to-grayscale
f ( c , ) 
 m
i
i
i
r
2
rb
gb
2
2
g
b
Multivariate Polynomial Mapping Function
• Parametric color-to-grayscale
f ( c , ) 
 m
i
i
i
gb
r
2
g
2
b
2
Multivariate Polynomial Mapping Function
• Parametric color-to-grayscale
f ( c , ) 
 m
i
i
i
r
2
g
2
b
2
Multivariate Polynomial Mapping Function
• Parametric color-to-grayscale
f ( c , ) 
 m
i
i
i
g
2
b
2
Multivariate Polynomial Mapping Function
• Parametric color-to-grayscale
f ( c , ) 
 m
i
i
i
b
2
Multivariate Polynomial Mapping Function
• Parametric color-to-grayscale
f ( c , ) 
 m
i
i
i
Multivariate Polynomial Mapping Function
• Parametric color-to-grayscale
f ( c , ) 
 m
i
i
i
0.1550
0.8835
0.3693
0.1817
0.4977
-1.7275
-0.4479
0.6417
0.6234
Multivariate Polynomial Mapping Function
• Parametric color-to-grayscale
f ( c , ) 
 m
i
i
i
0.1550
0.8835
0.3693
0.1817
0.4977
-1.7275
-0.4479
0.6417
0.6234
Our Model
g xy  g x  g y  f  c x ,    f  c y , 
    m
i
ix
 m iy 
i
• Objective function:
m ax

 
( x , y ) N

G  xy , 
x,y
2
  1    G   
x,y
,
xy
2

Numerical Solution
m ax

( x , y ) N
 m in 

Define:

E 


 x , y G   xy , 
2

 1   x , y  G   xy , 


( x , y ) N
In   x , y G  xy , 

 
( x , y ) N

2
  1    G   
In   x , y G  xy , 

x,y
2

2

,
xy
2
 
  1   x , y  G   xy , 

2
 
Numerical Solution
m in E 

 E 
 j


 E 


   m
i
( x , y ) N
xi

0
 m yi   m xj  m yj    1  2  x , y   m xj  m yj   x , y  0
i
 x,y 
 x, yG  x, y ,
 x, yG  x, y ,
2

2
 1   x , y

 G 
,
x,y
2

Numerical Solution
Initialize : 
   m
i
( x , y ) N
i
xi
 m yi   m xj  m yj    2  x , y  1   m xj  m yj   x , y
Numerical Solution
   m
i
( x , y ) N
xi
 m yi   m xj  m yj    2  x , y  1   m xj  m yj   x , y
i
obtain 
 x,y 
 x, yG  x, y ,
 x, yG  x, y ,
2
  1  
2
x,y

 G 
,
x,y
2

Numerical Solution
   m
i
( x , y ) N
xi
 m yi   m xj  m yj    2  x , y  1   m xj  m yj   x , y
i
obtain 
obtain  x , y
 x,y 
 x, yG  x, y ,
 x, yG  x, y ,
2
  1  
2
x,y

 G 
,
x,y
2

Numerical Solution
   m
i
( x , y ) N
xi
 m yi   m xj  m yj    2  x , y  1   m xj  m yj   x , y
i
obtain 
obtain  x , y
 x,y 
 x, yG  x, y ,
 x, yG  x, y ,
2
  1  
2
x,y

 G 
,
x,y
2

Numerical Solution
   m
i
( x , y ) N
xi
 m yi   m xj  m yj    2  x , y  1   m xj  m yj   x , y
i
obtain 
obtain  x , y
 x,y 
 x, yG  x, y ,
 x, yG  x, y ,
2
  1  
2
x,y

 G 
,
x,y
2

Numerical Solution
   m
i
( x , y ) N
xi
 m yi   m xj  m yj    2  x , y  1   m xj  m yj   x , y
i
obtain 
obtain  x , y
 x,y 
 x, yG  x, y ,
 x, yG  x, y ,
2
  1  
2
x,y

 G 
,
x,y
2

Numerical Solution (Example)
Iter 1 r
0.33
g
b
0.33 0.33
2
rg
rb
gb
r
0.00
0.00
0.00
0.00
g
2
b
2
0.00 0.00
Numerical Solution (Example)
Iter 2 r
0.97
g
b
0.91 0.38
rg
rb
-3.71
2.46
gb
r
2
-4.01 -4.02
g
2
b
2
4.00 0.79
Numerical Solution (Example)
Iter 3 r
g
b
rg
rb
1.14 -0.25 1.22 -1.55 -1.53
gb
r
2
-3.51 -1.18
g
2
3.32
b
2
0.69
Numerical Solution (Example)
Iter 4 r
g
b
rg
rb
gb
1.33 -1.61 2.10 1.35
-0.36
-1.61
r
2
-1.69
g
2
b
2
1.70 0.29
Numerical Solution (Example)
Iter 5 r
g
b
rg
1.52 -2.25 2.46 2.69
2
rb
gb
r
-1.38
-0.30
-1.95
g
2
0.79
b
2
-0.02
Numerical Solution (Example)
Iter 13 r
g
b
1.98 -3.29 3.02
rg
rb
gb
5.94
-3.38
2.81
r
2
g
2
b
2
-2.91 -1.56 -0.96
Numerical Solution (Example)
Iter 14 r
g
b
1.99 -3.31 3.03
rg
rb
gb
6.03
-3.42
2.89
r
2
g
2
b
2
-2.95 -1.62 -0.98
Numerical Solution (Example)
Iter 15 r
g
b
2.00 -3.32 3.04
rg
rb
gb
6.10
-3.45
2.94
r
2
g
2
-2.98 -1.67
b
2
-1.00
Results
Input
Ours
[Rasche et al. 2005] [Kim et al. 2009]
Results
Input
Ours
[Rasche et al. 2005] [Kim et al. 2009]
Results
Input
Ours
[Rasche et al. 2005] [Kim et al. 2009]
Results
Input
Ours
[Rasche et al. 2005] [Kim et al. 2009]
Results (Quantitative Evaluation)
• color contrast preserving ratio (CCPR)
CCPR=
#  ( x , y ) | ( x , y )    , | g x  g y |  
|  |
the set containing all neighboring pixel pairs with
the original color difference    .

x,y
Results (Quantitative Evaluation)
  10
Our Results (Quantitative Evaluation)
 x,y  
  10
Results (Quantitative Evaluation)
 x,y  
 x,y   & g x,y  
  10
Results (Quantitative Evaluation)
Number: 38740
Number: 24853
 x,y  
 x,y   & g x,y  
  10
Results (Quantitative Evaluation)
Number: 38740
CCPR=
Number: 24853
24853
38740
 64.2%
Results (Quantitative Evaluation)



Results (contrast boosting)
substituting our grayscale image for the L channel in the Lab space
Results (contrast boosting)
substituting our grayscale image for the L channel in the Lab space
Conclusion
• A new color-to-grayscale method that can well
maintain the color contrast.
• Weak color constraint.
• Polynomial Mapping Function for global
mapping.
The End
Limitations
• Color2gray is very subjective visual experience.
Contrast enhancement may not be acceptable for
everyone.
• Compared to the naive color2grayscale mapping, our
method is less efficient due to the extra operations.
An arguable result
Running Time
• For a 600 × 600 color input, our Matlab
implementation takes 0.8s
• A C-language implementation can be 10 times
faster at least.