#### Transcript Image Processing

1 • Light and the EM spectrum • The H.V.S. and Color Perception

### What is an Image ?

• An image is a projection of a 3D scene into a 2D

projection plane

.

• An image can be defined as a 2 variable function I(x,y) , where for each position (x,y) in the projection plane, I(x,y) defines the light intensity at this point.

2

### Camera trial #1

scene film Put a piece of film in front of an object.

3 source: Yung-Yu Chuang

### Pinhole camera

pinhole camera scene barrier film Add a barrier to block off most of the rays.

• It reduces blurring • The pinhole is known as the aperture • The image is inverted 4 source: Yung-Yu Chuang

5

The Pinhole Camera Model (where) ( x,y ) 6 d (x,y,z) d –

focal length x

1  

y w

     0 0 0 1 0 0 0  1 /

d

0 0      0  

X Y Z

1   Y center of projection (pinhole) d X Z

7 Shading Model: Given the illumination incident at a point on a surface, what is reflected?

• The factors determining the shading effects are: – The light source properties: • Positions, Electromagnetic Spectrum, Shape.

– The surface properties: • Position, orientation, Reflectance properties.

– The eye (camera) properties: • Position, orientation, Sensor spectrum sensitivities.

8

### The Light Properties

9 Newton’s Experiment, 1665 Cambridge.

Discovering the fundamental spectral components of light.

(from Foundations of Vision: Brian Wandell, 1995.

10 A prism

Electromagnetic Radiation - Spectrum 11 Wavelength in nanometers (nm)

12 Electromagnetic Wave

Monochromators Monochromators measure the power or energy at different wavelengths 13

Spectral Power Distribution (SPD) The

Spectral Power Distribution

function e(  ) (SPD) of a light is a which defines the relative energy at each wavelength.

1 0.5

0 400 500 600 Wavelength (  ) 700 14

Examples of Spectral Power Distributions 1 1 0.5

1 0 400 500 600 Blue Skylight 700 0.5

1 0 400 500 600 700 Tungsten bulb 0.5

0.5

15 0 400 500 600 700 Red monitor phosphor 0 400 500 600 700 Monochromatic light

### The Surface Properties

Reflected Light Incoming Light Transmitted Light 16 • Interactions between light and matter depends on the physical characteristics of light as well as the matter.

• Three types of interactions: – Reflection – Absorption – Transmittance

The Bidirectional Reflectance Distribution Function (BRDF) • A BRDF describes how much light is reflected when light makes contact with a certain material

: quantity of light reflected in direction (  e ,  e )

BRDF

L E

(  ( 

e i

, ,  

i e

, ,   ) )

: quantity of light arriving from direction (  i ,  i ) 17

Simplified Model Incident light normal Specular reflection  Diffuse reflection 18

Diffuse

(

lambertian

)

reflection

Reflected randomly between color particles reflection is equal in all directions.

Specular reflection

mirror like reflection at the surface

19 Different Types of Surfaces

Simplified rendering models: reflectance Often are more interested in relative spectral composition than in overall intensity, so the spectral BRDF computation simplifies a wavelength-by-wavelength multiplication of relative energies.

.*

B. Freeman, and Foundations of Vision, by Brian Wandell,

=

21 Spectral Property of Lambertian Surfaces Yellow Red 1 0.8

0.6

0.4

0.2

1 0.8

0.6

0.4

0.2

400 500 600 Blue 700 1 0.8

0.6

0.4

0.2

400 1 0.8

0.6

0.4

0.2

500 600 Gray 700 400 500 600 700 400 Wavelength (nm) 500 600 700 Surface Body Reflectances (albedo)

Some reflectance spectra Forsyth, 2002

### The Eye Properties

   23  Cornea תינרק Pupil ןושיא Iris תיתשק Retina תיתשר Cornea Lens Pupil Iris Ocular Muscle Fovea Vitreous Humor Optic Disc Optic Nerve Retina

24

25

### The Visual Pathway

Retina Optic Nerve Optic Chiasm Lateral Geniculate Nucleus (LGN) Visual Cortex

26

### Eye v.s. Camera

Yaho Wang ’s slides

27 bipolar ganglion

### The Human Retina

cones rods horizontal amacrine light

• Retina contains 2 types of photo-receptors – Cones: • Day vision, can perceive color tone – Rods : • Night vision, perceive brightness only 28

Cones: • High illumination levels (Photopic vision) • • Sensitive to color (there are three cone types: L,M,S) Produces high-resolution vision • 6-7 million cone receptors, located primarily in the central portion of the retina Cone Spectral Sensitivity 1 0.75

0.5

0.25

29 0 400 500 600 Wavelength (nm) L M S 700 A side note: • Humans and some monkeys have three types of cones (trichromatic vision); most other mammals have two types of cones (dichromatic vision).

• Marine mammals have one type of cone.

• Most birds and fish have four types. •Lacking one or more type of cones result in color blindness.

Rods: • Low illumination levels (Scotopic vision).

• Highly sensitive (respond to a single photon).

• Produces lower-resolution vision • 100 million rods in each eye.

• No rods in fovea.

Rod Spectral Sensitivity 1 30 0.75

0.5

0.25

0 400 500 600 Wavelength (nm) 700

### Photoreceptor Distribution

Foveal Periphery photoreceptors 31 rods S - Cones L/M - Cones

Cone Receptor Mosaic (Roorda and Williams, 1999) 32 L-cones M-cones S-cones

Cone ’s Distribution: • L-cones (Red) occur at about ~65% of the cones throughout the retina.

• M-cones (green) occur at about ~30% of the cones.

• S-cones (blue) occur at about ~2-5% of the cones (

Why so few

?).

33 18 x 10 4 rods cones 14 10 Distribution of rod and cone photoreceptors 6 2 -60 -40 -20 0 fovea 20 40 60 Degrees of Visual Angle

### The Cone Responses

Assuming

Lambertian

Surfaces Sensors Illuminant Surface Output

M S L

   

l

(  )

e

(  )

k

(  ) 

m

(  )

e

(  )

k

(  ) 

s

(  )

e

(  )

k

(  ) 34 e(  ) – Fixed, point source illuminant k(  ) –surface’s reflectance l(  ),m(  ),s(  ) – Cone responsivities

Metamer -

two lights that appear the same visually. They might have different SPDs (spectral power distributions).

Tungsten light 800 Monitor emission 200 400 100 0 400 500 600 0 700 Wavelength (nm) 400 500 600 700 35 The phosphors of the monitor were set to match the tungsten light.

### The Trichromatic Color Theory

Trichromatic:

“tri”=three “chroma”=color color vision is based on three primaries (i.e., it is 3D).

Thomas Young (1773-1829) A few different retinal receptors operating with different wavelength sensitivities will allow humans to perceive the number of colors that they do.

Suggested 3 receptors.

Helmholtz & Maxwell (1850) Color matching with 3 primaries.

36

### Color Matching Experiment

• • Given a set of 3 primaries, one can determine for every spectral distribution, the intensity of the guns required to match the color of that spectral distribution.

The 3 numbers can serve as a color representation.

test match

T(

) + + R(

) G(

) + B(

)

Primaries 37

T

rR

gG

bB

 

### Color matching experiment 1

38 from: Bill Freeman

### Color matching experiment 1

p 1 p 2 39 p 3 from: Bill Freeman

### Color matching experiment 1

p 1 p 2 40 p 3 from: Bill Freeman

### Color matching experiment 1

The primary color amounts needed for a match p 1 p 2 41 p 3 from: Bill Freeman

### Color matching experiment 2

42 from: Bill Freeman

### Color matching experiment 2

p 1 p 2 43 p 3 from: Bill Freeman

### Color matching experiment 2

p 1 p 2 44 p 3 from: Bill Freeman

### Color matching experiment 2

We say a “negative” amount of p 2 was needed to make the match, because we added it to the test color’s side.

The primary color amounts needed for a match: p 1 p 2 p 3 p 1 p 2 p 3 p 1 p 2 45 p 3 from: Bill Freeman

Color matching experiment for Monochromatic lights 1 1 1 0.5

0 400 500 600 700 0.5

0 400 500 600 700 0.5

0 400 500 600 700 Primary Intensities 46

The

### Color

Matching Functions (CMF) 3 2 1 b(  ) g(  ) r(  ) 0 400 500 600 Wavelength (nm) 700 Stiles & Burch (1959) Color matching functions. Primaries are: 444.4 525.3 and 645.2

47

Problems

: Some perceived colors cannot be generated. This is true for

any

choice of visible primaries.

The superposition principle Foundations of Vision, by Brian Wandell, Sinauer Assoc., 1995 48 from: Bill Freeman

Observation

- Color matching is linear: – if (S  P) then (S+N  P+N) – if (S  P) then (  S   P) • Let T(  )=c  (   0 )+d  (   1 ) a double chromatic color: How should we adjust the 3 primaries?

d c

r

49 

c r

0 

d r

1 ;  0  1

g

c g

0 

d g

1 ;

b

c b

0 

d b

###  

1

• Outcome 1 : Any T(  ) can be matched:

r

 

T d

 ;

g

 

T d

 ;

b

 

T

• Outcome 2

:

CMF can be calculated for any chosen primaries U(  ), V(  ), W(  ):

u

 

v w

 

a

1   

b

1

c

1

a

2

b

2

c

2

a

3

b c

3 3     

r g b

  50

d

51

### The CIE Color Standard

• The

CIE

(Commission Internationale d ’Eclairage) defined in 1931 three hypothetical lights X, Y, and Z whose matching functions are positive everywhere:

### Tristimulus

• Let X, Y, and Z be the tri-stimulus values.

• A color can be specified by its trichromatic coefficients, defined as

x

X y

X X Y Z z

X Z Z Z

X ratio Y ratio Z ratio Two trichromatic coefficients are enough to specify a color (x + y + z = 1).

CIE Chromaticity Diagram Input light spectrum y From: Bahadir Gunturk x 53

CIE Chromaticity Diagram Input light spectrum y From: Bahadir Gunturk x 54

CIE Chromaticity Diagram Input light spectrum y From: Bahadir Gunturk x 55

CIE Chromaticity Diagram Input light spectrum From: Bahadir Gunturk y Boundary 380nm x 700nm 56

CIE Chromaticity Diagram Input light spectrum Boundary From: Bahadir Gunturk 57

CIE Chromaticity Diagram Light composition From: Bahadir Gunturk 58

CIE Chromaticity Diagram Light composition Light composition From: Bahadir Gunturk 59

60

### The sRGB Color Standard

• The sRGB is a device-independent color space. It was created in 1996 by HP and Microsoft for use on monitors and printers.

• It is the most commonly used color space.

• It is defined by a transformation from the xyz color space.

Color matching predicts matches, not appearance 61

62 Color Appearance

63 Color Appearance

64 Color Appearance

65

### Color Spaces

RGB Color Space (additive) • Define colors with (r, g, b) ; amounts of red, green, and blue 66

CMY Color Space (subtractive) • Cyan, magenta, and yellow are the complements of red, green, and blue – We can use them as filters to subtract from white – The space is the same as RGB except the origin is white instead of black 67

### Color names for cartoon spectra

400 500 600 700 nm 400 500 600 700 nm 400 500 600 700 nm 400 500 600 700 nm 400 500 600 700 nm From: B. Freeman 400 500 600 700 nm

400 500 600 700 nm When colors combine by

the color spectra. Example color displays that follow this mixing rule: CRT phosphors, multiple projectors aimed at a screen, Polachrome slide film.

400 500 600 700 nm Red and green make … Yellow!

400 500 600 700 nm

### Subtractive color mixing

400 500 600 700 nm 400 500 600 700 nm When colors combine by

multiplying

the color spectra. Examples that follow this mixing rule: most photographic films, paint, cascaded optical filters, crayons.

Cyan and yellow (in crayons, called “blue” and yellow) make … 400 500 600 700 nm Green!

71

Red Yellow Magenta 72 Green Cyan Blue

### HSV color space

• Hue - the chroma we see (red, green, purple).

• Saturation - how pure is the color (how far the color from gray ).

• Value (brightness) - how bright is the color.

73

### HSV color space

Saturation 74 Hue Value

75