Transcript Computational Vision 493.69 – 051

3-D Computer Vision
CSc 83029
Photometric Stereo & Shape from Shading
3-D Computer Vision CSc83029 / Ioannis Stamos
Photometric Stereo & Shape from
Shading
 Technique for recovering 3-D shape
information from image intensity (brightness)
 We will discuss:
 Reflectance maps.
 Photometric stereo.
 Shape from shading.
3-D Computer Vision CSc83029 / Ioannis Stamos
Radiometry and Reflectance
Image irradiance
E ( p )  L( P )
Brightness falloff
 d 
2
  cos 4 ( )
4 f 
1 / F-number of lens
Scene radiance
We assume (calibration is needed) that:
E ( p )  L( P )
3-D Computer Vision CSc83029 / Ioannis Stamos
Lambertian Reflectance Model
I(x,y)
p
n
θi
v
s
P
'
'
k cos i  k ( n  s)


L( P)   ( n  s)
L( P ) 
or:
E (p)   (n  s)
A Lambertian sphere
k : Source brightness
ρ’: Surface albedo (reflectance)
ρ : Effective albedo (absorbs
source
brightness)
E(p)  R ,s (n) : REFLECTANCE MAP
3-D Computer Vision CSc83029 / Ioannis Stamos
Lambertian Reflectance Model
I(x,y)
p
n
θi
v
s
P
A Lambertian sphere
E(p)  R ,s (n) : REFLECTANCE MAP
Relates Image Irradiance E to surface
orientation for given source direction
and surface reflectance
3-D Computer Vision CSc83029 / Ioannis Stamos
Representation of surface normal
Z
n
ry
Unit vector
Z=Z(x,y)
rx
y
x
Appendix A.5 (Trucco)
N  rx  ry  ( p, q,1)
n  N / || N ||
n
( p, q,1)
p 2  q2  1
Z
Z
p
,q  
x
y
3-D Computer Vision CSc83029 / Ioannis Stamos
Gradient Space (p,q)
Source
z
1.0
 ps , qs ,1
q
s
p
n
 p, q,1  N
y
n  N / || N ||
x
Surface normal can be represented by a point (p,q) on a plane!
Source direction can be represented by a point (ps,qs)!
**We want to calculate (p,q) from intensity I(x,y)
3-D Computer Vision CSc83029 / Ioannis Stamos
Gradient Space (p,q)
Source
z
1.0
 ps , qs ,1
q
s
n
p
x
Surface normal n 
Source direction s 
 p, q,1  N
y
n  N / || N ||
 p, q,1
p 2  q2  1
 ps , qs ,1
cosi  n  s 
1  pps  qqs
1  p 2  q2 1  ps  qs
ps  qs  1
2
2
Assumption: SOURCE DIR. IS CONSTANT FOR ENTIRE SCENE.
3-D Computer Vision CSc83029 / Ioannis Stamos
2
2
Reflectance Map (Lambertian)
OR:
I ( x , y )  E ( p) 
1  pps  qqs
1 p  q
2
2
1  ps  qs
2
2
 R1,s ( p, q)
Source
z
s
p
ISO-BRIGHTNESS
q CONTOUR
Constant θi
y
x
3-D Computer Vision CSc83029 / Ioannis Stamos
Reflectance Map (Lambertian)
ISO-BRIGHTNESS
CONTOURS
R1,s ( p, q)  0.8
( ps , qs )
i  90o
NOTE: R(p,q) is maximum when (p,q)=(ps,qs)
3-D Computer Vision CSc83029 / Ioannis Stamos
Reflectance Map (Lambertian)
Examples.
Where is
the source
with respect
to the
sphere?
3-D Computer Vision CSc83029 / Ioannis Stamos
Reflectance Map (Glossy Surfaces)
3-D Computer Vision CSc83029 / Ioannis Stamos
Shape from Shading
ISO-BRIGHTNESS
CONTOURS
R1,s ( p, q)  0.8
( ps , qs )
i  90o
PROBLEM: Given 1) source direction ( ps , qs )
2) surface reflectance (ρ)
3) one intensity image I(x,y)
R ,s ( p, q)
Can we find unique surface orientation (p,q)?
Two reflectance maps?
R2( p, q)  r
R1( p, q)  r
3-D Computer Vision CSc83029 / Ioannis Stamos
Two reflectance maps?
R2( p, q)  r
R1( p, q)  r
Intersections:
2 solutions for
p and q.
What if we don’t know the albedo?
3-D Computer Vision CSc83029 / Ioannis Stamos
Photometric Stereo
Use multiple light sources to resolve ambiguity
In surface orientation.
Note: Scene does not move – Correspondence
between points in different images is easy!
Notation: Direction of source i: s i or ( psi , qsi )
Image intensity produced by source i: Ii ( x, y)
3-D Computer Vision CSc83029 / Ioannis Stamos
Lambertian Surfaces (special case)
n  ( n x , n y , nz )
si  ( sxi , s yi , szi )
Use THREE sources in directions s1, s2 , s3
Image Intensities measured at point (x,y):
I1   s1  n
I 2   s2  n
I 3   s3  n
s1T 
 I1 
 I     sT  n
 2
 2
sT3 
 I 3 
 
S
 I1 
n  S 1  I 2   N
 
 I 3 
N
n
N
N
3-D Computer Vision CSc83029 / Ioannis Stamos
orientation
albedo
Photometric Stereo: RESULT
INPUT
orientation
albedo
From Surface Orientations to Shape
Integrate needle map
3-D Computer Vision CSc83029 / Ioannis Stamos
Calibration Objects & Look-up Tables
Calibration: Useful when Reflectance Map Equations
are uknown.
Use Calibration sphere of known size and same reflectance
as scene objects.
 p, q 
Each point on the sphere
has a unique known
orientation.
3-D Computer Vision CSc83029 / Ioannis Stamos
Calibration Objects & Look-up Tables
Illuminate the sphere with one source at a time and
obtain an image.
Each surface point with orientation (p,q) produces
three images (I1, I2, I3)
Generate a LOOK-UP TABLE (I1, I2, I3) -> (p,q)
I3
(p,q)
ARRAY
I1
I2
For an object of uknown shape but same reflectance,
obtain 3 images using same sources. For each image
point use LUT to map (I1, I2, I3) -> (p,q)
Photometric Stereo: Remarks
(1) Reflectance & illumination must be known a-priori.
(2) Local Method.
(3) Major Assumption: No interreflections.
Concave surfaces exhibit
interreflections.
3-D Computer Vision CSc83029 / Ioannis Stamos
Shape from Shading
 Can we recover shape from a Single
Image?
3-D Computer Vision CSc83029 / Ioannis Stamos
Human Perception of Shape from
Shading
We assume light source is above us.
Ramachandran 88
3-D Computer Vision CSc83029 / Ioannis Stamos
Human Perception of Shape from
Shading
We assume light source is above us.
Ramachandran 88
3-D Computer Vision CSc83029 / Ioannis Stamos
Human Perception of Shape from
Shading
Surface boundaries have strong influence on perceived shape
3-D Computer Vision CSc83029 / Ioannis Stamos
Finding the Illumination
Direction
Assumption: The surface normals of
the scene are distributed uniformly in
3-D space
cos  
P  ,   
2
 ,  
I

Illumination direction
Mean intensity
 cos
4
1
I 2   2 1  3 cos2 
6
Iy
 tan
Ix


Mean square intensity
Mean image gradient
Shape from Shading
R  p, q  
1  pps  qqs
1  p 2  q 2 1  ps  qs
2
2
2
2
2

(
p

p

q

q
)

(
I
(
x
,
y
)

R
(
p
,
q
))
x
y
 x
y
2
2
Smoothness constraint
2
2
2
   ( ( px 2  p y 2 3-DqComputer
x  q y )  ( I ( x, y )  R( p, q)) )dxdy
Vision CSc83029 / Ioannis Stamos
Shape from Shading
e    ( px  p y  q2 x  q2 y )  ( I ( x, y )  R( p, q))2
2
2
Calculus of Variations -> Discrete Case:
e
R
 2 4 pkl  pkl   2I (k , l )  R( pkl , qkl ) 
0
pkl
p
e
R
 2 4qkl  qkl   2I (k , l )  R( pkl , qkl ) 
0
qkl
q
pkl
Is the average of the 4 neighboring values
1
R

I (k , l )  R( pkln , qkln ) 
4
p
1
R

qkln 1  qkln 
I ( k , l )  R( pkln , qkln ) 
4
q
pkln 1  pkln 
Iterative solution
3-D Computer Vision CSc83029 / Ioannis Stamos
Shape from Shading
We know the normal at the contour.
This provides boundary conditions.
OCCLUDING BOUNDARY
3-D Computer Vision CSc83029 / Ioannis Stamos