Transcript Scan line algorithm - University of California, Merced

EECS 274 Computer Vision
Sources, Shadows, and Shading
Surface brightness
• Depends on local surface properties
(albedo), surface shape (normal), and
illumination
• Shading model: a model of how brightness
of a surface is obtained
• Can interpret pixel values to reconstruct its
shape and albedo
• Reading: FP Chapter 2, H Chapter 11
Radiometric properties
• How bright (or what color) are
objects?
• One more definition: Exitance
of a light source is
– the internally generated power
(not reflected) radiated per
unit area on the radiating
surface
• Similar to radiosity: a source
can have both
– radiosity, because it reflects
– exitance, because it emits
• Independent of its exit angle
• Internally generated energy
radiated per unit time, per unit
area
E ( P )   Le ( P ,  0 ,  o ) cos  0 d 

• But what aspects of the
incoming radiance will we
model?
– Point, line, area source
– Simple geometry
Radiosity due to a point sources
 
 
r
2
Radiosity due to a point source
•
As r is increased, the rays leaving
the surface patch and striking the
sphere move closer evenly, and
the collection changes only
slightly, i.e., diffusive reflectance,
or albedo
•
Radiosity due to source
B P    d solid angle Exitance term  cos  i
 
  d   E cos  i
r
2
Nearby point source model
B P    d solid angle Exitance term  cos  i
 
  d   E cos  i
r
 N P   S P  

  d P 
2
 r P 

2
•
•
•
•
The angle term, can be written in terms of N and S
N: surface normal
ρd: diffuse albedo
S: source vector - a vector from P to the source, whose length is the
intensity term, ε2E
– works because a dot-product is basically a cosine
Point source at infinity
• Issue: nearby point source
gets bigger if one gets closer
– the sun doesn’t for any
reasonable assumption
• Assume that all points in the
model are close to each other
with respect to the distance to
the source
• Then the source vector doesn’t
vary much, and the distance
doesn’t vary much either, and
we can roll the constants
together to get:
 N P   S P  

B ( P )   d  P 
2
 r P 

N P   S P  N  ( S 0  S ( P ))

2
( r0  r ( P )) 2
r P 

N  S0
2
r0

N  S0
2
r0


( N  S ( P )) rP
1  2
  
r0


 N S
B ( P )   d  P  N  P   S 
Line sources
Infinitely long narrow cylinder with constant exitance
radiosity due to line source varies with inverse distance, if the source
is long enough
Area sources
• Examples: diffuser boxes,
white walls
• The radiosity at a point due to
an area source is obtained by
adding up the contribution over
the section of view hemisphere
subtended by the source
– change variables and add up
over the source
Radiosity due to an area source
• ρd is albedo
• E is exitance
• r is distance
between points Q
and P
• Q is a coordinate
on the source
E (Q ) 
1

Le(Q )
 
  P  L Q, QP cos  d
BP    d P  Li P, QP cos  i d

d
e
i

 E Q  
  d P  
 cos  i d
 

dAQ 

 E Q  
 d  p  
 cos  i  cos  s 2 
 
r 

source 
cos  i cos  s
  d P   E Q 
dAQ
2
r
source
Shading models
• Local shading model
– Surface has radiosity due only
to sources visible at each
point
– Advantages:
• often easy to manipulate,
expressions easy
• supports quite simple
theories of how shape
information can be
extracted from shading
• Global shading model
– Surface radiosity is due to
radiance reflected from other
surfaces as well as from
surfaces
– Advantages:
• usually very accurate
– Disadvantage:
• extremely difficult to infer
anything from shading
values
Local shading models
• For point sources at infinity:
B( P) 

 B ( P)
s
ssources visible from P

( P) N ( P)  S s
d
ssources visible from P
• For point sources not at infinity
B( P) 

ssources visible from P
 d ( P)
N (P)  S (P)
rs ( P ) 2
Shadows cast by a point source
• A point that can’t see the
source is in shadow (self cast
shadow)
• For point sources, the
geometry is simple (i.e., the
relationship between shape
and shading is simple)
• Radiosity is a measurement of
one component of the surface
normal
B( P) 

 B ( P)
s
ssources visible from P

( P) N ( P)  S s
d
ssources visible from P
Analogous to the geometry of
viewing in a perspective camera
Area source shadows
Are sources do not produce dark
shadows with crisp boundaries
1.
2.
3.
Out of shadow
Penumbra (“almost shadow”)
Umbra (“shadow”)
Photometric stereo
• Assume:
– A local shading model
– A set of point sources that are infinitely distant
– A set of pictures of an object, obtained in
exactly the same camera/object configuration
but using different sources
– A Lambertian object (or the specular
component has been identified and removed)
Monge patch
Projection model for surface recovery - Monge patch
In computer vision, it is often known as height map, depth map, or
dense depth map
Image model
• For each point source, we
know the source vector (by
assumption)
• We assume we know the
scaling constant of the linear
camera (i.e., intensity value is
linear in the surface radiosity)
• Fold the normal and the
reflectance into one vector g,
and the scaling constant and
source vector into another Vj
• Out of shadow:
I j ( x, y )  kB( x, y )
 k ( x, y )( N ( x, y )  S j )
  ( x, y ) N ( x, y )  ( kS j )
 g ( x, y )  V j
• g(x,y): describes the surface
• Vj: property of the illumination
and of the camera
• In shadow:
I j ( x, y )  0
From many views
• From n sources, for each of which Vi is known
V1T 
 T
V 
V  2 

V T 
 n 
I j ( x, y )  kB( x, y )
 k ( x, y )( N ( x, y )  S j )
  ( x, y ) N ( x, y )  ( kS j )
 g ( x, y )  V j
• For each image point, stack the measurements
i ( x , y )  ( I 1( x , y ), I 2( x , y ),  , I n ( x , y )) T
• Solve least squares problem to obtain g
i( x, y )  V g ( x, y )
One linear system per point
 I1 ( x, y )  V1T 

  T
 I 2 ( x, y )  V2 
       g ( x, y )

  
 I ( x, y )   V T 
 n
  n 
Dealing with shadows
i( x, y )  V g ( x, y )
 I1 ( x, y )  V1T 

  T
I
(
x
,
y
)
 2
 V2 
       g ( x, y )

  
 I ( x, y )   V T 
 n
  n 
 I12 ( x, y )   I1 ( x, y )
0

0  V1T 
 2
 
 T 
I 2 ( x, y ) 
  V2 
 I 2 ( x, y )   0
g ( x, y )
   





0   

 
 I 2 ( x, y )   0
 V T 

0
I
(
x
,
y
)
n
 n
 
 n 
Known
Known
Known
Unknown
Recovering normal and
reflectance
• Given sufficient sources, we can solve the previous
equation (e.g., least squares solution) for g(x, y)
• Recall that g(x, y) = (x,y) N(x, y) , and N(x, y) is
the unit normal
• This means that alberdo x,y) =||g(x, y)||
• This yields a check
– If the magnitude of g(x, y) is greater than 1, there’s a
problem
• And N(x, y) = g(x, y) / x,y)
Five synthetic images
Generated from a sphere in a orthographic view from the same viewing position
Recovered reflectance
||g(x,y)||=ρ(x,y): the alberdo value should be in the range of 0 and 1
Recovered normal field
For viewing purpose, vector field is shown for every 16th pixel in each direction
Parametric surface tangents
x
v
v
u
x
u
Shape from normals
• Recall the surface is written as
( x , y , f ( x , y ))
• Parametric surface
x  x, y  y , z  f ( x, y )
 f 
 f 
ru  i   k , rv  j   k
 x 
 y 
f
f
N  r u  rv  i  j  k
x y
• This means the normal has the
form:

N ( x, y )  

  fx 

1
  f 
 y

2
2
f x  f y  1 

1


• If we write the known vector g
as
 g1 ( x , y ) 


g ( x, y )   g 2 ( x, y ) 
 g ( x, y ) 
 3

• Then we obtain values for the
partial derivatives of the
surface:
f x( x , y )  g1 ( x , y ) / g 3 ( x , y )
f y ( x, y )  g 2 ( x, y ) / g 3 ( x, y )
Shape from normals
• Recall that mixed second partials are equal --- this gives us a check.
We must have:
 ( g 1 ( x , y ) / g 3 ( x , y ))  ( g 2 ( x , y ) / g 3 ( x , y ))

y
x
(or they should be similar, at least)
• Known as integrability test
• We can now recover the surface height at any point by integration
along some path, e.g.
v
u
0
0
f (u , v )   f y ( 0, y ) dy   f x ( x , v ) dx c
Recovered surface by integration
v
u
0
0
f (u , v )   f y ( 0, y ) dy   f x ( x , v ) dx c
The illumination cone
What is the set of n-pixel images of an object under all
possible lighting conditions (at fixed pose)?
(Belhuemuer and Kriegman IJCV 99)
x2
x1
Single light source image
xn
N-dimensional
Image Space
The illumination cone
What is the set of n-pixel images of an object under all
possible lighting conditions (but fixed pose)?
Proposition: Due to the superposition of images, the set of
images is a convex polyhedral cone in the image space.
Illumination Cone
x2
Single light source images:
Extreme rays of cone
2-light source
image
x1
xn
N-dimensional
Image Space
Generating the illumination cone
For Lambertian surfaces, the illumination cone is determined by the 3D
linear subspace B(x,y), where
I ( x , y )   max(  ( x , y ) n ( x , y )  s i ,0 )   max( B ( x , y )  s i ,0 )
i
i
When no shadows, then I ( x , y )   B ( x , y )  si
i
Use least-squares to find 3D linear subspace,
subject to the constraint
fxy=fyx (Georghiades, Belhumeur, Kriegman, PAMI, June, 2001)
3D linear subspace
ax,y fx(x,y)
fy(x,y)
albedo (surface normals)
Original (Training) Images
Surface. f(x,y) (albedo
textured mapped on surface)
Image-based rendering: Cast
shadows
Single Light Source
Face Movie
Yale face database B
• 10 Individuals
• 64 Lighting Conditions
• 9 Poses
=> 5,760 Images
Variable lighting
Limitation
• Local shading model is a poor description of
physical processes that give rise to images
– because surfaces reflect light onto one another
• This is a major nuisance; the distribution of light
(in principle) depends on the configuration of
every radiator; big distant ones are as important
as small nearby ones (solid angle)
• The effects are easy to model
• It appears to be hard to extract information from
these models
Interreflections - a global
shading model
• Other surfaces are now area sources - this
yields:
Radiosity at a surface  Exitance  Radiosity due to other surfaces
cos  i cos s
B ( x )  E ( x )   d ( x )  B(u)
Vis ( x, u ) dAu
2
r ( x , u )
all other
sources
• Vis(x, u) is 1 if they can see each other, 0
if they can’t
What do we do about this?
• Attempt to build approximations
– Ambient illumination
• Study qualitative effects
– reflexes
– decreased dynamic range
– smoothing
• Try to use other information to control
errors
Ambient illumination
• Two forms
– Add a constant to the radiosity at every point in the
scene to account for brighter shadows than predicted
by point source model
• Advantages: simple, easily managed (e.g. how would you
change photometric stereo?)
• Disadvantages: poor approximation (compare black and
white rooms
– Add a term at each point that depends on the size of
the clear viewing hemisphere at each point
• Advantages: appears to be quite a good approximation, but
jury is out
• Disadvantages: difficult to work with