Transcript Document

Polarization-based Inverse
Rendering from Single View
Daisuke Miyazaki
Robby T. Tan
Kenji Hara
Katsushi Ikeuchi
Modeling cultural assets
Geometrical
Photometrical
Environmental
Integrated framework for obtaining 3 types of information
2
Related work
Geometry Photometry Environment
Tominaga et.al. 2000


Zheng et.al. 1991



Nayar et.al. 1996



Sato et.al. 1999


Ramamoorthi et.al. 2001


Nishino et.al. 2001


Hara et.al. 2002




Proposed method

3
Outline
1. Reflection components
separation
2. Shape from polarization
using diffuse light
3. Light source estimation
from intensity peak
2
4. Reflection parameters
estimation by l.s.m.
Minimize
Ks , σ
rendered
image
real
image
4
1. Reflection components separation
Dichromatic reflection model
Incident light
Surface
normal
Specularly
reflected light
Diffusely
reflected light
Air
Object
6
Reflection components separation
[Tan2002]
Input
Diffuse
•Shape
Specular
•Illumination
•Reflection parameters
7
2. Shape from polarization
Related work
Koshikawa 1979
Wolff 1990
Rahmann et.al. 2001
Miyazaki et.al. 2002
Proposed method
Object
Opaque
Reflection View
Specular 1
Opaque
Diffuse
Opaque
Diffuse
Transparent Specular
2
2~5
2
Opaque
1
Diffuse
9
Polarization
Incident light
Specularly
reflected light
Diffusely
reflected light
Air
Object
10
Surface normal
Camera
Polarizer
q
Azimuth angle 
Object
11
Azimuth angleφ and intensity difference
255
Imax
Intensity
Imin
0
1
2
Rotation
angle
360 of
polarizer
Azimuth angle 
-ambiguity
12
Propagation
object
Determination of azimuth angle 
Propagate φ from occluding boundary to inner
part of object area (Assumption: smooth surface)
[Ikeuchi&Horn1981]
Cannot apply to “dimples”(=perfect concave)
13
Zenith angleθ and DOPρ
1
DOP ρ
Degree
Of
Polarization
I max  I min

I max  I min
2

n  1 n  sin 2 q

2
2  2n 2  n  1 n  sin 2 q  4 cos q
ρ
0
θ
n 2  sin 2 q
Zenith angle θ
90°
14
Modification
0.5
DOP ρ
Degree
Of
Polarization
0
2
2

n  1 n  Isin
max q I min
Definition

of DOP: 2 
2
2
2  2n 2  n  1 n  sin 2 qI 4 cos
q
n

sin
q
max  I min
I max  I min
Modified DOP:  
I max  I min  u
u: modification factor
•Raises DOP
•Assumption
•Closed smooth object
•“u” is constant
u
Zenith
angle
90° θ
15
Surface normal
φ
θ
Surface normal
16
Height
• Relaxation method
[Ikeuchi1984]
2

 H
  H


p


q
  x   y  dxdy
2
Minimize:
Surface n   p
normal
 q 1
H
H
q

Gradient p 
y
x
T
1 p2  q2
Iteratively update:
H
where,
( i 1)
Height H
1  p q 
( x, y )  H ( x, y )    
4  x y 
(i )
17
3. Illumination estimation
Illumination sphere
Light source is represented in polar coordinate system (θ, φ)
θ=0°
L2=(θ2, φ2)
L1=(θ1, φ1)
L3=(θ3, φ3)
φ=90°
θ=90°
φ=180°
θ=90°
φ=0°
Object
φ=270°
θ=180°
19
Illumination estimation
Detect position of intensity peak
Determine light source orientation from the peak
1.Project to (θ, φ)-space
2.Thresholding
3.Detect intensity peak
20
4. Reflection parameters estimation
Torrance-Sparrow reflection model
Surface
Incident normal
light
Bisector
α
θi
θr
View
I  K d  cosθi  K s
Diffuse reflection
Object surface
Known: θi, θr, α
1
cosθr
e
α2

2 2
Specular reflection
Unknown:
•Diffuse reflection scale; Kd
•Specular reflection scale; Ks
•Surface roughness; σ
22
Reflection parameters estimation
Solve the following least-square problem
by steepest-descent method
2
Minimize
Ks, σ
Ks
rendered
image
1
cosθr
e

real
image
α2
2 2
23
Experimental result
Input
Azimuth angleφ
Intensity I
DOPρ
25
Result of shape estimation
26
Result of illumination estimation
Actual illumination distribution
Estimated illumination distribution
27
Rendering result
Input
Synthesized
image
Rendered image under
different illumination & view
28
Result for another object
Input
Synthesized
image
Estimated
shape
Rendered image under
different illumination & view
29
Conclusions
• Estimated geometrical, photometrical,
environmental information in one integrated
framework
– Shape from polarization
– Surface reflection parameters from iterative
computation
– Illumination from intensity peak
30
Application to digital archiving project
• Multiple View
• Modeling a statue in a room
– IBR with
• surface normal
• reflection parameters
Photorealistic preservation
31
Fin
(c) Daisuke Miyazaki 2003
All rights reserved.
http://www.cvl.iis.u-tokyo.ac.jp/
D. Miyazaki, R. T. Tan, K. Hara, K. Ikeuchi,
"Polarization-based Inverse Rendering from Single
View," in Proceedings of International Symposium on
the CREST Digital Archiving Project, pp.51-65, Tokyo,
Japan, 2003.05