Transcript Reflectance Imaging: A Simple Approach to Capturing Surface Detail Tom Malzbender

Reflectance Imaging:
A Simple Approach to
Capturing Surface Detail
Tom Malzbender
Hewlett-Packard
Laboratories
Texture Mapping
[ Catmull ‘74 ]
Pro:
• Photographic
input
• Simplicity
• Hardware Support
Con:
• Unrealistic Silhouettes
• Static Lighting
Bump Mapping
[ Blinn 78 ]
Pro:
• Lighting
Variations
Con:
• Per-pixel lighting
computation
• Filtering is Problematic
• Procedural Synthesis
(Not image-based.)
[ Rushmeier ‘97 ]
PTM Demonstration:
Advantages:
• Image based unlike bump mapping
• Simpler to evaluate than bump mapping
• Can leverage mip mapping
PTM Demonstration:
Top: Polynomial Texture Map
Bottom: Conventional Texture Map
Advantages:
• Image based unlike Bump Mapping
• Simpler to evaluate than Bump Mapping
• Can leverage Mip Mapping
Acquiring PTM’s Photographically
• Fixed object, fixed camera.
• Limited to Diffuse Objects.
*Third acquisition device being fabricated currently
Modeling Pixel Color Changes Directly
(u,v)
Light Direction Space
Modeling Pixel Color Changes Directly
(u,v)
Light Direction Space
L(u,v;lu ,l v )  a0 lu  a1lv  a2lu lv  a3lu  a4 lv  a5
2
2
Polynomial Texture Mapping
PTM: Store RGB per pixel and
Store polynomial coefficients
(a0-a5) per textel:
L(u,v;lu ,l v )  a0 lu  a1lv  a2lu lv  a3lu  a4 lv  a5
2
2
R = L R'
G = L G'
B = L B'
Why Polynomials?
• Compact Representation
• Consist solely of multiplies and adds.
• Cheap to evaluate on both modern CPUs and VLSI
Light Direction Parametrization
L(u,v;lu ,l v )  a0 lu  a1lv  a2lu lv  a3lu  a4 lv  a5
2
u,v
a0-a5
lu,lv
2
- texture coordinates
- fitted coefficients stored in texture map
- projection of light direction into texture plane
v
N
L
(lu,lv)
u
* lu,lv can be
scan-converted
without normalization.
lv
lu
PTM Formats
Per Pixel Storage
Format
• LRGB -
• RGB
-
• ENC
-
...
+
R,G,B
Index to L.U.T. storing Polynomial Coefficients
2D Applications
• Enhancement of Cuneiform
Tablets w/ Zuckerman USC
• PTMs for Short Image Sequences
• PTMs for Depth of Focus Effects.
Surface Normal Extraction
Yields maximum
surface brightness
Surface Normal Extraction
For a diffuse object, coordinates of
(lu,lv) that maximize luminance
yield local surface normals.
Setting
L L

0
u v
lu0,lv0
lu 0 
yields:
lv 0 
Providing a surface normal per textel:

2
2
N  (lu 0 , lv 0 , 1  lu 0  lv 0 )
a2 a4  2a1a3
4a0a1  a2 2
a 2 a3  2a0 a 4
4a0a1  a2
2
Specular Enhancement
Cuneiform tablet courtesy of
Dr. Bruce Zuckerman at U.S.C.
Diffuse Gain - a reflection transformation that:
• Keeps the surface normal fixed.
• Increases the second derivative
of the reflectance function by g.
a0 '  ga0
a1 '  ga1
a2 '  ga2
a3 '  (1  g )( 2a0lu 0  a2lv 0 )  a3
a4 '  (1  g )( 2a1lu 0  a2lv 0 )  a4
a5 '  (1  g )( a0lu 0 2  a1lv 0 2  a2lu 0lv 0 ) 
( a 3  a 3 ' ) l u 0  ( a 4  a 4 ' ) l v 0  a5
Light Direction Extrapolation
• Input images are collected across a
hemisphere of light directions, i.e. -1<= lu,lv <=1
• PTM’s can be evaluated outside of the hemisphere,
( lu,lv < -1 or lu,lv > 1 )
Forensic Applications
Java PTM viewer
• Written by Clifford Lyon, Harvard University
• Easy to add new datasets.
• http://materialobjects.com/browser/PTMBrowser_1.html
Forensic Applications
National Gallery London
Joe Padfield
Light Arm
Real Time
Enhancement
• Basler A504kc 500 F/sec camera.
• Real-time transfer to HP workstation.
• Processing on GPU, Nvidia Gforce 7800 GTX.
• 60 f/sec throughput.
• Normal transforms (unsharp mask), Relighting.
PTM Object Movies - POM
• Viewpoint Variation
• Lighting Variation
• Surface Enhancements
• Currently lacking in view interpolation.
Collaboration w/
Alberto Proenca, Univ. Minho, Portugal
Cultural Heritage Imaging, California
Collection Archéologie du Musée
de l'Hospice du Grand St. Bernard
Highlight-based PTMs
Developed with Mark Mudge,
Cultural Heritage Imaging
V
N
L
f
z
Sx
1.0
x
Conclusions + Web Tools
•
Reflectance functions are easy to collect and model.
•
Transforming reflectance functions can help visualization.
•
Tools are available at hpl.hp.com/ptm
sample PTMs
PTM viewer
Polynomial Fitter
PTM format document