Acquiring the Reflectance Field of a Human Face Paul Debevec, Tim Hawkins, Chris Tchou, H.P. Duiker, Westley Sarokin, and Mark Sagar UC Berkeley / USC Institute for Creative Technologies / LifeF/X www.debevec.org

Related Work

Hanrahan and Krueger .

Reflection from Layered Surfaces due to

Subsurface Scattering.

SIGGRAPH 93 Bregler et al .

Video Rewrite.

SIGGRAPH 97 Guenter et al .

Making Faces.

SIGGRAPH 98 Pighin et al .

Synthesizing Realistic Facial Expressions from Photographs

. SIGGRAPH 98 Sagar et al .

The Jester

. SIGGRAPH 99 ET Marschner et al .

Reflectance

Measurements of Human Skin.

1999

The Reflectance Field

The Reflectance Field

The Reflectance Field

The Reflectance Field

R i ( u i ,v i ,

q

i ,

f

i )

incident light field

The Reflectance Field

R i ( u i ,v i ,

q

i ,

f

i )

incident light field

R r ( u r ,v r ,

q

r ,

f

r )

radiant light field

The Reflectance Field

R i ( u i ,v i ,

q

i ,

f

i ; u r ,v r

8D reflectance field

,

q

r ,

f

r )

4D Slices of the 8D Reflectance Field

R i ( u i ,v i ,

q

i ,

f

i ; u r ,v r ,

q

r ,

f

r )

The Light Stage

The Light Stage: 60-second exposure

Light Stage Data

Original Resolution: 64  32 Lighting through image recombination: Haeberli ‘92, Nimeroff ‘94, Wong ‘97

Light Stage Results

Environments from the

Light Probe Image Gallery www.debevec.org

Reflectance Functions

Lighting Reflectance Functions

normalized light map reflectance function lighting product 1 rendered pixel 1 DCT Basis Smith and Rowe.

Compressed domain processing of JPEG-encoded images

. 1996

Interactive Lighting Demo SIGGRAPH 2000 Creative Applications Laboratory

Interactive Lighting Demo SIGGRAPH 2000 Creative Applications Laboratory

Changing the Viewpoint

Reflection of Light from Skin Specular Component:

After Hanrahan ‘93

Color of light, shiny, brighter near grazing, maintains polarization Subsurface Component: Color of skin, diffuse, desaturated near grazing, scrambles polarization

Separating Reflectance Components using Crossed Polarizers

Normal Image Subsurface Component Colorspace techniques - Sato ‘94, Nayar ‘97 Specular Component

Subsurface Component

Surface Normal Estimate

Original RF

Transforming a Reflectance Function

Specular Component => Torrance Sparrow microfacet distribution Shifted and Scaled Specular Final RF Comparison RF

Point-Source Comparison

Original Image Novel Viewpoint

Spatially-Varying Reflectance Parameters

Surface Normals

n

Diffuse Albedo r

d

Specular Intensity

k s

Specular Roughness a

Compositing Test

Original Image Light Probe Rendered Face Composite

4. Ongoing Work

•

Animate the faces

•

Capture more spectral samples

•

Use high-speed cameras to achieve real-time capture

5. Conclusion We have presented:

 The light stage apparatus for capturing slices of the reflectance field of the human face  A technique for correctly relighting faces and objects with arbitrary illumination  A technique for extrapolating human reflectance to novel viewpoints

Thanks Digital Media Innovation Program Interactive Pictures Corporation Alias|Wavefront UCB Digital Digital Media/New Genre Program ONR/BMDO Cornell Program of Computer Graphics Berkeley Millennium Project and Shawn Brixey, Bill Buxton, Larry Rowe, Jessica Vallot, Patrick Wilson, Melanie Levine, Eric Paulos, Christine Waggoner, Holly Cim, Eliza Ra, Bryan Musson, David Altenau, Marc Levoy, Maryann Simmons, Henrik Wann Jensen, Don Greenberg, Pat Hanrahan, Randal Kleiser, Chris Bregler, Michael Naimark, Dan Maas, Steve Marschner, and Kevin Binkert.

In-plane Reflectometry Measurements

 Subsurface exhibits chromaticity falloff  Specular is monochromatic with Torrance Sparrow microfacet behavior q

i

q

r

Subsurface q

i

q

r

Subsurface

+

Specular

Reflectometry Experiment

The Jester

SIGGRAPH 99 Electronic Theater Mark Sagar et al. - LifeF/X, Inc.

Performance and Text: Jessica Vallot

Changing the Viewpoint Model from Structured Lighting

Traditional Approach

 Derive

reflectance parameters

each point on the face’s surface for  Map the parameters onto a

geometric model

of the face  Render using traditional methods

Reflectance Function Mosaic