Transcript Shape and Shading - North Dakota State University

Shape and Shading
Koenderink & Van Dorn
Chapter 72 of The Visual Neurosciences
Introduction
• Observers are interested in
– Geometry of objects
– Material identification
– Light field: the primary and secondary
sources of radiation and how radiation
pervades space
• What we have: an image
• What we are interested in: the scene
Introduction
• Current topic deals with only the static,
achromatic, monocular observer
• Ignoring
– Observer motion
– Object motion
– Color
– Binocular
– Material properties
Introduction
• We want to know how object shape can
be determined from monochrome
pictures.
The light field
• Light model: rays (particle model)
• Radiators
– Primary: luminous
– Secondary
• Reflecting
• Scattering
The light field
• Volume (ray) density of radiation: the
length of all rays crossing a volume,
divided by that volume.
• Or the density of photons crossing that
volume during a given time interval.
The light field
• Net flux vector:
• Number of photons crossing a unit area
per unit time.
• Calculated as proportional to cos(),
where  is the angle between a surface
patch normal and the beam direction.
• Such a definition is appropriate only for
small patches, of course.
The light field
• The light passing thru any such circular
patch defines a tube.
• Such tubes may be curved.
The light field
• One might also consider the rays of light
passing thru a point.
• There are zero such rays.
• However, we can define something
called radiance.
• <see handbook>
The light field
• Radiant flux: the total energy emitted by
a source, or thru some area.
• Irradiance: the radiant flux per unit area
(radiant flux density). W/m2
• Radiance: the flux density per solid unit
angle. (W/m2)/sr, where sr = A/r2. This
is effectively the energy passing thru the
solid angle of one steradian.
The light field
• In these terms, the light field is simply
the radiance distribution.
• I.e., for any point, we can consider the
rays of light that impinges upon it from
all directions.
The light field
• Rays
– Origins: primary radiators
– End: black surfaces
• Air, smoke, fog, etc. may also absorb
ray energy but such effects are ignored
in this treatment.
Objects in the light field
• Photons are scattered at object surfaces.
• Probability of any photon reflecting is a
function of the incident angle and the viewing
(or exit) direction.
• The scattered radiance from a point on a
surface, to a particular viewing direction,
divided by the irradiance from a given
direction, of the surface is called the
bidirectional reflectance distribution function
(BRDF).
• BRDF depends on 4 angular parameters.
Objects in the light field
• BRDFs tend to be complex functions of
the 4 angles.
• Many psychophysical studies and
computer models ignore this fact.
• A perfectly white surface has a BRDF of
1/ (integral)
Objects in the light field
• Illumination of a matt sphere by
directional light
– Shading (attitude effect)
– Cast shadow
– Body shadow
• Disc appearance from the direction of
illumination: disc with dark edge
Objects in the light field
• Natural illumination environments
– Combination of directional light (solar) and
diffuse.
– Approximately uniform hemispherical
diffuse beam.
• Very little body shadow
• Ground side will be darker (vignetting)
– Ganzfield
• Whiteout
Photometric effects
• Levels of scale
– Whole scene
– Object
– Texture
• Incomplete description
• Some averaging
– Subtexture
• Material properties
• Incorporated into BRDF
Photometric effects
• Context
– Whole scene
• Object
– Texture
» Subtexture
• Examples
– Shape from shading affected by contours
– Shape from shading with illumination direction
information
• Scene cues
– Specularities
– Shadow directions
– Degree of diffusion in shadows
Photometric effects
• Texture and illumination cues
• Directional illumination
– Texture most apparent near body shadow
• Diffuse illumination
– Texture due to cracks and pits
– Darkness here not due to attitudinal
mechanisms but vignetting mechanisms.
Photometric effects
• Specularities
• Light may be scattered by underlying
substrates for some angles and reflected for
others.
• Disappear in diffuse light fields
• Directional illumination combined with surface
texture can produce arrays of specularities
– Example: shining ripples on a lake
– Orange vs. tomato
Photometric effects
• Inter-reflections
• Light bounces from one object onto
others.
• Ties scene together and indicates
relations between objects.
Photometric effects
• Deviations from Lambertian
– Backscatter
– Asperity scattering
– Translucence
Photometric effects
• Backscatter
– Light projected onto textures tends to be reflected
back more to the source.
– Why? Shadows are not visible to the source.
– Viewed from other angles, shadows decrease
average luminance.
– Attitudinal shading is decreased, making object
look flatter.
Photometric effects
• Asperity scattering
– Reflections from hair tips
– Common on plants and animals
– Furs: hairs are ~parallel with surface provide distinct
patterns of specularities.
– Asperity occurs when hairs stick straight up.
– More light is reflected to the viewer when there are more tips
per unit view angle (as near contours).
– Again: Attitudinal shading is decreased, making object look
flatter.
– Can create light edges.
– Together with Lambertian shading, explains
• The existence of odd order filters in V1
• The use of line drawings in art.
Photometric effects
• Translucence
– Most materials are translucent at the micro scale.
– Thus, BRDF concepts do not exactly apply.
– Light is not reflected from a point but enters at one point and
exits at a slightly different point.
– However, if we consider BRDF as reflection from small discs,
then micro translucence and microstructures within a surface
can explain a BRDF.
The structure of pictures
• The radiance distribution from any point determines
the set of possible picture that can be taken from that
point.
• However, cameras have their own properties.
–
–
–
–
Dynamic range
Resolution
MTF
Iimited field of view
• Computer screens, film and printers also have their
own limitations on dynamic range, resolution etc.
• Not mentioned: luminance response functions
Shape from shading
• Definition: invariant under a group of
transformations.
• Equivalently: what’s in the scene, rather than
what’s in the image.
• In this analysis, viewpoint and other
parameters are held constant.
• Illumination varies.
Shape from shading
•
•
•
•
Concave / convex ambiguities.
Illumination direction / shape confound
Other cues can help resolve these problems.
Only bumps share inter-reflections with the parent
plane.
• Only dents have internal inter-reflections.
• Many such details may be missing in computer
graphic implementations, affecting the associated
psychophysics.
Shape from shading
• Cues in collimated light fields
–
–
–
–
Body shadow
Textural quality of the body shadow edge
Structure of specularities
Structure of contour edge (serrated or not)
• Orange vs tomato ( same macro shape )
Shape from shading
• Cues in diffuse light fields
– Overall contour shape
– Textural effects due to vignetting
– Shading
• Due to vignetting not attitudinal
• Lambertian is not realistic
– Vignetting is not taken into account.
– Inter-reflections are not taken into account.
Shape from shading
• Shape from Lambertian shading
– Illumination is assumed directional and the same at all points in the
scene
– Light returned determines attitude
– Surface normals can be considered as unit vectors on the unit
sphere (Gauss maps)
• Gauss map examples
– Plane: point on sphere, degenerate
– Cylinders and cones: curves on sphere, degenerate
– All others: 2D patches on sphere, 1 to 1 locally
• Isophotes
– Lines of equal luminance
– Correspond to circles on the Gauss sphere
Shape from shading
• The nature of non 1 to 1 maps
– Sphere is multiply covered
– Think of a cloth with folds
– Folds give rise to critical points (min, max, saddle) in the
irradiance field
– Folds of the Gauss map correspond to inflections of the
surface, parabolic curves
– Such parabolic curves bound the convex, concave and
saddle regions of the surface
– The pattern of such parabolic curves can act as a description
of the shape.
Psychophysical results
• Little definitive
• Extreme stimulus reduction
– Studying images with only one type of cue (say shading)
– Doesn’t provoke strong impressions of shape
– Examples
• Silhouettes
• Line drawings
• Indication of shadows and lit areas
– The more cues available, the more subjects agree
– Is the real problem interaction?
• Lack of realism
– Published work with computer graphics using unknown or undescribed algorithms
Open problems
• “Majority of literature is irrelevant due to incomplete description
of the stimuli, extreme stimuli reduction, or invalid paradigms.”
• Stimuli must be complex / natural
• Control experiments by using complex / natural images and by
varying only a single cue.
• Some stimuli are not generic
– Ellipsoids have isophotes and shadow boundaries are planar
curves.
– No other shapes do.
– Degenerate Gauss maps
• Veridicality
– Observer’s response is due to two factors
• Idiosyncratic differences
• Cues
– These are often lumped together.
Open problems
• How to draw theories from computer vision models
for use in human research?
– This is difficult because a computer might return a surface
mesh.
– What is the human equivalent?
• What about shading in art?
– It differs from actual shading yet we understand it.
• Shape impressions of observers need to be recorded
rather than simple yes / no forced choices.