Transcript week14-models.ppt

Models and Matching
Methods of modeling objects and their
environments;
Methods of matching models to sensed
data for recogniton
Stockman CSE/MSU Fall 2008
1
Some methods to study








Mesh models (surface)
Vertex-edge-face models (surface)
Functional forms: superquadrics (surface)
Generalized cylinders (volume)
Voxel sets and octrees (volume)
View class models (image-based)
Recognition by appearance (image-based)
Functional models and the Theory of
affordances (object-oriented)
Stockman CSE/MSU Fall 2008
2
Models are what models do
Stockman CSE/MSU Fall 2008
3
What do models do?
Stockman CSE/MSU Fall 2008
4
Vertex-edge-face models
Polyhedra and extensions;
Model the surface of objects
Stockman CSE/MSU Fall 2008
5
Vertex-Edge-Face model
Stockman CSE/MSU Fall 2008
6
Sample object
All surfaces are planar
or cylindrical
Stockman CSE/MSU Fall 2008
7
Matching methods





Hypothesize point correspondences
Filter on distances
Compute 3D alignment of model to data
Verify positions of other model points,
edges, or faces. You can now do this!
LOTS of work in the literature on this!
Can work for many industrial objects
(and human faces perhaps!)
Stockman CSE/MSU Fall 2008
8
Triangular meshes
Very general and used by most CAD
systems.
Stockman CSE/MSU Fall 2008
9
Texture-mapped mesh dog
3D SURFACE MODEL
Courtesy of Kari Puli
SURFACE PLUS TEXTURE
With each triangle is a mapping of its vertices
into pixels [r, c] of a color image. Thus any
point of any triangle can be assigned a color
[R, G, B]. There may be several images
available to create these mappings.
Stockman CSE/MSU Fall 2008
10
Meshes are very general
They are usually verbose and often are too detailed for many
operations, but are often used in CAD. (Volumetric cube models are
actually displayed here: made from many views by Kari Pulli.)
Stockman CSE/MSU Fall 2008
11
Modeling the human body for
clothing industry and …
Multiple Structured light scanners used: could
this be a service industry such as Kinkos?
Actually cross sections
of a generalized
cylinder model.
Stockman CSE/MSU Fall 2008
12
Mesh characteristics
+ can be easy to generate from scanned data
Stockman CSE/MSU Fall 2008
13
Making mesh models
Stockman CSE/MSU Fall 2008
14
Marching cubes
http://www.exaflop.org/docs/marchcubes/ (James Sharman)
Raster scan through image
F(r, c).
Look for adjacent pixels, one
above threshold and one
below threshold.
Interpolate real coordinates
for f(x, y) = t in between
"Marching Cubes: A High Resolution 3D Surface Construction Algorithm",
William E. Lorensen and Harvey E. Cline,
Computer Graphics (Proceedings of SIGGRAPH '87), Vol. 21, No. 4, pp. 163-169.
Stockman CSE/MSU Fall 2008
15
Marching in 3D space F(s, r, c)
Some voxel
corners are
above
threshold t
and some
are below.
Stockman CSE/MSU Fall 2008
16
PhD work by Paul Albee 2004



Used Argonne National Labs scanner
High energy, high resolution planar
Xrays penetrate object rotating on a
turntable
Computer aided tomography
synthesizes a 3D volume of densities
with voxel size of about 5 microns
Stockman CSE/MSU Fall 2008
17
Segmentation of Scutigera a
tiny crablike organism
Slice j of material density
F( sj, r, c )
“thresholded” volume
Stockman CSE/MSU Fall 2008
18
Some common 3D problems







analyze blood vessel structure in head
capture structure and motion of vertebrae
of spine
analyze porosity and structure of soil
analyze structure of materials
automatic segmentation into regions
automatic correspondence of 3D points at
two instants of of time
3D volume visualization and virtual tours
Stockman CSE/MSU Fall 2008
19
Scanning technique abstraction
CCD camera (row)
material sample
X-ray planes
scintillator
Pin head
X-rays partly absorbed by sample;
excite scintillator producing one row
in the camera image; rotate sample
a few degrees and produce another
row; 3D reconstruction using CT
rotate
Stockman CSE/MSU Fall 2008
20
Scutigera: a tiny crustacean
• organism is smaller than 1 mm
• scanned at Argonne
• volume segmented and
meshed by Paul Albee
• roughly ten million triangles
to represent the surface
• anaglyph created for 3D
visualization
(view with stereo glasses)
Stockman CSE/MSU Fall 2008
21
Presentation of Results to User


Can explore the 3D data using rotation/translation
Can create stereo images from 3D data
Stockman CSE/MSU Fall 2008
22
Physics-based models
Can be used to make meshes;
Meshes retain perfect topology;
Can span spots of bad or no data
Stockman CSE/MSU Fall 2008
23
Physics-based modeling
Stockman CSE/MSU Fall 2008
24
Forces move points on the
model; halt at scanned data
Stockman CSE/MSU Fall 2008
25
Fitting an active contour to
image data
Stockman CSE/MSU Fall 2008
26
Balloon model for closed
object surface
Courtesy of Chen and Medioni
Stockman CSE/MSU Fall 2008
27
Balloon evolution
• balloon stops at
data points
• mesh forces
constrain neighbors
• large triangles split
into 4 triangles
• resulting mesh has
correct topology
• hard CS part is
detecting when
balloon should be
stopped by data
point
Stockman CSE/MSU Fall 2008
28
Physics-based models
Can also model dynamic behavior of solids (Finite Element Methods)
Stockman CSE/MSU Fall 2008
29
Tagged MRI: 3D interest
points can be written to body!
The MRI sensor tags
living tissue and can
sense its movement.
Motion of a 3D
tetrahedral finite
elements model can then
be analyzed. FMA model
attempts to model the
real physics of the heart.
Work by Jinah Park and
Dimitry Metaxes.
Stockman CSE/MSU Fall 2008
30
Algorithms from computer graphics
make mesh models from blobs



Marching squares applied to some
connected image region (blob)
Marching cubes applied to some
connected set of voxels (blob)
See a CG text for algorithms: see the
visualization toolkit for software
Stockman CSE/MSU Fall 2008
31
The octree for compression
Stockman CSE/MSU Fall 2008
32
Generalized cylinders
Stockman CSE/MSU Fall 2008
33
Generalized cylinders
• component parts have axis
• cross section function describes variation along axis
• good for articulated objects, such as animals, tools
• can be extracted from intensity images with difficulty
Stockman CSE/MSU Fall 2008
34
Extracting a model from a
segmented image region
Courtesy of Chen and Medioni
Stockman CSE/MSU Fall 2008
35
Interpreting frames from video



Can we match a frame region to a
model?
What about a sequence of frames?
Can we determine what actions the
body is doing?
Stockman CSE/MSU Fall 2008
36
Generalized cylinders
Stockman CSE/MSU Fall 2008
37
View class models
Objects modeled by the distinct
views that they can produce
Stockman CSE/MSU Fall 2008
38
“aspect model” of a cube
Stockman CSE/MSU Fall 2008
39
Recognition using an aspect model
Stockman CSE/MSU Fall 2008
40
View class model of chair
2D Graph-matching (as in Ch 11) used to evaluate match.
Stockman CSE/MSU Fall 2008
41
Side view classes of Ford Taurus
(Chen and Stockman)
These were made in the
PRIP Lab from a scale
model.
Viewpoints in between
can be generated from x
and y curvature stored on
boundary.
Viewpoints matched to
real image boundaries
via optimization.
Stockman CSE/MSU Fall 2008
42
Matching image edges to model limbs
Could recognize car model at stoplight or
gate or in car wash.
Stockman CSE/MSU Fall 2008
43
Appearance-based models
Using a basis of sub images;
Using PCA to compress bases;
Eigenfaces (see older .pdf slides 14C)
Stockman CSE/MSU Fall 2008
44
Function-based modeling
Object-oriented;
What parts does the object have;
What behaviors does it have;
What can be done with it?
(See plastic slides of Louise Starks’s work.)
Stockman CSE/MSU Fall 2008
45
Louise Stark: chair model


Dozens of CAD models of chairs
Program analyzes model for
* stable pose
* seat of right size
* height off ground right size
* no obstruction to body on seat
* program would accept a trash can
(which could also pass as a container)
Stockman CSE/MSU Fall 2008
46
Theory of affordances: J.J. Gibson




An object can be “sittable”: a large number of
chair types, a box of certain size, a trash can
turned over, …
An object can be “walkable”: the floor, ground,
thick ice, bridge, ...
An object can be a “container”: a cup, a hat, a
barrel, a box, …
An object can be “throwable”: a ball, a book, a
coin, an apple, a small chair, …
Stockman CSE/MSU Fall 2008
47
Minski’s theory of frames
(Schank’s theory of scripts)



Frames are learned expectations –
frame for a room, a car, a party, an
argument, …
Frame is evoked by current situation –
how? (hard)
Human “fills in” the details of the
current frame (easier)
Stockman CSE/MSU Fall 2008
48
Make a frame for my house






Item
Item
Item
Item
Item
Item
1
2
3
4
5
6
Stockman CSE/MSU Fall 2008
49