Transcript Vision Computing

Shape from Contour
Recover 3D information
Muller-Lyer illusion
Linear perspective - Ponzo illusion
Shape
1. Shape is a stable property of objects
2. The difficulty lies in finding a
representation of global shape that is
general enough to deal with the wide
variety of objects in the real world
3. Its most significant role is in object
recognition, where geometric shape along
with colour and texture provide the most
significant cue to enable us to identify
objects, classify what is in the image as
an example of some class one has seen
before
Shape from Contour
 After edge detection, an important question
in visual recovery is to deduce the 3-D
structure of scene from its line drawing.
 an inherent ambiguity exists because under
perspective projection the line drawing of
any scene event, for example, a depth
discontinuity, can restrict the location of the
event only to a narrow cone of rays (C1)
figure C1 - An infinite number of 3D drawings
can give rise to the same image
The goal of the shape-from-contour module is to
derive information about the orientation of the
various different faces.
A line drawing of a scene consisting of polyhedra. Shaded
surfaces are shadows
reference
 Winston, Patrick (1992) Artificial
Intelligence (3rd Edition), Reading
(Mass): Addison-Wesley Pub. Co
Chapter 12; pp249-272
Symbolic Constraints and
Propagation
 Ambiguities
– multi-interpretations for an individual
components of an input
– Enormous possible combinations of the
components
• Some of them may not occur
 The use of constraints
– To reduce the complexity
• Analyse the problem domain to determine what the
constraints are
• Apply a constraint satisfaction algorithm
Edges
 An Obscuring Edge – A boundary between
objects, or between object and the
background
 A Concave Edge - An Edge between two
faces that form an acute angle when viewed
from the outside of the object
 A Convex Edge – An edge between two
faces that form an obtuse angle when
viewed from outside the object.
The Scope of Lines
 At moment, not consider cracks between
coplanar faces and shadow edge between
shadows and the background
 But the approach is extensible to handle
these other types
 we consider only figures composed
exclusively of trihedral vertices, which are
vertices at which exactly three planes come
together. (figure S1)
 You need to know this assumption
Figure
S1 - trihedral
Some trihedral
figures
Some
figures
figure S1 - Some Nontrihedral figures
Determining the Constraints
 how to recognize individual objects in a
figure - our objective
 to label all the lines in the figure so that we
know which ones correspond to boundaries
between objects
 a set of four labels that can be attached to a
give line. (S2)
+ convex line
- concave line
> Boundary line with interiors to the right (down)
< Boundary line with interiors to the right (up)
Figure S2 - Line labelling conventions
Figure S2 - An example of line labelling
Four Trihedral Vertex Types
 the number of ways of labeling a figure composed of N
lines is 4N – how to find the correct one?
 the number of possible line labellings would be
reduced, if
– constrains on the kinds of vertices
– constrains on the lines
– every line must meet other lines at a vertex at each of its
ends.
 For the trihedral figures there are only four
configurations that describe all the possible vertices.
(Figure S3)
The four trihedral vertex types (S3)
Labels and Their Constraints
 the maximum number of ways that each
of the four types of lines might combine
with other lines at a vertex
 there are 208 ways to form a trihedral
vertex
 But, in fact, only a very small number of
these labelings can actually occur in line
drawings representing real physical
objects (S4)
A figure occupying one octant (S4)
S5
Illustration example
 Octants –
 Trihedral figure may differ in the number of octants that
they fill and in the position (which must be one of the
unfilled octants) from which they are viewed –
 Any vertex that can occur in a trihedral figure must
correspond to such a division of space with some
number (between one and eight) of octants filled, which
is viewed from one of the unfilled octants
 So to find all the vertex labelling that can occur, we need
only consider all the ways of filling the octants and each
of the ways of viewing those fillings, and then record the
types of the vertices that we find.
Possible Trihedral Vertex Labelings(S5)
 we get a complete list of the possible
trihedral vertices and their labellings
(figure S6)
 the 208 labellings that were theoretically
possible, only 18 are physically possible.
 a severe constraint on the way that lines
in drawings corresponding to real figures
can be labeled.
S6
Label set (18)
Label set (16)
W1
Waltz Procedure
1.
2.
3.
4.
5.
pick one vertex and find all the labellings that are
possible for it.
move to an adjacent vertex and find all of its possible
labellings. The line from the first vertex to the second
must end up with only one label, and that label must be
consistent with the two vertices it enters.
inconsistent ones can be eliminated.
Continue with another adjacent vertex. Constraints
arises from this labelling and these constraints can be
propagated back to vertices that have already been
labelled, so the set of possible labellings for them is
further reduced.
This process proceeds until all the vertices in the figure
have been labelled. (figure W1, W12)
W12
Convergent Intelligence
 Thus symbolic constraint propagation offers a
plausible explanation for one kind of human
information processing, as well as a good way for
computer to analyse drawings. This idea suggest
the following principle:
 The world manifests constraints and regularities.
If a computer is to exhibit intelligence, it must
exploit those constraints and regularities, no
matter of what the computer happens to be made.
W12
Waltz algorithm
 Please see the algorithm in the extra note (the
handout in class or download from the module
website)
 This algorithm will always find the unique,
correct figure labeling if one exists. If a figure is
ambiguous, however, the algorithm will
terminate with at least one vertex still having
more than one labeling attached to it.
 was applied to a larger class of figures in which
cracks and shadows might occur.
 the usefulness of the algorithm increases as the
size of the domain increases and thus the ratio
of physically possible to the theoretically
possible vertices decreases.
W10
W11
A Sample Example of the Labelling Process
Static Constraints and Dynamic
Constrains
 Static constrains:
– do not need to be represented explicitly as part of
a problem state.
– They can be encoded directly into the linelabeling algorithm.
 dynamic constrains
– describe the current options for the labeling of
each vertex.
– will be represented and manipulated explicitly by
the line-labeling algorithm.
Labelling and Reality
 Successful labeling is a necessary condition
for realizability as an object in trihedral
vertex world, but not in a world that allows
vertexes with more than three faces.
 Successful labeling is not a sufficient
condition for realizability as an object in a
three-faced vertex world (M C Escher, local
and global)
Extensibility
 Shadow areas can be of great use in
analyzing the scene that is being portrayed
 When these variations are considered,
there become more than eighteen
allowable vertex labelings
 But the ratio of physical allowable vertices
to theoretically possible ones becomes
even smaller than 18/208.
 Thus this approach can be extended to
larger domains
Many lines and junction labels are needed
to handle shadows and cracks
 Shadows help determine where an
object rests against others (Cr1)
 Concave edges often occur where two
or three objects meet. It is useful to
distinguish among the possibilities by
combining the minus label with the one
or two boundary labels that are seen
when the objects are separated (Cr2)
CR1
CR1
CR2
Illumination Increase Label
Count and Tightens Constraint
 There are now 11 ways that any
particular line may be labelled
 32=9 illumination combinations for each
of the 11 line, giving 99 total
possibilities. (Cr3)
 Only 50 of these combinations are
possible.
CR3
Summary
 Understand the major difficulties that
confront programs designed to perform
perceptual tasks
 describe the use of constraint satisfaction
procedure as one way of surmounting some
of those difficulties.
 perceptual abilities are essential in the the
construction of intelligent robots/systems