Transcript Interactive Visual Pattern Recognition System

Computer Assisted Visual
InterActive Recognition
(CAVIAR)
Jie Zou
RPI ECSE DocLab
Advisor:
Committee:
April 16, 2004
Prof. George Nagy
Prof. Qiang Ji
Prof. Robert B. Kelley
Prof. Mukkai Krishnamoorthy
Jie Zou, Doclab ECSE, RPI
1
Agenda
Introduction
Related research
CAVIAR methodology
Interactive segmentation
CAVIAR – flower recognition system
CAVIAR – face recognition system
Conclusions
April 16, 2004
Jie Zou, Doclab ECSE, RPI
2
Agenda
Introduction
Related research
CAVIAR methodology
Interactive segmentation
CAVIAR – flower recognition system
CAVIAR – face recognition system
Conclusions
April 16, 2004
Jie Zou, Doclab ECSE, RPI
3
Motivation
All operational systems require human
assistance (preprocessing, handling
rejects).
CAVIAR makes parsimonious use of
human visual talent throughout the
process rather than only at the
beginning or the end.
April 16, 2004
Jie Zou, Doclab ECSE, RPI
4
Scope of CAVIAR
Visual pattern recognition only
Each CAVIAR system addresses a
specific domain
Many class classification
Low throughput
April 16, 2004
Jie Zou, Doclab ECSE, RPI
5
Research Goals
Allocation of human and machine
responsibilities
Mathematical model
Framework and design principles
Prototype CAVIAR systems
April 16, 2004
Jie Zou, Doclab ECSE, RPI
6
Agenda
Introduction
Related research
CAVIAR methodology
Interactive segmentation
CAVIAR – flower recognition system
CAVIAR – face recognition system
Conclusions
April 16, 2004
Jie Zou, Doclab ECSE, RPI
7
Content-Based Image Retrieval
Typical search of CBIR system




Submit a query image.
Specify the relative importance of the
features.
Relevance feedback (label the retrieved
images as acceptable or not-acceptable).
Iterates until user finds the desired image.
April 16, 2004
Jie Zou, Doclab ECSE, RPI
8
CBIR vs. CAVIAR
CBIR
CAVIAR
Subjective retrieval
Objective classification
User judges retrieval results
Statistical decision boundary
User weights features
Machine weights features
Narrow domain
Broad domain
Relevance feedback
Relevance feedback
Model adjustment
April 16, 2004
Jie Zou, Doclab ECSE, RPI
9
Flower Recognition
Little research on automatic flower
recognition

M. Das, R. Manmatha, and E.M. Riseman,
“Indexing flower patent images using
domain knowledge," IEEE Intelligent
Systems, vol. 14, no. 5, pp. 24-33, 1999.
April 16, 2004
Jie Zou, Doclab ECSE, RPI
10
Face Recognition
Started in 1960’s. Now, most active
pattern recognition application
Eigenface, dominant method
Geometrical feature models are
appropriate for interactive recognition
April 16, 2004
Jie Zou, Doclab ECSE, RPI
11
Agenda
Introduction
Related research
CAVIAR methodology
Interactive segmentation
CAVIAR – flower recognition system
CAVIAR – face recognition system
Conclusions
April 16, 2004
Jie Zou, Doclab ECSE, RPI
12
Psychophysics
Attneave (1954): “the nature of
redundancy in visual stimulations”, and
“information is concentrated along
contours.”
Miller (1956): plus or minus 7
April 16, 2004
Jie Zou, Doclab ECSE, RPI
13
Allocation of Human and
Machine Responsibilities
Conventional System
April 16, 2004
Jie Zou, Doclab ECSE, RPI
CAVIAR
14
Notation
CAVIAR state
Model parameters
Features
Index vector
Training set
Label
April 16, 2004
Jie Zou, Doclab ECSE, RPI
15
Formal Description (1)
Finite state machine
Initial state

Model building

Feature extraction

Indexing
April 16, 2004
Jie Zou, Doclab ECSE, RPI
created by:
16
Formal Description (2)
Model manipulation leads to a state
transition from state n to state n+1:

Model building

Feature extraction

Indexing
April 16, 2004
Jie Zou, Doclab ECSE, RPI
,
17
Formal Description (3)
The task can terminate at any state by
identification.
April 16, 2004
Jie Zou, Doclab ECSE, RPI
18
Illustration (Video)
April 16, 2004
Jie Zou, Doclab ECSE, RPI
19
Agenda
Introduction
Related research
CAVIAR methodology
Interactive segmentation
CAVIAR – flower recognition system
CAVIAR – face recognition system
Conclusions
April 16, 2004
Jie Zou, Doclab ECSE, RPI
20
Notation
Parametric boundary
Exact boundary
Foreground region
Background region
Radius vector
intersects
at
, and
at
.
April 16, 2004
Jie Zou, Doclab ECSE, RPI
or
or
21
Training – Color Distributions
April 16, 2004
Jie Zou, Doclab ECSE, RPI
22
Training – Circle Parameter
Distributions
April 16, 2004
Jie Zou, Doclab ECSE, RPI
23
Training – Deviation of Circular
Model From Exact Boundary
ß=5.52
April 16, 2004
Jie Zou, Doclab ECSE, RPI
24
Automatic Segmentation –
Circle Partition
Use a circle to isolate a region,
which contains mostly flower colors.
April 16, 2004
Jie Zou, Doclab ECSE, RPI
25
Automatic Segmentation –
Generate Boundary Likelihood Map
Distance to the circle
Magnitude of color gradient
Boundary pixels should be close to the circle, and have
high color gradient.
April 16, 2004
Jie Zou, Doclab ECSE, RPI
26
Automatic Segmentation –
Deformation on BLM
Circle center = a foreground seed, and
four corner pixels = background seeds
Foreground and background regions
compete with each other to expand.
Eventually, converge to the watershed
of the seed pixels on the BLM.
April 16, 2004
Jie Zou, Doclab ECSE, RPI
27
Advantage of BLM over Gradient
Map
April 16, 2004
Jie Zou, Doclab ECSE, RPI
28
Examples of the Result of
Automatic Segmentation
April 16, 2004
Jie Zou, Doclab ECSE, RPI
29
Interactive Correction (Video)
April 16, 2004
Jie Zou, Doclab ECSE, RPI
30
Segment Flower Pictures with
Interactive Correction
1078 flowers from 113 species
Borjan Gagoski and Adam Callahan
5.7 seed pixels, 15.2 seconds per picture
Greenie Cheng
Histogram of the number of manual seeds
Histogram of time
30.0%
20.0%
18.0%
25.0%
16.0%
20.0%
14.0%
15.0%
12.0%
10.0%
8.0%
10.0%
6.0%
4.0%
5.0%
2.0%
April 16, 2004
Jie Zou, Doclab ECSE, RPI
78
72
66
60
54
48
42
36
30
24
18
12
6
0
39
36
33
30
27
24
21
18
15
12
9
6
0.0%
3
0
0.0%
31
Agenda
Motivation
Related research
Interactive segmentation
CAVIAR methodology
CAVIAR – flower recognition system
CAVIAR – face recognition system
Conclusions
April 16, 2004
Jie Zou, Doclab ECSE, RPI
32
Flower Database
320 by 240 resolution
Highly variable illumination, and
complex background
216 samples from 29 classes for
development
612 samples from 102 classes for
evaluation
April 16, 2004
Jie Zou, Doclab ECSE, RPI
33
Rose Curve Model
Parametric curve with
six parameters.
Flowers are composed
of petals, which have
circular symmetry.
When n=0, rose curve
reduces to circle.
April 16, 2004
Jie Zou, Doclab ECSE, RPI
34
Classification Features
number of petals.
the ratio of outer to inner radius.
first three order moments of the hue and
saturation histograms
April 16, 2004
Jie Zou, Doclab ECSE, RPI
35
CAVIAR-Flower GUI
Adjust inner circle radius by
dragging this control point.
Move the rose
curve by
dragging this
control point.
Change the petal number
using this ComboBox.
Adjust outer circle radius by
dragging this control point.
April 16, 2004
Jie Zou, Doclab ECSE, RPI
36
CAVIAR-Flower (Video)
April 16, 2004
Jie Zou, Doclab ECSE, RPI
37
Evaluation
CAVIAR compared to human-alone and
machine-alone
Machine learning

Decision directed approximation
Finite state machine calibration
April 16, 2004
Jie Zou, Doclab ECSE, RPI
38
Experimental Protocol
Thanks to Borjan Gagoski
April 16, 2004
Jie Zou, Doclab ECSE, RPI
39
CAVIAR Compared to HumanAlone and Machine-Alone
Accuracy vs Time
100.00%
Accuracy
80.00%
60.00%
40.00%
20.00%
0.00%
0.00
5.00
10.00 15.00 20.00 25.00 30.00 35.00 40.00
Time
Human-alone
CAVIAR
Machine-alone
Significantly reduce the recognition time
compared to human-alone
Significantly increase the accuracy compared
to machine-alone
April 16, 2004
Jie Zou, Doclab ECSE, RPI
40
Accuracy of Initial Automatic
Recognition
Accuracy After Automatic Recognition
60%
50%
40%
30%
20%
10%
0%
T2
T3
Max
T4
Min
T5
Median
T2 – 5 labeled
T3 – 1 labeled
T4 – 1 labeled + 2 pseudo
T5 – 1 labeled + 4 pseudo
April 16, 2004
Jie Zou, Doclab ECSE, RPI
41
Rank Order after Initial
Automatic Recognition
Rank Order After Automatic Recognition
16.0
14.0
12.0
10.0
8.0
6.0
4.0
2.0
0.0
T2
T3
Max
T4
Min
T5
Median
T2 – 5 labeled
T3 – 1 labeled
T4 – 1 labeled + 2 pseudo
T5 – 1 labeled + 4 pseudo
April 16, 2004
Jie Zou, Doclab ECSE, RPI
42
Time of Interactive
Recognition
Time
30.0
25.0
20.0
15.0
10.0
5.0
0.0
T2
T3
Max
T4
Min
T5
Median
T2 – 5 labeled
T3 – 1 labeled
T4 – 1 labeled + 2 pseudo
T5 – 1 labeled + 4 pseudo
April 16, 2004
Jie Zou, Doclab ECSE, RPI
43
Accuracy of Interactive
Recognition
Accuracy
100%
95%
90%
85%
80%
75%
T2
T3
Max
T4
Min
T5
Median
T2 – 5 labeled
T3 – 1 labeled
T4 – 1 labeled + 2 pseudo
T5 – 1 labeled + 4 pseudo
April 16, 2004
Jie Zou, Doclab ECSE, RPI
44
Observations about Machine
Learning
Initialized with a single training samples per
class.
Self-learning: user classified pseudo-labeled
samples improve the performance.
Performance of T5 is close to T2, suggesting
that instead of initializing with many training
samples, we can trust the system’s self
learning.
April 16, 2004
Jie Zou, Doclab ECSE, RPI
45
Calibration of Finite State
Machine
60.0%
50.0%
40.0%
30.0%
20.0%
10.0%
0.0%
0
1
2
3
Original
4
5
6
7
8
9
10
Geometrical Distribution with P=.549
52% samples are immediately confirmed.
90% samples are identified by 3 adjustments.
The probability of success on each
adjustment is just over one half.
April 16, 2004
Jie Zou, Doclab ECSE, RPI
46
Summary of CAVIAR-Flower
Parameterized rose curve to model the
flowers.
Display the rose curve and let user
adjust it if necessary.
The evaluation of the system shows
advantages of CAVIAR system.
April 16, 2004
Jie Zou, Doclab ECSE, RPI
47
Agenda
Motivation
Related research
Interactive segmentation
CAVIAR methodology
CAVIAR – flower recognition system
CAVIAR – face recognition system
Conclusions
April 16, 2004
Jie Zou, Doclab ECSE, RPI
48
CAVIAR - Face
Not to implement a state-of-the-art face
recognition system.
To demonstrate the wider applicability
of CAVIAR methodology.
400 FERET pictures of 200 subjects
(“ba” and “bk” series)
April 16, 2004
Jie Zou, Doclab ECSE, RPI
49
Face model and features
Face model contains
only two pupils.
An automatic facial
feature detection
program locates the
other 26 points.
Thanks to Yan Tong,
Zhiwei Zhu, and
Dr. Qiang Ji.
April 16, 2004
Jie Zou, Doclab ECSE, RPI
50
Face model and features
Similarity transform
all 28 points to
place two pupils at
(-1,0) and (1,0).
The normalized
coordinates of the
26 points are the
features for
classification.
April 16, 2004
Jie Zou, Doclab ECSE, RPI
51
Interactive Recognition
April 16, 2004
Jie Zou, Doclab ECSE, RPI
52
Interactive Recognition
April 16, 2004
Jie Zou, Doclab ECSE, RPI
53
Experiments
Three types of experiments: human
browsing only, BAAR, and CAVIAR
200 training samples, one from each
person
200 test samples, one from each person
15 subjects, each classifies 40 faces
April 16, 2004
Jie Zou, Doclab ECSE, RPI
54
CAVIAR Compared to HumanAlone, Machine-Alone, and BAAR
Accuracy vs. Time
100%
Accuracy
80%
60%
40%
20%
0%
0.0
10.0
20.0
30.0
40.0
50.0
60.0
70.0
80.0
Time (s)
Human Only
April 16, 2004
Machine Only
Jie Zou, Doclab ECSE, RPI
BAAR
CAVIAR
55
Summary of CAVIAR-Face
A simple face recognition system
Still clearly shows the advantages of
CAVIAR approach
April 16, 2004
Jie Zou, Doclab ECSE, RPI
56
Agenda
Motivation
Related research
Interactive segmentation
CAVIAR methodology
CAVIAR – flower recognition system
CAVIAR – face recognition system
Conclusions
April 16, 2004
Jie Zou, Doclab ECSE, RPI
57
Contributions
Domain specific geometrical models to
mediate between human and computer
Allocation of human and machine in
interactive visual recognition task



Model building (primarily human)
Feature extraction (primarily machine)
Classification (collaborative)
Finite state machine
Two experimental CAVIAR systems
April 16, 2004
Jie Zou, Doclab ECSE, RPI
58
CAVIAR Design Principles
Machine tries its best to infer the model
and the candidates.
Machine should display its results, and
let the user correct any errors.
Machine always accepts human
correction.
April 16, 2004
Jie Zou, Doclab ECSE, RPI
59
Observations
CAVIAR system can significantly reduce the time
compared to unaided human, and significantly
increase the accuracy compared to unaided machine
(without years of R&D).
CAVIAR system can be initialized with a single
training sample per class.
CAVIAR system demonstrates self-learning ability.
CAVIAR is better than BAAR, and provides more
opportunities for machine learning from human
intervention.
April 16, 2004
Jie Zou, Doclab ECSE, RPI
60
Directions For Future Work
Other applications
Mobile CAVIAR
Collaborative learning
Machine suggestion to human
April 16, 2004
Jie Zou, Doclab ECSE, RPI
61
Other applications – Face
Face under head rotation,
occlusion, and changes of
illumination and facial
expression
Elastic bunch graph
model, and others
April 16, 2004
Jie Zou, Doclab ECSE, RPI
62
Other applications – Skin
Diseases
Nearly 1000
diagnoses
Big image atlases
available


John Hopkins
dermatology image
atlas
University of Erlangen
April 16, 2004
Jie Zou, Doclab ECSE, RPI
63
Other applications – Fish
Black Crappie
Atlantic Sturgeon
Alabama Shad
Blue Gill
U.S. Fish & wild life service
April 16, 2004
Jie Zou, Doclab ECSE, RPI
64
Other Applications
Only a few successful visual recognition
applications: OCR, fingerprint, ??.
CAVIAR may make others viable.
Education
April 16, 2004
Jie Zou, Doclab ECSE, RPI
65
Mobile
CAVIAR
Courtesy to Abhishek Gattani
April 16, 2004
Jie Zou, Doclab ECSE, RPI
66
Mobile CAVIAR
Client-server architecture, with a handheld computer as a client, connecting to
an Internet server.
The easy accessibility of multiple
pictures poses an interesting
information fusion problem.
April 16, 2004
Jie Zou, Doclab ECSE, RPI
67
Collaborative Learning
When using CAVIAR,
also collect training
samples
Pattern identified by
some users can benefit
peer users
Extension of the
concept of Open Mind
Initiative
April 16, 2004
Jie Zou, Doclab ECSE, RPI
68
Machine Suggestion to Human
Currently, machine suggestions only
annoy users.
Eventually may be able to make useful
suggestions to acquire or refine
particular features.
April 16, 2004
Jie Zou, Doclab ECSE, RPI
69
Thank you!
Questions and Comments
April 16, 2004
Jie Zou, Doclab ECSE, RPI
70
Motivation
Successful visual pattern recognition systems
require several years R&D.
All operational systems require human
assistance (preprocessing, handling rejects).
Pronounced difference between human and
machine.


Human: gestalt tasks, object-background
separation, etc.
Machine: computing complex features, evaluating
conditional probabilities, etc.
April 16, 2004
Jie Zou, Doclab ECSE, RPI
71
CAVIAR Introduction
It may be more effective to make parsimonious use of human visual
talent throughout the process rather than at beginning or end.
April 16, 2004
Jie Zou, Doclab ECSE, RPI
72
Human and Machine Visual
Perception
Visual perception is a kind of
computation.
Two theories of human visual
perception.


Recognition by components (RBC).
View-based recognition.
Machine visual perception is still in its
infancy.
April 16, 2004
Jie Zou, Doclab ECSE, RPI
73
Human Computer Interaction
HCI is a wide topic.



Graphic user interface
Attentive user interface
Exploratory data analysis
 Graphically display the data.
 For designing a classifier, rather than actual classification.

Pattern recognition systems with human in the
loop
 Preprocessing
 Handling rejects
April 16, 2004
Jie Zou, Doclab ECSE, RPI
74
CAVIAR vs. CBIR
CAVIAR is objective classification, CBIR is
subjective retrieval.
Users arbitrarily emphasizing particular
features is not a good idea in CAVIAR.
CAVIAR is on narrow domain, CBIR is usually
on broad domain.
Relevance feedback is useful in CAVIAR, but
more precise human-computer
communication can be established through
domain-specific models.
April 16, 2004
Jie Zou, Doclab ECSE, RPI
75
Unsupervised Decision-directed
Approximation
Assume that the decision of the classifier is
the true label of the unknown.
Works well if:


Little overlap among the class-conditional
densities.
Initial classifier is reasonably good.
Used in our experiment.

Benefits from human intervention.
April 16, 2004
Jie Zou, Doclab ECSE, RPI
76
Allocation of Human and
Machine Responsibilities
April 16, 2004
Jie Zou, Doclab ECSE, RPI
77
Allocation Human and
Machine Responsibilities
We give up strong segmentation for
parametric model fitting.
Model building is primarily a human
responsibility.
Feature extraction is done primarily by
machine.
Classification requires human-machine
collaboration.


Human adjusts the model instance.
Final human confirmation.
April 16, 2004
Jie Zou, Doclab ECSE, RPI
78
CAVIAR Compared to Machinealone and Human-alone (1)
Cost function is a convex combination
of error, E, and time, T.
Machine-alone: E=EM, T=0.
Human-alone: E=0, T=TH.
CAVIAR: Ei, Ti.
April 16, 2004
Jie Zou, Doclab ECSE, RPI
79
CAVIAR Compared to Machinealone and Human-alone (2)
April 16, 2004
Jie Zou, Doclab ECSE, RPI
80
CAVIAR Compared to BrowsingAfter-Automatic (BAA)
The user browses to find the correct
candidate.
April 16, 2004
Jie Zou, Doclab ECSE, RPI
81
CAVIAR Compared to BrowsingAfter-Automatic (BAA)
In principle, CAVIAR can never be
worse than BAA.
The practical scenarios are much more
complicated, and very difficult to model.
CAVIAR provides more opportunity for
machine learning from human
intervention.
April 16, 2004
Jie Zou, Doclab ECSE, RPI
82
Automatic Image Segmentation
Intensively studied for decades, no “off-theshelf” solution.
Segmentation methods includes:




Clustering.
Edge-linking.
Region splitting and merging.
Hybrid optimization.
General image segmentation is extremely
difficult.
April 16, 2004
Jie Zou, Doclab ECSE, RPI
83
Automatic Segmentation –
Circle Partition
α=0.025.
Use a circle to isolate a region, which contains mostly flower colors.
April 16, 2004
Jie Zou, Doclab ECSE, RPI
84
Interactive Image Segmentation
From human initialization, deform
automatically to the object boundary.


Snakes or active contours.
Seeded region growing.
Human and computer collaborate step
by step.


Intelligent scissors.
Intelligent paint.
April 16, 2004
Jie Zou, Doclab ECSE, RPI
85
Image Segmentation Evaluation
Analytical methods.

Analyze the principles and properties.
Empirical methods.

Compare the results to the ground-truth
segmentation.
We believe that segmentation can only
be evaluated according to its purpose.
April 16, 2004
Jie Zou, Doclab ECSE, RPI
86
Strong Segmentation Vs.
Weak Segmentation
Strong segmentation

Division of the image into regions; region F
contains the pixels of object O, and nothing else.
F=O.
Weak segmentation

Partition of the image into conspicuous object
region F without locating the precise boundary.
F≈O.
Strong segmentation may not be necessary
for pattern recognition.
April 16, 2004
Jie Zou, Doclab ECSE, RPI
87
Outline of the Interactive
Segmentation Procedure
Training



Color distributions.
Circle parameter distributions.
Statistics of the deviation of the exact boundary
from the circle.
Automatic segmentation



Fit circle by maximizing posterior.
Generate boundary likelihood map.
Watershed of seeds is the boundary.
Interactive correction

Shriek or expand the foreground region by
introducing more seeds.
April 16, 2004
Jie Zou, Doclab ECSE, RPI
88
Interactive Correction
April 16, 2004
Jie Zou, Doclab ECSE, RPI
89
Evaluation of Automatic
Segmentation
174 test samples, 187 training samples
from 29 classes.
Region discrepancy (RD): 25%.
Boundary discrepancy (BD): 8.7 pixels.
Classification rank order (RO): 3.84.
April 16, 2004
Jie Zou, Doclab ECSE, RPI
90
Rank Order is Not Correlated with
RD and BD
Scatter plot of BD and RD
1.2
1
RD
0.8
0.6
0.4
0.2
0
0
10
20
30
40
50
BD
Scatter plot of RD and RO
12
12
10
10
8
8
RO
RO
Scatter plot of BD and RO
6
6
4
4
2
2
0
0
0
10
20
30
40
50
0
0.4
0.6
0.8
1
1.2
RD
BD
April 16, 2004
0.2
Jie Zou, Doclab ECSE, RPI
91
Experiments Comparing Strong
and Weak Segmentation
We compared classification results of
strong segmentation and two (rose
curve and circle) weak segmentations.
Test 612 unknown samples, with 510
training samples.
April 16, 2004
Jie Zou, Doclab ECSE, RPI
92
Automatic Segmentation is Not
Reliable.
The classification result of each kind of manual
segmentation is much better than that of the
corresponding automatic segmentation.
April 16, 2004
Jie Zou, Doclab ECSE, RPI
93
With Reliable Segmentation
The improvement in classification from strong
segmentation is limited even if reliable segmentation
can be achieved.
April 16, 2004
Jie Zou, Doclab ECSE, RPI
94
When Reliable Segmentation cannot be
Achieved
The simpler weak segmentation is preferred if
reliable segmentation can not be achieved.
April 16, 2004
Jie Zou, Doclab ECSE, RPI
95
Strong Segmentation may not be
necessary for Pattern Recognition
Strong segmentation is generally
difficult to achieve either automatically
or interactively.
Strong segmentation is obviously overfitted to a particular sample, not the
class that the sample belongs to.
Our experiments empirically show this.
April 16, 2004
Jie Zou, Doclab ECSE, RPI
96
Summary of Interactive Image
Segmentation
A procedure for model-based interactive
segmentation.
The segmentation results approaches the
“exact” boundary with little human efforts.
Extensions.



For highly textured image.
Adjusting circle after human clicking.
More elaborate models for both parametric
partition and boundary likelihood map.
April 16, 2004
Jie Zou, Doclab ECSE, RPI
97
Analytical Solution to Fitting a
Rose Curve
April 16, 2004
Jie Zou, Doclab ECSE, RPI
98
Training
April 16, 2004
Jie Zou, Doclab ECSE, RPI
99
Recognition
April 16, 2004
Jie Zou, Doclab ECSE, RPI
100
Accuracy of Initial Automatic
Recognition
April 16, 2004
Jie Zou, Doclab ECSE, RPI
101
Rank Order after Initial
Automatic Recognition
April 16, 2004
Jie Zou, Doclab ECSE, RPI
102
Top-3 Accuracy of Initial
Automatic Recognition
April 16, 2004
Jie Zou, Doclab ECSE, RPI
103
Time of Interactive
Recognition
April 16, 2004
Jie Zou, Doclab ECSE, RPI
104
Accuracy of Interactive
Recognition
April 16, 2004
Jie Zou, Doclab ECSE, RPI
105
Rank Order After Automatic
and Interactive Recognition
Median Rank Order after automatic and
interactive recognition
14
12
10
8
6
4
2
0
II
III
IV
Auto
V
Final
Subjects do adjust the rose curves,
which reduces the median rank orders.
April 16, 2004
Jie Zou, Doclab ECSE, RPI
106
CAVIAR Compared to BAA
Accuracy vs Time
100.00%
Accuracy
95.00%
90.00%
85.00%
80.00%
0.00
5.00
10.00
15.00
20.00
25.00
Time
CAVIAR
BAA
Initial automatic classification is good. The
average rank order is 6.6 (2 clicks).
The subjects are first-time user, not able to
always find the right strategy.
April 16, 2004
Jie Zou, Doclab ECSE, RPI
107
Training
Manually entered the positions of 28
points for each of 200 training pictures.
The difficulty of face recognition with a geometrical face
model is due to unreliable automatic feature point detection.
Human correction of eye centers is already very helpful.
April 16, 2004
Jie Zou, Doclab ECSE, RPI
108
New Human-Computer
Interfaces
Quality of model parameters.



Associate confidence measures to model
parameters, which relates to classification
features.
Assume the parameters adjusted by human have
high confidence.
Occlusion can be communicated to the computer.
Categorical features.


Divide a family into clusters. (male/female).
Category-specific classifiers.
April 16, 2004
Jie Zou, Doclab ECSE, RPI
109