Transcript Final Presentation: Cascaded Classifier for Automatic Crater Detection

Cascaded Classifier for
Automatic Crater Detection
Henry Z. Lo
Advisor: Wei Ding
Domain Scientist: Tomasz Stepinski
Knowledge Discovery Lab
University of Massachusetts Boston
Overview
• Introduction:
o Cascading classifier.
o Experimental road map.
• Experiments:
o Tests on feature sets.
o Tests on positive example training set content.
o Tests on negative example training set size.
o Tests on negative example training set content.
• Discussion:
o Implications of results.
o Unresolved issues.
o Future directions.
Cascading Classifier
• Architecture:
o Layers of Adaboost classifiers.
o Each layer trained on the FP of previous layer.
o Input must be accepted by all, sequentially, to be
considered a crater.
o Rejection can happen at any stage.
Cascading Classifier
• Features:
o Exclusively uses Haar-like features.
o Can be calculated in constant time.
o Contrast based.
o Scanned over entire subwindow.
Cascading Classifier
• Implementation:
o Used OpenCV implementation.
o Free and open source.
o Many variables:
 Number of layers.
 "Minimum hit rate" - false positive rate.
 "Max false alarm" - false negative rate.
 3 feature sets.
Experimental Road Map
• Tweak for performance:
o OpenCV parameters.
o Features.
o Training set.
• The following OpenCV parameters improve performance:
o Minimum hit rate.
o Max false alarm.
o Number of layers.
• Still need to tweak features and training sets for:
o Training time.
o Generalizability.
• L
Experimental Road Map
Experimental Road Map
• Each of these factors will be tested individually for effect on
precision, recall, and F1.
• We avoid studying interaction effects for simplicity.
• In the future, we will investigate how to combine different
features and test sets for optimal result.
Experimental Road Map
• We use tile 3-24 for both training and testing.
• This tile was chosen for its relatively smooth surface.
• Future studies will test on other tiles as well.
Feature Set Variation
Feature Set Variation
• OpenCV offers 3 different feature sets:
o CORE:
1a, 1b, 2a, 2c.
o BASIC:
CORE + 2b, 2d, 3a
o ALL:
all features
• Since ALL is a superset of CORE and BASIC, it should
perform best.
Feature Set Variation
• In recall, CORE and BASIC
outperformed ALL.
• In precision and F1, the exact
opposite was true.
Haar Features
Haar Features
• Inclusion of tilted features beneficial to performance.
• More features than those given may provide further benefit.
• It is not obvious how to create Haar features in OpenCV.
• Postponing creation of specialized Haar features.
Ground Truth Windows
Ground Truth Windows
• Positive examples contained tightly cropped craters.
• No crater rims or surrounding area.
• Experimented with including area around craters.
•
• Range: 1x crater radius - 2x crater radius, in steps of .1.
1.0
1.2
1.4
1.6
1.8
2.0
Ground Truth Windows
• As the subwindow
increased, precision and
F1 increased.
• However, recall suffered.
Negative Example Set Size
Negative Example Set Size
• All classifiers tested were trained on 300 negative examples.
• By providing the classifier with more negative examples, we
give it more information.
• Performance should increase with more negative examples.
• Tested classifiers trained on 300, 400, 500, 600, and 700
negative examples.
Negative Example Set Size
• F1 and precision increase
with more negative
examples.
• Recall decreases.
Negative Example Manipulation
Negative Example Manipulation
• The idea is to put some false positives back into the training
set.
• This will teach the classifier using its own mistakes.
• However, selecting the false positives is rather difficult, as
we will see later.
Result Implications
• Window scaling has the most noticeable effect on F1, recall,
and precision.
• Next most important is the feature set used.
• The number of negative training examples is the least
important; however, this may be due to the small range of
values being tested.
Future Directions
• Once optimal features and training sets are found, we can
manipulate OpenCV variables.
• Recall that the classifier may be improved by the following:
o
More layers in the classifier.
o
Setting the minimum hit rate (recall).
o
Setting the max false alarm rate (precision).
•
•
• Time complexity of classifier training requires further study.
Future Directions
• Further exploration of cascaded classification algorithm:
o
Testing classifier on other tiles.
• Exploration of other object detection algorithms.
o
Neural networks.
Questions?