Transcript Boda: Towards a Framework for Evaluating Accuracy/Efficiency Tradeoffs in Object Detection Matt Moskewicz, Forrest Iandola, and Kurt Keutzer Berkeley Parlab {moskewcz, forresti, keutzer}@eecs.berkeley.edu Author Author Author Rethinking how.

Boda: Towards a Framework for Evaluating
Accuracy/Efficiency Tradeoffs in Object Detection
Matt Moskewicz, Forrest Iandola, and Kurt Keutzer
Berkeley Parlab
{moskewcz, forresti, keutzer}@eecs.berkeley.edu
Author
Author
Author
Rethinking how we evaluate computer vision systems
Introduction




Boda Framework
Real-world object detection systems require a
balance of efficiency and accuracy
Computer vision pipelines each consist of a
series of reusable building blocks
Today's object detection evaluation software
typically produces a precision-recall curve and
not much else
Boda Design and Goals
 Efficiently and transparently evaluate object detectors
 Native implementation in C++ for efficiency
 Preserve full, runnable computer vision pipelines
 Visualize intermediate steps in computer vision pipelines
 Proof-of-concept use cases:
Boda will efficiently compute a variety of metrics
on individual stages in object detection pipelines



Deformable parts models
PASCAL-style object detection evaluations
Input Image
Find interesting image regions [4]
Extract Deep Convnet feature descriptors [5]
SVM classifier
Timers /
Profiling
Designing Object Detectors

Example Object
Detection Pipeline
CLI
OTHER_MODE
Today's object detectors have little correlation
between speed and accuracy
struct MODE {
uint32_t param1;
main( void ) { … } };
Tests (XML)
Boda will explore/evaluate new designs that
balance speed and accuracy
primary output (e.g.
mAP / Dets)
Regression Diffs
"Car," "Boat," etc.
Evaluating Object Detectors
 Current software [3] for evaluating
object detectors is inefficient and gives
limited feedback
 It's time to rethink the system for
evaluating object detectors
Object Detection Building Blocks
[2]
Decision Trees
Cascaded
Deformable Parts Models (DPMs) [1]
Deep Convolutional
Neural Networks [5]
References
1. P. Felzenszwalb, R. Girshick, D. McAllester, D. Ramanan. Object Detection with
Discriminatively Trained Part Based Models. PAMI, 2010.
2. P. Dollár, C. Wojek, B. Schiele and P. Perona. Pedestrian Detection: An Evaluation of the
State of the Art. PAMI, 2012.
3. M. Everingham, et al. The PASCAL Visual Object Classes (VOC) Challenge, IJCV, 2010.
4. J.R.R. Uijlings et al. Selective Search for Object Recognition. IJCV, 2013.
5. R. Girshick, J. Donahue, T. Darrell, and J. Malik. Rich feature hierarchies for accurate
object detection and semantic segmentation. ArXiv Tech Report, 2013.
6. P. Felzenszwalb, R. Girshick, D. McAllester. Cascade Object Detection with Deformable
Part Models. CVPR, 2010.
Quantized Convolution
Selective Search [4]
Histograms of Oriented Gradients