Transcript Super-Patches Image Tessellation with Arbitrarily Shaped Patches Eddie K. H. Ng, University of Toronto Joint work with Brendan Frey.

Super-Patches
Image Tessellation with Arbitrarily
Shaped Patches
Eddie K. H. Ng, University of Toronto
Joint work with Brendan Frey.
Motivations



Patch-based models have been successfully
applied to many different areas of computer
vision.
Patch size / shape selection remains a skillful
labor and confined to simple geometric shapes.
Causes problems in part-based object
recognition.
Eddie K. H. Ng
CIAR Summer School on Learning and Vision
Aug 15 to 19, 2006
Goals


Learn a set of coherent patches of arbitrary
shapes and sizes.
Use the resulting patches to enhance
performance of various tasks in computer
vision.
Eddie K. H. Ng
CIAR Summer School on Learning and Vision
Aug 15 to 19, 2006
Inspiration

Super-pixels [Ren
& Malik, 2003]


Create a local,
coherent entity while
maintaining overall
structure
Reduce computational
complexity.
X. Ren and J. Malik. Learning a classification model for segmentation. In Proc. 9th Int. Conf. Computer Vision, volume 1,
pages 10-17, 2003.
* Image taken from http://www.cs.sfu.ca/~mori/research/superpixels/.
Eddie K. H. Ng
CIAR Summer School on Learning and Vision
Aug 15 to 19, 2006
Basic Concepts [1]


The shape and size of each patch is specified
through a kernel function (e.g. Gaussian (m, s)).
The set of kernel functions compete for the
ownership each pixel.
Initial Conditions
Input Image
Eddie K. H. Ng
Patches after Competition
Input Image
CIAR Summer School on Learning and Vision
Aug 15 to 19, 2006
Basic Concepts [2]


The tessellation is then a map showing the
respective ownerships.
Free parameter - number of patches *
Patches after Competition
Input Image
Eddie K. H. Ng
Tessellated Image
CIAR Summer School on Learning and Vision
Aug 15 to 19, 2006
Basic Concepts [4]


But how does the patches compete with each
other?
Each patch is mapped to a region in the Epitome.
Prior knowledge of Image Classes
(epitome)
Input Image
Eddie K. H. Ng
Tessellated Image
CIAR Summer School on Learning and Vision
Aug 15 to 19, 2006
Basic Concepts [5]

Cost Function:
C
R ,G , B

l
2
N pixels
N patches


 ~



~
   I l xi    ki  l Tk  xi  
k

 
 i 

e zki
 ki 
z ji
j e

1 ~ ~ T 1 ~ ~
zki    xi  m k   k  xi  m k 
2
Optimize cost function using conjugate gradient
Eddie K. H. Ng
CIAR Summer School on Learning and Vision
Aug 15 to 19, 2006
Run Demo
Other Explored Paths

Variations on the cost function:
Garbage Model
 Isotropic Patches
 Shape Regularization





Grow-and-Merge
Different formulation of the problem
Other kernel functions
Other learning algorithms
Eddie K. H. Ng
CIAR Summer School on Learning and Vision
Aug 15 to 19, 2006
References

Epitome


Low-level vision


W.T. Freeman, E.C. Pasztor, O. T. Carmichael, Learning Low-level vision, International Journal of
Computer Vision 40(1), 25-47, 2000
Super-pixels


N. Jojic, B. J. Frey, A. Kannan, Epitomic analysis of appearance and shape , ICCV 2003.
X. Ren and J. Malik. Learning a classification model for segmentation. In Proc. 9th Int. Conf. Computer
Vision, volume 1, pages 10-17, 2003.
Web resources


www.psi.utoronto.ca
http://research.microsoft.com/~jojic/epitome.htm
Eddie K. H. Ng
CIAR Summer School on Learning and Vision
Aug 15 to 19, 2006