Transcript Local SVM Classification Based on Triangulation

CS558 Project
Local SVM Classification based
on triangulation
(on the plane)
Glenn Fung
Outline of Talk
 Classification problem on the plane
 All of the recommended stages were applied:
 Sampling
 Ordering:
 Clustering
 Triangulation
 Interpolation (Classification)
SVM: Support vector Machines
 Optimization: Number of training points increased
 Evaluation:
Checkerboard dataset
Spiral dataset
Classification Problem in R 2
 Given m points in 2 dimensional space
 Represented by an m-by-2 matrix A
 Membership of each A i in class +1 or –1
SAMPLING:
1000 randomly sampled points
ORDERING:
Clustering
 A Fuzzy-logic based clustering algorithm was used.
 32 cluster centers were obtained
250
200
150
100
50
0
-50
-50
0
50
100
150
200
250
ORDERING:
Delaunay Triangulation
 Algorithms to triangulate and to get the Delaunay triangulation from HWKs 3
and 4 were used.
 Given a point,the random point approach is used to localize the triangle that
contains it.
200
150
100
50
0
0
50
100
150
200
Interpolation:
SVM
 SVM : Support Vector Machine Classifiers
 A different nonlinear Classifier is used for each triangle
 The triangle structure is efficiently used for both training
and testing phases and for defining a “simple” and fast
nonlinear classifier.
What is a Support Vector Machine?
 An optimally defined surface
 Typically nonlinear in the input space
 Linear in a higher dimensional space
 Implicitly defined by a kernel function
What are Support Vector Machines
Used For?
 Classification
 Regression & Data Fitting
 Supervised & Unsupervised Learning
(Will concentrate on classification)
Support Vector Machines
Maximizing the Margin between Bounding
Planes
w
x 0w = í + 1
A+
A-
x 0w = í à 1
2
jj wjj 2
The Nonlinear Classifier
 The nonlinear classifier:
K (x 0; A 0)D u = í
K (A; A 0) : R m â n â R n â m7
à ! R mâ m
 Where K is a nonlinear kernel, e.g.:
 Gaussian (Radial Basis) Kernel :
0
K ( A; A ) i j =
"
à ö kA i à A j k 22
; i; j = 1; . . .; m
 The i j -entry of K (A; A 0) represents the “similarity”
of data points A i and A j
Reduced Support Vector Machine Algorithm
öu
Nonlinear Separating Surface: K (x 0; Aö0)D
ö= í
(i) Choose a random subset matrix A 2 R mâ n of
entire data matrix A 2 R mâ n
(ii) Solve the following problem by the Newton
method with corresponding D ú D :
min
(u; í ) 2 R
÷
kp(e à
2
m+ 1
öu
D (K (A; A 0)D
ö à eí ); ë) k 22 + 12ku
ö; í k 22
(iii) The separating surface is defined by the optimal
solution ( u; í ) in step (ii):
öu
K (x 0; Aö0)D
ö= í
How to Choose A in RSVM?
 A is a representative sample of the entire dataset
 Need not be a subset of A
 A good selection of A may generate a classifier using
very small m
 Possible ways to choose A :
 Choose m random rows from the entire dataset A
 Choose A such that the distance between its rows
exceeds a certain tolerance
 Use k cluster centers of A + and A à as A
Obtained Bizarre “Checkerboard”
Optimization: More sampled points
Training parameters adjusted
Result: Improved Checkerboard
Nonlinear PSVM: Spiral Dataset
94 Red Dots & 94 White Dots
Next:Bascom Hill
Some Questions
 Would it work for B&W pictures (regression instead of
classification?
 Aplications?