Transcript Week 4
COLOR-ATTRIBUTES-RELATED
IMAGE RETRIEVAL
WEEK 4
Student: Kylie Gorman
Mentor: Yang Zhang
GMM AND FISHER VECTOR
CODE
STEP ONE: CONCATENATE THE FEATURE
MATRICES OF EACH IMAGE
STEP TWO: APPLY GMM FUNCTION
Generate mean, covariance, and prior mode
probabilities
Mean
Covariance
Prior Mode
Probabilities
STEP THREE: CREATE FISHER VECTORS
encoding = vl_fisher(new', means, covariances, priors);
REPEAT PROCESS
FINAL STEPS
Calculate feature matrix of each image, isolating
the object first
Concatenate matrices
Apply PCA function to preprocess data
Multiply each individual feature matrix by result
Concatenate output into 1 matrix
Apply GMM function and obtain mean,
covariance, and prior mode probabilities
Apply Fisher Vector to each individual result to
obtain vectors that are the same size
Use those fisher vectors for 11 SVM’s (one for
each color)
COMPLETE STEPS
Using Ebay Data (omitting binary images)
Use all Google Data (from 30 to 100 images per color)
Increase cluster size in GMM from 10 to 128
SVM
LINEAR SVM
First tried it with libsvm code
MATLAB Function: svmtrain (Training, Group)
Training: Data to be processed (transpose matrix)
Group: Specifies +1 or -1 data
Use SVM for each color (black, blue, brown,
green, grey, orange, pink, purple, red, white,
yellow)
Changed to fitcsvm(X,Y)
SVM TRAIN OUTPUT
FITCSVM OUTPUT
CLASSIFY DATA
MATLAB Function:
svmclassify(SVMStruct,Sample)
Use SVMStruct from svmtrain (from each color)
Sample: Concatenated Ebay Fisher Vectors
Changed to predict(SVMModel, X)
SVMModel from fitcsvm
SVM CLASSIFY OUTPUT
Column vector with the same number of rows as
Sample. Each entry (row) in Group represents the
class of the corresponding row of Sample.
PREDICT OUTPUT
Returns Label and
Score
CURRENT PROGRESS
CALCULATE PRECISION
Calculate 12 highest scores for each color, using
first column only
Determine if each score is a correct match by
checking indices
Calculate each color’s precision
FUTURE GOALS
Fix Binary function
Try process with new data set
Data set available: Fahad Shahbaz Khan, Rao
Muhammad Anwer, Joost van de Weijer, Andrew D.
Bagdanov, Maria Vanrell, Antonio M. Lopez
Image retrieval test
Object Detection