Transcript slides
Tagging of digital historical
images
Authors:
A. N. Talbonen ([email protected])
A. A. Rogov ([email protected])
Petrozavodsk state university
General tagging model
Tag DB
Object
selection
Image
collection
Tag
attribution
Object
DB
Indexing
File
Tags
I1
……
I2
……
Full-text
index
General research features
Research is based on analysis of
image collection of White Sea-Baltic
Sea Canal provided by National
museum of Karelia
Collection consists of about 8k images
with resolution 75 dpi.
1. Face tagging
General features
Predominance of small-sized objects (width is less than
40 pixels)
No database
Available expert
Distribution of object’s size
1. Face tagging
General algorithm
Object (face) detection.
Computing of pairwise distances
between objects.
Tagging (for each object):
The system displays a list of the most
similar objects.
The expert determines a relationship
between objects
Object tags are specified
1. Face tagging
Face detection features
There is OpenCV library (OpenCvSharp in
C#) and it’s method
cv::CascadeClassifier::detectMultiScale
(haarDetectObject in C#) (Viola-Jones
implementation) being used for face
detection
Viola-Jones method parameters are used to
affect on precision and recall on face
detection results
There is face recognition method based on
Local Binary Patterns being used to
improve the quality of Viola-Jones results
1. Face tagging
Face detection diagram
Source image
Face detection
Object
Face
objects
Fake
objects
Training set
Recognition
Object is
a face
Yes
Insert in result
collection
1. Face tagging
Local binary patterns (LBP)
Original LBP filter
Advanced LBP filters
1. Face tagging
Local binary patterns
Uniform codes
(patterns)
Rotation invariant
codes
1. Face tagging
Local binary patterns
Computing of face object histogram
Weight matrix
1. Face tagging
Face detection experiment
The purpose is to find the LBP modification with the
best detection rates
Experiment features:
Sample of 1070 images
Assessing features
Fake object when:
Object is not a face
Faces are recognized weakly
Faces turned at an angle greater than 90 degrees
Face object when:
Object is a face
Object is an image of people: portraits, paintings,
sculptures
12 different LBP modifications were used
1. Face tagging
Face detection experiment results
1. Face tagging
Face recognition experiment
Purpose is to find the LBP
modification with the best face
recognition rates
Experiment features
Training set contains 19 objects including
3 relevant pairs of face objects and 1
relevant pair of fake objects
10 LBP modifications were used
1. Face tagging
Face recognition experiment
1)
2)
3)
4)
5)
6)
7)
8)
9)
10)
11)
12)
13)
14)
15)
16)
17)
18)
19)
Pairs: {1, 15}, {3, 14}, {4, 13}, {7, 9}
1. Face tagging
Face recognition experiment results
Modification
Precision
LBP8,1
0,38
LBP16,1
0,25
LBP8,2
0,50
LBP16,2
0,50
LBP8,3
0,50
LBP16,3
0,75
ri
Взвешенный LBP16,3
0,50
riu
Взвешенный LBP16,3
0,38
u
Взвешенный LBP16,3
0,63
Взвешенный LBP16,3
1,00
1. Face tagging
Face comparing
Training set object’s histograms:
Objects at position (row, col): (1,1) and (3, 4) correspond to
fake objects and have similar histograms very different from the rest
2. Texture tagging
General features
The classifier with tags based on
moments is built
Texture searching is based on the
built classifier
Search involves finding the segments
corresponding to different textures
Minimal segment size to be include in
result is 100 pixels
2. Texture tagging
Moment-based segmentation
Moment calculation function:
Source image I
Moment image M00
Moment image M10
Moment image M01
2. Texture tagging
Moment-based segmentation
Moment feature calculation function:
F00
Binary segmentation example
F10
Precision: 96,7 %
F01
2. Texture segmentation
Implementation features
Each moment is an image
Moment computing is based on
library OpenCV and it’s method
cv::filter2D
Parameter seek is based on
developed experiment
2. Texture tagging
Parameter seek example
Moment
window size
Moment feature
Window size
Sigma
Precision
9
49
0,01
95,285
9
39
0,005
95,1782
9
39
0,02
95,1752
9
44
0,005
95,1355
9
49
0,015
95,1324
14
14
0,02
93,8416
14
14
0,005
93,7103
14
19
0,005
92,7826
14
19
0,015
92,7826
14
34
0,015
92,5293
14
29
0,015
92,395
14
34
0,02
92,3248
24
24
0,02
87,9639
39
19
0,01
87,9639
2. Texture tagging
Classifier features
Set of textures of several classes is
given
Each class is assigned a set of tags
Each image is subjected to a
separate texture search
Each texture found adds appropriate
set of tags to the source image
2. Texture tagging
Example
Source image
2. Texture tagging
Example
Classifier example
Classifier textures example
2. Texture tagging
Experiment
Purpose is to evaluate the search quality
Experiment features
Sample of 100 images
Classifier contains 2 textures
House roof
House wall
2. Texture tagging
Search quality evaluate method
Single texture estimations:
R
F
ij
Prj
i
ij
i
R
E
ij
; Re j
ij
General estimations:
R
Pr
F
ij
i, j
ij
i, j
i
i
R
; Re
E
ij
i, j
ij
i, j
Eij
Fij
- Flag of belonging to
assessed collection
- Flag of belonging to
search result
Rij Eij Rij
Flag of relevance
2. Texture tagging
Experiment results
Thanks for your attention!