Transcript Gabor filter with LBP

Local descriptors and similarity measures for frontal
face recognition: A comparative analysis
小组成员：周稻祥
报告人：周稻祥
About the Author: Witold Pedrycz
 Department of Electrical and Computer Engineering, University of Alberta, Canada
 Professor & Canada Research Chair & IEEE Fellow & Professional Engineer
 Research interests and activities:
Software Engineering
System modelling and knowledge discovery
Reconfigurable and evolvable architectures.
Pattern recognition
 Personal Homepage：http://www.ece.ualberta.ca/~pedrycz/index.html
About the Author: Marek Reformat
 A member of the IEEE and ACM.
 A member of program committees of several conferences related to computational
intelligence and software engineering.
 Actively involved in North American Fuzzy Information Processing Society (NAFIPS).
 Research interests and activities:
Knowledge extraction and knowledge representation
Semantic-based intelligent systems
Decision support
Software quality and maintenance
 Personal Homepage：
http://www.ece.ualberta.ca/~reform/index.html
Contents
Main process
Local descriptors
Gabor filter with LBP
Similarity measures & Experimental
results
Taxonomy
On averaged
ILBP,MBLBP,GR
AB
Rotation
invar
Shifted LBP
On pixel
LBP,CS,RI
Psychological
WLD
Three Dimen
VLBP,LBP-TOP
Local
descriptors
Ternary
LTP,DLTP
Distance
based
TPLBP,FPLBP
Selete subset
U2,DLBP,SELBP
Multiresoluti
onGabor,MB,GR
AB
Derivative patterns
ELGBP,LDP
Gabor phase
quantization
HGPP,LGPDP,LGXP
local descriptors
on Gabor filtered
image
Gabor magnitude
LGBP,MHLVP,LGBPHS,MUL
GBP
Gabor magnitude &
phase
ELGBP,MBP
3D
GV-LBPTOP
Main process
1：选择合适的局部描述子，一般是选择基准点或者每一个像素点
2：局部描述子提升形成整体描述子，如果是基于每个像素得到的描述
子，通常是把每个区域的描述子进行连接。
3：通过某种相似度量，对未知图像的描述子与已知图像的描述子进行
匹配
Main process
𝑉𝑖 =[𝑥1 , 𝑥2 , … , 𝑥𝑀 ]
23
11
43
98
12
99
78
21
65
Local pattern:
pixel level
description
V=[𝑉1 , 𝑉2 , … , 𝑉𝑁 ]
Histogram 𝐻𝑖 of
Local patterns
H=[𝐻1 , … , 𝐻𝑅 ]
Contents
Main process
Local descriptors
Gabor filter with LBP
Similarity measures & Experimental
results
Baisc LBP etc
Local binary patterns:
Basic LBP
Circular LBP
Uniform-Ri LBP
𝑃−1
𝑟𝑖𝑢2
𝐿𝐵𝑃𝑃,𝑅
=
𝑝=0
𝑠 𝑔𝑝 − 𝑔𝑐
𝑃+1
𝑖𝑓 𝑈(𝐿𝐵𝑃𝑃,𝑅 ) ≤ 2
𝑜𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒
Where
𝑈 𝐿𝐵𝑃𝑃,𝑅
= 𝑠 𝑔𝑃−1 − 𝑔𝑐 − 𝑠 𝑔0 − 𝑔𝑐
𝑃−1
Elongated LBP
+
𝑠 𝑔𝑝 − 𝑔𝑐 − 𝑠 𝑔𝑝−1 − 𝑔𝑐
𝑝=1
B
57
A
=2
New Variants：
Dominant LBP
Bilinear interpolation
Statistically Effective LBP
of a pixel
Hamming LBP
ILBP
Local binary patterns:
Threshold 100.11
2𝑃+1 - 1 Bins
ELBP
Local binary patterns:
L1
L2 L3 L4
Sign
Magnitude
Threshold
CLBP
Local binary patterns:
Original image
Center gray level
clbp_C
Local difference
S
M
clbp_S
clbp_M
clbp_map
clbp_Histogram
Sign
Magnitude
classifier
LTP
Local binary patterns:
1
0
1
23
11
1
2
0
-1
1
0
0
1
0
5
98
12
14
1
DLTP=| LTPU-LTPL|=135-40=95
0
0
23
21
6
1
1
0
1
-1
0
0
0
0
LTP
1
(AELTP)
Soft-LBP
Local binary patterns:
𝑆1,𝑑
0
𝑥
𝑥 = 0.5 + 0.5 𝑑
1
SLBP(𝑥𝑐 , 𝑦𝑐 , ℎ) =
𝐻𝑆𝐿𝐵𝑃(𝑖) =
𝑥 < −𝑑
−𝑑 ≤𝑠 ≤𝑑
𝑃−1
𝑝=0 [𝑏𝑝
𝑥>𝑑
𝑥 ∗ 𝑠1,𝑑 𝑖𝑝 − 𝑖𝑐 + (1 − 𝑏𝑝 𝑥 ) ∗ 𝑠0,𝑑 𝑖𝑝 − 𝑖𝑐 ]
𝑆𝐿𝐵𝑃(𝑥, 𝑦, 𝑖)
𝑥,𝑦
𝑆0,𝑑 𝑥 =1-𝑆1,𝑑 𝑥
𝑖 = 1,2, … , 𝑃 − 1
SILT
Gabor filter with LBP:
P
63
𝟔𝟖
42
64
64
27
61
95
8𝟑
[64(1-t)
64(1+t)]
t=0.
1
00
00
00
00
10
10
01
01
MB-LBP
Local binary patterns:
Integral image
𝑖𝑖 𝑥, 𝑦
𝑖(𝑥 ′ , 𝑦 ′ )
=
𝑥 ′ ≤𝑥,𝑦≤𝑥
D=4+1-(2+3)
GARB
Local binary patterns:
GRAB(General Region Assigned to binary)
1
2
8
某个阈值 如5
3
4
7
6
5
1
2
4
3
5
6
7
8
solving the orientation problem
small variation in edge angles cause smaller
variations in the binary representation
#：noise & variations & rotation tolerant operator
CS-LBP
Local binary patterns:
CS-LB𝑃𝑅,8 (pc)= 𝑠(𝑃0 -𝑃4 )*20 +𝑠(𝑃1 -𝑃5 )* 21 +
𝑠(𝑃2 -𝑃6 )* 22 +𝑠(𝑃3 -𝑃7 )* 23
1, 𝑥 ≥ 𝜏
𝑠 𝑥 =
0, 𝑥 < 𝜏
TP-LBP
Local binary patterns:
TPLB𝑃𝑅,𝑃,𝑤,𝛼 (p)=
𝑃
𝑖=0 𝑠(𝑑(𝐶𝑖
TPLB𝑃𝑅,8,3,2 (p)=
𝑠(𝑑(𝐶0 , 𝐶𝑝 )-𝑑(𝐶2 , 𝐶𝑝 ))*20 +
𝑠(𝑑(𝐶1 , 𝐶𝑝 )-𝑑(𝐶3 , 𝐶𝑝 ))*21 +
𝑠(𝑑(𝐶2 , 𝐶𝑝 )-𝑑(𝐶4 , 𝐶𝑝 ))*22 +
𝑠(𝑑(𝐶3 , 𝐶𝑝 )-𝑑(𝐶5 , 𝐶𝑝 ))*23 +
𝑠(𝑑(𝐶4 , 𝐶𝑝 )-𝑑(𝐶6 , 𝐶𝑝 ))*24 +
𝑠(𝑑(𝐶5 , 𝐶𝑝 )-𝑑(𝐶7 , 𝐶𝑝 ))*25 +
𝑠(𝑑(𝐶6 , 𝐶𝑝 )-𝑑(𝐶0 , 𝐶𝑝 ))*26 +
𝑠(𝑑(𝐶7 , 𝐶𝑝 )-𝑑(𝐶1 , 𝐶𝑝 ))*27
, 𝐶𝑝 ) - 𝑑(𝐶𝑖+∝𝑚𝑜𝑑𝑃 , 𝐶𝑝 ))*2𝑖
𝑠 𝑥 =
1, 𝑥 ≥ 𝜏
0, 𝑥 < 𝜏
FP-LBP
Local binary patterns:
FPLB𝑃𝑅1,𝑅2,𝑃,𝑤,𝛼 (p)=
𝑃
𝑖=0 𝑠(𝑑(𝐶1,𝑖 , 𝐶2, 𝑖+∝
𝑚𝑜𝑑𝑝 )
- 𝑑(𝐶1,𝑖+𝑃/2, , 𝐶2,
FPLB𝑃𝑅,8,3,2 (p)=
𝑠(𝑑(𝐶1,0 , 𝐶2,1 )-𝑑(𝐶1,4 , 𝐶2,5 ))*20 +
𝑠(𝑑(𝐶1,1 , 𝐶2,2 )-𝑑(𝐶1,5 , 𝐶2,6 ))*21 +
𝑠(𝑑(𝐶1,2 , 𝐶2,3 )-𝑑(𝐶1,6 , 𝐶2,7 ))*22 +
𝑠(𝑑(𝐶1,3 , 𝐶2,4 )-𝑑(𝐶1,7 , 𝐶2,0 ))*23
𝑃
𝑖+ 2 +𝛼
𝑖
))*2
𝑚𝑜𝑑𝑃
LDP
Local binary patterns:
85
32
26
313
97
503
0
53
50
10
537
0
399
1
60
38
45
161
97
161
0
1：Robust against Gaussian white noise and
non-monotonic illumination changes
2：Rotation invariant
0
1
1
0
0
Local binary patterns:
VLBP
Local binary patterns:
VLBP
Local binary patterns:
LBP-TOP
Local binary patterns:
LBP-TOP
Local binary patterns:
LBP-TOP
Local binary patterns:
LBP-TOP
Contents
Main process
Local descriptors
Gabor filter with LBP
Similarity measures & Experimental
results
Gabor filter with LBP
∞
F(ω) =
𝑓 𝑥 exp −𝑖ω𝑥 𝑑𝑥 连续
−∞
1
F(ω) =
𝑀
𝑀−1
𝑓 𝑥 exp(−𝑖ω𝑥/𝑀) 离散
𝑥=0
Gabor filter with LBP
∞
F(𝑢, 𝑣) =
∞
𝑓 𝑥, 𝑦 exp −𝑖(𝑢𝑥 + 𝑣𝑦 ) 𝑑𝑥𝑑𝑦
−∞ −∞
1
F(𝑢, 𝑣) =
𝑀𝑁
𝑀−1 𝑁−1
𝑥=0 𝑥=0
𝑢𝑥 𝑣𝑦
𝑓 𝑥, 𝑦 exp(−𝑖( + )
𝑀
𝑁
二维离散傅里叶变换
连续
离散
音
调
时
间
振
幅
频
率
低频
高频
频
域
分
布
=
+
空
间
分
布
=
+
Gabor filter with LBP
Gabor核函数：
Dennis
Gabor, 1946年提出窗口傅里叶变换进行时频分析，其中
||𝐾𝜇,𝑣 ||2
窗口函数就是高斯函数。
𝜑
𝑥, 𝑦 =
exp(−||𝐾 ||2 ||𝑧||2 /2𝜎 2 )(exp(𝑖𝐾 ) − exp −𝜎 2 /2 )
𝜇,𝑣
𝜎2
其中：Z=(x,y) , 𝐾𝜇,𝑣 =
𝜇,𝑣
(𝐾𝑣 𝑐𝑜𝑠𝜙𝜇 , 𝐾𝑣 𝑠𝑖𝑛𝜙𝜇 )𝑇 , 𝐾𝑣
𝜇,𝑣
= 𝑓𝑚𝑎𝑥 /2
𝑣
2
𝐺𝜇,𝑣 𝑥, 𝑦 = 𝑖 𝑥, 𝑦 ∗ 𝜑𝜇,𝑣 𝑥, 𝑦 = 𝐴𝜇,𝑣 𝑥, 𝑦 exp(𝑖𝜃𝜇,𝑣 𝑥, 𝑦 )
𝜇 = 0, … , 𝜇𝑚𝑎𝑥 − 1; 𝜈 = 0, … , 𝑣𝑚𝑎𝑥 − 1, 𝑣𝑚𝑎𝑥 =5, 𝜇𝑚𝑎𝑥 =8, 𝜙𝜇 =μ𝜋/8;
卷积定理：
𝑖 𝑥, 𝑦 ∗ 𝜑 𝑥, 𝑦 ⇔ 𝐼 𝑢, 𝑣 𝜙(𝑢, 𝑣)
𝑖 𝑥, 𝑦 𝜑 𝑥, 𝑦 ⇔ 𝐼 𝑢, 𝑣 ∗ 𝜙 (𝑢, 𝑣)
HGPP
Gabor filter with LBP:
GGP𝑃𝑣𝑅𝑒 (𝑥, 𝑦)
GGP𝑃𝑣𝐼𝑚 (𝑥, 𝑦)
𝑅𝑒
LGP𝑃𝑢,𝑣
(𝑥, 𝑦) =
𝐼𝑚
LGP𝑃𝑢,𝑣
(𝑥, 𝑦) =
7
𝑅𝑒
𝑃
𝑖=0 𝑢,𝑣
7
𝐼𝑚
𝑃
𝑖=0 𝑢,𝑣
=
=
𝜇𝑚𝑎𝑥 −1 𝑅𝑒
𝑃𝑖,𝑣
𝑖=0
𝑥, 𝑦 ∗ 2𝑖
𝜇𝑚𝑎𝑥 −1 𝐼𝑚
𝑃𝑖,𝑣
𝑖=0
𝑥, 𝑦 ∗ 2𝑖
10 GGPP
𝑅𝑒
𝑥, 𝑦 𝑋𝑂𝑅 𝑃𝑢,𝑣
𝑥𝑖 , 𝑦𝑖 ∗ 2𝑖
𝐼𝑚
𝑥, 𝑦 𝑋𝑂𝑅 𝑃𝑢,𝑣
𝑥𝑖 , 𝑦𝑖 ∗ 2𝑖
此时用的是实部与虚部而非幅值与相位值
80 GGPP
Gabor filter with LBP:
HGPP
GGPP
LGPP
1
1
0
0
1
1
0
1
0
LGPD
Gabor filter with LBP:
P
|∆𝜃𝑢.𝑣 𝑍
𝟎. 𝟖𝝅 𝟎. 𝟏𝝅
1. 𝟓𝝅
1. 𝟗𝝅 𝟎. 𝟓𝝅
1. 𝟔𝝅
0.9𝝅
𝟏
1.3𝝅 𝟎. 𝟑𝝅
𝟏
0
1
此时用的是相位值
0
0
1
𝟏
LGXP
Gabor filter with LBP:
𝐿𝐺𝑋𝑃𝑢.𝑣 𝑍𝑐 =
𝑃
𝑖−1
𝑖
2
𝐿𝐺𝑋𝑃
𝑢.𝑣
𝑖=1
𝑖
𝐿𝐺𝑋𝑃𝑢.𝑣
={0,1}
𝟎. 𝟖𝝅 𝟎. 𝟏𝝅
1.6𝝅 𝟎. 𝟗𝝅
0.9𝝅
𝟏
1. 𝟓𝝅
0. 𝟔𝝅
𝜋
0,
2
,
𝜋
,𝜋
2
3𝜋
𝜋,
2
,
3𝜋
, 2𝜋
2
0
1
1.3𝝅 𝟎. 𝟑𝝅
𝟎
0
1
0
𝐻 = [𝐻𝑢0, 𝑣0,1, 𝐻𝑢0, 𝑣0,𝑚 , 𝐻𝑢𝑚𝑎𝑥, 𝑣𝑚𝑎𝑥 ,1 , 𝐻𝑢𝑚𝑎𝑥, 𝑣𝑚𝑎𝑥 ,𝑚 ]
此时用的是相位值
𝟎
Contents
Main process
Local descriptors
Gabor filter with LBP
Similarity measures & Experimental
results
Similarity measures
Fusing sub-region
1: All sub-regional histograms
concatenated
2: Sub-regions of two images are compared
pair-wise
and the results are aggregated
Fusing sub-region
A:Feature level
fusion
I1
I2
LGBP_mag
I1
I2
LGX
P
Fusing sub-region
I1
I1
LGBP_mag
I2
I2
LGXP
Others measure:
1:cosine distance
measure
2:LDA
Reduction
dimensionality
1: AdaBoost
2: Borda count
Thank you