Transcript AUTOMATIC TARGET RECOGNITION AND DATA FUSION

AUTOMATIC TARGET
RECOGNITION AND
DATA FUSION
March 9th, 2004
Bala Lakshminarayanan
Outline
 Introduction
 Distributed processing
 DSN topologies
 Data fusion
 SFTB project
 Classification results
Bala Lakshminarayanan, 9th March
2
Introduction
 Automatic Target Recognition
 Classify civilian targets with high accuracy




7 targets
3 sensors (IR, Grayscale, Acoustic)
3 nodes
3 scenarios
 Nodes are placed along the road
Bala Lakshminarayanan, 9th March
3
Processing paradigms…(1)
 Centralized processing – fusion center




High communication bandwidth
Higher network cost
Non-optimal processing, esp. when sensor
coverage does not overlap
Central node dependence
Bala Lakshminarayanan, 9th March
4
Processing paradigms…(2)
 Distributed processing






Redundancy – accurate classification
Lesser network cost
Reduced bandwidth requirement though
increased communication between nodes
Better response to rapid changes
Needs proper architecture
Energy efficiency
Bala Lakshminarayanan, 9th March
5
Serial topology
Event (H)
y2
y1
Sensor 1
u1
Sensor 2
yn
u2
Sensor n
un
yi : Local observation
ui : Local decision variable
n : Number of sensors
u0 : Global decision variable
Bala Lakshminarayanan, 9th March
6
Parallel topology
Event (H)
y2
y1
Sensor 1
u1
yn
Sensor 2
u2
Sensor n
un
• Classifier Selection model
• Each classifier is an “expert”
• For feature x, classifier in its vicinity is given highest credit
Bala Lakshminarayanan, 9th March
7
Parallel topology
Event (H)
y1
Sensor 1
u1
yn
y2
Sensor 2
Sensor n
un
u2
Fusion Center
u0
• Classifier Fusion model
• All classifiers trained over entire
feature space
• Competitive model
Bala Lakshminarayanan, 9th March
8
Data fusion…(1)
 Disparate sensors used for data collection
 Need to integrate results – fuse sensor data
to give user ability to decide better
 Objective of fusion is to give one reliable,
robust decision rather than many uncertain
decisions
 Fusion levels



Temporal
Multi-modal
Multi-sensor (from different nodes)
Bala Lakshminarayanan, 9th March
9
Data fusion…(2)
 Temporal fusion
 Independent frames
 Majority voting
 Multi-modality fusion
 Different sensing modalities, all exposed to whole
feature space
 Competitive rather than complementary
 BKS algorithm
 Multi-sensor fusion
 Handles faulty sensors
 MRI algorithm
Bala Lakshminarayanan, 9th March
10
Data fusion…(3)
Multi sensor fusion
(From different
nodes)
Multi-modality fusion
(From different
sensing modalities)
Temporal fusion
(From different
frames)
Multi-modality fusion
(From different
sensing modalities)
Temporal fusion
(From different
frames)
Temporal fusion
(From different
frames)
Temporal fusion
(From different
frames)
IR NODE
(Frame1, frame2,
Frame3….frameN)
Bala Lakshminarayanan, 9th March
GRAYSCALE NODE
(Frame1, frame2,
Frame3….frameN)
11
BKS Classification…(1)
 Behaviour Knowledge Space
 Aggregates results obtained from individual
classifiers
 Statistically, gives the optimal result
 OCR on 46,451 numerals shows BKS
outperforms voting, Bayesian and DempsterShafer
 These approaches require the independence
assumption – not so in real applications
Bala Lakshminarayanan, 9th March
12
BKS Classification…(2)
 Independence assumption


All classifiers are assumed to be equal
Information for fusion is taken from confusion
matrix of single classifier
 BKS avoids the independence assumption by
concurrently recording decisions of all
classifiers
 Behaviour of all classifiers recorded on a
knowledge space - BKS
Bala Lakshminarayanan, 9th March
13
BKS Classification…(3)
Feature Vector
x  Rn
Classifier
D: R n  [0,1]c  {0}
Class Label
M D ( x)
Crisp Classifier, Fuzzy classifier,
Possibilistic classifier
Decision can he hardened using the
maximum membership rule
D( x)  k  M D K ( x)  max( M Di ( x)) i
Bala Lakshminarayanan, 9th March
14
BKS Classification…(4)
 Majority voting



Class labels are crisp or hardened
Crisp label most represented is assigned to x
Ties are broken randomly
Bala Lakshminarayanan, 9th March
15
BKS Classification…(5)
 BKS


s1, s2, …, sL are crisp labels assigned to x by
classifiers D1, D2, …, DL respectively
Every combination of labels is an index to an
LUT
Bala Lakshminarayanan, 9th March
16
BKS Classification…(6)
 Example of BKS
 c = 3, L = 2, N = 100
s1, s2
Number from each
class
Label
1,1
10/3/3
1
1,2
3/0/6
3
1,3
5/4/5
1,3
2,1
0/0/0
0
2,2
1/16/6
2
2,3
4/4/4
1,2,3
3,1
7/2/4
1
3,2
0/2/5
3
3,3
0/0/6
3
Bala Lakshminarayanan, 9th March
17
SFTB Framework
Start
Inputs-nodeID, scenario…
Grab frames from dataset
Continuous, non continuous
Segment using bgSubtract()
Background image
Extract features invMoment()
Normalize, write database files
Classify
Bala Lakshminarayanan, 9th March
readData(),
knn()
End
18
Results…(1)
 30 frames from each node – 5 testing, 25
training
 7 targets
 3 scenarios
 2 nodes (IR, Grayscale)
 6 databases with 210 feature vectors
 2 versions – database1, database2
Bala Lakshminarayanan, 9th March
19
Results…(2)
Targets
1
2
3
4
5
6
7
1
4
-
-
-
-
-
1
2
-
5
-
-
-
-
-
3
-
-
4
-
-
1
-
4
-
-
-
5
-
-
-
5
-
-
-
-
1
-
4
6
-
2
2
-
-
1
-
7
3
-
-
-
-
-
2
Node in23, Scenario 1, k=7,
Classification accuracy 62.86%
Node px12, Scenario 1, k=7,
Classification accuracy 62.86%
Targets
1
2
3
4
5
6
7
1
3
-
1
-
1
-
-
2
-
5
-
-
-
-
-
3
1
-
4
-
-
-
-
4
-
-
2
2
1
-
-
5
-
-
-
-
4
-
1
6
1
1
-
-
-
3
-
7
-
-
-
-
-
2
3
Bala Lakshminarayanan, 9th March
20
Results…(3)
Scenario
k=7
k=9
k=11
1
62.86
68.57
68.57
6
68.57
68.57
71.43
25
88.57
88.57
80.00
Scenario
k=7
k=9
k=11
1
62.86
45.71
51.43
6
57.14
54.28
57.14
25
48.57
48.57
34.28
Bala Lakshminarayanan, 9th March
Node PX12
Node IN23
21
Future work
 Perform fusion between IR and grayscale
image data
 Perform fusion between images from different
scenarios
Bala Lakshminarayanan, 9th March
22
References
 X. Wang, H. Qi, S. S. Iyengar, “Collaborative multi-modality
target classification in distributed sensor networks”, International
Conference on Information Fusion (ICIF), July 2002
 R. Viswanathan, P.K. Varshney, “Distributed Detection with
multiple sensors: Part 1-Fundamentals”, Proceedings of the
IEEE, Vol 85, Jan 1997
 L.I. Kuncheva, J.C. Bezdek, Robert P.W. Duin, “Decision
templates for multiple classifier fusion: an experimental
comparison”, Pattern Recognition, Vol 34, 2001
 Y.S. Huang, C.Y. Suen, “A method for combining multiple
experts for the recognition of unconstrained handwritten
numerals”, IEEE Transactions on Pattern Analysis and Machine
Intelligence, Vol 17, Jan 1995
Bala Lakshminarayanan, 9th March
23