Transcript A Comparative Study of Some Multiple Expert Recognition

Multiple Classifier Combination for
Character Recognition: Revisiting
the Majority Voting System and its
Variations
A. F. R. Rahman and H. Alam
BCL Technologies Inc.
USA
M.C. Fairhurst
University of Kent
UK
Basic Problem Statement
Given a number of experts working on the
same problem, is group decision superior to
individual decisions?
Ghosts from the Past…
• Jean-Charles de Borda (1781) •CC. L. Dodgson (Lewis Carrol)
• N. C. de Condorcet (1785) (1873)
• M. W. Crofton (1885)
• Laplace (1795)
• E. J. Nanson (1907)
• Issac Todhunter (1865)
• Francis Galton (1907)
•
Is Democracy the answer?
• Infinite Number of Experts
• Each Expert Should be Competent
How Does It Relate to Character
Recognition?
Each Expert has its:
• Strengths and Weaknesses
• Peculiarities
• Fresh Approach to Feature Extraction
• Fresh Approach to Classification
• But NOT 100% Correct!
Practical Resource Constraints
Unfortunately, We Have Limited
• Number of Experts
• Number of Training Samples
• Feature Size
• Classification Time
• Memory Size
Solution
• Clever Algorithms to Exploit Experts
– Complimentary Information
– Redundancy: Check and Balance
– Simultaneous Use of Arbitrary Features and
Classification Routines
Question?
– Recent trend is towards complicated decision
combination schemes
– Exhaustive Classifier Selection
– Theoretical analysis in place of empirical
methods
How sophisticated (read “complex”)
algorithms do we really need?
Majority Voting Philosophy
• Should the decision agreed by the majority
of the experts be accepted without giving
due credit to the competence of the experts?
---- OR ---• Should the decision delivered by the most
competent expert be accepted without
giving any importance to the majority
consensus?
[1] Simple Majority Voting
Classifier
Classifier
Classifier
Classification
Decision
Classification
Decision
Classification
Decision
Decision Fusion : Counting Individual Votes to Support a
Decision
Final Decision
Decision accepted if
at least k of the
experts agree, where
n
 2 1
k 
 n 1
 2
If n is even,
If n is odd.
[2] Weighted Majority Voting
Overall
Weight of
the
Classifier
Classifier
Classification
Decision
Overall
Weight of
the
Classifier
Classifier
Classification
Decision
Overall
Weight of
the
Classifier
Classifier
Classification
Decision
Decision Fusion: Counting Weighted Votes for Individual Decisions to Support a Final
Decision
Final Decision
[2] Weighted Majority Voting
(Contd.)
So if k th decision to assign the unknown pattern to the i th class is
denoted by d ik with 1  i  m, m being the number of classes, then the
final combined decision d icm supporting assignment to the i th class
takes the form of:
d icom 

k 1, 2,...,n
k
* d ik
The final decision d comis therefore:
d com  maxi1,2,..,m dicom
[3] Class-wise Weighted Majority
Voting
Class-based
Weight of
the
Classifier
Classifier
Classification
Decision
Class-based
Weight of
the
Classifier
Classifier
Classification
Decision
Class-based
Weight of
the
Classifier
Classifier
Classification
Decision
Decision Fusion: Counting Class-based Weighted Votes for Individual Class Decisions to
Support a Final Decision
Final Decision
[4] Restricted Majority Voting
(Top Choice)
Overall
Classifier
Weight
Classifier
Overall
Classifier
Weight
Classifier
Overall
Classifier
Weight
Best Classifier Selection
Decision Fusion: Selection of Best Decision Delivered by Best
Classifier
Final Decision
Classifier
[4] Restricted
Majority
Voting
(Generalized)
[5] Class-wise Best Decision
Selection
Class-based
Classifier
Weight
Classifier
Class-based
Classifier
Weight
Classifier
Class-based
Classifier
Weight
Best Class-based
Classifier Selection
Decision Fusion: Selection of Best Class-based Decision
Delivered by Best Class-wise Classifier
Final Decision
Classifier
[6] Enhanced Majority Voting
[7] Ranked Majority Voting
• Not only the top choice, but ranked list of other classes
• Takes account of the negative votes cast by the experts against
a particular decision.
• Each expert not only supplies the top choice (class) decision,
but also supplies the ranking of all the other choices
considered.
• The idea is to translate this ranking into ``scores" which
would be comparable across all the decisions by all the
experts.
[7] Ranked Majority Voting: Continued
(Class Set Reordering)
• Highest Rank: Take the highest assigned
rank
• Borda Count: Sum of the number of classes
ranked below it by each classifier.
• Regression Method
Selection of a Database
• NIST Handwritten Characters
• Collected Off-line
• Total 34 Classes (0-9, A-Z, no Distinction
between 0/O and I/1)
• Total Samples of Over 34,000 characters
• Size Normalized to 32X32
Performance of the Classifiers
Expert
Accepted Recog.
Error
Rej.
FWS
97.35
78.76
18.59
2.65
MPC
97.62
85.78
11.84
2.38
BWS
95.50
72.31
23.19
4.50
MLP
95.13
82.31
12.82
4.87
Performance of the Combination
Combination Method
Accepted
Recog.
Error
Rej.
Simple
96.59
90.59
6.00
3.41
Weighted
96.85
90.64
6.21
3.15
Class-wise Weighted
96.86
90.70
6.16
3.14
Restricted Top Choice
95.68
88.97
6.71
4.32
Class-wise Best Decision
96.76
89.64
6.79
3.24
Restricted Generalized
96.54
90.63
5.91
3.46
Enhanced (ENOCORE)
97.14
90.91
6.23
2.86
Ranked (Borda)
96.99
90.77
6.22
3.01
Committee
95.98
89.63
6.35
4.02
Regression
97.68
90.68
6.83
2.32
Comparative Study
Method
Accepted Recogn.
Error
Reject
BKSM
96.20
90.84
5.36
3.80
Sum
Rule
GA
96.40
90.21
6.19
3.60
96.36
92.39
3.97
3.64
Best of
MVS
97.14
90.91
6.23
2.86
Conclusions
• Majority Voting Solutions can be very versatile
and adaptive
• Different variations may be adopted for different
problem domains
• The Majority Voting configuration is generic
• Majority Voting Systems may be as applicable to
any task domains with equal effectiveness as other
complicated solutions