Transcript Document
Handwritten Characters
Recognition Based on an
HMM Model
7/20/2015
1
Work Based-on Articles
“A Study of Hidden Markov Models for Offline Recognition of Handwritten
Characters” .
B. Gosselin & A. Paggiaro
“A Tutorial on Hidden Markov Models and
Selected Applications in Speech
Recognition”.
Lawrence R. Rabiner
7/20/2015
2
Project Goal
Application of the Principles
Described in Article [1] ,Using
Theory From Article [2].
7/20/2015
3
Elements of an HMM
N , the Number of States in the Model.
Denoted as S s1 , s 2 ,...,s N .
2. M , the Number of Distinct Observation
Symbols Per State ,
Denoted as V v1 , v2 ,...,vM .
3. The State Transition Probability
Distribution, Denoted as A aij ,
Where
aij Pr(qt s j | qt 1 si ) .
1.
7/20/2015
4
Elements of an HMM (Cont.)
4. The Observation Symbol Probability
Distribution in State j , Denoted as
where ,
B {b j (k )}
b j (k ) Pr(Ot vk | qt s j ).
5. The Initial State Distribution , Denoted as
Π {i }
Where ,
i P r(q1 si ) .
6. The Model Notation is :
{ A, B, }
7/20/2015
5
3 Basic Problems of HMM
Problem 1 ( Evaluation )
How Do We Efficiently Compute
Pr(O | )?
Solution to This , Allows Us to Choose the
Model Which Best Matches the
Observations Sequence , O .
7/20/2015
6
3 Basic Problems of HMM (Cont.)
Problem 2 ( Decoding )
Given Sequence , O , and a Model , ,
How Do We Choose Optimally the State
Sequence , Q ,Best Matches the
Observations , O ?
Solution to This , Enables Us to Learn
About the Physical Meaning of the Hidden
Part of the Model ( e.g the States ).
7/20/2015
7
3 Basic Problems of HMM (Cont.)
Problem 3 ( Learning )
How Do We Adjust the Model Parameters
A, B, in Order to Maximize Pr(O | )
Per Each Class of Characters ?
Solution to This , Enables Us to Optimize
The Model Parameters , In Order to Have
Best Description of How a Given
Observation Sequence Comes About .
7/20/2015
8
Creating My Database
1. Creating Digital Image of Handwritten
Characters.
2. Isolation of the Characters.
3. Binarization ( Using Local Threshold ).
3
7/20/2015
2
1
9
My Database Specifications
•
•
•
•
Samples From 8 Different People.
918 Characters ( ~ 8 x 5 x 27 ).
Character’s Size - 0.7 x 0.6 [Cm] ,
( 0.24 x 0.27 [Inch] ).
Scanner – 300 DPI , 256 Gray-Level .
7/20/2015
10
Applying HMM Procedure
Top Level
Binary
Images
Pre
Process
Feature
Extraction
Designing
Discrete
HMMs
7/20/2015
11
Pre - Processing
Goal –
Provide Skeleton Image of Each Character.
Binary
Images
7/20/2015
Invert
Colors
Create
Skeleton
Algorithm
12
Examples Of Skeletons
7/20/2015
Original
Skeleton
13
Feature Extraction
Reduce Amount of Data Held in Characters.
Provide Attractive Representation (Using
Oriented Search Principle Method).
95 Pairs (X,Y)
7/20/2015
14 Pairs (X,Y)
14
Oriented Search Principle
Goals –
“Chipper” Characters Representation.
Trying to Retrieve Writing Dynamics.
Algorithm Steps –
Finding First Pixel ( TBLR Scan ).
Specifying the Next Likely Pixel ( Iteratively ).
Conclusion of Line Segment.
( By Distortion TH or Number of Pixels ).
4. Normalizing Ordinates ( According to the
Initial Width & Height of Character ).
1.
2.
3.
7/20/2015
15
Oriented Search Principle (Cont.)
Illustration –
7/20/2015
16
Feature Extraction - Example
0
60
Observation
Sequence
60
O
X1
5 1 5 5 0 3 53 0 3 0 4 0
Y1
5 5 4 0 2 5 2 53 0 3 5 4 5
1 5 1 5 3 53 03 0 4 0 6 0
X2
Y2
7/20/2015
=
40 5
2 53 0 3 5 4 5 4 5
17
Example of Extremities
Ordinates Distribution
7/20/2015
18
Feature Extraction - Summarize
Advantages
Disadvantages
Preserve Shape of
Characters.
Reduce Amount of
Data.
Retrieve Dynamic
Level of Writing.
Over Sensitivity to
First Pixel.
Significant Variance
of Segments Number.
Significant Variance
of Segments Order.
Requires HMM Design !!
7/20/2015
19
Designing Discrete HMM
( Per Each Class )
Observations
Sequences
Create
ACM
7/20/2015
Initialize HMM
Structure
Training
HMM
20
Designing Discrete HMM ( Cont. )
Average Character Model
A Representative Model For Each
Character Defined By All Obs.
Goals
Defines the Number of States in the HMM
Structure.
Initializes the State Transitions Distributions.
Gives Physical Meaning to the States of the
HMM ( Each State Correspond to Segment ).
7/20/2015
21
Designing Discrete HMM ( Cont. )
Average Character Model (Cont.)
Creation Procedure :
1.
2.
3.
4.
5.
Definition of Segments Number, TH.
Applying VQ Process ( Local,Mean+Std ).
Association of Segments to ACM
Segments.( Using Distance Measure ).
Updating ACM ( According to Calculation of
Mean Value of All Grouped Segments ).
Goto Step 3 ( Unless TH Achieved ).
7/20/2015
22
Designing Discrete HMM ( Cont. )
Average Character Model (Cont.)
Distance Measure Definition
: d min(d1 , d 2 )
d1 ( x01 x02 ) 2 ( y01 y02 ) 2 ( x11 x12 ) 2 ( y11 y12 ) 2
d 2 ( x11 x02 ) 2 ( y11 y02 ) 2 ( x01 x12 ) 2 ( y01 y12 ) 2
d1
7/20/2015
d2
23
Designing Discrete HMM ( Cont. )
Initialize HMM Structure
N = Segments On ACM.
Si ith ACM Segment.
(Random Selection)
Example:
{O1av , O2av ,...,ONav }
{S1 , S 2 ,.......,S N }
7/20/2015
24
Designing Discrete HMM ( Cont. )
Segments Alignment Process
Goal -
Matching Each Segment of Each
Character to the Ones of the ACM.
Training Char.
7/20/2015
ACM
25
Designing Discrete HMM ( Cont. )
Alignment Process Results
Each Observations Sequence’s Segment Is
Indexed According to ACM Closest Segment.
AP
Example : {O1}
{S , S ,...,S }
1
1
N
{O } {S 4 , S3 ,...,S1}
2
AP
AP
{OK }
{S2 , S1 ,...,S5}
7/20/2015
O [ 4KxT ]
S [ KxN ]
26
Designing Discrete HMM ( Cont. )
Initialize HMM Parameters
According to Matrix S :
Number of Si ' s in First Column of S
Initial { } i
K
Transitions from Si to S j
Initial { A } aij
Transitions from Si to Any State
Observations VK in S j
Initial { B }
b j (k )
Total Observations in S j
7/20/2015
27
Designing Discrete HMM ( Cont. )
Training HMM Procedure
Training
Data {O}
Calculate
Q {q1 , q2 ,...,qT }
Max{P(Q | O, ) }
( Viterbi Path )
7/20/2015
Adjust HMM
Parameters
{A, B, }
28
Designing Discrete HMM ( Cont. )
Training The HMM ( Cont.)
?
Dist ( , ) Th
Yes
No
Save
7/20/2015
29
Classification System
HMM
א
Skeleton
Char.
P(O | )
HMM
Feature
א
MLP
בP(O | )
Extraction
Image
ת
HMM
ת
7/20/2015
P(O | )
30
Improve Classification Performance
Classification Decision Calculation Using the
Forward-Backward Procedure :
N
P(O | ) T (i )
i 1
t (i ) P(O1 ,.....,Ot , qt Si | )
N2 T Calculations Vs.
7/20/2015
2T NT Calculations !!
31
Improve Classification ( Cont. )
1. Adding the Probability That an Observations
Sequence ends on a given State Si :
N
P(O | ) (T (i ) f (i))
i 1
f (i) P(qT Si )
7/20/2015
32
Improve Classification ( Cont. )
2. Adding the Probability That the Observations
Sequence Length is T :
N
P(O | ) ( T (i ) f (i )) n(T )
i 1
n( x) P(T x)
7/20/2015
33
Improve Classification ( Cont. )
3. Adding “Punishment” Probability to Each
State, Si ,of the Model ,That Doesn’t Take
Place in a Sequence Of Observations :
( Improve Discrimination [ ב, ) ] ר.
N
P (O | ) ( T (i ) f (i )) n(T ) (1 c(i ))
i 1
i , S{qt* }
c(i ) P( S i {q j }) , 1 i N , 1 j T
{qt*}
Viterby Optimal State Sequence.
7/20/2015
34
Improve Classification ( Cont. )
4. Adding the Probability Distribution of the
“Discrete” Aspect Ratio ( Width / Height ).
( Avoid Discrimination Problem Due to the
Normalization of Character Dimensions ).
N
P (O | ) ( T (i ) f (i )) n(T ) (1 c(i )) r ( R )
i 1
i , S { qt* }
r ( X ) P ( r X ) , X 0.2 : 0.1 : 2
7/20/2015
35
Classification ( DB Properties )
Performed on NIST3 Database.
1579 of Each Class English
Unconstrained Uppercase Handwritten
Characters.
1324 – Training Set, 235 – Testing Set.
Total Number of Characters : 41,054.
( 1579 X 26 ).
7/20/2015
36
Classification ( Total Results )
Improve
Number
Recog.
Rate
(%)
7/20/2015
None
1
2
78.6 79.6 80.3
3
4
83.2
84.4
37
Classification ( Results Per Char. )
7/20/2015
38
My Goals
Implementation of those
Principles (on MATLAB).
Using My “Small”
Database.
Having Same Recognition
Rate.
7/20/2015
39