Transcript Document

Handwritten Characters
Recognition Based on an
HMM Model
7/20/2015
1
Work Based-on Articles
“A Study of Hidden Markov Models for Offline Recognition of Handwritten
Characters” .
B. Gosselin & A. Paggiaro
 “A Tutorial on Hidden Markov Models and
Selected Applications in Speech
Recognition”.
Lawrence R. Rabiner

7/20/2015
2
Project Goal
Application of the Principles
Described in Article [1] ,Using
Theory From Article [2].
7/20/2015
3
Elements of an HMM
N , the Number of States in the Model.
Denoted as S  s1 , s 2 ,...,s N .
2. M , the Number of Distinct Observation
Symbols Per State ,
Denoted as V  v1 , v2 ,...,vM .
3. The State Transition Probability
Distribution, Denoted as A  aij ,
Where
aij  Pr(qt  s j | qt 1  si ) .
1.
7/20/2015




 
4
Elements of an HMM (Cont.)
4. The Observation Symbol Probability
Distribution in State j , Denoted as
where ,
B  {b j (k )}
b j (k )  Pr(Ot  vk | qt  s j ).
5. The Initial State Distribution , Denoted as
Π  {i }
Where ,
 i  P r(q1  si ) .
6. The Model Notation is :
  { A, B, }
7/20/2015
5
3 Basic Problems of HMM
Problem 1 ( Evaluation )
How Do We Efficiently Compute
Pr(O |  )?
Solution to This , Allows Us to Choose the
Model Which Best Matches the
Observations Sequence , O .
7/20/2015
6
3 Basic Problems of HMM (Cont.)
Problem 2 ( Decoding )
Given Sequence , O , and a Model ,  ,
How Do We Choose Optimally the State
Sequence , Q ,Best Matches the
Observations , O ?
Solution to This , Enables Us to Learn
About the Physical Meaning of the Hidden
Part of the Model ( e.g the States ).
7/20/2015
7
3 Basic Problems of HMM (Cont.)
Problem 3 ( Learning )
How Do We Adjust the Model Parameters
A, B,  in Order to Maximize Pr(O |  )
Per Each Class of Characters ?
Solution to This , Enables Us to Optimize
The Model Parameters , In Order to Have
Best Description of How a Given
Observation Sequence Comes About .
7/20/2015
8
Creating My Database
1. Creating Digital Image of Handwritten
Characters.
2. Isolation of the Characters.
3. Binarization ( Using Local Threshold ).
3
7/20/2015
2
1
9
My Database Specifications
•
•
•
•
Samples From 8 Different People.
918 Characters ( ~ 8 x 5 x 27 ).
Character’s Size - 0.7 x 0.6 [Cm] ,
( 0.24 x 0.27 [Inch] ).
Scanner – 300 DPI , 256 Gray-Level .
7/20/2015
10
Applying HMM Procedure
Top Level
Binary
Images
Pre
Process
Feature
Extraction
Designing
Discrete
HMMs
7/20/2015
11
Pre - Processing
Goal –
 Provide Skeleton Image of Each Character.
Binary
Images
7/20/2015
Invert
Colors
Create
Skeleton
Algorithm
12
Examples Of Skeletons
7/20/2015
Original
Skeleton
13
Feature Extraction

Reduce Amount of Data Held in Characters.
 Provide Attractive Representation (Using
Oriented Search Principle Method).
95 Pairs (X,Y)
7/20/2015
14 Pairs (X,Y)
14
Oriented Search Principle
Goals –
 “Chipper” Characters Representation.

Trying to Retrieve Writing Dynamics.
Algorithm Steps –
Finding First Pixel ( TBLR Scan ).
Specifying the Next Likely Pixel ( Iteratively ).
Conclusion of Line Segment.
( By Distortion TH or Number of Pixels ).
4. Normalizing Ordinates ( According to the
Initial Width & Height of Character ).
1.
2.
3.
7/20/2015
15
Oriented Search Principle (Cont.)
Illustration –
7/20/2015
16
Feature Extraction - Example
0
60
Observation
Sequence
60
O
X1
5 1 5 5 0 3 53 0 3 0 4 0
Y1
5 5 4 0 2 5 2 53 0 3 5 4 5
1 5 1 5 3 53 03 0 4 0 6 0
X2
Y2
7/20/2015
=
40 5
2 53 0 3 5 4 5 4 5
17
Example of Extremities
Ordinates Distribution
7/20/2015
18
Feature Extraction - Summarize
Advantages
Disadvantages


Preserve Shape of
Characters.
 Reduce Amount of
Data.
 Retrieve Dynamic
Level of Writing.
Over Sensitivity to
First Pixel.
 Significant Variance
of Segments Number.
 Significant Variance
of Segments Order.
Requires HMM Design !!
7/20/2015
19
Designing Discrete HMM
( Per Each Class )
Observations
Sequences
Create
ACM
7/20/2015
Initialize HMM
Structure
Training
HMM
20
Designing Discrete HMM ( Cont. )
Average Character Model
A Representative Model For Each
Character Defined By All Obs.
Goals

Defines the Number of States in the HMM
Structure.
 Initializes the State Transitions Distributions.
 Gives Physical Meaning to the States of the
HMM ( Each State Correspond to Segment ).
7/20/2015
21
Designing Discrete HMM ( Cont. )
Average Character Model (Cont.)
Creation Procedure :
1.
2.
3.
4.
5.
Definition of Segments Number, TH.
Applying VQ Process ( Local,Mean+Std ).
Association of Segments to ACM
Segments.( Using Distance Measure ).
Updating ACM ( According to Calculation of
Mean Value of All Grouped Segments ).
Goto Step 3 ( Unless TH Achieved ).
7/20/2015
22
Designing Discrete HMM ( Cont. )
Average Character Model (Cont.)
Distance Measure Definition
: d  min(d1 , d 2 )
d1  ( x01  x02 ) 2  ( y01  y02 ) 2  ( x11  x12 ) 2  ( y11  y12 ) 2
d 2  ( x11  x02 ) 2  ( y11  y02 ) 2  ( x01  x12 ) 2  ( y01  y12 ) 2
d1
7/20/2015
d2
23
Designing Discrete HMM ( Cont. )
Initialize HMM Structure

N = Segments On ACM.
 Si  ith ACM Segment.
(Random Selection)
Example:
{O1av , O2av ,...,ONav }
{S1 , S 2 ,.......,S N }
7/20/2015
24
Designing Discrete HMM ( Cont. )
Segments Alignment Process
Goal -
Matching Each Segment of Each
Character to the Ones of the ACM.
Training Char.
7/20/2015
ACM
25
Designing Discrete HMM ( Cont. )
Alignment Process Results
Each Observations Sequence’s Segment Is
Indexed According to ACM Closest Segment.
AP
Example : {O1} 
{S , S ,...,S }
1
1
N
{O } {S 4 , S3 ,...,S1}
2
AP
AP
{OK } 
{S2 , S1 ,...,S5}
7/20/2015
O [ 4KxT ]
S [ KxN ]
26
Designing Discrete HMM ( Cont. )
Initialize HMM Parameters
According to Matrix S :
Number of Si ' s in First Column of S
 Initial {  }   i 
K
Transitions from Si to S j
 Initial { A }  aij 
Transitions from Si to Any State
Observations VK in S j
 Initial { B } 
b j (k ) 
Total Observations in S j
7/20/2015
27
Designing Discrete HMM ( Cont. )
Training HMM Procedure
Training
Data {O}
Calculate
Q  {q1 , q2 ,...,qT }
Max{P(Q | O,  ) }
( Viterbi Path )
7/20/2015
Adjust HMM
Parameters
  {A, B, }
28
Designing Discrete HMM ( Cont. )
Training The HMM ( Cont.)
?
Dist ( ,  )  Th
Yes
 
No
Save  
7/20/2015
29
Classification System
HMM
‫א‬
Skeleton
Char.
P(O |  )
HMM
Feature
‫א‬
MLP
‫ ב‬P(O |  )
Extraction
Image
‫ת‬
HMM
‫ת‬
7/20/2015
P(O |  )
30
Improve Classification Performance
Classification Decision Calculation Using the
Forward-Backward Procedure :
N
P(O |  )   T (i )
i 1
 t (i )  P(O1 ,.....,Ot , qt  Si |  )
N2 T Calculations Vs.
7/20/2015
2T NT Calculations !!
31
Improve Classification ( Cont. )
1. Adding the Probability That an Observations
Sequence ends on a given State Si :
N
P(O |  )   (T (i )   f (i))
i 1
 f (i)  P(qT  Si )
7/20/2015
32
Improve Classification ( Cont. )
2. Adding the Probability That the Observations
Sequence Length is T :
N

P(O |  )   ( T (i )   f (i ))  n(T )
 i 1

n( x)  P(T  x)
7/20/2015
33
Improve Classification ( Cont. )
3. Adding “Punishment” Probability to Each
State, Si ,of the Model ,That Doesn’t Take
Place in a Sequence Of Observations :
( Improve Discrimination [ ‫ב‬,‫ ) ] ר‬.
 N

P (O |  )   ( T (i )   f (i ))  n(T )   (1  c(i ))
 i 1

i , S{qt* }
c(i )  P( S i  {q j }) , 1  i  N , 1  j  T
{qt*}
 Viterby Optimal State Sequence.
7/20/2015
34
Improve Classification ( Cont. )
4. Adding the Probability Distribution of the
“Discrete” Aspect Ratio ( Width / Height ).
( Avoid Discrimination Problem Due to the
Normalization of Character Dimensions ).
N

P (O |  )   ( T (i )   f (i ))  n(T )   (1  c(i ))  r ( R )
 i 1

i , S { qt* }
r ( X )  P ( r  X ) , X  0.2 : 0.1 : 2
7/20/2015
35
Classification ( DB Properties )




Performed on NIST3 Database.
1579 of Each Class English
Unconstrained Uppercase Handwritten
Characters.
1324 – Training Set, 235 – Testing Set.
Total Number of Characters : 41,054.
( 1579 X 26 ).
7/20/2015
36
Classification ( Total Results )
Improve
Number
Recog.
Rate
(%)
7/20/2015
None
1
2
78.6 79.6 80.3
3
4
83.2
84.4
37
Classification ( Results Per Char. )
7/20/2015
38
My Goals
Implementation of those
Principles (on MATLAB).
Using My “Small”
Database.
Having Same Recognition
Rate.
7/20/2015
39