Associative memory
Download
Report
Transcript Associative memory
Pattern Association
Introduction
Pattern association involves associating a new
pattern with a stored pattern.
Is a “simplified” model of human memory.
Types of associative memory:
Heteroassociative memory
Autoassociative memory
Hopfield Net
Bidirectional Associative Memory (BAM)
Introduction
These are usually single-layer networks.
The neural network is firstly trained to store a
set of patterns in the form s : t
s represents the input vector and t the
corresponding output vector.
The neural network is then tested on a set of
data to test its “memory” by using it to
identify patterns containing incorrect or
missing information.
Introduction
Associative memory can be feedforward
or recurrent.
Autoassociative memory cannot hold an
infinite number of patterns. Factors
that affect this:
Complexity of each pattern
Similarity of input patterns
Heteroassociative Memory
Architecture
w11
x1
.
w1j
.
wi1
.
w1m
wij
xi
. wim
wnj
w
n1
.
.
wnm
xn
y1
.
.
.
yj
.
.
.
ym
Heteroassociative Memory
The inputs and output vectors s and t
are different.
The Hebb rule is used as a learning
algorithm or calculate the weight matrix
by summing the outer products of each
input-output pair.
The heteroassociative application
algorithm is used to test the algorithm.
The Hebb Algorithm
Initialize weights to zero, wij =0, where
i = 1, …, n and j = 1, …, m.
For each training case s:t repeat:
xi = si , where i=1,...,n
yi = tj, where j = 1, .., m
Adjust weights wij(new) = wij(old) + xiyj,
where i = 1, .., n and j = 1, .., m
Exercise
Train a heteroassociative neural network to
store the following input and output vectors:
1 -1 -1 -1
1 1 -1 -1
-1 -1 -1 1
-1 -1 1 1
1 -1
1 -1
-1 1
-1 1
Test the neural network using all input data
and the following input vector: 0 1 0 -1
Autoassociative Memory
The inputs and output vectors s and t
are the same.
The Hebb rule is used as a learning
algorithm or calculate the weight matrix
by summing the outer products of each
input-output pair.
The autoassociative application
algorithm is used to test the algorithm.
Autoassociative Memory
Architecture
w11
x1
.
w1i
.
wi1
.
w1n
wii
xi
. win
wni
. wn1
.
wnn
xn
y1
.
.
.
yi
.
.
.
yn
Exercise
Store the pattern 1 1 1 -1 in an autoassociative
neural network.
Test the neural network on the following input:
1 1 1 -1
-1 1 1 -1
1 -1 1 -1
1 1 -1 -1
1111
0 0 1 -1
0 1 0 -1
0110
The Hopfield Neural Network
Is a recurrent associative memory neural
network.
Application algorithm
Exercise: Store the pattern [1 1 1 0] using a
Hopfield neural network. Test the neural
network to see whether it is able to correctly
identify an input vector with two mistakes in
it: [0 1 1 0]. Note θi=0, for i=1,..,4
Bidirectional Associative
Memory (BAM)
Consists of two layers, x and y.
Signals are sent back and forth between both
layers until an equilibrium is reached.
An equilibrium is reached if the x and y
vectors no longer change.
Given an x vector the BAM is able to produce
the y vector and vice versa.
Application algorithm
BAM Exercise
Store the vectors representing the
following patterns using a BAM:
[ 1 -1 1] with the output vector [1 -1]
[-1 1 -1] with the output vector [-1 1]
θi=0, θj=0 for i = 1,..3 and j=1..2