Transcript Associative Pattern Memory (APM)

Associative Pattern Memory (APM)
Larry Werth
July 14, 2007
Introduction and Background of APM
•
•
•
•
•
Human Associative Pattern Memory
Computer Implemented APM
Basis for Two Successful Startup Companies
Six Patents Granted and Others Pending
Successful Implementation of NKS
Objective of My Presentation
•
•
•
•
Describe the APM Concept & Implementation
Describe its Advantages / Features
Identify Types of Applications
Describe its Current Status and Future Goals
Origin of Concept
•
•
•
•
Randomly Connected Neural Network Models
States Sequence Terminates in a Cycle
Randomly Map Each State to an Input Pattern
Sampled Pattern Value & Current State
Determine Next State
• The Ultimate Cycle Represents the Input
Pattern
• Cycles Form the Basis of the APM
Cycle Properties
Randomly Connected DFA’s
Total States (N)
100
1,000
10,000
100,000
1,000,000
10,000,000
100,000,000
1,000,000,000
Expected#
Terminal
States(S)
12
40
125
396
1,253
3,963
12,533
39,632
Expected#
Number
Cycles(C)
3
4
5
6
8
9
10
11
Expected#
Transition
States(T)
7
20
63
198
627
1982
6267
19817
Fraction
Terminal
States(F)
.12
.040
.0125
.00396
.001253
.000396
.0001253
.0000396
Conceptual Implementation of APM
State Array
Train Pattern
(Write to Cycle Addresses)
Pattern Address
Current
State
Address
Input
Pattern
Array
Next
State
Address
Response Array
Next State Array (Value = 0)
Pattern Value
Next State Array (Value = 1)
State Array:
Filled with Random Pattern Addresses
Next State Arrays : Filled with Random State Addresses
Response Array: Assigned Responses to Patterns
Respond to Pattern
(Read From Cycle Addresses)
Solution to Multiple Cycles
• Introduce a Refractory Period
• A State Can Not Occur Again Until After a
Specified Number of Steps
• Establishes a Minimum Cycle Length
• Assures One Cycle Per Input Pattern
Independent of Initial State
• Input Pattern is Represented By a Single
Sequence of Random Addresses in Memory
Minimum Cycle Length Example
• Number of States: 1,000,000
• Minimum Cycle Length: 3,700
• Probability of a Second Cycle of 3,700 in
Length: 1 in 1,000,000
Based on the probability of not picking one of
3700 in 1,000,000 after 3700 tries.
Response/Recognition Capacity
• During Training Desired Responses are Written
to Cycle Addresses in Response Memory
• Problem: Response Memory Fills UP Quickly
• Any Cycle Address has Memory of Previous
Input Sample Values
• Do Not Need to Use All Cycle Addresses
• Solution: Vertical Sensors
Vertical Sensor Cycle Detection
Vertical
Sensors
Detect
Presence
Absence
of Cycle
Upper
Memory
Plane forms
New Input
Pattern
Based on
Sensor Status
State Addresses
Plane With Cycle
Vertical Sensor Implementation
•
•
•
•
•
•
Number of States: 1,000,000
Minimum Cycle Length: 3,700
One of 270 Addresses are in Cycles
Vertical Sensor Field Size: 135
Probability Field Contains Cycle Address: .5
Vertical Sensor Determines Bit Status of Hash
Values that Addresses Response Memory
Fuzzy Hash
• Similar Input Patterns Produce Similar Cycles
• Similar Input Patterns Generate the same or
Similar Hash Codes
• Multiple Independent Hash Codes are
Generated By One Cycle (One Input Pattern)
• A Voting Mode For Response Identification
Contributes to Fuzzy Recognition
Advantages of Using Cycles
•
•
•
•
Creates a Fuzzy Hash
Simple and Fast Implementation
Common Language for Different Pattern Types
Spatial and Temporal Integration to Form New
Higher Level Input Patterns
• Automatic Segmentation of Time Varying
Patterns
Applications
• Actual Applications: Hand Printed Character
Recognition, Machine Vision, Video
Compression, Financial Pattern Forecasting
• Signal Processing – Vector Quantization
• Video Surveillance – Smart Cameras
• Video Object Tracking
• Stereo Vision
Current Status and Objectives
• Software Library Written in C/C++
• Objective: General Purpose Tool for Pattern
Recognition Development
• Looking for a Business Partner
• Software Will be Available on Our Web Site
www.netwerth.net