Transcript Sparse distributed memory for ‘conscious’ software agents

Soft Computing @ Yonsei
Cognitive Systems Research 2006
Sparse distributed memory
for ‘conscious’ software agents
Ashraf Anwar, Stan Franklin
SCALB
Tae-min, Jung
2015-07-20
Software Agent / Tea-min Jung
1
Soft Computing @ Yonsei
Outline
•
Introduction
– Sparse distributed memory
– CMattie
•
•
Autonomous, cognitive, and ‘conscious’ agents
Sparse Distributed Memory
– SDM as a control structure for autonomous agents
• The SDM agent
• Modeling the world using SDM
• Including action in the world model
– Some psychological aspects of SDM
• Cued behavior
• Learning to act
•
•
•
•
CMattie, a ‘conscious’ clerical agent
SDM in CMattie
Results and statistics
Conclusions & Future research
2015-07-20
Software Agent / Tea-min Jung
2
Soft Computing @ Yonsei
Introduction
•
Sparse distributed memory (SDM)
– Content addressable memory developed by Kanerva (1988a)
– Useful model of human associative memory
•
Using Agent Domain
– Cognitive software agents
– Conscious software agents
•
CMattie
– Clerical software agent
– Uses SDM for associative memory
2015-07-20
Software Agent / Tea-min Jung
3
Soft Computing @ Yonsei
Introduction : CMattie
•
Feature
– Clerical Software Agent
– Human-like Agent
– CMattie’s ‘consciousness’ mechanism follows global workspace theory
•
SDM
– Extended version of SDM
– Build, Store, Retrieve
• Percepts, Behaviors, Emotions
2015-07-20
Software Agent / Tea-min Jung
4
Soft Computing @ Yonsei
Autonomous, cognitive, and ‘conscious’ agents
•
Autonomous Agent
– Sensing
• Situated in an environment
– Action
• Can affect environment
•
A cognitive agent
– One of Autonomous agents
– More Cognitive capabilities
• Problem Solving, Planning, Learning, Perception, Emotions, Etc.
•
A ‘conscious’ agent
– Extra functionality ‘consciousness’
– Global workspace theory (Baars, 1998, 1997)
– Broadcast of Information
• Recruit processes
• Can Help in handling the current novel or problematic situation
2015-07-20
Software Agent / Tea-min Jung
5
Soft Computing @ Yonsei
Sparse Distributed Memory
•
What is SDM?
– Kanerva, 1988
– Sparse allocation of storage locations
– Use Vast binary address space
•
2n Semantic space where n is both the word length and the dimension of the
address space
– The distributed nature of information storage and retrieval
•
Associative Memory
– Auto-associative
• Used to retrieve itself
• The contents and the addresses belong to the same space and are used
alternatively
– hetero-associative
• Stored in sequences
•
The word length (dimension of the space)
– Represented by one or more bits in a Boolean vector or binary string of
length n
2015-07-20
Software Agent / Tea-min Jung
6
Soft Computing @ Yonsei
Sparse Distributed Memory (cont’d)
•
Content addressable
– Items in memory can be retrieved by using part of their contents as a cue
– Do not having to know their actual addresses in memory as in traditional
computer memory.
•
Boolean spaces
– The set of all Boolean vectors (points) of some fixed length, n, called the
dimension of the space
•
Hamming Distance
– Boolean geometry uses a metric
– distance between two points is the number of coordinates at which they
differ
Example : d((1,0,0,1,0), (1,0,1,1,1)) = 2
2015-07-20
Software Agent / Tea-min Jung
7
Soft Computing @ Yonsei
Sparse Distributed Memory (cont’d)
2015-07-20
Software Agent / Tea-min Jung
8
Soft Computing @ Yonsei
Sparse Distributed Memory (cont’d)
2015-07-20
Software Agent / Tea-min Jung
9
Soft Computing @ Yonsei
SDM as a control structure for autonomous agents
•
The SDM agent
– Learning and Adaptation
• Autonomous agent interacts with its environment, and records its interaction
•
Patterns
– Binary vectors
– Large Size
– Usage
• Address
• Datum
•
Addressing the memory
– Need not be exact
– Address patterns use write address attract
2015-07-20
Software Agent / Tea-min Jung
10
Soft Computing @ Yonsei
Modeling the world using SDM
•
Subjective experience
– Merge internal and external ‘pictures’
•
Internal
– Scan our surrounding for overall cues and fill in much of the detail from the
internal model
•
External
– Sensory Input
2015-07-20
Software Agent / Tea-min Jung
11
Soft Computing @ Yonsei
Modeling the world using SDM
2015-07-20
Software Agent / Tea-min Jung
12
Soft Computing @ Yonsei
Modeling the world using SDM
•
The world model Update
– (i) The pattern held in the focus at time t is used to address the memory,
activating a set of memory locations.
– (ii) The response read from those locations is the memory prediction of the
sensory input at time t+1.
– (iii) If the prediction agrees with the sensory input, there is no need to adjust
the memory, and the read pattern simply becomes the contents of the focus
at time t+1.
– (iv) If the prediction disagrees with the sensory input, a third correct pattern is
computed from them, e.g. an average, and it becomes the content of the
focus at time t11. However, before it is used to address the memory at time
t11, it is written in the locations from which the faulty output was just read
(the locations selected at time t).
2015-07-20
Software Agent / Tea-min Jung
13
Soft Computing @ Yonsei
Modeling the world using SDM
•
Leaning
– Correction patterns are written into memory over time
– The memory builds a better model of the world
•
Result
– Better understanding capacity to store information
– i.e. experience.
2015-07-20
Software Agent / Tea-min Jung
14
Soft Computing @ Yonsei
Including action in the world model
•
Action
– Animals
• perform actions means learning to reproduce sequences of patterns that drive the
muscles
– Agent
• the agent’s own actions can be included in the world model by storing motor
sequences in memory in addition to sensory sequences
2015-07-20
Software Agent / Tea-min Jung
15
Soft Computing @ Yonsei
Some psychological aspects of SDM : Cued behavior
•
Assume
– Focus
• 80% for sensory input
• 20% for effectors
– Stimulus sequence <A, B, C>
– Response sequence <X, Y, Z>
•
Generated Focus
– <Aw, BX, CY, dZ>
– <AW, BX, CY, DZ>
•
Guess<A, B, K, L>
–
–
–
–
–
2015-07-20
K and L are quite different from C and D
BX will retrieve CY, and action Y will be executed
Senses will report K instead of C
Focus will be HY instead of CY (H is some combination of C and K)
DZ will not be retrieved (failure of action Z)
Software Agent / Tea-min Jung
16
Soft Computing @ Yonsei
Cued behavior (cont’d)
•
This failure can be explained as follows.
– (i) An environment monitoring system ceases to act when the proper cues are
no longer present.
– (ii) A system that monitors its own action effects, stops acting when the
effects no longer confirm the system expectations.
•
Result
– The system will initiate the ‘thought’ in the focus and then block off the
present
– The memory will then retrieve into the focus the likely consequences of the
contemplated actions
2015-07-20
Software Agent / Tea-min Jung
17
Soft Computing @ Yonsei
Some psychological aspects of SDM : Learning to act
•
Success
– How well it predicts the world
– Action correction is much harder than perceptual correction
•
Learning to act
– The System stores actions in a way that increase the likelihood of finding
good states and of avoiding bad ones
2015-07-20
Software Agent / Tea-min Jung
18
Soft Computing @ Yonsei
CMattie, a ‘conscious’ clerical agent
•
CMattie
– A ‘conscious’ software agent
– To Manage seminar announcements in the Mathematical Sciences
Department at the University of Memphis
•
Aims
– To discuss the use of SDM in CMattie
•
Work
– Send weekly seminar announcements via email to a mailing list that they
maintain
– Receive and understand email messages in natural language from seminar
organizers and attendees
– Handle their information and requests, and compose and email the weekly
department seminar announcements to their lists
2015-07-20
Software Agent / Tea-min Jung
19
Soft Computing @ Yonsei
CMattie, a ‘conscious’ clerical agent (cont’d)
•
Perception
– Understanding email messages
in natural language
•
Codelets
– perception action, emotions,
metacognition, etc.
– Small pieces of code
– Running independently
– Simple specific task to
accomplish
2015-07-20
Software Agent / Tea-min Jung
20
Soft Computing @ Yonsei
CMattie, a ‘conscious’ clerical agent (cont’d)
•
Memory
– Produce suggested actions and emotional states
– SDM
• Long term associative memory
– CBM (Case Based Memory)
• Episodic memory
•
PRs
– Perception Registers (PRs).
– When an email message comes in Selected
2015-07-20
Software Agent / Tea-min Jung
21
Soft Computing @ Yonsei
SDM in CMattie
•
•
Function of SDM in CMattie
Step 1
– Suggesting actions
• Emotional status : Incoming Percepts (Values in the perception registers)
– Previous experience
• Response to the observed values in the perception registers.
– Auto-associative SDM
• Relies on the dominance, in size, of the percepts over the behaviors and emotions
•
Step 2
– Providing for defaults for missing information in perception registers, based
on previous history and previous associations built over time.
•
Step 3
– Removal of noise and correction of errors in percepts, based on previous
history and previous associations built over time.
2015-07-20
Software Agent / Tea-min Jung
22
Soft Computing @ Yonsei
SDM in CMattie (cont’d)
•
•
The output from SDM has three distinct parts.
(i)
– One part fills a set of registers identical to the perception registers. Defaults
for missing fields are taken from here.
•
(ii)
– A second contains the suggested behavior in response to the percept
received.
•
(iii)
– The third influences CMattie emotional status based again on the percept
received.
2015-07-20
Software Agent / Tea-min Jung
23
Soft Computing @ Yonsei
Enhancements to original SDM in CMattie
•
Writing important or emotionally involved percepts into memory
multiple times
– emphasize the specific percept by giving it more repetitions internally in the
memory
– It also resembles a human going over emotional situations again and again in
his or her mind.
•
Using error detection and correction
– Allow for original information retrieval was considered and tested
– The Problem with this approach is that special encoding for various words
need to be considered to allow for enough hamming distance between
various words
– Word encoding
2015-07-20
Software Agent / Tea-min Jung
24
Soft Computing @ Yonsei
Tuning of SDM parameters in CMattie
•
•
•
•
•
•
Address space dimension (word length): 736
Number of hard locations: 3000
Critical distance for divergence: 155
Access radius for read / write: 331
Convergence distance: 147
Maximum number of reading iterations: 10
2015-07-20
Software Agent / Tea-min Jung
25
Soft Computing @ Yonsei
Enhancements to original SDM in CMattie
•
Use of hashing to encode various field values into uniform
– Fixed length hash keys.
– Example : All various length speaker names are hashed using fixed length
– Avoid having to deal with arbitrary long names.
•
Assigning weight or precedence to certain fields in the PRs
–
–
–
–
2015-07-20
Adding extra word length for them
Semantic space
Making it more capable of retrieving missing fields when used as a partial cue
Implementation-dependent
Software Agent / Tea-min Jung
26
Soft Computing @ Yonsei
Run Trace
•
Example
– Email message is a request to add a name to the mailing list
•
Step1
– The incoming email message (sensory input) is analyzed and understood by
the perception module
– Moved into the appropriate perception registers (PRs).
– PRs
• ‘EMPTY’, ‘EMPTY’, ‘EMPTY’, ‘EMPTY’, ‘EMPTY’, ‘EMPTY’, ‘EMPTY’, ‘EMPTY’, ‘EMPTY’,
‘addToListMessage’, ‘[email protected]’, ‘EMPTY’ ‘EMPTY’ ‘EMPTY’ ‘EMPTY’
‘EMPTY’ ‘EMPTY’
•
Step 2
– The contents of the PRs are passed to the input foci of SDM and CBM
•
Step 3
– SDM and CBM go into reading cycles. Note that the behavior and emotion
fields are EMPTY’ when reading
2015-07-20
Software Agent / Tea-min Jung
27
Soft Computing @ Yonsei
Run Trace (cont’d)
•
Step 4
– Upon convergence, the output from SDM and CBM is returned to the
appropriate registers in their respective foci
– Suggestions for the behavior and its activation
– Let us assume that the SDM output behavior is ‘Update Seminar List
Recipients’ with high enough activation e.g. 0.9, to make it execute
•
Step 5
– ‘Consciousness’ plays its role and the content of the focus including some
output from memory, gets broadcast
– This makes the information available to the behavior net as well as the
emotion module
•
Step 6
– The emotion module extracts the emotion part of the broadcast (originally
from the memory output)
2015-07-20
Software Agent / Tea-min Jung
28
Soft Computing @ Yonsei
Run Trace (cont’d)
•
Step 7
– The behavior codelets extract the behavior part of the broadcast (originally
from the memory output)
•
Step 8
– The emotion module determines the new emotional status for the agent and
puts it into the input focus overwriting the ‘EMPTY’ placed by perception
•
Step 9
– The behavior net determines the new behavior activation for the agent and
puts it into the input focus overwriting the ‘EMPTY’ placed by perception
•
Step 10
– SDM and CBM go into a write cycle to update the agent memory with the
new input percept along with the corresponding emotion and behaviors
2015-07-20
Software Agent / Tea-min Jung
29
Soft Computing @ Yonsei
Results and statistics
2015-07-20
Software Agent / Tea-min Jung
30
Soft Computing @ Yonsei
Conclusions
•
SDM has proven to be a successful
tool for associative memory in
‘conscious’ software agents
2015-07-20
Software Agent / Tea-min Jung
31
Soft Computing @ Yonsei
Future research
•
•
Associations based upon relatively small cues, as we human do.
For SDM to converge
– Sufficiently large portion of a previously written word must be presented to
the memory as an address.
•
Future Work
– We are currently working on a new version of SDM that addresses this
problem
– If successful, this new technique should solve one of the major open
problems for the use of SDM as associative memory
2015-07-20
Software Agent / Tea-min Jung
32