Transcript Introduction to Cognitive Science

Computational Cognitive
Modelling
COGS 511-Lecture 3
SOAR and ACT-R
17.07.2015
COGS 511
1
Related Readings
Course Pack:
• Anderson et al.’s (2004) An Integrated Theory of the Mind
• Lehman et al. (2006). A Gentle Introduction to SOAR

See also (optional)
 Anderson and Lebiere (2003). The Newell Test for a Theory of
Cognition. Behavioral and Brain Sciences 26, 587-640.
 Anderson J.R. (2007) How Can the Human Mind Occur in the Physical
Universe? OUP. (Latest book on ACT-R as a unified theory)
 Chapter 2 of Polk and Seifert
 Anderson and Lebiere (1998) Atomic Components of Thought.
Lawrence-Erlbaum (The previous one..)
Some slided are adopted from Lebiere’s Introductory tutorial, see http://act-r.psy.cmu.edu
Thanks to Evgueni Stepanov for letting me use his drawings;see Masters theses by Stepanov and Özyörük.
17.07.2015
COGS 511
2
SOAR
States, Operators and Reasoning
 Descendant of General Problem
Solver (1963)
 Software is in now version 9.0.1
with 9.1 and 9.2 betas...(as of
March, 2010)

17.07.2015
COGS 511
3
SOAR’s Cognitive Design
Principles






Be goal oriented.
Use symbols and abstractions.
Be flexible and exhibit adaptive
behaviour.
Learn from experience and environment.
What aspects of cognitive behaviour is
missing here?
Unrealistic aspects: e.g. Forgetting in
SOAR results when new associations
prevent old ones from firing.
17.07.2015
COGS 511
4
17.07.2015
COGS 511
A gentle intro to SOAR-1999
5
Soar as a Production
System



All long term memory used to be composed of
productions. Now semantic and episodic memory
exists although there is a heavy weight towards
procedural knowledge.
Conflict resolution – all satisfied productions put
their contents to working memory, so rules are
allowed to fire in parallel, but at the level of
operator proposals only.
Rules do not take actions by themselves, they
propose actions (i.e. operators)
17.07.2015
COGS 511
6
A gentle intro to SOAR-1999
17.07.2015
COGS 511
7
Elements of SOAR




Problem Spaces – Restricting the arena of
action to what is relevant domain
knowledge
Goals – Knowledge of objectives
Operators – Knowledge about actions
States - An Internal Representation of a
situation

Working Memory Elements-Feature and
Values, where values of features may be
features themselves
17.07.2015
COGS 511
8
Problem space
…
S1 f1 v1
f1 v2
goal
S12
f1 v1
f1 v2
operator
S91 f1 v1
f1 v2
S2 f1 v1
f1 v2
initial S0 f1 v1
state
f1 v2
S3 f1 v1
f1 v2
goal state
S30 f1 v1
f1 v2
S80 f1 v1
f1 v2
…
goal state
(Lehman et al., 2006)
17.07.2015
COGS 511
9
SOAR’s Decision Cycle


Perception is asynchronous wrt to decision
Recognize/Elaborate


Decide




Match Working Memory against “if”s in LTM; parallel
firings- ends with quiescence-all the knowledge that
can be elicited in the current context is in WM
Evaluate preferences – symbolic (better, best,
acceptable, worst, prohibit) or numeric; apply the
chosen operator
Act
A single operator per decision cycle
~~100msec (typical decision cycle); comparison
w. Behavioural data in terms of decision cycles
17.07.2015
COGS 511
10
17.07.2015
COGS 511
A Gentle Intro to SOAR (1999)
11
Impasses


If a successful decision cannot be made,
an impasse arises, resolving this impasse
becomes a subgoal of the original goal
Predetermined set of impasses



Operator tie impasse – more than one
acceptable preference
No change impasse- A new operator can not
be selected
Conflict impasse – Conflicting preferences
17.07.2015
COGS 511
12
Chunking



Learning associations via examining the
preimpasse environment and the solution
to the impasse
Other learning styles used to be built on
chunking
Now, reinforcement learning



Operates on operator selection rules
Learns rules that tests features
Learns expected rewards
17.07.2015
COGS 511
13
Perception/Motor Interface
fastserve
serve
Working
Memory
…
curvedserve
Resolve
tie
Curved
serve
Long-term Memory
a1…a5, a7…a9
Adapted from (Lehman
et al., 2006)
Soar Syntax
If I exist, then write “Hello World” and
halt.
sp {hello-world
(state <s> ^type state)

(write “Hello World”)
(halt)}
17.07.2015
COGS 511
15
SOAR Content Theories and
Applications






R1 SOAR – an Expert systems that configures
computer systems
Designer-SOAR- algorithm generation from
specification
NL-Soar and LG-Soar – About natural language
comprehension and production
Nasa Test Director – NTD- Soar
TacAir Soar- Soar MOUTBOT – military behaviour
models (TacAir Soar > 8000 rules)
Hybrids: EASE – Elements of SOAR, ACT-R and
EPIC
17.07.2015
COGS 511
16
Advantages and
Disadvantages of SOAR
Parsimony – single long term
memory, single learning rule (also a
disadvantage?)-has changed in Soar
9- Addition of episodic and semantic
memory + reinforcement and
concept learning
 Symbolic (also a disadvantage?utility, frequency etc. is not
accounted for-also revised recently)

17.07.2015
COGS 511
17
Some of the Recent
Advances to SOAR





Soar Technology – still freely available.
New interface, better editing and
debugging facilities and kernel allowing
better interaction with other agents and
applications
Development of experimental simulation
agents
Implementation of reinforcement learning
and preliminary coverage of emotions
http://sitemaker.umich.edu/soar
17.07.2015
COGS 511
18
Some Further
Developments
Port of NLSOAR to Ver. 9 with
minimalist syntax and access to
Wordnet and corpora...
 SOAR in Java
 iSOAR on iPhone
 SOAR and Robotics
 Bayesian – Causal hybrids into
SOAR

17.07.2015
COGS 511
19
History of the ACT-R framework
Adaptive Control of Thought-Rational
Predecessor
HAM
Declarative memory only
Theory versions
ACT-E
ACT*
ACT-R
Added productions
Learning and subsymbolic part
Further development 1993+
ACT-R 2.0
ACT-R 3.0
ACT-R 4.0
ACT-RN
ACT-R/PM
ACT-R 5.0
ACT-R 6.0
(1993)
Implementations
17.07.2015
Neural-network implementation
EPIC’s perception-motor added
PM integration, theory updates
Current version
COGS 511
20
ACT-R Models by Topic Area
I. Perception & Attention
1. Psychophysical Judgements
2. Visual Search
3. Eye Movements
4. Psychological Refractory Period
5. Task Switching
6. Subitizing
7. Stroop
8. Driving Behavior
9. Situational Awareness
10. Graphical User Interfaces
II. Learning & Memory
1. List Memory
2. Fan Effect
3. Implicit Learning
4. Skill Acquisition
5. Cognitive Arithmetic
6. Category Learning
7. Learning by Exploration
and Demonstration
8. Updating Memory &
Prospective Memory
9. Causal Learning
III. Problem Solving & Decision Making
1. Tower of Hanoi
2. Choice & Strategy Selection
3. Mathematical Problem Solving
4. Spatial Reasoning
5. Dynamic Systems
6. Use and Design of Artifacts
7. Game Playing
8. Insight and Scientific Discovery
IV. Language Processing
1. Parsing
2. Analogy & Metaphor
3. Learning
4. Sentence Memory
V. Other
1.
2.
3.
4.
5.
6.
7.
Cognitive Development
Individual Differences
Emotion
Cognitive Workload
Computer Generated Forces
fMRI
Communication, Negotiation,
Group Decision Making
http://act-r.psy.cmu.edu/publications/index.php
Visit http://act.psy.cmu.edu/papers/ACT-R_Models.htm link.
ACT-R Architecture
Intentional Module
Declarative Module
Goal Buffer
Retrieval Buffer
Matching
core
production
system
productions
Selection
Execution
perceptualmotor
system
Perceptual-Motor Buffers
Perceptual-Motor Modules
External World
(Anderson et. al, 2004)
17.07.2015
COGS 511
22
ACT-R Buffers
1. Goal Buffer
-represents where one is in the task
-preserves information across production cycles
2. Retrieval Buffer
-holds information retrieval from declarative memory
-seat of activation computations
3. Visual Buffers
-location
-visual objects
4. Auditory, Vocal, and Manual Buffers
Modules and Core Production System communicate via buffers. A
buffer can hold only one unit of information at a time.
17.07.2015
COGS 511
23
Basic Elements of ACT-R




Chunks are schema-like units of declarative
knowledge. They have types and slots which can
contain chunks themselves as values.
Chunks can be created by productions or
encodings in buffers.
Productions are basic units of procedural
knowledge.
Any module can create/use chunks. Not all
chunks are in Declarative Memory but Decl.
Memory chunks cannot be changed from within
a production; chunks merge into Decl Memory
from buffers (ACT-R 6.0)
17.07.2015
COGS 511
24
Productions
Modules operate in parallel and
asynchronously but one production
fires at each cycle. Production cycle
is approximated at 50 ms.
 Productions can recognize
information in buffers, can make
requests to buffers, and update
them.

17.07.2015
COGS 511
25
ACT-R: Knowledge Representation
DOG
animal
fact1
class
IF
the goal is to check classification
and the question was received
and the animal is dog
and the kind to check is mammal
THEN
try to remember
whether there is a fact that
animal dog is
classified as a mammal
and go on to the next step.
17.07.2015
MAMMAL
fact1
isa category-fact
animal DOG
class MAMMAL
(p check-category
=goal>
isa
classification
state “questioned”
animal dog
kindof mammal
==>
+retrieval>
isa
category-fact
animal dog
class mammal
=goal>
isa
classification
state “requesting”)
Courtesy of Stepanov,E.
COGS 511
26
Attending to a Word in Two Productions
(P find-next-word
=goal>
ISA
comprehend-sentence
word
nil
==>
+visual-location>
ISA
visual-location
screen-x lowest
attended nil
=goal>
word
looking
)
(P attend-next-word
=goal>
ISA
comprehend-sentence
word
looking
=visual-location>
ISA
visual-location
==>
=goal>
word
attending
+visual>
ISA
visual-object
screen-pos =visual-location
)
 no word currently being processed.
 find left-most unattended location
 update state
 looking for a word
 visual location has been identified
 update state
 attend to object in that location
See ACT-R Tutorial for full description
Subsymbolic ACT-R

Chunk retrieval depends on






Base level activation, which rises and falls acc. to
practice and delay
Contextual activation, association strength with slots of
the current goal and attentional weighting (depends on
fan)
(partial) matching to retrieval specifications
Noise
A chunk will be retrieved only if its activation is
over a threshold.
The lower the activation of a chunk, the longer it
takes to retrieve it (latency).
17.07.2015
COGS 511
28
Subsymbolic ACT-R

Which production is selected to fire
depends on its utility: past
successes and failures of that
specific production for the
achievement of the current goal, the
current goal’s importance and an
estimate of the cost of the
production given in seconds.
17.07.2015
COGS 511
29
ACT-R: Subsymbolic
CHUNK i
Bi
=goal>
isa classification
Wj animal
DOG
Wj kindof
MAMMAL
state
Sji
animal S
ji
DOG
Sji class
class
MAMMAL
questioned
+retrieval>
isa category-fact
Pk animal
DOG
Pk class
MAMMAL
Production Utility Equation
Activation Equation
17.07.2015
fact1
isa category-fact
animal DOG
class MAMMAL
Mki
Ui  PiG  Ci  
Ai  Bi  WjSji  PkMki  1   2
j
COGS 511
k
Courtesy of Stepanov,E.
30
Parameters
Default values of parameters are
either taken from empirical data or
are working approximations induced
from a number of models (e.g.
decay 0.5)
 Issue: All parameters can be set,
but should you?

17.07.2015
COGS 511
31
Learning in ACT-R

Production Compilation: Successive
productions are built into a single
production that has the effect of
both except when perceptual-motor
dependencies are in effect.
Subsymbolic values are updated
accordingly.
17.07.2015
COGS 511
32
Recent Developments

Emphasis on finding neural anchors for
the concepts in the ACT-R model:




to acquire new sources of data to guide the
development of the theory eg via predictions
of BOLD functions in fMRI.
Integration of ACT-R with a computational
neuroscience toolkit (Nengo)
Decision Tree Learning for production
matching
Threaded Cognition – managing a set of
goals
17.07.2015
COGS 511
33
ACT-R 6.0




Mostly a reimplementation but the same
theory.
Uniform and better module-buffer
structure; a better vision module
Support for multiple models; faster than
ACT-R 5.0 etc
New utility learning mechanism via
temporal difference reinforcement
learning
17.07.2015
COGS 511
34
Other Recent Applications
ACT-R and Semantic Web
Integration
 ACT-R on a Robot
 Module for competitive and
sequential tasks: Stroop, Picture
Word Interference...
 Other modules in development:
temporal, metacognitive, spatial...

17.07.2015
COGS 511
35
Newell Test for a Theory of
Cognition





Flexible behaviour
Real time
performance
Adaptive
behaviour
Vast Knowledge
Base
Dynamic
Behaviour
17.07.2015







Knowledge
Integration
Natural Language
Learning
Development
Evolution
Consciousness
Brain Realization
COGS 511
36
Some Further General
Developments

MindModeling@Home project –
A volunteer computing project
 3-D Brain (ACT-R); situated language
and active vision (ACT-R); fatigue
research (ACT-R)

17.07.2015
COGS 511
37
Lecture 4

Connectionist and Dynamic Modelling
Paradigms
• Readings: McLeod et al. Chaps 1,5,7
• Eliasmith. The Third Contender in Thagard,
Chap. 13
HW is to be posted next week – do
tutorials (esp. 1,2 and 3 during this
week)
 See Forum activity on project readings

17.07.2015
COGS 511
38