Transcript CAP6938 Neuroevolution and Developmental Encoding Leaky Integrator Neurons and CTRNNs Dr. Kenneth Stanley October 25, 2006

CAP6938
Neuroevolution and
Developmental Encoding
Leaky Integrator Neurons and
CTRNNs
Dr. Kenneth Stanley
October 25, 2006
Artificial Neurons are a Model
 n

• Standard activation model H j     xi wij 
 i 1

• But a real neuron doesn’t have an activation level
– Real neurons fire in spike trains
Wolfgang Maass, http://www.tu-graz.ac.at/igi/maass
– Spikes/second is a rate
– Therefore, standard activation can be thought of as outputting a
firing rate at discrete timesteps (i.e. rate encoding)
What is Lost in Rate Encoding?
• Timing information
• Synchronization
• Activity between discrete timesteps
30 Neurons Firing in a monkey’s striate cortex
From Krüger and Aiple [Krüger and Aiple, 1988].
Reprinted from www.igi.tugraz.at/ maass/123/node2.html
Spikes Can Be Encoded Explicitly
•
•
•
•
•
Leaky integrate and fire neurons
Encode each individual spike
Time is represented exactly
Each spike has an associated time
The timing of recent incoming spikes determines
whether a neuron will fire
• Computationally expensive
• Can we do almost as well without encoding
every single spike?
Yes: Leaky Integrator Neurons
(CTRNNS; Continuous Time
Recurrent Neural Networks)
• Idea: Calculate activation at discrete steps
but describe rate of change on a
continuous scale
• Instead of activating only based on input,
include a temporal component of
activation that controls the rate at which
activation goes up or down
• Then the neuron can react to changes in a
temporal manner, like spikes
Activation Rate Builds and Decays
Input to neuron
Activation
Level
(i.e. spike
rate)
Output over time
time
• Incoming activation causes the output
level to climb over time
• We can sample the rate at any discrete
granularity desired
• A view is created of temporal dynamics
without full spike-event simulation
What is Leaking In a Leaky
Integrator?
Leaking activation level
(membrane potential)
Activation
Level
(i.e. spike
rate)
time
• The neuron loses potential at a defined rate
• Each neuron leaks at its own constant rate
• Each neuron integrates at the same constant
rate as well
Leaky Integrator Equations
Leak
• Expressing rate of change of activation
level:
• Apply Euler Integration to derive discretetime equivalent
• Expressing current activation in terms of
activation on previous discrete timestep:
Real time
Between
steps
Equations from: Blynel, J., and Floreano, D. (2002). Levels of dynamics neural controllers. In
Proceedings of the Seventh International Behavior on From Animals to Animats, 272–281.
What Can a CTRNN Do?
• With the right time constants for each
neuron, complex temporal patterns can be
generated
• That is, the time constants are a new
parameter (inside nodes) that can evolve
• More powerful than a regular RNN
• Capable of generating complex temporal
patterns with no input and no clock
Pattern Generation for What?
• Walking gaits with no input!
Evolution of central pattern generators for
bipedal walking in a real-time physics
environment
T Reil, P Husbands - Evolutionary
Computation, IEEE Transactions on, 2002
Reil and Husbands Went on to
Found the Company NaturalMotion
Pattern Generation for What?
• Salamander walking gait
Ijspeert A.J.: A connectionist central
pattern generator for the aquatic
and terrestrial gaits of a simulated
salamander, Biological Cybernetics,
Vol. 84:5, 2001, pp 331-348.
• Wing flapping
Evolution of neuro-controllers for
flapping-wing animats - group of 2 »
JB Mouret, S Doncieux, L Muratet, T
Druot, JA … - Proceedings of the
Journees MicroDrones, Toulouse, 2004
Maybe Good for Other Things with
Temporal Patterning
• Music?
• Tasks that we typically do not conceive in
terms of patterns?
• Learning tasks (better than a simple RNN?;
Blynel and Floreano 2002 paper)
• Largely unexplored
• How far away from the benefits of a true
spiking model?
Leaky NEAT
• There is a rough, largely untested leakyNEAT at
the NEAT Users Group files section:
– http://groups.yahoo.com/group/neat/files/
– Introduces a new activation function and new time
constant parameter in the nodes
• A new leaky-rtNEAT will soon be available too
• The topology of most CTRNNs in the past was
determined completely by the researcher
Next Topic: Non-neural NEAT,
Closing Remarks on Survey Portion of
Class
• Complexification and protection of innovation in non-neural
structures
• Example: Cellular Automata neighborhood functions
• What have we learned, what is its significance, and where does the
field stand?
Reading: Mitchell Textbook pp. 44-55 (Evolving Cellular Automata)
think about: How would NEAT apply to this task?