Transcript CAP6938 Neuroevolution and Developmental Encoding Overview of NEAT and CPPNs Dr. Kenneth Stanley August 28, 2006

CAP6938
Neuroevolution and
Developmental Encoding
Overview of NEAT and
CPPNs
Dr. Kenneth Stanley
August 28, 2006
Why an Overview?
•
•
•
•
Preparation for declaring projects
Get an idea what is possible
Sufficient background without details
Technical details coming later in semester
NEAT: NeuroEvoluton of
Augmenting Topologies
• Evolutionary algorithm applied to evolving
artificial neural networks (ANNs)
• Evolution (i.e. genetic algorithms) is a way
to optimize ANN topology and weights
• NEAT evolves increasingly complex ANNs
• Able to solve difficult control and decisionmaking tasks
Evolution:Survival of the Roundest
Gen 1
Select as parents
Gen 2
Select as parents
Gen 3
Champ!
NeuroEvolution
• Evolve ANNs
• Evaluated for performance on a task
–
–
–
–
–
–
–
Controlling a vehicle or robot
Drawing a picture
Recognizing an image
Warning about danger
Playing a board game
Playing a video game
Playing music
• The fittest survive and reproduce
ANNs can Sense and React
NEAT Features
• NEAT is unique: Networks complexify over
evolution
Crash Warning Champ
• So they can find the right complexity for the task
• NEAT solves problems with systems of diverse
networks using historical marking and speciation
• A first step in evolving complex systems
NEAT Applications
• NEAT makes new applications possible
and has set performance records
Double Pole Balancing Record
Vehicle Warnings
Keepaway Record
NERO: Real-time Neuroevolution
in a Video Game
Robot Duel
Go
Hopper
Not Just About Control and
Classification
Interactive Picture Evolution
Interactive Drum Pattern Evolution
Interactive Particle Effect Evolution
Interactive Chord Progression Evolution
Guitar Effect-Pedal Emulation
Non-neural NEAT
• ANNs aren’t the only thing that can
complexify
• Anything with arbitrary structure can
potentially be evolved
– Cellular Automata
– Lego Structures
– FSMs
– L-systems
L-System Plant Geometry Generator
by Hung-Wen Chen
What Can’t NEAT Do?
• “Regular” NEAT is a direct encoding
• It doesn’t encode regularities or symmetries
• Regularity and symmetry are believed to be
essential to representing and searching complex
structures: gene reuse
• Giant complex structures in nature are
composed of patterns
• Patterns have regularities
• Indirect encodings capture such information
efficiently
Indirect Encoding
• Most are developmental encodings
– Based on embryo to adult development
– “Artificial Embryogeny”
– This is how encoding works in nature
• Many have been proposed
• None close to natural complexity
Proposed Indirect Encodings
What’s missing?
• None implement complexification as in
NEAT: maybe genes can be added and
marked like in NEAT
– For example: L-system NEAT?
• Something’s still missing…
A Recent Discovery
• Drawn by a NEAT-evolved neural
network!?
Compositional Pattern Producing
Networks (CPPNs)
• Networks of functions composed to
produce patterns with regularities
• An abstraction of natural development
without developmental simulation
• Complexifying repeating patterns,
regularities, symmetries, repetition with
variation, etc.
CPPN-NEAT
•
•
•
•
A new kind of indirect encoding
Elegantly captures regularities
Convert patterns into phenotypes
An entirely new research area
Exploiting Regularity Without Development by Kenneth O. Stanley, To appear in: Proceedings of the AAAI Fall
Symposium on Developmental Systems. Meno Park, CA: AAAI Press, 2006 (8 pages)
Music is a Pattern over Time
• A CPPN is a pattern generator that can
produce repetition with variation, just like
music
• A CPPN can produce several related
patterns from an underlying simpler
pattern (multiple instruments)
• Music generation is a promising
demonstration
Homework for 8/30/06
• Neural network weight-learning algorithms
Fausett pp. 39-80 (in Chapter 2)
Fausett pp. 289-316 (in Chapter 6)
Chapter 1 of Sutton (on RL): Online Book ch.1
Optional: Kaelbling RL survey paper
• Remember proposals/partners due 9/11