Transcript Slide 1

Brain Awareness Week at the European Parliament
How our brain works:
Recent advances in the theory of brain function
1
“Objects are always imagined as being present in the field of
vision as would have to be there in order to produce the same
impression on the nervous mechanism” - Hermann Ludwig
Ferdinand von Helmholtz
Geoffrey Hinton
From the Helmholtz machine to
the Bayesian brain and selforganization
Thomas Bayes
Richard Feynman
Hermann Haken
2
temperature
What is the difference between a
snowflake and a bird?
Phase-boundary
…a bird can avoid surprises
3
What is the difference between snowfall and a flock of birds?
Ensemble dynamics and swarming
…birds (biological agents) stay in the same place
They resist the second law of thermodynamics, which says
that their entropy should increase
4
But what is the entropy?
T
…entropy is just average surprise H   dt S (t ) S (t )   ln p( s | m)
0
s 
A 
Low surprise (I am usually here)
High surprise (I am never here)
This means biological agents must self-organize to minimise surprise. In
other words, to ensure they occupy a limited number of (attracting) states
5
But there is a small problem… agents cannot measure their surprise
s 
?
But they can measure their free-energy, which is always bigger than surprise
F (t )  S (t )
This means agents should minimize their free-energy. So what does this mean?
6
What is free-energy?
…free-energy is basically prediction error
sensations – predictions
= prediction error
where small errors mean low surprise
7
How can we minimize prediction error (free-energy)?
sensations – predictions
Prediction error
Change
sensory input
Change
predictions
Action
Perception
…prediction errors drive action and perception to suppress themselves
8
But where do predictions come from?
…they come from the brain’s model of the world
Sensory input
Models (hypotheses)
Prediction
error
This means the brain models and predicts its sensations
(cf, a Helmholtz machine).
9
So how do prediction errors change predictions?
sensory input
Forward connections convey
feedback
Adjust hypotheses
Prediction errors
Predictions
prediction
Backward connections return
predictions
…by hierarchical message passing in the brain
10
Why hierarchical message passing?
cortical layers
Specialised
cortical areas
…because the brain is organized hierarchically, where each
level predicts the level below
11
David Mumford
Hierarchical message passing in the brain
Forward prediction error
 (i ) v
 (i 2)v
 (i 1)v
 (i 1) x
 (i ) x
 (i 1)v
 (i ) v
s(t )
 (i ) x
 (i1) x
Backward predictions
12
What about action?
predictions
Reflexes to action
dorsal root
sensory error
s(a)
action
ventral horn
a
Action can only suppress (sensory) prediction error.
This means action fulfils our (sensory) predictions
13
Summary
•Biological agents resist the second law of thermodynamics
•They must minimize their average surprise (entropy)
•They minimize surprise by suppressing prediction error (free-energy)
•Prediction error can be reduced by changing predictions (perception)
•Prediction error can be reduced by changing sensations (action)
•Perception entails recurrent message passing in the brain to optimise predictions
•Action makes predictions come true (and minimises surprise)
Examples:
Perception (birdsongs)
Action (goal-directed reaching)
Policies (the mountain car problem)
14
Making bird songs with Lorenz attractors
Syrinx
Sonogram
Frequency
Vocal centre
 v1 
v 
v2 
0.5 1
1.5
time (sec)
causal states
hidden states
15
Perception and
message passing
prediction and error
20
15
10
5
0
-5
10
v
20
30
40
50
60
causal states
Backward predictions
20
x
15
stimulus
10
5000
5
4500
s(t )
4000
Forward prediction error


v
0
x
-5
3500
-10
3000
hidden states
10
20
30
40
50
60
20
2500
2000
15
0.2
0.4
0.6
time (seconds)
0.8
10
5
0
-5
10
20
30
40
50
60
16
Frequency (Hz)
Perceptual categorization
Song a
Song b
Song c
time (seconds)
17
Hierarchical models of birdsong: sequences of sequences
v1(1)
v2(1)
sonogram
Syrinx
Frequency (KHz)
Neuronal hierarchy
0.5
1
1.5
Time (sec)
18
Simulated lesions and hallucinations
percept
LFP
Frequency (Hz)
LFP (micro-volts)
60
40
20
0
-20
-40
no top-down messages
LFP
Frequency (Hz)
LFP (micro-volts)
60
40
20
0
-20
-40
-60
no lateral messages
Frequency (Hz)
LFP (micro-volts)
LFP
0.5
1
1.5
time (seconds)
60
40
20
0
-20
-40
-60
0
500
1000
1500
2000
peristimulus time (ms)
19
 (1) x
Action, predictions and priors
 (2)v
 (1)v
 (1) x
 (1)v
visual input
V 
s  w
J 
(0, 0)
x1
Descending
sensory prediction
error
J1
proprioceptive input

(1)v
x 
s   1w
 x2 
x2
J2
V  (v1 , v2 , v3 )
a
20
The mountain car problem
The environment
The cost-function
0.7
Adriaan Fokker
Max Planck
0.6
height
0.5
 ( x)
0.4
0.3
c ( x, h )
0.2
0.1
0
-2
-1
0
1
2
position
True equations of motion
x

x 
f  

1
 x  (a)   x  8 x
( x)
(h)
position
happiness
Policy (predicted motion)
 x   x 
f  

 x cx   x 
“I expect to move faster when cost is positive”
21
a
Exploring & exploiting the environment
With cost
(i.e., exploratory
dynamics)
22
Policies and prior expectations
Using just the free-energy principle and a simple gradient ascent scheme, we have solved
a benchmark problem in optimal control theory using a handful of learning trials.
If priors are so important, where do they come from?
23
…we inherit them
Darwinian evolution of virtual block
creatures. A population of several
hundred creatures is created within a
supercomputer, and each creature is
tested for their ability to perform a given
task, such the ability to swim in a
simulated water environment. The
successful survive, and their virtual genes
are copied, combined, and mutated to
make offspring. The new creatures are
again tested, and some may be
improvements on their parents. As this
cycle of variation and selection continues,
creatures with more and more successful
behaviours can emerge.
The selection of adaptive predictions
24
Time-scale
Free-energy minimisation leading to…
10 3 s
Perception and Action: The optimisation of neuronal and
neuromuscular activity to suppress prediction errors (or freeenergy) based on generative models of sensory data.
100 s
103 s
106 s
1015 s
Learning and attention: The optimisation of synaptic gain and
efficacy over seconds to hours, to encode the precisions of
prediction errors and causal structure in the sensorium. This
entails suppression of free-energy over time.
Neurodevelopment: Model optimisation through activitydependent pruning and maintenance of neuronal connections that
are specified epigenetically
Evolution: Optimisation of the average free-energy (free-fitness)
over time and individuals of a given class (e.g., conspecifics) by
selective pressure on the epigenetic specification of their
generative models.
25
Thank you
And thanks to collaborators:
Jean Daunizeau
Lee Harrison
Stefan Kiebel
James Kilner
Klaas Stephan
And colleagues:
Peter Dayan
Jörn Diedrichsen
Paul Verschure
Florentin Wörgötter
26