Transcript USC Brain Project Specific Aims

Laurent Itti:
CS564 - Brain Theory and Artificial Intelligence
Lecture 1. Introduction and Brain Overview
Reading Assignments:*
TMB2: Chapters 1; 2.4
HBTNN:
I.1. Introducing the Neuron (Arbib)
* Unless indicated otherwise, the TMB2 material is the required reading, and
the other readings supplementary.
Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview
1
CS 564: Brain Theory and Artificial Intelligence
URL: http://iLab.usc.edu/classes/2002cs564/ for
syllabus, instructor and TA information, handouts,
homework and grades
Instructor:
Laurent Itti; itti@pollux (Office Hour: Mon 3-5, HNB30A)
TA:
Yoo-Hee Shin [email protected]
This course provides a basic understanding of brain function, and of artificial
neural networks which provide tools for a new paradigm for adaptive parallel
computation.
No background in neuroscience is required, nor is specific programming
expertise, but knowledge of C++ will be useful for homeworks.
Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview
2
Texts and Grading
Text:
M.A. Arbib, 1989, The Metaphorical Brain 2:
Neural Networks and Beyond, Wiley-Interscience.
Supplementary reading:
M.A. Arbib, Ed., 1995, The Handbook of Brain Theory and Neural Networks,
MIT Press (paperback).
One mid-term and a final will cover the entire contents of the readings so far as
well as the lectures.
The final exam will cover all of the course, but emphasizing material not
covered in the mid-term.
Distribution of Grades:
Homeworks: 40%; Mid-term: 30%; Final Exam: 30%.
Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview
3
Syllabus Overview
Introduction
Overview
Charting the brain
The Brain as a Network of Neurons
x 1(t)
w1
x 2(t)

w2
w
xn(t)
Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview
axon
y(t+1)
n
4
Syllabus Overview
Introduction (cont.)
Experimental techniques
Introduction to Vision
Schemas
Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview
5
Syllabus Overview
Basic Neural Modeling & Adaptive Networks
Didday Model of Winner-Take-All
Hopfield networks
E = - ½  ij sisjwij +  i sii
Adaptive networks: Hebbian learning;
Perceptrons; landmark learning
Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview
6
Syllabus Overview
Neural Modeling & Adaptive Networks (cont.)
Adaptive networks: gradient descent
and backpropagation
Reinforcement learning
Competition and cooperation
     
     
     
Visual plasticity; self-organizing
feature maps; Kohonen maps
Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview
7
Syllabus Overview
Examples of Large-scale Neural Modeling
System concepts
 x (t ) 0 1 
q (t )  
  
 x(t ) 0 0
 x(t ) 0 
 x (t )  1  u (t )

  m 
de la y
FEF
FOn
PPc tr
ms
switch
PP
qv
sm
vm
vs
VisCx
sm
Model of saccadic eye movements
CD
TH
LG N
vm
SNR
vs
sm
de la y
FEFvs
FEFms
SC
vs
ms
qv
FOn
wta
ey e movement
FEFvs
FEFms
B rainstem
Saccade
G enerator
Retina
VisInput
Feedback and the spinal cord;
mass-spring model of muscle
Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview
8
Syllabus Overview
Large-scale Neural Models of Vision
Early visual processing
C
OA
Depth perception
L
+qmax
OA
-qmax
B
A
R
D
qL
qR
qLB, qLD
qRA, qRD
-qmax +qmax
q0
qLA, qLC
q0
qRB, qRC
Optic flow
Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview
9
Syllabus Overview
Large-scale Neural Models of Vision (cont.)
Visual attention
Object recognition
Scene perception
Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview
10
Syllabus Overview
AIP
AIP
Other Advanced Neural Modeling
Dorsal
dorsal/ventral
Stream:
streams
Affordances
Ways to grab
this “thing”
Reaching, grasping and affordances
Task Constraints
Task
Constra ints ( F6)
(F6)
Working Memory
W orking
(46?) Me mory (46)
F5
Instruction Stimuli
Ventral
Stream:
Recognition
(F2) Stim uli (F2)
Instruction
“It’s a mug”
PFC
IT
Cerebellar adaptation
reach programming
Memory and consciousness
grasp programming
Parietal
Cortex
How (dorsal)
Visual
Cortex
Inferotemporal
Cortex
Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview
What (ventral)
11
Syllabus Overview
Applications and Outlook
Towards highly-capable
robots
Overview and summary
Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview
12
Three Frameworks
Artificial intelligence (AI): build a
“packet of intelligence” into a machine
Cognitive psychology: explain human behavior by interacting processes
(schemas) “in the head” but not localized in the brain
Brain Theory: interactions of components of the brain - computational neuroscience
- neurologically constrained models: e.g., networks of neurologically localized
schemas
and abstracting from them as both Artificial intelligence and Cognitive
psychology:
- connectionism: networks of trainable “quasi-neurons” to provide “parallel
distributed models” little constrained by neurophysiology
- abstract (computer program or control system) information processing
models
Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview
13
The Aim of the Course
To gain an understanding of biological neurons as the basis for:
Brain Theory: modeling interactions of components of the brain,
especially more or less realistic biological neural networks localized in
specific brain regions
Connectionism in both Artificial intelligence (AI) and Cognitive
psychology: modeling artificial neural networks -- networks of trainable
“quasi-neurons” -- to provide “parallel distributed models” of intelligence in
humans, animals and machines
This lecture: A tourist’s guide to the brain ;-)
Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview
14
A motivating theme: Vision
Vision as a progressive change in representation
Marr (1982): through 2 ½ D primal sketch
Because vision is by far the most studied sense (because
it is easy to experiment with), we will use it as a basis
for many examples of models studied in this course.
Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview
15
Vision and the brain
Ryback et al, 1998
Roughly speaking, about half of
the brain is concerned with vision.
Although most of it is highly automated and unconscious, vision hence
is a major component of brain function.
Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview
16
Vision, AI and robots
1940s: beginning of Artificial Intelligence
Sm
input
m
neuron
M
out put
McCullogh & Pitts, 1942
i wixi  
Perceptron learning rule (Rosenblatt, 1962)
Backpropagation
Hopfield networks (1982)
Kohonen self-organizing maps
…
Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview
17
Vision, AI and Robots
1950s: beginning of computer vision
Aim: give to machines same or better vision capability as ours
Drive: AI, robotics applications and factory automation
Initially: passive, feedforward, layered and hierarchical process
that was just going to provide input to higher reasoning
processes (from AI)
But soon: realized that could not handle real images
1980s: Active vision: make the system more robust by allowing the
vision to adapt with the ongoing recognition/interpretation
Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview
18
A tourist’s guide to the brain
Gross anatomy
Non-neural structures
Major cortical areas
Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview
19
Central vs. Peripheral Nervous System
The brain is not the entire nervous systems; there is also
the spinal cord, many peripheral “ganglia” (small
clusters of neurons), and neurons extend connections to
locations all over the body (e.g., sensory neurons, motor neurons).
Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview
20
Autonomic Nervous System
Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview
21
Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview
22
Axes in the brain
Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview
23
The “Bauplan” for the Mammalian Brain
Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview
24
Medical Orientation Terms for Slices
Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview
25
Main Arterial Supply to the Brain
Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview
26
Arterial Supply is Segmented
Occlusion/damage to one artery will affect specific brain
regions. Important to remember for patient studies.
Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview
27
Ventricular System
Ventricules: Cavities filled with fluid inside and around
the brain. One of their functions is to drain garbage out
of the brain.
Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview
28
Cortical Lobes
Sulcus (“fissure” if very large): Grooves in folded cortex
Gyrus: cortex between two sulci
1 sulcus, many sulci; 1 gyrus, many gyri
Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview
29
Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview
30
Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview
31
Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview
32
Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview
33
Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview
34
Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview
35
Neurons
Cell body (soma): where computation takes place
Dendrites: input branches
Axon: unique output (but may branch out)
Synapse: connection between presynaptic axon and
postsynaptic dendrite (in general).
Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview
36
Electron Micrograph of a Real Neuron
Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview
37
Neurons and Synapses
Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview
38
Grey and White Matters
Grey matter: neurons (cell bodies), at outer surface of brain
White matter: interconnections, inside the brain
Deep nuclei: clusters of neurons deep inside the brain
Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview
39
Major Functional Areas
Primary motor: voluntary movement
Primary somatosensory: tactile, pain, pressure, position, temp., mvt.
Motor association: coordination of complex movements
Sensory association: processing of multisensorial information
Prefrontal: planning, emotion, judgement
Speech center (Broca’s area): speech production and articulation
Wernicke’s area: comprehension of speech
Auditory: hearing
Auditory association: complex
auditory processing
Visual: low-level vision
Visual association: higher-level
vision
Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview
40
Major Functional Areas
Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview
41
A View of the Monkey Brain
Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview
42
Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview
43
http://www.radiology.wisc.edu/Med_Students/neuroradiology/fmri/
Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview
44
Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview
45
Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview
46
Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview
47
Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview
48
Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview
49
Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview
50
Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview
51
Limbic System
Cortex “inside” the brain.
Involved in emotions, sexual behavior, memory, etc
(not very well known)
Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview
52
Major Functional Areas
Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview
53
Visual Input to the Brain
Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview
54
Eye and retina
Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview
55
Human Visual System
Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview
56
Primary Visual Pathway
Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview
57
Layered Organization of Cortex
Cortex is 1 to 5mm-thick, folded at the surface of the brain
(grey matter), and organized as 6 superimposed layers.
Layer names:
1: Molecular layer
2: External granular layer
3: External pyramidal layer
4: internal granular layer
5: Internal pyramidal layer
6: Fusiform layer
Basic layer functions:
Layers 1/2: connectivity
Layer 4: Input
Layers 3/5: Pyramidal cell bodies
Layers 5/6: Output
Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview
58
Layered Organization of Cortex
Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview
59
Slice through the thickness of cortex
1
2
3
4
5
6
Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview
60
Columnar Organization
Very general principle in cortex: neurons processing similar “things” are
grouped together in small patches, or “columns,” or cortex.
In primary visual cortex…
as in higher (object
recognition) visual areas…
and in many, non-visual, areas as well (e.g., auditory, motor, sensory, etc).
Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview
61
Retinotopy
Many visual areas are organized as retinotopic maps: locations next
to each other in the outside world are represented by neurons close
to each other in cortex.
Although the topology is thus preserved, the mapping typically is highly non-linear
(yielding large deformations in representation).
Stimulus shown on screen…
and corresponding
activity in cortex!
Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview
62
Retinotopy
Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview
63
Mammalian and Frog Visual Systems
Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview
64
Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview
65
Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview
66
Interconnect
Felleman & Van Essen, 1991
Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview
67
Interconnect…
(other source)
Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview
68
More on Connectivity
Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview
69
Varieties of Vertebrate Brains
Snake
Catfish
Frog
Alligator
Primitive Mammal
Goose
Horse
Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview
70
Outlook
There is a lot to learn about the brain!
… but don’t feel overwhelmed, we will smoothly
introduce all new concepts.
Principled theoretical and engineering methods will allow us to abstract
some of these complications.
Starting with fundamental techniques, we will then study fairly
complex, large-scale neural models.
Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview
71