Transcript Attention Changes the Representation? Visual Changes with Attention Auditory Changes with Task Shamma Tone-Discrimination Task.

Attention Changes the Representation?
Visual Changes with Attention
Auditory Changes with Task
Shamma Tone-Discrimination Task
Top Down Attention: Automatically Learning the Attention Field
* Reynolds and Heeger , Neuron , 09.
The Attention Spotlight
Attention
Dominant Spike Frequencies
Speech Separation is NOT the
Right Approach for Perception
Malcolm Slaney
Problems
• Continuation
– Tone and Noise
– Parliament Cough
• What do you hear?
– Waveforms?
– Ideas?
Speech Examples
Wedding
Sine
Natural
Miriam Makeba
Change in Perception
• Old
– Cocktail Party Effect
– Sound Separation
• New
– Situational Awareness
– Sound Understanding
A Real-Time Cocktail Party
Merve, Trevor, Kailash,
Barbara, Mounya, Ozlem,
Nuno, DeLiang, Malcolm, Shih-Chii, Tobi
A Cocktail Party
Blah Blah
Blah.
Blah Blah
Blah.
Attending to Conversations
28 11 39
83 sex 23
68 23 34
56 81
Where do I attend?
?
?
? ?
?
?
?
Task: Recognize the highest valued (two digit) numbers
Male
Female
Scene Analysis Experiments
Time
Overlapping Speakers
Two-digit Sentences (even digit at the end)
Template matching (utterance dependent)
Cognition
Male/Female
Salience
Recognized Digits
Attention
ASR
Binaural
Receiver
ITD
Histogram
Novelty/
Salience
Saliency Maps
• Longer tones better
• Missing parts salient
• Modulation more
salient
• Forward masking
works
ITD Histogram and Salience
Interaural Time Delay – Correlation vs. Delay
Time
Time
ITD via a Silicon Cochlea
Real-Time Movie
Smart
• Listen for most
promising
• Stay if first digit > 5
• Switch if digit < 5
Male digits are: 98 52 94 34 32 56 14 38
54 94 14 38 58 36 32 38
Female digits are: 54 98 52 12 54 52 56
58 16 14 36 58 16 52 58 52
ADHD
• Switch always
• Anytime there is a
salient event,
switch to active
channel
Male digits are: 98 52 94 34 32 56 14 38
54 94 14 38 58 36 32 38
Female digits are: 54 98 52 12 54 52 56
58 16 14 36 58 16 52 58 52
Scene Analysis Engineering
Cognition
(Python)
HTML (male/female)
HTML (digits)
ASR
(Matlab)
HTML (sound)
State: Direction to attend,
Digits recognized
Task: Switch attention
based on recognition and
saliency
Binaural
(jAER)
HTML (Salience)
UDP (ITD)
Novelty
(Python)
Shortcuts
• Saliency
– Binaural onset instead of monaural saliency
• Recognition
– Template matching instead of HMM
• Cognition
– Two digit sentences, easy semantics
• Sound Separation
– Monaural input instead of delay-and-add
Male
Female
Scene Analysis Experiments
Time
Issues
Binaural – Beam forming not enough
Saliency – Harmonics not represented
Attention – Recognition with spotlight?