Transcript ICIS 2004 What you see is not always what you get

What you see is not always what you get: Showing effects of event structure
Peter Gordon, Alisa Matlin, Nicholas Joy, Erin Aylward, Janet Eisenband
Teachers College, Columbia University
Introduction
• Is there a distinction between
seeing and knowing when
employing looking paradigms in
infant research?
• For infants to understand an
event, they must be looking at
the relevant aspects of the event
in question
• This establishes gaze direction
as a necessary component of
event understanding
• Recent studies have
established that gaze direction
predicts whether infants show
increased looking when obscured
objects do not re-emerge from a
screen in the expected manner
(Johnson, 2003)
• But can event representation
be dissociated from gaze
direction?
• In other words, is gaze
direction a sufficient condition for
event understanding?
The Case of SHOW
The GIVE/HUG Effect
• When 8 to 10 month olds see an event of GIVING, they
treat a toy undergoing transfer of possession with special
status
• The act of GIVING requires an agent, a transferred
• When habituated to GIVING, they show recovery of
looking time when the toy is missing in test video stimuli
•In a similar manner, the act of SHOWING requires an
agent, an object being shown, and an experiencer
• This is because the object is RELEVANT to the action
•Again, the object is highly relevant in defining the act of
SHOWING
object, and a recipient. In this sense, the toy is highly
relevant to the action, whereas it is not relevant for HUG
• Infants do not show recovery of looking time when the
missing object is IRRELEVANT to the action, such as when
two people are hugging and one is carrying a toy (Fig. 1)
•As in the case of GIVE, eye tracking revealed that infants
looked at the toy when watching a video of SHOWING
(Fig. 5)
Eye Tracking
•However, infants did not show recovery of looking time
when the toy was removed on test trials (Fig. 2)
• Eye tracking of infants as they watched the GIVE event
video revealed that, during the transfer of possession, they
looked at the toy more than any other element in the scene
Why Not?
• When the toy was missing in the test video, they
continued to look at where the toy had been in the original
video (Fig. 3)
• A valid representation of SHOW requires Theory of Mind
understanding that one person intends to transfer
information to the other through a visual medium
• With the HUG videos, infants looked at the toy much less
during the interaction phase, and almost never when it was
no longer in the video (Fig. 4)
•Infants under 12 months are unlikely to understand this
relationship
• Low level cues attract infants’ attention to the action
around the toy, but this is insufficient to trigger
dishabituation to the change in the event. Seeing and
understanding are dissociated in this case
Conclusion
• Gaze direction to the toy during events predicted whether
infants showed recovery of looking time in the habituation
test
Habituation
Habituation Data
Event Videos
•They see it but they just don’t get it
Eye Tracking Data
Test
% looking to toy location
Fig 3 Give
0.7
Fig. 1 Ten Month Olds: GIVE vs. HUG
Percent Looking time
Mean Looking time (sec)
45
Give
40
Hug
35
With Toy
0.6
30
Without Toy
0.5
0.4
0.3
0.2
0.1
25
Approach
20
GIVE w/
Retreat
GIVE w/o
0
Approach
Interaction
Departure
15
10
Fig 4 Hug
5
0
Hab1
Hab2
Hab3
Hab4
Hab5
Hab6
Avg Old Avg New
Trials
Approach
Fig. 2 Ten Month Olds: GIVE vs. SHOW
HUG w/
Retreat
HUG w/o
Percent Looking time
0.70
With Toy
0.60
Without Toy
0.50
0.40
0.30
0.20
0.10
0.00
Approach
Interaction
Departure
40
Give
35
Show
Fig 5 Show
0.7
30
Percent Looking times
Mean Looking time (sec)
45
25
20
15
10
5
0
Hab1
Hab2
Hab3
Hab4
Hab5
Trials
Hab6
Avg Old Avg New
SHOW w/
SHOW w/o
With Toy
0.6
Without Toy
0.5
0.4
0.3
0.2
0.1
0
Approach
Interaction
Departure