Transcript When the Shoe Fits: Cross-Situational Learning in

When the Shoe Fits:
Cross-Situational Learning in
Realistic Learning Environments
Tamara N. Medina1
John Trueswell1
Jesse Snedeker2
Lila Gleitman1
Institute for Research in Cognitive Science, University of Pennsylvania
2Department of Psychology, Harvard University
1
1998, Third Edition, Completely Revised
“A baby hears a word like
“shoes”, for example, over and
over again in daily life as the one
constant sound in a large variety
of statements. In one day you
may say to him:”
“That word “shoes” is the one
sound which occurs in all those
• “Where are your shoes?”
sentences
is always
• “Oh, what and
dirtyitshoes!”
associated
• “Let’s takewith
your those
shoes things
off.” that
onput
hisyour
feet.shoes
Eventually
he will
•go“I’ll
on.”
thenice
spoken
sound with
•associate
“Look what
new shoes.”
the objects and when he has
made that association, he will
have learned what the word
“shoes” means.”
Cross-Situational Learning
“Find a set of possible meanings in each situation and
intersect those sets across all situations in which a word
occurs to determine the meaning for that word.”
Siskind, J.M. (1996, Cognition)
“It’s not so easy!”
– Augustine, Locke, Quine, Gleitman, Fodor, Siskind, etc.
Frame / Level of Description
Animal? Dog? Terrier? Fido? Friendly?
Referential Uncertainty
Which referent?
Frame Problem Solved?
• Xu & Tenenbaum (2007): learn appropriate extensions
of a word via Bayesian inference (note “suspicious
coincidences”)
“VASH”
“VASH”
“VASH”
Reference Problem Solved?
• Yu & Smith (2007): learn word-object associations in
spite of “referential uncertainty”
“DOON” … “VASH” … “MIPEN” … “ZANT”
“VASH” ??
“VASH” !
Goal: Explore cross-situational word
learning using naturalistic settings: both
the cluttered and potentially uninformative
or misleading environments and these
somewhat more transparent ones.
Overview
• Adaptation of the Human Simulation
Paradigm (Gillette et al., 1999)
• Norming Study
• Current Study
• 2 measures to evaluate word learning
Adaptation of Human Simulation
Paradigm (Gillette et al., 1999)
• Selected stimuli based on results of earlier
norming study:
Gertner, Y., Fisher, C., Gleitman, G., Joshi, A., & Snedeker, J. (In
progress). Machine implementation of a verb learning algorithm.
• Large video corpus of parent-child
interactions in natural settings (home,
outdoors, etc.):
Snedeker, J. (2001). Interactions between infants (12-15 months)
and their parents in four settings. Unpublished corpus.
Norming Study
•
•
•
•
Identified 48 most frequently occurring content words.
Randomly selected six instances of each word.
Each instance was edited into a 40-second “vignette”.
Sound turned off.
– Visual context only cue to word meaning, placing viewers in the
situation of the early word learner.
• Utterance of target word indicated by a BEEP.
Gertner, Y., Fisher, C., Gleitman, G., Joshi, A., & Snedeker, J. (In progress).
Machine implementation of a verb learning algorithm.
(silence)
(silence)
<BEEP>
30 sec
(silence)
10 sec
Drawings courtesy of Emily Trueswell
No opportunity for cross-situational learning in norming study
8% correct
83% correct
Low Informative
(<33% correct)
High Informative
(>50% correct)
Subject Guesses
(Target Word = Shoe)
Subject Guesses
(Target Word = Horse)
0% correct
…
Subject Guesses
(Target Word = Shoe)
90% of Vignettes = Low Informative (typical)
7% of Vignettes = High Informative (atypical)
Gertner, Y., Fisher, C., Gleitman, G., Joshi, A., & Snedeker, J. (In progress).
Machine implementation of a verb learning algorithm.
Questions
• Does the observation of multiple naturalistic
learning instances generate a gradually
increasing learning curve?
• With regard to informativeness, given only the
Low Informative vignettes, is cross-situational
learning successful? Or is a High Informative
instance necessary?
• If learners are building an interpretation across
instances, does it matter when the High
Informative instance occurs?
Current Study
Similar to norming study,
BUT allows for cross-situational learning
(silence)
Current Study Allows for
Cross-Situational Learning
(silence)
“VASH”
30 sec
(silence)
10 sec
Opportunity for cross-situational learning
…
“VASH”
(Target Word = Shoe)
“MIPEN”
(Target Word = Horse)
“VASH”
(Target Word = Shoe)
Subject makes guess
Subject makes guess
Subject makes guess
Subject rates
Confidence (1 to 5)
Subject rates
Confidence (1 to 5)
Subject rates
Confidence (1 to 5)
Final Conjectures and Confidence Ratings for each word
Manipulated the distribution of
informative events
• For each of 8 Target nouns, there were:
– 1 “High Informative” vignette (>50% of participants correct in
norming study)
– 4 “Low Informative” vignettes (<33% of participants correct in
norming study)
• 4 Filler nouns
– 5 “Low Informative” vignettes
• Participants assigned to one of 4 orders:
–
–
–
–
High Informative First:
H-L-L-L-L
High Informative Middle: L-L-H-L-L
High Informative Last:
L-L-L-L-H
High Informative Absent: L-L-L-L-L
(fifth vignette is a repeat of the first)
Accuracy Across Vignettes
Accuracy Across Vignettes
Accuracy Across Vignettes
Accuracy Across Vignettes
Accuracy Across Vignettes
Interim Summary: What have
we learned about learning?
1. Gradual learning from
partially informative
instances is small to
nonexistent.
2. Successful learning depends
on the presence of a High
Informative instance.
(Epiphany!)
3. Low Informative instances
have a corrupting influence
on later-occurring High
Informative instances.
(Cross-situational learning of
the bad sort.)
Epiphany!
Successful learning depends on the
presence of a High Informative instance.
• Explicit and immediate insight?
• After using evidence from later instances?
– High Informative instance provides key for
interpreting later instances.
Confidence on Correct Guesses
across Vignettes
Confidence on Correct Guesses
across Vignettes
Confidence on Correct Guesses
across Vignettes
Confidence on Correct Guesses
across Vignettes
Implications
• Shape of the word learning curve may be
very different than what cross-situational
learning models (e.g., Yu & Smith, 2007) have
suggested:
Rapid
Incremental
Implications
• Successful word learning from cross-situational
observation requires the occurrence of a highly
informative instance.
– But must it occur first? Logically, no!
• Greater delay between instances of a novel word:
Every day is a new day.
• Multiple High Informative learning instances.
– Previous studies which show striking rapid word learning
are such cases.
• Less weight on interpretations of Low Informative
instances.
Implications
• A High Informative instance is the first step
in successful cross-situational word
learning.
– Prior Low Informative instances might not be
remembered over time.
– Later Low Informative instances become
useful (confirmatory evidence?)
• Supported by rising confidence levels after a High
Informative vignette.
Cross-situational learning does work,
but only when the shoe fits.
Accuracy
Confidence
Perseverance of First Guess