Transcript Slide 1

Today
Nonlearnable production
grammars: ranking reversals
and reinterpretation of contrast
Fijian
Malayalee English
Korean liquids
First--review
Our job in processing speech…
Turn sound waves (vibration of air
molecules) into meaning.
To do this, we need to interpret the acoustic
cues in the sound waves in order to
translate sound waves into phonological
representations.
From sound to meaning
Acoustic Form
(perception)
Phonological Representation
(production)
Phonetic Representation
Questions
When a speaker mispronounces a foreign
word, is the mispronunciation due to
misproduction?
misperception?
Japanese example
English
[beɪsbɔl]
Japanese
[be:suboru]
First Possibility: Accurate Perception,
Inaccurate Production
•
•
•
Japanese listener hears [beɪsbɔl].
/beɪsbɔl/ is input to Japanese production
grammar, which disallows certain
segments and consonant codas.
Production grammar transforms
/beɪsbɔl/ to [be:suboro].
Second Possibility: Inaccurate
Perception, Accurate Production
•
•
•
Japanese listener hears [be:suboru].
/be:suboru/ is input to Japanese
production grammar, and is a legal
Japanese structure.
Output of production grammar is
[be:suboru].
Evidence for Misperception of /r-l/
Miyagawa et al. (1975) (and others):
English listeners perceived /r-l/
continuum categorically;
Japanese listeners did not have clear
category boundaries for /r/ vs. /l/.
Effect of Native Language
In English, the [r] –[l] contrast is significant:
‘rip’ vs. ‘lip’
In Japanese, this contrast is not significant:
Japanese employs a single liquid
pronounced more like [r] in certain
contexts, more like [l] in other contexts.
Misperception even with
good production
Sheldon & Strange (1982):
Japanese students of English who
produced an English-like /r/-/l/ contrast
still performed poorly on discrimination
(even of their own [ra], [la] tokens).
Evidence for misperception of
consonant sequences as CVC
Dupoux et al. (1999):
Japanese listeners did not reliably
distinguish forms like [ebzo] – [ebuzo]
(they heard ‘illusory vowels’).
French listeners DID distinguish [ebzo][ebuzo].
Effect of Native Language
In French (and English), the contrast
between
VCCV and VCVCV is significant:
‘NAFTA’ vs. ‘taffeta’
In Japanese, no such contrast exists
because obstruents ([f], [b], [z], etc.) are
not possible in syllable coda.
Persistent misperception
Takagi & Mann (1995): adult Japanese
speakers who had lived in the US and
been using English daily for more than 12
years still performed below English
speakers on perception tests.
Our basic question
Do foreign language modification patterns
come from misperception or
misproduction?
And how can you tell (short of doing
perception experiments for each case)?
Criterion: Learnability

OT provides a model of acquisition.

This model allows us to determine
whether a particular grammar is learnable
from a set of input data.
Acquisition of a production
grammar
THE LEARNER’S TASK:
To discover constraint rankings.
DEFAULT RANKING
M >> F
e.g., NoComplexOnset >> Max (don’t delete)
Predicts underlying /sno/ mapped to [so].
Constraint reranking: error-driven
If the learner hears a marked structure (e.g.
[sno]), (s)he will rerank the constraints.
Max >> NoComplexOnset
Learning Fx >> Fy
In order to learn relative ranking of faithfulness
constraints, learner must have evidence of
unfaithful mappings.
F >> F

If /sno/ > [so], then deletion is the preferred
repair.
Dep (don’t insert) >> Max (don’t delete)

If /sno/ > [sono], then insertion is the
preferred repair.
Max (don’t delete) >> Dep (don’t insert)
Is a grammar learnable?
Rankings must be either
universal default (M>>F)
or
learnable from the data
Case Study 1: Fijian stop
adaptation (Kenstowicz 2003)
Fijian:
‘balloon’ [balun] > [mbaluni] (*[paluni])
Fijian NL (native language)
grammar

M >> F
constraint banning fully oral stop [b]
outranks faithfulness
What choices do Fijians have as substitutes
for [b]?
Fijian Native Language Phonemes
Voiceless unaspirated
/p/
Voiced prenasalized
/mb/
Why do Fijians choose /b/ > [mb],
not [p]?
Modification grammar:
Ident[voice] >> Ident[nasal]
(maintaining voice specification is more
important than maintaining nasal
specification)
Where does this ranking come
from?

Ident[voice] >> Ident[nasal]

From English?
From Fijian?
From universal grammar?


PROBLEM!
Universal default (Steriade 2001):
Ident[nasal] >> Ident[voice]
Fijian adapters
Ident[voice] >> Ident[nasal]
Alternative analysis:
misperception

Kenstowicz 2003: Word-initial Fijian stops
are often only slightly prenasalized (or not
at all prenasalized).

NL perception grammar interprets wordinitial voiced stop as prenasalized.
(Voicing is a cue for prenasalization).
Fijian stop adaptation as NL
transfer

Fijian NL perception grammar interprets
the English /p-b/ contrast in terms of
Fijian /p-mb/ contrast.
Case Study 2: Malayalee English
Malayalee English (Mohanan &
Mohanan 2003)
English intervocalic voiceless stops
Rippo:RT ‘report’
bekkar
‘baker’
pæ:kket ‘packet’
Malayalam NL grammar
Markedness constraint: *V[-voice]V:
no voiceless stops in intervocalic position
(also active in Korean)
M >> F
Malayalee English grammar
Ident(voice) >> Ident(length)
Maintaining voicing specification is more
important than maintaining length
specification
PROBLEM!
Malayalam ranking:
*VkV>>Ident(length) >> Ident(voice)
/VkV/ > [VgV]
Malayalee English Ranking:
*VkV>> Ident(voice) >>Ident(length)
/VkV/ > [VkkV]
Learnability problem

Evidence for reranking of length
faithfulness and voicing faithfulness

No evidence from Malayalam
No evidence from English

BUT--
The adaptation pattern is explainable as
transfer of Malayalam perception
grammar.
Malayalee English, continued
English intervocalic voiced stops
bæ:bu:n ‘baboon’
iRigeet
‘irrigate’
figaR
‘figure’
Alternative analysis:
misperception
Malayalam speakers interpreted English
contrast in terms of Malayalam contrast
English
voice contrast
VkV
VgV
Malayalam
length contrast
/VkkV/ > [VkkV]
/VkV/ > [VgV]
Malayalam cues for singletongeminate contrast (Local &
Simpson (1999)
1. Voicing: singletons are voiced
intervocalically.
2. Vowel duration: vowels longer before
singletons than before geminates
mean V duration:
76.5 msec before singleton
58.8 msec before geminates
English cues for voicing contrast
1. Voicing
2. Vowel duration: vowel is longer
before voiced consonant.
Cue Confusion

English:
shorter vowel > voiceless consonant.
lack of voicing > voiceless consonant.

Malayalam:
shorter vowel > geminate consonant.
lack of voicing > geminate consonant.
Note: English voice contrast is
not maintained in all contexts
‘possible’ > [pɔ:sibL]
‘impossible’ > [imbɔ:sibL]
Malayalee English ranking:
*NC[-voice] >> Ident(voice)Foreign>>
*V[-voice]V
How could this be learned?
Interim summary

Fijian and Malayalee English adaptation
patterns required specific production
grammar rankings.

These rankings were not learnable from the
data of either the native or the foreign
language.

In Fijian, the adaptation rankings
contradicted putative universal rankings (so
they cannot be default).

In Malayalee English, the adaptation
rankings contradicted the NL rankings.

In both languages, the adaptation pattern
could be better understood as interpretation
of the foreign language acoustic cues in
terms of a native language contrast.
If misproduction does stem from
misperception…
At what level of processing does
misperception occur?
Sources of Misperception
Do listeners
•
•
accurately perceive the acoustic
differences, but miscategorize them?
OR
fail to perceive acoustic differences that
are relevant for native language
contrasts?
Native Language Neural Commitment
Hypothesis (Kuhl 2004)
Early exposure to a language produces a
“neural commitment” to the acoustic cues
that are important for that language.
This neural commitment leads to efficient
processing of NL contrasts (but not of FL
contrasts).
But what if you start early?
Pallier et al. (1997) tested ability of fluent
Spanish-Catalan bilinguals to
discriminate [e] and [ɛ], which contrast in
Catalan but not in Spanish.
Subjects included 2 groups
•
•
bilinguals with Spanish as first language
bilinguals with Catalan as first language
Both groups
• used both languages daily
• were highly proficient in both languages
• had been exposed to the second
language by age 6
Even for fluent bilinguals, mother
tongue shaped perception
Subjects whose first language exposure
was to Spanish were unable to reliably
perceive the e/ɛ distinction.
In contrast, subjects whose first exposure
was to Catalan DID perceive this contrast.
Hypothesis
Early exposure to a language makes the
speech processing system sensitive
(only?) to those acoustic cues that
distinguish linguistically significant
contrasts in the native language.
Case Study 3: Korean Pronunciation of
English [r], [l]
[r] – [l] alternations in Korean
tal
tar-i
‘moon’
‘moon (nominative)’
/l/ and /r/ do not contrast in Korean
/l/ occurs in syllable coda
(word-finally and before a consonant)
/r/ occurs in syllable onset
(word-initially and between vowels)
Korean Liquid Alternation (Lee 2001)
a. /l/
tal
mal
sal.ku
‘moon’
‘horse’
‘apricot’
ta.r-i
ma.r-i
sa.ram
‘moon (nom.)’
‘horse (nom.)’
‘person’
b. /r/
How do Korean speakers interpret the
English /l/ - /r/ contrast?
Possibility One: Accurate Perception,
Inaccurate Production
According to Korean grammar:
Both /l/ and /r/ should be pronounced
as /l/ in syllable coda,
as /r/ in syllable onset.
Possibility One: Accurate Perception,
Inaccurate Production
Korean production grammar forces
intervocalic [l] and [r] to be pronounced
as [r] in syllable onset:
‘cherry’
‘cello’
heard as
heard as
/ceri/
pronounced as
[ce.ri]?
/celo/ pronounced as [ce.ro]?
Possibility Two: Inaccurate Perception,
Accurate Production
Koreans do not perceive [r] – [l] contrast:
‘cherry’
‘cello’
heard as
heard as
/ceri/
pronounced as
[ce.ri]?
/cero/ pronounced as [ce.ro]?
If Korean speakers simply map
English phonemes to Korean
phonemes…
The [r] – [l] contrast should not be
maintained by Korean speakers—it
should be lost either in perception or in
production.
But what actually happens?
Korean Adaptation of English liquids
(Kenstowicz 2005, Oh 2005)
intervocalic /r/ > [r]
kʰo:rasɨ
orenʒi
misɨtʰeri
‘chorus’
‘orange’
‘mystery’
BUT
intervocalic /l/ > [ll]
cʰello
sillikʰon
kʰolla
‘cello’
‘silicon’
‘cola’
Though there is some variation

[kilo, killo]
‘kilo’
Oh (2005): influence of spelling
orthographic ‘ll’ > [ll] 99.9%
orthographic ‘l’ > [ll] 84%
So strong trend toward [ll], regardless of
spelling.

Puzzle
/r/ - /l/ contrast is not significant in Korean,
but Koreans maintain it in English words.
English [r] > Korean [r] (kʰo:rasɨ ‘chorus’)
English [l] > Korean [ll] (kʰolla ‘cola’)
But Korean has [r] – [ll] contrast
Oh (2005)
mu.ri
mə.ri
‘group’
‘head’
mul.li
məl.li
‘physics’
‘far’
English Contrast
Korean Contrast
/r/
>
/r ~ l/
/l/
>
/ll/
although English (initial) [r] has longer F1
transition than [l] (O’Connor et al. 1957)
Does the change from /l/ to /ll/ take
place in production?
Optimality Theory says it can’t.
Korean Native Language Grammar
Lee (2001)
[l] only if moraic (attached to coda) >>
DepMora (don’t change duration) >>
Ident[lateral] (don’t change laterality)
(=>To repair illegal structure V[l]V, change it
to V[r]V)
To get pattern in which V[l]V is repaired by
changing it to V[ll]V, we would need to
rerank the constraints.
Reranking to describe /l/ > [ll]
Native Language ranking:
[l]=moraic >> Ident(length) >> Ident(lat)
Interlanguage Ranking:
[l] =moraic >> Ident(lat) >> Ident(length)
Problem
There is no data that would cause the
learner to rerank these constraints in this
way—nothing about English provides
evidence that changing duration is a
better way to repair an illegal structure
than changing laterality.
So..
Analysis assuming accurate perception,
inaccurate production is ruled out.
The theory of grammar learning provides no
way for learners to come up with a
grammar that would transform V[l]V to
V[ll]V.
Inaccurate perception?
Do Korean listeners actually hear English
[VlV] as [VllV]?
The relevant cues for Korean listeners:
1. presence/absence of laterality
(mainly F3)
2. consonant duration
muri = [-lateral, short]
mulli = [+lateral, long]
‘group’
‘physics’
Questions

How important is duration in signalling
the VrV – VllV contrast for Korean
listeners?

Would a difference in laterality alone be
enough to signal this contrast?
Kim (2007): [lateral] alone is enough
Subjects: monolingual speakers of Korean
Kim (2007) Stimuli
1. Real words containing VllV, e.g. [mulli]
(legal in Korean).
2. Real words containing VrV, e.g. [muri]
(legal in Korean).
3. ‘Words’ containing VlV, e.g. [muli]
(illegal in Korean), made by shortening
the [l] in V[ll]V words.
Kim (2007) Task
Listeners heard (edited) words like [muli].
Forced choice: Did you hear
[muri] (‘group’) or [mulli] (‘physics’)?
Results
[VlV] heard as
[VllV]
324
90%
[VrV]
36
10%
Kim (2007) results suggest that
Korean listeners pay at least as much
attention to the laterality as to consonant
duration.
Laterality alone is enough to cause Korean
listeners to classify even a short [l] as [ll].
BUT
Does this mean that Korean listeners
actually do not hear the differences
between English [l] and Korean [ll]?
ERP
Event-Related Potentials: A non-invasive
method of measuring electrical activity in
the brain during low level processing
(Luck 2005).
Electrode Cap
Mismatch Negativity
The Mismatch Negativity (MMN) indicates
brain response to change in an auditory
stimulus.
MMN is elicited even in the absence of
attention to stimulus (e.g., while watching
a silent movie).
Oddball Paradigm
ba - ba - ba - ba - pa
standard
deviant
MMN is typically elicited 150-250 msec after
onset of deviance.

Discrimination tasks tap into conscious
categorization.

ERP taps into involuntary, pre-attentive
brain reactions to change in auditory
stimuli.
MMN and Contrast
Various experiments have shown much
stronger MMN to acoustic differences that
cross NL phoneme boundaries.
Phillips et al. 2000

MEG investigation of stimuli varying on
[tæ-dæ] continuum.

English listeners responded differently
when standard/deviant boundary crossed
[t-d] phoneme boundary than when it fell
within a category, though degree of
acoustic variation was the same.
Stronger MMN/MMF with NL contrast




Japanese, French: [ebzo] vs. [ebuzo]
(Dehaene-Lambertz et. al. 2000)
Japanese, English: [ra] vs. [la]
(Zhang et. al. 2003)
Hungarian, Finnish: [æ] vs. [e]
(Winkler et. al. 1999)
Hindi, English: [pa] vs. [ba]
(Sharma and Dorman 2000)
Hwang, Broselow, Squires tested
Korean responses to
1. enne vs. ene (possible NL contrast)
[kannan] ‘newborn’ vs. [kanan] ‘poverty’
2. elle vs. ele (not a possible NL contrast,
because *[ele] is illegal)
Supported by NSF grant BCS-07460227
Subjects

10 native speakers of Korean, now
students at SUNY Stony Brook, in US
6months – 3 years

All proficient speakers of English
Task

Subjects watched a silent movie.
Subjects heard through headphones:
enne – enne - enne- enne- ene – enne…
or
elle – elle – elle – elle – ele – elle – elle …
(8 blocks of 300 trials = 2400 trials)

onset of deviance: 88ms.
Same acoustic difference

Both the [ene] and [ele] stimuli were
created by shortening the duration of the
long consonants in [enne], [elle].

Exactly the SAME durational change for
[enne]
=
[elle]
98ms.
[ene]
=
[ele]
48ms.
Same acoustic difference, but different
MMN responses (preliminary results)



Change from [enne] to [ene]: Stronger
MMN
Change from [elle] to [ele]: Weaker MMN
STRONGER MMN RESPONSE to the
cross-category change ([enne] – [ene])
than to the within-category change ([elle][ele]).
Subject:
Neuroscan
EEG file: 4.avg Recorded : 16:05:09 29-Feb-2008
SCAN 4.3
Grand
of -responses
to Standards
and Deviants
Rate - 500 Hz, HPF - 0.1
Hz, LPFAverage
- 30 Hz, Notch
60 Hz
Printed
: 12:51:42 17-Apr-2008
*4.avg
2.avg
Electrode: FZ
ele
5.0
Standard
Deviant
4.0
3.0
2.0
1.0
µV
0.0
-1.0
-2.0
Subject: -3.0
EEG file: -4.0
8.avg Recorded : 16:05:09 29-Feb-2008
Rate - 500-5.0Hz, HPF - 0.1 Hz, LPF - 30 Hz, Notch - 60 Hz
*8.avg-200.0
6.avg
-100.0
0.0
100.0
200.0
300.0
ms
Neuroscan
SCAN 4.3
Printed : 12:50:25 17-Apr-2008
400.0
500.0
600.0
700.0
400.0
500.0
600.0
700.0
Electrode: FZ
ene
5.0
4.0
3.0
2.0
1.0
µV
0.0
-1.0
-2.0
-3.0
-4.0
-5.0
-200.0
-100.0
0.0
100.0
200.0
300.0
ms
• Black line: response to ele after ele,
or ene after ene
• Red line: response to ele after elle,
or ene after enne
Subject:
Neuroscan
EEG file: 4.avg Recorded : 16:05:09 29-Feb-2008
SCAN 4.3
Grand
of -responses
to Standards
and Deviants
Rate - 500 Hz, HPF - 0.1
Hz, LPFAverage
- 30 Hz, Notch
60 Hz
Printed
: 12:51:42 17-Apr-2008
*4.avg
2.avg
Electrode: FZ
ele
5.0
Standard
Deviant
4.0
3.0
2.0
1.0
µV
0.0
-1.0
-2.0
Subject: -3.0
EEG file: -4.0
8.avg Recorded : 16:05:09 29-Feb-2008
Rate - 500-5.0Hz, HPF - 0.1 Hz, LPF - 30 Hz, Notch - 60 Hz
*8.avg-200.0
6.avg
-100.0
0.0
100.0
200.0
300.0
ms
Neuroscan
SCAN 4.3
Printed : 12:50:25 17-Apr-2008
400.0
500.0
600.0
700.0
400.0
500.0
600.0
700.0
Electrode: FZ
ene
5.0
4.0
3.0
2.0
1.0
µV
0.0
-1.0
-2.0
-3.0
-4.0
-5.0
-200.0
-100.0
0.0
100.0
200.0
300.0
ms
Subject:
Neuroscan
EEG file: 2diff.avg Recorded
16:05:09 29-Feb-2008
SCAN 4.3
Grand:Difference
Wave (Deviant-Standard)
Rate - 500 Hz, HPF - 0.1 Hz, LPF - 30 Hz, Notch - 60 Hz
Printed : 12:54:21 17-Apr-2008
*2diff.avg
4diff.avg
Electrode: FZ
5.0
MMN
Ele
Ene
4.0
3.0
2.0
1.0
µV 0.0
-1.0
-2.0
-3.0
-4.0
-5.0
-200.0 -100.0
0.0
100.0
200.0
300.0
ms
400.0
500.0
600.0
700.0
• Shows difference in response to elleele change vs. enne-ene change.
Subject:
Neuroscan
EEG file: 2diff.avg Recorded
16:05:09 29-Feb-2008
SCAN 4.3
Grand:Difference
Wave (Deviant-Standard)
Rate - 500 Hz, HPF - 0.1 Hz, LPF - 30 Hz, Notch - 60 Hz
Printed : 12:54:21 17-Apr-2008
*2diff.avg
4diff.avg
Electrode: FZ
5.0
MMN
Ele
Ene
4.0
3.0
2.0
1.0
µV 0.0
-1.0
-2.0
-3.0
-4.0
-5.0
-200.0 -100.0
0.0
100.0
200.0
300.0
ms
400.0
500.0
600.0
700.0
Subject:
EEG file: 2diff.avg Recorded : 16:05:09 29-Feb-2008
Rate - 500 Hz, HPF - 0.1 Hz, LPF - 30 Hz, Notch - 60 Hz
Neuroscan
SCAN 4.3
Printed : 12:54:21 17-Apr-2008
Grand Difference Wave (Deviant-Standard)
*2diff.avg
4diff.avg
Electrode: FZ
5.0
Ele
Ene
4.0
3.0
2.0
1.0
µV
0.0
-1.0
-2.0
-3.0
-4.0
-5.0
-200.0
-100.0
0.0
100.0
200.0
300.0
ms
400.0
500.0
600.0
700.0
250ms
250.00 ms
ele
+2.5
+2.5
+2.2
+2.2
+1.9
+1.9
+1.6
+1.6
+1.3
+1.3
+0.9
+0.9
+0.6
+0.6
+0.3
+0.3
0
0
-0.3
-0.3
-0.6
-0.6
-0.9
-0.9
-1.3
-1.3
-1.6
-1.6
-1.9
-1.9
-2.2
-2.2
-2.5
250.00 ms
ene
-2.5
Our original question:
Why VrV > VrV, but VlV > VllV?
kʰo:rasɨ
‘chorus’
kʰolla
‘cola’
Inaccurate Perception: Why Korean
listeners interpret ‘cola’ as [kolla]

Korean listeners hear laterality of ‘cola’.

This is a stronger cue than durational
cue—in Korean, any lateral segment
between vowels must be long.
Mishearing in early processing?

ERP data suggests Korean speakers’
brains respond less strongly to the
durational difference in [ele] vs. [elle] than
in [ene] vs. [enne].

Native Language Neural Commitment
Hypothesis (Kuhl 2004).
Implications for Foreign Language
Teaching

If students can’t even hear some of the FL
contrasts, will perception training help
them?
It’s not all bad news
•
Dehaene-Lambertz et al. (2000): at later
time windows (where conscious
processing may be involved), Japanese
speakers responded to ebzo-ebuzo
contrast more like French speakers .
It’s not all bad news

Tremblay et al. (1997): training DID
improve MMN response to non-native
contrast.
But problems do persist, at least for some
non-native contrasts.
Interim Summary

Formal phonology tells you which production
grammar patterns are learnable.

3 unlearnable patterns (in Fijian, Malayalee
English, Korean) result from transfer of
perception grammar—foreign language
acoustic cues are interpreted in terms of NL
contrasts.

This misperception seems to occur at early
levels of processing, and to persist even after
extensive exposure to foreign language.

However, listeners may be able to overcome
this at higher levels of processing.