Transcript Slide 1

Computer-Assisted Listening and Speaking Tutor
Jacques Koreman
Åsta Øvregaard
Egil Albertsen
Sissel Nefzaoui
Eli Skarpnes
Dept. of Language and Communication Studies
Styringsdialog,
November 2009
NKUL10, 7 May 19
2010
slide 1
Outline
• What is CALST?
• Three parts:
• Basic vocabulary: enabling simple communication
• Contrastive listening training: recognizing the sounds
of Norwegian, including different dialects
• Pronunciation training: speaking to be understood
• Future work:
• Analysing problems foreigners have
• Automatic evaluation of pronunciation errors
Styringsdialog, 19 November 2009
slide 2
What is CALST?
CALST is a collaborative project on computer-assisted
pronunciation teaching (CAPT) for Norwegian.
• Time/place chosen by L2-learner
• Individualized learning (for L1 and variety of Norwegian)
• Prestige (pronunciation errors)
• Combined with classroom teaching
pronunciation
immigration
Styringsdialog,
November 2009
NKUL10, 7 May 19
2010
communication
integration
slide 3
Cross-disciplinary collaboration
VOX
IMDi
Research
ISK
Norwegian as L2
Phonetics
Practice
Technology
ISK + ILN
courses for students
and employees
EVO
immigrants
Styringsdialog,
November 2009
NKUL10, 7 May 19
2010
Norgesuniversitet
HF, NTNU
CALST
KTH
basic CAPT system
(www.speech.kth.se/ville)
technical support
slide 4
CALST work packages
1. Development of basic lexicon for Norwegian
• Recordings of one male/one female speaker for 4 Norwegian
dialects (Østlandet, Vestlandet, Trøndelag and Nord-Norge):
role model, no single standard
• Alignment of talking face with speech signal
• Selection and creation of pictures
2. Contrastive listening:
• Phonological contrastive analysis (what phonemes) and
phonetic analysis (how realized) for several L1
• Depends on dialect
3. Self-monitoring of pronunciation
• First learn to hear, then learn to speak
Styringsdialog,
November 2009
NKUL10, 7 May 19
2010
slide 5
Basic vocabulary learning
• First step towards communication
• Simple and intuitive user interface
• Train and test mode
• Extra information on flash cards:
• English translation
• Declinations and inflexions
Styringsdialog, 19 November 2009
slide 6
Basic vocabulary: selection criteria
A1 Has a basic vocabulary repertoire of isolated words and
phrases related to particular concrete situations.
A2 Has a sufficient vocabulary for the expression of basic
communicative needs.
Has a sufficient vocabulary for coping with simple survival
needs.
Has sufficient vocabulary to conduct routine, everyday
transactions involving familiar situations and topics.
Common European Framework of Reference for Languages:
Learning, Teaching, Assessment, http://www.coe.int/T/DG4/
Portfolio/documents/Framework_EN.pdf, p.110.
Styringsdialog, 19 November 2009
slide 7
Basic vocabulary
Ny i Norge
(n = 1579)
493
På vei
(n = 1485)
233
452
608
192
245
euroFluent
(n = 1541)
496
total n = 2719
Ellingsen og Mac Donald (2004), På vei. Oslo: J. W. Cappelens forlag A/S.
euroFluent (2008), www.eurofluent.net.
Manne og Nilsen (2004), Ny i Norge. Bergen: Forlaget Fag og Kultur AS.
Styringsdialog, 19 November 2009
slide 8
Basic vocabulary
• Comparison with basic vocabulary og Lexin, a webbased dictionary developed specially for immigrants
under the auspices of Utdanningsdirektoratet: 349
additional words.
• Some lacking cardinal and ordinal numbers 1-20, all
tens up to 100, etc.
• Words from this total word set were used to build up
semantic categories.
• Within a semantic category, the grammatical category
of the words was the same.
• Additional (non-semantic) categories for strong verbs,
weak verbs, years (numbers) and phrases.
Styringsdialog, 19 November 2009
slide 9
Semantic categories (16-32 words)
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.
17.
18.
19.
20.
Animals
Emotions
Family 1
Family 2
Colours
Geography (topography)
Houses (buildings)
Household 1 (living room, etc.)
Household 2 (kitchen, etc.)
Clothing
Countries and continents
Body
Health
Food 1 (elementary)
Food 2 (expanded)
Mathematics
Nationality
Plants
Position and direction 1 (adv.)
Position and direction 1 (prep.)
Styringsdialog, 19 November 2009
32
30
28
21
16
24
23
18
29
32
24
27
16
32
32
19
25
19
16
16
21. Tools 1 (personal belongings)
22. Tools 2 (kitchen, workshop)
23. Travel
24. Sport and sparetime
25. Numbers 1 (cardinal)
26. Numbers 2 (similar-sounding)
27. Numbers 3 (similar-sounding)
28. Numbers 4 (ordinal)
29. Time 1 (weekdays, months)
30. Time 2 (time of day)
31. Education
32. Weather and climate
33. Work
34. Economy
35. Years (culturally relevant)
36. Weak verbs
37. Strong verbs
38. Phrases
39. Pronouns
32
32
20
28
52
42
38
32
23
18
24
20
25
30
18
32
27
16
21
slide 10
Basic vocabulary selection
• All words were given priority 1-3 on the basis av
pedagogical considerations (cf. CEF A1 and A2 criteria):
1. necessary (1200 words), 2 useful (700 words), 3. nice to have (1200 words).
Category:
Priority 1
Priority 2
Priority 3
Utdanning
skole
fag
opplæring
Familie
mor
samboer
ektepar
Dyr
fisk
torsk
bikkje
• All words were given a visualizability value 1-3 by the
person who was hired to produce the pictures.
Category:
Visualizability 1
Visualizability 2
Visualizability 3
Utdanning
tavle
naturfag
utdanning
Familie
barn
slekt
skilsmisse
Geografi
øy
kyst
område
→ 1000-word basic vocabulary
Styringsdialog, 19 November 2009
slide 11
Visualizing words
• Simple, stylized pictures for quick perception.
• Taken from UVic’s Language Teaching Clipart Library
database, and expanded by the project (310 drawn by
hand, computer drawn).
• First drawn on paper, scanned in and coloured using Paint
and PhotoFiltre.
• Neutral to gender and culture – if possible.
• Drawn on transparent background to allow green (“correct”)
or red filling (“incorrect”).
• Some categories not use individual pictures for each word,
but instead use a composite picture, e.g. to express family
relationships in a family tree or to visualize states in a map
of the world.
Styringsdialog,
November 2009
NKUL10, 7 May 19
2010
slide 12
Norwegian dialects
• Norwegian has a large number of different dialects
(so what, so do other languages)
• And no real standard pronunciation variant
(e.g. English, German and Dutch do)
• There is no tradition in Norway to accommodate to
problems in understanding dialects (overstatement!)
• Learners of Norwegian have to deal with this in
everyday communication situations
• Standard classroom situation: training in listening to
and speaking Urban East Norwegian (østlandsk)
• CALST: choice between 4 main dialect regions with a
male and a female speaker (role model) for each, or
combination of dialects
Styringsdialog,
November 2009
NKUL10, 7 May 19
2010
slide 13
Next step: contrastive listening training
• What is easy for a Norwegian speaker/listener, may be
difficult for an L2-learner:
• “bi-by-bu”
• Retroflexion: “har det” (“ha det”)
• Aspiration: [ph, th, kh]
• And do Norwegians have a pronunciation problem in
Norwegian: “7.” = “20.”?
(The loss of this opposition means one problem less for
foreigners! – but at the expense of many misunderstandings)
• We’re all foreigners, almost everywhere:
• “blue eyes” or “blue ice”?
• “very well”
Styringsdialog,
November 2009
NKUL10, 7 May 19
2010
slide 14
Selecting focus groups
• Original proposal: contrastive analysis of most frequent
and problematical foreign languages in comparison to
Norwegian
• Analysis of exam results for Norwegian courses in last
5 years underway: test results and pronunciation grade
analyzed for
• native language background
• gender
• age
• position (exchange student, Ph.D., NTNU employee)
• study program or institute
• Also based on data/experience from UiO and EVO
Styringsdialog,
November 2009
NKUL10, 7 May 19
2010
slide 15
Contrastive analysis: thinking big?
• UPSID: UCLA Phonological Segment Inventory
Database contains phonemic inventory of Norwegian
compared to 450 (!) other languages
• Can we derive an automatic analysis for each of this
automatically?
web.phonetik.uni-frankfurt.de/upsid.html
• Other information on the web: syllable structure, word
stress, tone, … (SOWL)
• Use L1-L2 differences to guide users of the CAPT
system through exercises
…or let them also do “easy” exercises for familiar
contrast for motivation?
Styringsdialog,
November 2009
NKUL10, 7 May 19
2010
slide 16
Possible follow-ups
1. CALST follow-up programmed
• Test data from CALST on server (also after project!)
• logged together with student background data:
gender, age, language background, length of residence,
drop-out rate, grade, etc.
• new project on Comparative Analysis of various L1?
2. CAPT’N
• Cross-disciplinary project submitted to NFR in 2008:
technology + phonetics
• Aim: automatic analysis of learners’ pronunciation of
problematical sound contrasts + phonetic feedback.
• Project rejected: “too applied” + cross-disciplinary project
possibly difficult to review
• IET resubmitting the technological part of this project.
Styringsdialog,
November 2009
NKUL10, 7 May 19
2010
slide 17
Computer-Assisted Listening and Speaking Tutor
Thank you for your attention!
Styringsdialog,
November 2009
NKUL10, 7 May 19
2010
slide 18