Transcript No Slide Title

Kathy McCoy
Artificial Intelligence
Natural Language Processing
Applications for People with Disabilities
Primary Research Areas

Natural Language Generation – problem of choice.





Deep Generation --- structure and content of coherent text
Surface Generation – particularly using TAG (multi-lingual
generation and machine translation)
Discourse Processing
Second Language Acquisition
Applications for people with disabilities affecting
their ability to communicate
Projects

ICICLE – CALL system for teaching English as a
second language to ASL natives (Chris Pennington,
Rashida Davis, Mike Bloodgood, David Derman)

Augmentative Communication – Center for
Applied Science and Engineering in Rehabilitation
(ASEL) – Word Prediction and Contextual Information


Text Summarization (Greg Silber)
Multi-lingual Sentence Generation (Raymond
Kozlowski, Vijay Shanker)

Graph-to-Text (Sandee Carberry, Dan Chester,
Stephanie Elzer, Nancy Green)
Developing Intelligent
Communication Aids for
People with Disabilities
Kathleen F. McCoy
Computer and Information Sciences &
Center for Applied Science and
Engineering in Rehabilitation
University of Delaware
Augmentative Communication

Intervention that gives non-speaking person an
alternative means to communicate
User Population
 May have severe motor impairments





Unable to speak
Unable to write
Cannot use sign language
May have cognitive impairments and/or developmental
disabilities
May be too young to have developed literacy skills
Row-Column Scanning
Row-Column Scanning II
Language Representation: Words
Still Need to Spell!
Predicting Fringe Vocabulary
Word Prediction of Spelled Words (infrequent
context-specific words)
Methods
 Statistical NLP Methods
 Syntax/Semantic Filters
 Other Contextual Clues


Geographic Location, Time of Day, Conversational
Partner, Topic of Conversation
Language Representation: Phrases
Delivering Text in Context – What
must Technology Facilitate?
Context: Video Store want Grisham’s “A Time to
Kill”
Prestored Message: Many of Grisham’s books
have been made into movies.
Choice:
 Edit message to be perfect
 Let listener know and edit to be perfect
 Deliver message quickly as is
Hypothesis: Perception Different
Depending on Kind of Mismatch
Grice Theory of Language Maxims




Quantity – amount of information provided
Quality – truth value and adequacy of message
Relation – relevance
Manner – way message is delivered
Experiments




Book Store Situation
Actors playing roles
AAC User wants to purchase books
Subjects are clerks, asked to put themselves in
clerk’s position
Findings



Some maxims are more important than others
Need to develop technologies to support
prestored text delivery obeying the important
maxims
Current work: looking at repair strategies and
how the repair process can be supported
Modeling the Acquisition of
English in the ICICLE System
Kathleen F. McCoy
Department of Computer and Information Sciences
University of Delaware
People

Current People





Recent Graduate


Chris Pennington
Mike Bloodgood
Rashida Davis
David Durmond
Lisa Masterman Michaud
Others


Greg Silber, Meghan Boyle, Mohamed Mostagir, Stephanie
Baker, Heejong Yi
Graduates: Matthew Huenerfauth, Jill Janofsky, Litza Stark,
David Schneider
The ICICLE Project
Interactive Computer Identification and
Correction of Language Errors


Interactive writing tutor for native signers of
American Sign Language (ASL)
Purpose: analyze student-written English texts
and provide individualized feedback and
instruction on grammar
The ICICLE Project

Cycle of user input, system response
student provides piece of text
 system analyzes text for grammatical
errors



system provides student with tutorial
instruction on the errors
student has opportunity to make
corrections and request re-analysis
The
ICICLE
System
Current Implementation
the student
enters text here
the system shows
which sentences
have errors
explanations
shown here
Writing From Deaf Students



Literacy is a serious issue for the Deaf population.
Lots of variation in level of acquisition.
Marked Differences from writing of hearing peers.



Dropped be: She really pretty.
Missing Possessives: She age is 13.
Subject/verb agreement, plural markers, determiners: She
really like go with friend to mall.
Work on ICICLE
Previous work focused on developing grammar
and mal-rules and modeling the user’s level of
acquisition (so different analyses can be found
depending on it)
Current Work
 Tutorial Responses
 Probabilistic Parsing
 NEED SYSTEM HELP!!!!!

What Mal-Rules do We Use?
“She is teach piano on Tuesdays.”

Beginner: Over-application of auxiliary IS,
missing simple present morphology:


Intermediate: Botched progressive tense:


She teaches piano on Tuesdays.
She is teaching piano on Tuesdays.
Advanced: Botched passive voice:

She is taught piano on Tuesdays.
Text Summarization
Greg Silber
Lexical Chain Interpretation



What is important to include in a summary?
Focus on nouns
Coherent text will repeat noun concepts



A particular noun concept may be referred to with different
lexical items. E.g., computer, machine, Sun
Developed a linear time algorithm for determining
interpretation of words that allow most “coherent text”
Those concepts repeated most often belong in a
summary
Current Status




Algorithm can pick out important noun
concepts
Current Question: What do you say about those
nouns????
Looking at picking out predicates that link
important noun concepts in a text
Generating a coherent summary