Transcript Ch. 14: Understanding Natural Language

14
14.0
14.1
Understanding Natural Language
The Natural Language
Understanding
Problem
Deconstructing
Language: A Symbolic
Analysis
14.2
Syntax
14.3
Syntax and Knowledge
with ATN parsers
14.4
Stochastic Tools for
Language Analysis
14.5
Natural Language
Applications
14.6
Epilogue and
References
14.7
Exercises
Additional source used in preparing the slides:
Patrick H. Winston’s AI textbook, Addison Wesley, 1993.
1
Chapter objective
• Give a brief introduction to deterministic
techniques used in understanding natural
language
2
An early natural language understanding
system: SHRDLU (Winograd, 1972)
3
It could converse about a blocks world
• What is sitting on the red block?
• What shape is the blue block on the table?
• Place the green cylinder on the red brick.
• What color is the block on the red block?
Shape?
4
The problems
• Understanding language is not merely
understanding the words: it requires inference
about the speaker’s goals, knowledge,
assumptions. The context of interaction is also
important.
 Do you know where Rekhi 309 is?
 Yes.
 Good, then please go there and pick up the documents.
 Do you know where Rekhi 309 is?
 Yes, go up the stairs and enter the semi-circular section.
 Thank you.
5
The problems (cont’d)
• Implementing a natural language
understanding program requires that we
represent knowledge and expectations of the
domain and reason effectively about them.
 nonmonotonicity
 belief revision
 metaphor
 planning
 learning
 …
Shall I compare thee to a summer’s day?
Thou art more lovely and more temperate:
Rough winds do not shake the darling buds of May,
And summer’s lease hath all too short a date:
Shakespeare’s Sonnet XVIII
6
The problems (cont’d)
• There are three major issues involved in
understanding natural language:
 A large amount of human knowledge is assumed.
 Language is pattern based. Phoneme, word, and
sentence orders are not random.
 Language acts are products of agents embedded in
complex environments.
7
SHRDLU’s solution
• Restrict focus to a microworld :
blocks world
• Constrain the language:
use templates
• Do not deal with problems involving
commonsense reasoning:
still can communicate meaningfully
8
Linguists’ approach
• Prosody: rhythm and intonation of language
• Phonology: sounds that are combined
• Morphology: morphemes that make up words
• Syntax: rules for legal phrases and sentences
• Semantics: meaning of words, phrases,
sentences
• Pragmatics: effects on the listener
• World knowledge: background knowledge of
the physical world
9
Stages of language analysis
1. Parsing: analyze the syntactic structure of a
sentence
2. Semantic interpretation: analyze the meaning
of a sentence
3. Contextual/world knowledge representation:
Analyze the expanded meaning of a sentence
For instance, consider the sentence:
Tarzan kissed Jane.
10
The result of parsing would be:
11
The result of semantic interpretation
12
The result of contextual/world knowledge
interpretation
13
Can also represent questions:
Who loves Jane?
14
Parsing using Context-Free Grammars
A bunch of rewrite rules:
1. sentence
2. noun_phrase
3. noun_phrase
4. verb_phrase
5. verb_phrase
6. article
7. article
8. noun
9. noun
10. verb
11. verb
 noun_phrase verb_phrase
 noun
 article noun
 verb
 verb noun_phrase
a
 the
 man
 dog
 likes
 bites
these are the nonterminals
these are the terminals
these are the symbols of the grammar
15
Parsing
• It is the search for a legal derivation of the
sentence.
• sentence  noun_phrase verb_phrase
 article noun verb_phrase
 The noun verb_phrase
 The man verb_phrase
 The man verb noun_phrase
 The man bites noun_phrase
 The man bites article noun
 The man bites the noun
 The man bites the dog
• Each intermediate form is a sentential form .
16
Parsing (cont’d)
• The result is a parse tree. A parse tree is a
structure where each node is a symbol from the
grammar. The root node is the starting
nonterminal, the intermediate nodes are
nonterminals, the leaf nodes are terminals.
• “Sentence” is the starting nonterminal.
• There are two classes of parsing algorithms
 top-down parsers: start with the starting symbol and try
to derive the complete sentence
 bottom-up parsers: start with the complete sentence and
attempt to find a series of reductions to derive the start
symbol
17
The parse tree
18
Parsing is a search problem
• Search for the correct derivation
• If a wrong choice is made, the parser needs to
backtrack
• Recursive descent parsers maintain backtrack
pointers
• Look-ahead techniques help determine the
proper rule to apply
• We’ll study transition network parsers (and
augmented transition networks)
19
Transition networks
20
Transition networks (cont’d)
• It is a set of finite-state machines representing
the rules in the grammar
• Each network corresponds to a single
nonterminal
• Arcs are labeled with either terminal or
nonterminal symbols
• Each path from the start state to the final state
corresponds to a rule for that nonterminal
• If there is more than one rule for a nonterminal
there are multiple paths from the start to the
goal (e.g., noun_phrase)
21
The main idea
• Finding a successful transition through the
network corresponds to replacement of the
nonterminal with the RHS
• Parsing a sentence is a matter of traversing
the network:
 If the label of the transition (arc) is a terminal, it must
match the input, and the input pointer advances
 If the label of the transition (arc) is a nonterminal, the
corresponding transition network is invoked recursively
 If several alternative paths are possible, each must be
tried (backtracking)---very much like nondeterministic
finite automaton---until a successful path is found
22
Parsing the sentence “Dog bites.”
23
Notes
• A “successful parse” is the complete traversal
of the net for the starting nonterminal from
sinitial to sfinal .
• If no path works, the parse “fails.” It is not a
valid sentence (or part of sentence).
• The following algorithm would be called using
parse(sinitial )
• It would start with the net for “sentence.”
24
The algorithm
Function parse(grammar_symbol);
begin
save pointer to current location in input stream;
case
grammar_symbol is a terminal;
if grammar_symbol matches the next word in the input stream then
return(success)
else begin
reset input stream
return(failure)
end;
25
The algorithm (cont’d)
… case …
…
grammar_symbol is a nonterminal;
begin
retrieve the transition network labeled by grammar_symbol
state := start state of network;
if transition(state) returns success
then return(success)
else begin
reset input stream;
return (failure)
end
end
end.
26
The algorithm (cont’d)
Function transition(current_state);
begin
case
current_state is a final state:
return (success)
current_state is not a final state:
while there are unexamined transitions out of current_state
do begin
grammar_symbol := the label on the next unexamined transition
if parse(grammar_symbol) returns (success)
then begin
next_state := state at the end of the transition;
if transition(next_state) returns (success);
then return(success)
end
end
return(failure)
end
27
end.
Modifications to return the parse tree
1. Each time the function parse is called with a
terminal symbol as argument and that terminal
matches the next symbol of input, it returns a
tree consisting of a single leaf node labeled
with that symbol.
2. When parse is called with a nonterminal, N, it
calls transition. If transition succeeds, it returns
an ordered set of subtrees. Parse combines
these into a tree whose root is N and whose
children are the subtrees returned by transition.
28
Modifications to return the parse tree (cont’d)
3. In searching for a path through a network,
transition calls parse on the label of each arc.
On success, parse returns a tree representing a
parse of that symbol. Transition saves these
subtrees in an ordered set and, on finding a
path through the network, returns the ordered
set of parse trees corresponding to the
sequence of arc labels on the path.
29
Comments of transition networks
• They capture the regularity in the sentence
structure
• They exploit the fact that only a small
vocabulary is needed in a specific domain
• If a sentence “doesn’t make sense”, it might
be caught by the domain information. For
instance, the answer to both of the following
questions is “there is none”
 “Pick up the blue cylinder”
 “Pick up the red blue cylinder”
30
The Chomsky Hierarchy and CFGs
• A CFG: a single nonterminal is allowed on the
left-hand side.
• CFGs are not powerful enough to represent
natural language
• Simply add plural nouns to the dogs world
grammar:
noun  men
noun  dogs
verb  bites
verb  like
“A men bites a dogs” will be a legal sentence
31
Options to deal with context
• Extend CFGs
• Use context-sensitive grammars (CSGs)
With CSGs the only restriction is that the RHS
is at least as long as the LHS
• Note that the one higher class, recursively
enumerable languages or Turing recognizable
languages is not an usually regarded as an
option
32
A context-sensitive grammar
sentence  noun_phrase verb_phrase
noun_phrase  article number noun
noun_phrase  number noun
number  singular
number  plural
article singular  a singular
article singular  the singular
article plural  the plural
singular noun  dog singular
singular noun  man singular
plural noun  men plural
plural noun  dogs plural
singular verb_phrase  singular verb
plural verb_phrase  plural verb
33
A context-sensitive grammar (cont’d)
singular verb  bites
singular verb  likes
plural verb  bite
plural verb  like
34
“The dogs bite”
sentence  noun_phrase verb_phrase
 article plural noun verb_phrase
 The plural noun verb_phrase
 The dogs plural verb_phrase
 The dogs plural verb
 The dogs bite
35
CSGs for practical parsing
1. The number of rules and nonterminals in the
grammar increase drastically.
2. They obscure the phrase structure of the
language that is so clearly represented in the
context-free rules
3. By attempting to handle more complicated
checks for agreement and semantic
consistency in the grammar itself, they lose
many of the benefits of separating the
syntactic and semantic components of
language
4. CSGs do not address the problem of building
a semantic representation of the text
36