Transcript Data-Oriented Semantics?

Data-Oriented Semantics?
Remko Scha
Institute for Logic, Language and Computation
University of Amsterdam
Semantic aspects of
Data-Oriented Parsing (DOP)
• A simple Data-Oriented Parsing model
• The Big Picture (with an application to semantics)
• Data-Oriented Semantic Interpretation
• Extended DOP models
Data-Oriented Parsing (DOP)
Memory-based approach to syntactic parsing and
disambiguation.
Basic idea: use the subtrees from a syntactically
annotated corpus directly as a stochastic
grammar.
Data-Oriented Parsing (DOP)
Simplest version: DOP1 (Bod 1992).
Annotated corpus defines
Stochastic Tree Substitution Grammar
Data-Oriented Parsing (DOP)
Simplest version: DOP1 (Bod 1992).
Annotated corpus defines
Stochastic Tree Substitution Grammar
(Slides adapted from Guy De Pauw,
University of Antwerp)
S
S
NP
S
VP
NP
killed
NP
NP
a
VP
killed
NP
S
Peter
raccoon
NP
S
NP
VP
killed
Peter
NP
a
raccoon
VP
Peter
VP
killed
VP
killed
NP
NP
a
raccoon
NP
a
NP
S
NP
VP
Peter
raccoon
S
S
NP
VP
NP
the
bear
ate
VP
VP
NP
ate
NP
ate
S
NP
S
VP
ate
honey
NP
NP
the
NP
VP
bear
ate
S
NP
honey
honey
NP
the
S
NP
NP
VP
NP
VP
the
bear
honey
ate
NP
VP
bear
Treebank
S
NP
S
NP
VP
Peter
killed
S
S
NP
NP
VP
killed
NP
VP
a
a
NP
VP
Peter
killed
NP
raccoon
S
S
VP
NP Peter
killed
NP
S
VP
S
raccoon
VP
NP
NP
S
VP
S
NP
NP
VP
NP
NP
killed
the
the
bear
ate
NP
NP
a
raccoon
NP
Peter
ate
NP
NP
NP
VP
VP
a
ate
NP
bear
NP
bear
S
the
killed
VP
VP
VP
ate
ate
VP
S
NP
VP
ate
NP
NP
NP
honey
honey
honey
raccoon
the
bear
NP
honey
Data-Oriented Parsing
Sentence to be parsed:
Peter killed the bear
S
S
NP
Peter
the
NP
NP
the
bear
bear
S
S
NP
Peter
killed
Peter
NP
NP
killed
VP
VP
NP
VP
VP
NP
NP
killed
NP
the
Peter
bear
1 parse; multiple derivations
VP
VP
killed
NP
the
bear
An annotated corpus defines a Stochastic
Tree Substitution Grammar
Probability of a Derivation:
Product of the Probabilities of the Subtrees
An annotated corpus defines a Stochastic
Tree Substitution Grammar
Probability of a Derivation:
Product of the Probabilities of the Subtrees
Probability of a Parse:
Sum of the Probabilities of its Derivations
An annotated corpus defines a Stochastic
Tree Substitution Grammar
Probability of a Derivation:
Product of the Probabilities of the Subtrees
Probability of a Parse:
Sum of the Probabilities of its Derivations
Disambiguation:
Choose the Most Probable Parse
Data-Oriented Parsing
Many computational issues
Many different models, with different
Annotation conventions
Fragment definitions
Probability calculations
Part II
The Big Picture
The Data-Oriented World View
Parsing is just an example: all of perception and
cognition may be usefully analyzed from a
data-oriented point of view.
The Data-Oriented World View
Parsing is just an example: all of perception and
cognition may be usefully analyzed from a
data-oriented point of view.
All interpretive processes are based on detecting
similarities and analogies with concrete past
experiences.
The Data-Oriented World View
Parsing is just an example: all of perception and
cognition may be usefully analyzed from a
data-oriented point of view.
All interpretive processes are based on detecting
similarities and analogies with concrete past
experiences.
For instance: Visual Processing
The Data-Oriented World View
Parsing is just an example: all of perception and
cognition may be usefully analyzed from a
data-oriented point of view.
All interpretive processes are based on detecting
similarities and analogies with concrete past
experiences.
For instance: Music Perception
(Cf. current work by Rens Bod)
The Data-Oriented World View
Parsing is just an example: all of perception and
cognition may be usefully analyzed from a
data-oriented point of view.
All interpretive processes are based on detecting
similarities and analogies with concrete past
experiences.
For instance: Lexical Semantics and Concept
Formation.
The Data-Oriented Perspective on Lexical Semantics
and Concept Formation.
A concept is the extensional set of its previously
experienced instances.
Classifying a new input under an existing
concept involves judging the input's similarity
to these instances.
Against:
Explicit definitions
Prototypes
The Data-Oriented Perspective on Lexical Semantics
and Concept Formation.
A concept is the extensional set of its previously
experienced instances.
Classifying a new input under an existing
concept involves judging the input's similarity
to these instances.
Against:
Explicit definitions
Prototypes
Learning
The Data-Oriented Perspective on Lexical Semantics
and Concept Formation.
Classifying a new input under an existing
concept involves judging the input's similarity
to the previously experienced instances of the
concept.
Controversial in linguistics and philosophy.
Cliché in Information Retrieval!
Part III
Data-Oriented Semantic Interpretation
Data-Oriented Semantic Interpretation
Add semantic annotations to the corpus trees.
(Van den Berg, Bod & Scha, 1994)
Add semantic annotations to the corpus trees
Assuming strict surface compositionality:
"daughter-notation".
d1(d2)
S
x d1(x, d2)
VP
d1(d2)
d1(d2)
NP
det
|
S P x  S: P(x)
every
NP
N
|
MEN
man
V
|
LOVE
loves
det
|
S P  x  S: P(x)
a
N
|
WOMEN
woman
Add semantic annotations to the corpus trees
Assuming strict surface compositionality:
"daughter-notation".
Semantic interpretation becomes trivial:
every subtree specifies how it combines the
semantic information from its daughter trees
Add semantic annotations to the corpus trees
Alternative way to do this: full formulas on the
non-terminal nodes.
– suitable for idioms
– otherwise not attractive
Add semantic annotations to the corpus trees
Semantic annotation helps with disambiguation:
use semantically refined syntactic categories.
(Bonnema, 1996)
Part IV
Extended DOP models
Extended DOP models
Scha (1990) about an imagined future DOP algorithm:
It will be especially interesting to find out how such an
algorithm can deal with complex syntactic
phenomena such as "long distance movement". It is
quite possible that an optimal matching algorithm
does not operate exclusively on constructions which
occur explicitly in the surface-structure; perhaps
"transformations" (in the classical Chomskyan sense)
play a role in the parsing process.
Extended DOP models
Bod & Kaplan: LFG-DOP
Lexical-Functional Grammar
Hoogweg: TIG-DOP
Tree-Insertion Grammar
(Cf. Tree-Adjoining Grammar)
Sima'an: The Tree-Gram Model (Markov-processes on
sister-nodes, conditioned on lexical heads)
Extended DOP models
Transformations?
Extended DOP models
Transformations?
"Which woman do you think that every man . . . ?"
Extended DOP models
Transformations?
"Which woman do you think that every man . . . ?"
"Which donkey do you think that every man . . . ?"
Extended DOP models
Transformations?
"Which woman do you think that every man loves?"
"Which donkey do you think that every man beats?"
Extended DOP models
Transformations?
Wh-movement, Passivization, Topicalization, Fronting,
Scrambling, . . .?
Move-Alfa?
Extended DOP models
Transformations?
Wh-movement, Passivization, Topicalization, Fronting,
Scrambling, . . .?
Move-Alpha?
Syntax comes back with a vengeance
Extended DOP models
Semantics for the extended DOP models:
– Future research
– Maintain compositionality
– Cf. F-structures in LFG (Bod, 4 p.m.)
The Data-Oriented Perspective on
Perlocutionary Effect
"The effect of a lecture depends on the habits of
the listener, because we expect the language
to which we are accustomed."
Aristotle, Metaphysics II: 12,13
Data-Oriented Parsing as a cognitive model
S
VP
NP
NP
det
|
every
N
|
man
V
|
loves
det
|
a
N
|
woman
Data-Oriented Parsing as a cognitive model
S
VP
NP
NP
det
|
every
N
|
man
V
|
loves
det
|
a
N
|
woman