Transcript Local Search for Bottom

A Pipeline Model for Bottom-Up
Dependency Parsing
Tenth Conference on Natural Language Learning, Shared Task
New York, USA, 2006
Ming-Wei Chang, Quang Do, Dan Roth
Computer Science Department
University of Illinois, Urbana-Champaign
1
Dependency Parsing Approach: Summary

Modified Shift-Reduced parser.

Actions are selected via a classifier
+ Extended action set
+ Look ahead search

Control Policy: Left to right, with step back

Dependency Types: A separate multiclass classifier

Multilingual: Convert non-projective languages to projective
[Nirve and Nilsson, 2005]
2
Motivation

Shift-Reduced Parsing as a pipeline model:


A classifier is used to determine which action to take
The decision at each stage depends on previous decisions.
+ Making a decision can rely on information acquired in
previous stages
- Making a decision can rely on incorrect
information
acquired in previous stages.

Viewed this way, we want to:
1.
2.
Reduce the number of decisions
Make local decisions more robust
3
Parsing Algorithm



Parsing from left to right; considering pair of (currently)
consecutive words (a,b) (with a<b)
For the pair (a, b), to become the child of a, b must be a complete
subtree.
Standard action set: Left, Right, Shift [Yamada and Matsumoto, 03],



Left: a is the parent of b
Right: b is the parent of a
Shift: the action is not Left or Right.
L
a
b


WaitLeft: a is the parent of b, but b
is the parent of other nodes. Action is deferred.
WaitRight (?): not needed !
x
b
b a
a
WL
+ We split Shift into Shift, WaitLeft, WaitRight

S
R
a
a
b
c
b
c
d
a
b
d
Control policy: Step Back


Provably, allows parsing in one pass over the sentence [ACL06]
Reduces the number of decisions
c
4
A Pipeline Model with Look Ahead Search

Pipilining decisions may result in error
accumulation:

The correct dependencies

w
If the algorithm
decides w  x
before x y and x  z,
we cannot recover the correct parent for y and z.
x
y
z


Correct early decisions are crucial


w
y
z
x
A look ahead search algorithm takes into
account future predicted actions
Local decisions are more robust
5
A Pipeline Model with Look Ahead – cont’



The search algorithm performs a search of length depth.
Additive scoring is used to score the sequence
The first action in this sequence is performed.
keep this
action
keep this
action
keep this
action
a0
a1
a0
a1
a
2
depth=1
depth=2
a0
a1
a
a
2
3
depth=3
6
Experiments (for Swedish)*


The effect of the new action.
The effect of look ahead search.
90
89.07
88.47
87.95
w/o WaitLeft
85
w WaitLeft
LS depth=3
82.31
81.48
81.82
80
Unlabeled Attachment Score
*
Labeled Attachment Score
For other languages, please refer to our paper.
7
Analysis

WaitLeft and the Look Ahead Search improve the parsing
results.
Results can be improved by:
 Selecting features and parameters more carefully


Currently we use exactly the same set of features and the same
parameters for all languages.
Using the FEAT column properly

The result for languages with FEAT column is generally worse than
the languages without FEAT column
8

Thank you !
9
It is not tractable to find the
global optimal predicted
sequence in the pipeline model
with the large depth.
In the pipeline framework,
the feature vector of current
decision depends on every
previous predictions.
The FEAT column
Average difference between our system with the best
systems
With Feats 4.5%, Without Feats 3.4%
10
Labeling the Dependency Type


A post-task after predicting the head for the tokens
in the sentence.
This is a multi-class classification.



Consider every edge of the tree
Classify the edge into several classes
The parents of the tokens which were labeled in the
first phase will be used as the features.
11
x
L
b
a
a
b
R
WL
a
a
b
b
c
12
90
89.07
88.47
87.95
w/o WaitLeft
85
w WaitLeft
LS depth=3
82.31
81.48
81.82
80
Unlabeled Attachment Score
Labeled Attachment Score
13
90
89.07
88.47
87.95
w/o WaitLeft
85
w WaitLeft
LS depth=3
82.31
81.48
81.82
80
Unlabeled Attachment Score
Labeled Attachment Score
14
90
88.47
87.95
w/o WaitLeft
w WaitLeft
85
LS depth=3
81.48
81.82
80
Unlabeled Attachment Score
Labeled Attachment Score
15
Experiments

We show the effect of the new action (on Swedish).
90
87.95
88.47
w/o WaitLeft
WaitLeft
85
w WaitLeft
WaitLeft
81.48
81.82
80
UAS
LAS
16
Experiments

The effect of look ahead search (on Swedish).
90
89.05 89.07
88.47
depth = 1
depth = 2
85
depth = 3
81.82
82.25 82.31
80
UAS
LAS
17