Transcript (PPT)

Barking Up the Wrong Treelength
Kevin Liu, Serita Nelesen, Sindhu Raghavan,
C. Randal Linder, and Tandy Warnow
IEEE TCCB 2009
Minimizing Treelength


Generalized

Input: set S of sequences and a function f(s, s') for
the edit distance between sequences s and s'

Output: A tree T, leaf-labelled by set S, with
additional sequences labelling the internal nodes of
T, so as to minimize treelength (total edit distance
on the edges of the tree)
Fixed Tree variant
POY




POY (from the American Museum of Natural History,
Ward Wheeler and colleagues) is the main software
for this.
Minimizing treelength is also known as “Direct
Optimization”
POY has passionate adherents who believe in
treelength
POY also has been heavily criticized
POY



Input: set S of sequences (unaligned), gap-open cost,
gap-extend cost, and transition/transversion ratio
Default settings for gap-open and gap-extend in POY
are “simple” (gap-open cost is 0)
POY can also be used to score a fixed input tree
under the desired treelength definition.
Ogden and Rosenberg 2007

Ogden and Rosenberg study compared POY 3.0 to
MP(ClustalW)

Model conditions – mostly 16 taxa (some 64 taxon trees),
K2P substitution model, short gaps (expected length 4)

Optimization Problem – Multiple edit distances, all on simple
gap penalties (gap-open cost is 0)

Performance metrics


Tree errors

Alignment errors

No mention of treelength
Result: MP(ClustalW) much more accurate than POY
O&R concluded that
Treelength is BAD!

O&R simulation study showed that POY alignments worse
than ClustalW more than 99% of the time, and POY trees less
accurate than ClustalW on average.

“Therefore, traditional multiple sequence alignment
approaches appear to vastly outperform direct
optimization-like approaches in terms of alignment
accuracy, at least for the data sets and parameter
settings that have been examined thus far.”

Ogden and Rosenberg 2007
Treelength is BAD!

“Although our data represents a fairly simple
case, for data sets similar to these the
traditional two-step approach will almost always
give a more accurate alignment and will most
likely recover equally or more accurate
phylogenetic relationships than direct
optimization as implemented in POY.”

Ogden and Rosenberg 2007
Our question
Does minimizing treelength work poorly in
general,
or
Is it minimizing treelength under simple gap
penalties that works poorly?
Gap penalties



Simple: a gap of length k costs kC
Affine: a gap of length k costs Copen+kCextend
Other types of penalties are possible
“Treelength not so bad!”
(paraphrasing Liu et al 2009)
Liu et al. 2009 show
 Treelength can be a good criterion, if based
upon affine gap penalty
 We developed POY*: a version of POY which
uses:


a particular affine gap penalty,
and a particular starting tree
Our Study 2008

Our study compares POY 4.0 to multiple
methods

Model conditions – 25 and 100 taxa, GTR+Gamma
for the substitution model, short and long gaps

Optimization Problem – Multiple edit distances,
based upon both simple and affine gap penalties

Results



Tree error
Alignment error
Treelength
Gap cost functions we studied



Simple1 – all mismatches and indels cost 1
Simple2 – indels cost 2, transversions cost 2 and
transitions cost 1
Affine – gap of length k costs 4 + k, transversions cost
2, and transitions cost 1
Simulation Study Overview

Model trees

Birth-death

Deviation from ultrametricity

Sequence evolution

Estimation of trees and alignments

Statistics
Simulation Study Overview

Model trees

Sequence evolution

GTR model of evolution from Tree of Life project

Gamma-distributed rates across sites

Gap model

Estimation of trees and alignments

Statistics
Simulation Study Overview



Model trees
Sequence evolution
Estimation of trees and alignments





POY
POY* - POY with particular starting tree (Probtree,
using a particular Affine gap penalty
Several two-phase methods (best alignments
followed by MP and ML)
PS (POY-score) on various trees
Statistics
Simulation Study Overview

Model trees

Sequence evolution

Estimation of trees and alignments

Statistics
1. Alignment error
2. Tree error
3. Treelength under each gap cost function
Simulation Study Model Conditions

4 model conditions

80 replicate datasets apiece

Different numbers of taxa allow us to explore
taxonomic sampling effects
Results –
Alignment
Errors


Simple vs. affine
penalties
Note: story
changes for
affine penalties,
especially on
long gap event
distribution
Alignment Error: ClustalW vs. POY*


POY* better than ClustalW over 50% in (b), and
90% of time under (a)
Compare with Ogden and Rosenberg, who find
ClustalW better than POY 99.9% of time
Results –
Alignment
Errors


PS is POY used to
estimate alignments
on various trees
Note: PS produces
worse alignments
than ClustalW if
simple gap cost
functions are used,
even if applied to
the true tree
Tree error
POY and POY* both use the same
gap penalty (affine)
Results shown on 100 taxon short
gap simulated datasets (results
for other models similar)
Tree Error
POY and POY* both use the same
gap penalty (affine)
Results shown on 100 taxon short
gap simulated datasets (results
for other models similar)
Tree error
POY and POY* both use the same
gap penalty (affine)
Results shown on 100 taxon short
gap simulated datasets (results
for other models similar)
How well does POY solve its
optimization problem?



We examine the treelength found by POY for
various model conditions
We let treelength be defined by simple1,
simple2, or affine
We compare treelengths found by POY to
treelengths achievable in each model condition
(as produced by scoring the true tree and other
trees)
Results – Simple Treelength Criteria
Results –
Affine
Treelength
Criterion
Results - Treelengths



POY search finds short trees for simple gap
penalties, but not for affine
Can we propose a better POY search for affine
penalties?
POY*
How well does POY solve its
optimization problem?


Simple gap penalties: excellent performance
Affine gap penalties: poor performance
But POY* optimizes both well.
The difference is just the starting tree.
Is it a good idea to optimize
treelength?


Simple gap penalties: NO! Worse trees and
worse alignments.
Affine gap penalties: Let’s see.
POY vs. POY* using affine gap
Insights
Simple gap penalties were a main cause behind Ogden
and Rosenberg's findings

Unable to obtain accurate POY alignments and trees under a simple
treelength criterion
Using affine penalties, POY*:



Obtains alignments that are more accurate than ClustalW 90% of long gap
datasets, 75% of medium, 55% of short
Has tree accuracy that is comparable to the best two-phase method (ML
on good alignments)
But poorer alignments than the best alignment methods (e.g., Probtree)
Conclusions


Distinguish between the optimization problem,
and the heuristic methods used for those
problems
The treelength optimization criteria chosen has
a significant impact on the tree and alignment
error

Simple alignment and trees aren't competitive
relative to two-phase methods, and improving
simple criteria treelengths doesn't get better trees

Affine criteria story is still open


Can we find shorter trees than two-phase trees?
How accurate are such shorter trees?
Questions?