Transcript SNAP: Fast, accurate sequence alignment enabling biological

SNAP: Fast, accurate sequence alignment
enabling biological applications
Ravi Pandya, Microsoft Research
ASHG 10/19/2014
SNAP
SNAP is fast *
Align 50x genome in 1.2 hours
(BWA-MEM = 11.75 hours)
Sort + index + markdup BAM in 2 hours
(samtools+sambamba = 4.25 hours)
SNAP is as accurate as BWA-MEM, Bowtie2, etc.
ROC on simulated data
% aligned on real data
Variant calls on real data
* NA12878:ERR194147, Azure D14 (16 cores, 112GB RAM, 800GB SSD)
Sequence alignment
The problem:
Given a read R and a reference genome G
Find the position in p in G that minimizes
EditDistance(R, G[p .. p + |R|])
SNAP solves this quickly and accurately because of:
Efficient system architecture
Reducing the number of comparisons
Reducing the cost of comparisons
System architecture
empty
full
align
sort
temp file
mergesort
index
mark
duplicates
compress
async read
async write
Bill Bolosky, MSR
The sequence alignment problem
The easy part:
97% of 20-mers
in the human genome
occur only once
but at only 75% of locations
CDF of per-read/pair alignment time, NA18705 169M pairs
(using deeper search parameters than current defaults)
100%
10% of reads
90%
80%
70%
Single
Equally
Weighted
Single
Time
Weighted
60%
The hard part:
The other 3% of 20-mers
and 25% of locations
50%
40%
Paired
Equally
Weighted
30%
20%
10%
0%
95% of time
Paired
Time
Weighted
Bill Bolosky, MSR
Hash table lookup
Build a multi-valued map (~30GB for hg19)
from all seeds S in G  all locations of S in G
330 reads/s
For all seeds in read, all locations of seed in genome,
Score implied alignment of read, keep the best
42x
14k reads/s
Ignore frequent seeds (>300 occurrences)
Only use a few seeds/read
Bill Bolosky, MSR
Fast scoring
6.6x
92k reads/s
O(n2)  Ukkonen O(nd), n=len, d=min(limit, actual)
Use limit = best score so far + 2 (for MAPQ)
1.2x
113k reads/s
Sort candidates by # of seed hits
1.4x
154k reads/s
(470x overall)
Skip locations with #seed misses > limit
Bill Bolosky, MSR
Paired-end alignment
Find & score candidate location pairs
C(R1:R2) = C(R1) ∩ C(R2) {± insert size}
Enumerate in O(h log n)
h = |C(R1) ∩ C(R2)|
n = |C(R1)| + |C(R2)|
Increases accuracy by allowing
much higher limit on seed occurrences
(e.g. 4k vs 300)
Results: simulated data
Mason-generated paired-end 100bp reads
Results: real data
NA18507 (Illumina HiSeq 50x)
* AWS cr1.8xlarge (32 cores, 244GB RAM, 2x120GB SSD)
Results: GATK variant calls
Broad GATK pipeline, curated NA12878 variant calls
Results: NIST Genome-in-a-Bottle
11.75
Appistry GATK pipeline, GIAB highly confident calls
Longer seeds are much faster, similar precision/recall
ERR194147*.fastq.gz, Azure D14 (16 cores, 112GB RAM, 800GB SSD)
Results: NIST Genome-in-a-Bottle
Lower confidence calls (qual>20, 2 platforms)
Highly confident
indel
snp
Aligner
Recall
Precision Recall
Precision
bwa-mem 97.24% 97.15% 99.57% 99.65%
snap-20
97.04% 97.48% 99.51% 99.57%
snap-24
97.04% 97.46% 99.52% 99.57%
snap-28
97.04% 97.45% 99.53% 99.57%
snap-32
97.00% 97.41% 99.51% 99.57%
Lower confidence
indel
snp
Aligner
Recall
Precision Recall
Precision
bwa-mem 96.38% 96.30% 99.00% 99.32%
snap-20
96.17% 96.68% 98.94% 99.25%
snap-24
96.17% 96.67% 98.95% 99.23%
snap-28
96.16% 96.62% 98.96% 99.21%
snap-32
96.11% 96.55% 98.94% 99.17%
Pathogen ID: SURPI (Charles Chiu, UCSF)
“This analysis of DNA sequences required just 96 minutes. A
similar analysis conducted with the use of previous generations
of computational software on the same hardware platform
would have taken 24 hours or more to complete, Chiu said.”
Charles Chiu, UCSF
SURPI
SNAP enables SURPI with:
Fast filtering mode
64-bit index for >40GB ntDB
Secondary mapping output
Acknowledgements
Microsoft Research
Bill Bolosky
Ravi Pandya
UC San Francisco
Taylor Sittler
Broad Institute
Christopher Hartl
UC Berkeley AMPLab
Matei Zaharia
Kristal Curtis
Armando Fox
Scott Shenker
Ion Stoica
David Patterson
Binaries, source, documentation (Apache 2.0 licensed)
http://snap.cs.berkeley.edu