Conditional Random Fields A brief description CS262 Lecture 9, Win07, Batzoglou HMMs and CRFs … … … … … K K K x1 x2 x3 … … K xN • Features used in objective function P(x, ) for.
Download ReportTranscript Conditional Random Fields A brief description CS262 Lecture 9, Win07, Batzoglou HMMs and CRFs … … … … … K K K x1 x2 x3 … … K xN • Features used in objective function P(x, ) for.
Conditional Random Fields A brief description CS262 Lecture 9, Win07, Batzoglou HMMs and CRFs 1 1 1 … 1 2 2 2 … 2 … … … K K K x1 x2 x3 … … K xN • Features used in objective function P(x, ) for an HMM: akl, ek(b), where b Vl(i) = Vk(i – 1) + (a(k, l) + e(l, i)) Let’s generalize g!!! CS262 Lecture 9, Win07, Batzoglou = Vk(i – 1) + g(k, l, xi) Vk(i – 1) + g(k, l, x, i) CRFs - Features i-1 x1 1. x2 x4 x5 x7 x6 x8 Define a set of features that may be important Number the features 1…n: • 2. x3 i f1(k, l, x, i), …, fn(k, l, x, i) features are indicator true/false variables Train weights wj for each feature fj Then, g(k, l, x, i) = CS262 Lecture 9, Win07, Batzoglou j=1…n fj(k, l, x, i) w j x9 x10 “Features” that depend on many pos. in x 1 2 3 4 5 6 x1 x2 x3 x4 x5 x6 1 2 3 4 5 6 x1 x2 x3 x4 x5 x6 HMM CRF … … • Score of a parse depends on all of x at each position • Can still do Viterbi because state i only “looks” at prev. state i-1 and the constant sequence x CS262 Lecture 9, Win07, Batzoglou Conditional Training • Hidden Markov Model training: Given training sequence x, “true” parse Maximize P(x, ) • Disadvantage: P(x, ) = P( | x) P(x) Quantity we care about so as to get a good parse CS262 Lecture 9, Win07, Batzoglou Quantity we don’t care so much about because x is always given Conditional Training P(x, ) = P( | x) P(x) P( | x) = P(x, ) / P(x) Recall Fj(x, ) = # times feature fj occurs in (x, ) = i=1…N fj(k, l, x, i) ; count fj in x, In HMMs, let’s denote by wj the weight of jth feature: wj = log(akl) or log(ek(b)) Then, HMM: CRF: CS262 Lecture 9, Win07, Batzoglou P(x, ) = exp[j=1…n wj Fj(x, )] Score(x, ) = exp[j=1…n wj Fj(x, )] Conditional Training In HMMs, P( | x) = P(x, ) / P(x) P(x, ) = exp[wTF(x, )] P(x) = exp[wTF(x, )] =: Z Then, in CRF we can do the same to normalize Score(x, ) into a prob. PCRF( | x) = exp[j=1…n wjF(j, x, )] / Z QUESTION: Why is this a probability??? CS262 Lecture 9, Win07, Batzoglou Conditional Training 1. We need to be given a set of sequences x and “true” parses 2. Calculate Z by a sum-of-paths algorithm similar to HMM • 3. We can then easily calculate P( | x) Calculate partial derivative of P( | x) w.r.t. each parameter wj (not covered—akin to forward/backward) • 4. Update each parameter with gradient descent Continue until convergence to optimal set of weights P( | x) = exp[j=1…n wjF(j, x, )] / Z is convex CS262 Lecture 9, Win07, Batzoglou Conditional Random Fields—Summary 1. Ability to incorporate complicated non-local feature sets • • 2. Do away with some independence assumptions of HMMs Parsing is still equally efficient Conditional training • • Train parameters that are best for parsing, not modeling Need labeled examples—sequences x and “true” parses (Can train on unlabeled sequences, however it is unreasonable to train too many parameters this way) • Training is significantly slower—many iterations of forward/backward CS262 Lecture 9, Win07, Batzoglou DNA Sequencing Some Terminology insert a fragment that was incorporated in a circular genome, and can be copied (cloned) vector the circular genome (host) that incorporated the fragment BAC read Bacterial Artificial Chromosome, a type of insert–vector combination, typically of length 100-200 kb a 500-900 long word that comes out of a sequencing machine coverage the average number of reads (or inserts) that cover a position in the target DNA piece shotgun the process of obtaining many reads sequencing from random locations in DNA, to detect overlaps and assemble CS262 Lecture 9, Win07, Batzoglou Sequencing and Fragment Assembly AGTAGCACAGA CTACGACGAGA CGATCGTGCGA GCGACGGCGTA GTGTGCTGTAC TGTCGTGTGTG TGTACTCTCCT 3x109 nucleotides 50% of human DNA is composed of repeats Error! Glued together two distant regions CS262 Lecture 9, Win07, Batzoglou What can we do about repeats? Two main approaches: • Cluster the reads • Link the reads CS262 Lecture 9, Win07, Batzoglou What can we do about repeats? Two main approaches: • Cluster the reads • Link the reads CS262 Lecture 9, Win07, Batzoglou What can we do about repeats? Two main approaches: • Cluster the reads • Link the reads CS262 Lecture 9, Win07, Batzoglou Sequencing and Fragment Assembly AGTAGCACAGA CTACGACGAGA CGATCGTGCGA GCGACGGCGTA GTGTGCTGTAC TGTCGTGTGTG TGTACTCTCCT 3x109 nucleotides A R B ARB, CRD or C CS262 Lecture 9, Win07, Batzoglou R D ARD, CRB ? Sequencing and Fragment Assembly AGTAGCACAGA CTACGACGAGA CGATCGTGCGA GCGACGGCGTA GTGTGCTGTAC TGTCGTGTGTG TGTACTCTCCT 3x109 nucleotides CS262 Lecture 9, Win07, Batzoglou Strategies for whole-genome sequencing 1. Hierarchical – Clone-by-clone i. ii. iii. Break genome into many long pieces Map each long piece onto the genome Sequence each piece with shotgun Example: Yeast, Worm, Human, Rat 2. Online version of (1) – Walking i. ii. iii. Break genome into many long pieces Start sequencing each piece with shotgun Construct map as you go Example: Rice genome 3. Whole genome shotgun One large shotgun pass on the whole genome Example: Drosophila, Human (Celera), Neurospora, Mouse, Rat, Dog CS262 Lecture 9, Win07, Batzoglou Whole Genome Shotgun Sequencing genome cut many times at random plasmids (2 – 10 Kbp) known dist cosmids (40 Kbp) ~500 bp CS262 Lecture 9, Win07, Batzoglou forward-reverse paired reads ~500 bp Fragment Assembly (in whole-genome shotgun sequencing) CS262 Lecture 9, Win07, Batzoglou Fragment Assembly Given N reads… Where N ~ 30 million… We need to use a linear-time algorithm CS262 Lecture 9, Win07, Batzoglou Steps to Assemble a Genome Some Terminology 1. Find overlapping reads read a 500-900 long word that comes out of sequencer mate pair a pair of reads from two ends 2. Merge some pairs of reads into of the same“good” insert fragment longer contigs contig a contiguous sequence formed by several overlapping reads with no gaps 3. Link contigs to form supercontigs supercontig an ordered and oriented set (scaffold) of contigs, usually by mate pairs 4. Derive consensus sequence consensus sequence derived from the sequene multiple alignment of reads in a contig CS262 Lecture 9, Win07, Batzoglou ..ACGATTACAATAGGTT.. 1. Find Overlapping Reads aaactgcagtacggatct aaactgcag aactgcagt … gtacggatct tacggatct gggcccaaactgcagtac gggcccaaa ggcccaaac … actgcagta ctgcagtac gtacggatctactacaca gtacggatc tacggatct … ctactacac tactacaca CS262 Lecture 9, Win07, Batzoglou (read, pos., word, orient.) (word, read, orient., pos.) aaactgcag aactgcagt actgcagta … gtacggatc tacggatct gggcccaaa ggcccaaac gcccaaact … actgcagta ctgcagtac gtacggatc tacggatct acggatcta … ctactacac tactacaca aaactgcag aactgcagt acggatcta actgcagta actgcagta cccaaactg cggatctac ctactacac ctgcagtac ctgcagtac gcccaaact ggcccaaac gggcccaaa gtacggatc gtacggatc tacggatct tacggatct tactacaca 1. Find Overlapping Reads • Find pairs of reads sharing a k-mer, k ~ 24 • Extend to full alignment – throw away if not >98% similar TACA TAGATTACACAGATTAC T GA || ||||||||||||||||| | || TAGT TAGATTACACAGATTAC TAGA • Caveat: repeats A k-mer that occurs N times, causes O(N2) read/read comparisons ALU k-mers could cause up to 1,000,0002 comparisons • Solution: Discard all k-mers that occur “too often” • Set cutoff to balance sensitivity/speed tradeoff, according to genome at hand and computing resources available CS262 Lecture 9, Win07, Batzoglou 1. Find Overlapping Reads Create local multiple alignments from the overlapping reads TAGATTACACAGATTACTGA TAGATTACACAGATTACTGA TAG TTACACAGATTATTGA TAGATTACACAGATTACTGA TAGATTACACAGATTACTGA TAGATTACACAGATTACTGA TAG TTACACAGATTATTGA TAGATTACACAGATTACTGA CS262 Lecture 9, Win07, Batzoglou 1. Find Overlapping Reads • Correct errors using multiple alignment TAGATTACACAGATTACTGA TAGATTACACAGATTACTGA TAGATTACACAGATTATTGA TAGATTACACAGATTACTGA TAG-TTACACAGATTACTGA insert A replace T with C TAGATTACACAGATTACTGA TAGATTACACAGATTACTGA TAG-TTACACAGATTATTGA TAGATTACACAGATTACTGA TAG-TTACACAGATTATTGA correlated errors— probably caused by repeats disentangle overlaps TAGATTACACAGATTACTGA TAGATTACACAGATTACTGA TAGATTACACAGATTACTGA In practice, error correction removes up to 98% of the errors CS262 Lecture 9, Win07, Batzoglou TAG-TTACACAGATTATTGA TAG-TTACACAGATTATTGA 2. Merge Reads into Contigs • Overlap graph: Nodes: reads r1…..rn Edges: overlaps (ri, rj, shift, orientation, score) Reads that come from two regions of the genome (blue and red) that contain the same repeat Note: of course, we don’t know the “color” of these nodes CS262 Lecture 9, Win07, Batzoglou 2. Merge Reads into Contigs repeat region Unique Contig Overcollapsed Contig We want to merge reads up to potential repeat boundaries CS262 Lecture 9, Win07, Batzoglou 2. Merge Reads into Contigs repeat region • Ignore non-maximal reads • Merge only maximal reads into contigs CS262 Lecture 9, Win07, Batzoglou 2. Merge Reads into Contigs • Remove transitively inferable overlaps If read r overlaps to the right reads r1, r2, and r1 overlaps r2, then (r, r2) can be inferred by (r, r1) and (r1, r2) CS262 Lecture 9, Win07, Batzoglou r r1 r2 r3 2. Merge Reads into Contigs CS262 Lecture 9, Win07, Batzoglou 2. Merge Reads into Contigs repeat boundary??? a sequencing error b … b a • Ignore “hanging” reads, when detecting repeat boundaries CS262 Lecture 9, Win07, Batzoglou Overlap graph after forming contigs CS262 Lecture 9, Win07, Batzoglou Unitigs: Gene Myers, 95 Repeats, errors, and contig lengths • Repeats shorter than read length are easily resolved Read that spans across a repeat disambiguates order of flanking regions • Repeats with more base pair diffs than sequencing error rate are OK We throw overlaps between two reads in different copies of the repeat • To make the genome appear less repetitive, try to: Increase read length Decrease sequencing error rate Role of error correction: Discards up to 98% of single-letter sequencing errors decreases error rate decreases effective repeat content increases contig length CS262 Lecture 9, Win07, Batzoglou 2. Merge Reads into Contigs • Insert non-maximal reads whenever unambiguous CS262 Lecture 9, Win07, Batzoglou 3. Link Contigs into Supercontigs Normal density Too dense Overcollapsed Inconsistent links Overcollapsed? CS262 Lecture 9, Win07, Batzoglou 3. Link Contigs into Supercontigs Find all links between unique contigs Connect contigs incrementally, if 2 links supercontig (aka scaffold) CS262 Lecture 9, Win07, Batzoglou 3. Link Contigs into Supercontigs Fill gaps in supercontigs with paths of repeat contigs CS262 Lecture 9, Win07, Batzoglou 4. Derive Consensus Sequence TAGATTACACAGATTACTGA TTGATGGCGTAA CTA TAGATTACACAGATTACTGACTTGATGGCGTAAACTA TAG TTACACAGATTATTGACTTCATGGCGTAA CTA TAGATTACACAGATTACTGACTTGATGGCGTAA CTA TAGATTACACAGATTACTGACTTGATGGGGTAA CTA TAGATTACACAGATTACTGACTTGATGGCGTAA CTA Derive multiple alignment from pairwise read alignments Derive each consensus base by weighted voting (Alternative: take maximum-quality letter) CS262 Lecture 9, Win07, Batzoglou Some Assemblers • PHRAP • Early assembler, widely used, good model of read errors • Overlap O(n2) layout (no mate pairs) consensus • Celera • First assembler to handle large genomes (fly, human, mouse) • Overlap layout consensus • Arachne • Public assembler (mouse, several fungi) • Overlap layout consensus • Phusion • Overlap clustering PHRAP assemblage consensus • Euler • Indexing Euler graph layout by picking paths consensus CS262 Lecture 9, Win07, Batzoglou Quality of assemblies—mouse Terminology: N50 contig length If we sort contigs from largest to smallest, and start Covering the genome in that order, N50 is the length Of the contig that just covers the 50th percentile. 7.7X sequence coverage CS262 Lecture 9, Win07, Batzoglou Quality of assemblies—dog 7.5X sequence coverage CS262 Lecture 9, Win07, Batzoglou History of WGA 1997 • 1982: -virus, 48,502 bp • 1995: h-influenzae, Let’s sequence the human 1 genome Mbp with the shotgun strategy • 2000: fly, 100 Mbp • 2001 – present That human (3Gbp), mouse (2.5Gbp), ratis*, chicken, dog, chimpanzee, several fungal genomes impossible, and a bad idea anyway Gene Myers CS262 Lecture 9, Win07, Batzoglou Phil Green Genomes Sequenced • http://www.genome.gov/10002154 CS262 Lecture 9, Win07, Batzoglou