Presentation is loading. Please wait.

Presentation is loading. Please wait.

Minimum Edit Distance Definition of Minimum Edit Distance.

Published byNicholas Logan Modified over 9 years ago

Similar presentations

Presentation on theme: "Minimum Edit Distance Definition of Minimum Edit Distance."— Presentation transcript:

1 Minimum Edit Distance Definition of Minimum Edit Distance

2 Dan Jurafsky How similar are two strings? Spell correction The user typed “graffe” Which is closest? graf graft grail giraffe Computational Biology Align two sequences of nucleotides Resulting alignment: Also for Machine Translation, Information Extraction, Speech Recognition AGGCTATCACCTGACCTCCAGGCCGATGCCC TAGCTATCACGACCGCGGTCGATTTGCCCGAC - AGGCTATCACCTGACCTCCAGGCCGA--TGCCC--- TAG-CTATCAC--GACCGC--GGTCGATTTGCCCGAC

3 Dan Jurafsky Edit Distance The minimum edit distance between two strings Is the minimum number of editing operations Insertion Deletion Substitution Needed to transform one into the other

4 Dan Jurafsky Minimum Edit Distance Two strings and their alignment:

5 Dan Jurafsky Minimum Edit Distance If each operation has cost of 1 Distance between these is 5 If substitutions cost 2 (Levenshtein) Distance between them is 8

6 Dan Jurafsky Alignment in Computational Biology Given a sequence of bases An alignment: Given two sequences, align each letter to a letter or gap -AGGCTATCACCTGACCTCCAGGCCGA--TGCCC--- TAG-CTATCAC--GACCGC--GGTCGATTTGCCCGAC AGGCTATCACCTGACCTCCAGGCCGATGCCC TAGCTATCACGACCGCGGTCGATTTGCCCGAC

7 Dan Jurafsky Other uses of Edit Distance in NLP Evaluating Machine Translation and speech recognition R Spokesman confirms senior government adviser was shot H Spokesman said the senior adviser was shot dead S I D I Named Entity Extraction and Entity Coreference IBM Inc. announced today IBM profits Stanford President John Hennessy announced yesterday for Stanford University President John Hennessy

8 Dan Jurafsky How to find the Min Edit Distance? Searching for a path (sequence of edits) from the start string to the final string: Initial state: the word we’re transforming Operators: insert, delete, substitute Goal state: the word we’re trying to get to Path cost: what we want to minimize: the number of edits 8

9 Dan Jurafsky Minimum Edit as Search But the space of all edit sequences is huge! We can’t afford to navigate naïvely Lots of distinct paths wind up at the same state. We don’t have to keep track of all of them Just the shortest path to each of those revisted states. 9

10 Dan Jurafsky Defining Min Edit Distance For two strings X of length n Y of length m We define D(i,j) the edit distance between X[1..i] and Y[1..j] i.e., the first i characters of X and the first j characters of Y The edit distance between X and Y is thus D(n,m)

11 Minimum Edit Distance Definition of Minimum Edit Distance

12 Minimum Edit Distance Computing Minimum Edit Distance

13 Dan Jurafsky Dynamic Programming for Minimum Edit Distance Dynamic programming: A tabular computation of D(n,m) Solving problems by combining solutions to subproblems. Bottom-up We compute D(i,j) for small i,j And compute larger D(i,j) based on previously computed smaller values i.e., compute D(i,j) for all i (0 < i < n) and j (0 < j < m)

14 Dan Jurafsky Defining Min Edit Distance (Levenshtein) Initialization D(i,0) = i D(0,j) = j Recurrence Relation: For each i = 1…M For each j = 1…N D(i-1,j) + 1 D(i,j)= min D(i,j-1) + 1 D(i-1,j-1) + 2; if X(i) ≠ Y(j) 0; if X(i) = Y(j) Termination: D(N,M) is distance

15 Dan Jurafsky N9 O8 I7 T6 N5 E4 T3 N2 I1 #0123456789 #EXECUTION The Edit Distance Table

16 Dan Jurafsky N9 O8 I7 T6 N5 E4 T3 N2 I1 #0123456789 #EXECUTION The Edit Distance Table

17 Dan Jurafsky N9 O8 I7 T6 N5 E4 T3 N2 I1 #0123456789 #EXECUTION Edit Distance

18 Dan Jurafsky N989101112111098 O8789 1110989 I76789 989 T65678989 11 N5456789101110 E43456789 9 T3456787898 N2345678787 I1234567678 #0123456789 #EXECUTION The Edit Distance Table

19 Minimum Edit Distance Computing Minimum Edit Distance

20 Minimum Edit Distance Backtrace for Computing Alignments

21 Dan Jurafsky Computing alignments Edit distance isn’t sufficient We often need to align each character of the two strings to each other We do this by keeping a “backtrace” Every time we enter a cell, remember where we came from When we reach the end, Trace back the path from the upper right corner to read off the alignment

22 Dan Jurafsky N9 O8 I7 T6 N5 E4 T3 N2 I1 #0123456789 #EXECUTION Edit Distance

23 Dan Jurafsky MinEdit with Backtrace

24 Dan Jurafsky Adding Backtrace to Minimum Edit Distance Base conditions: Termination: D(i,0) = i D(0,j) = j D(N,M) is distance Recurrence Relation: For each i = 1…M For each j = 1…N D(i-1,j) + 1 D(i,j)= min D(i,j-1) + 1 D(i-1,j-1) + 2; if X(i) ≠ Y(j) 0; if X(i) = Y(j) LEFT ptr(i,j)= DOWN DIAG insertion deletion substitution insertion deletion substitution

25 Dan Jurafsky The Distance Matrix Slide adapted from Serafim Batzoglou y 0 ……………………………… y M x 0 …………………… x N Every non-decreasing path from (0,0) to (M, N) corresponds to an alignment of the two sequences An optimal alignment is composed of optimal subalignments

26 Dan Jurafsky Result of Backtrace Two strings and their alignment:

27 Dan Jurafsky Performance Time: O(nm) Space: O(nm) Backtrace O(n+m)

28 Minimum Edit Distance Backtrace for Computing Alignments

29 Minimum Edit Distance Weighted Minimum Edit Distance

30 Dan Jurafsky Weighted Edit Distance Why would we add weights to the computation? Spell Correction: some letters are more likely to be mistyped than others Biology: certain kinds of deletions or insertions are more likely than others

31 Dan Jurafsky Confusion matrix for spelling errors

32 Dan Jurafsky

33 Weighted Min Edit Distance Initialization: D(0,0) = 0 D(i,0) = D(i-1,0) + del[x(i)]; 1 < i ≤ N D(0,j) = D(0,j-1) + ins[y(j)]; 1 < j ≤ M Recurrence Relation: D(i-1,j) + del[x(i)] D(i,j)= min D(i,j-1) + ins[y(j)] D(i-1,j-1) + sub[x(i),y(j)] Termination: D(N,M) is distance

34 Dan Jurafsky Where did the name, dynamic programming, come from? … The 1950s were not good years for mathematical research. [the] Secretary of Defense …had a pathological fear and hatred of the word, research… I decided therefore to use the word, “programming”. I wanted to get across the idea that this was dynamic, this was multistage… I thought, let’s … take a word that has an absolutely precise meaning, namely dynamic… it’s impossible to use the word, dynamic, in a pejorative sense. Try thinking of some combination that will possibly give it a pejorative meaning. It’s impossible. Thus, I thought dynamic programming was a good name. It was something not even a Congressman could object to.” Richard Bellman, “Eye of the Hurricane: an autobiography” 1984.

35 Minimum Edit Distance Weighted Minimum Edit Distance

36 Minimum Edit Distance Minimum Edit Distance in Computational Biology

37 Dan Jurafsky Sequence Alignment -AGGCTATCACCTGACCTCCAGGCCGA--TGCCC--- TAG-CTATCAC--GACCGC--GGTCGATTTGCCCGAC AGGCTATCACCTGACCTCCAGGCCGATGCCC TAGCTATCACGACCGCGGTCGATTTGCCCGAC

38 Dan Jurafsky Why sequence alignment? Comparing genes or regions from different species to find important regions determine function uncover evolutionary forces Assembling fragments to sequence DNA Compare individuals to looking for mutations

39 Dan Jurafsky Alignments in two fields In Natural Language Processing We generally talk about distance (minimized) And weights In Computational Biology We generally talk about similarity (maximized) And scores

40 Dan Jurafsky The Needleman-Wunsch Algorithm Initialization: D(i,0) = -i * d D(0,j) = -j * d Recurrence Relation: D(i-1,j) - d D(i,j)= min D(i,j-1) - d D(i-1,j-1) + s[x(i),y(j)] Termination: D(N,M) is distance

41 Dan Jurafsky The Needleman-Wunsch Matrix Slide adapted from Serafim Batzoglou x 1 ……………………………… x M y 1 …………………… y N (Note that the origin is at the upper left.)

42 Dan Jurafsky A variant of the basic algorithm: Maybe it is OK to have an unlimited # of gaps in the beginning and end: Slide from Serafim Batzoglou ----------CTATCACCTGACCTCCAGGCCGATGCCCCTTCCGGC GCGAGTTCATCTATCAC--GACCGC--GGTCG-------------- If so, we don’t want to penalize gaps at the ends

43 Dan Jurafsky Different types of overlaps Slide from Serafim Batzoglou Example: 2 overlapping“reads” from a sequencing project Example: Search for a mouse gene within a human chromosome

44 Dan Jurafsky The Overlap Detection variant Changes: 1.Initialization For all i, j, F(i, 0) = 0 F(0, j) = 0 2.Termination max i F(i, N) F OPT = max max j F(M, j) Slide from Serafim Batzoglou x 1 ……………………………… x M y 1 …………………… y N

45 Dan Jurafsky Given two strings x = x 1 ……x M, y = y 1 ……y N Find substrings x’, y’ whose similarity (optimal global alignment value) is maximum x = aaaacccccggggtta y = ttcccgggaaccaacc Slide from Serafim Batzoglou The Local Alignment Problem

46 Dan Jurafsky The Smith-Waterman algorithm Idea: Ignore badly aligning regions Modifications to Needleman-Wunsch: Initialization: F(0, j) = 0 F(i, 0) = 0 0 Iteration: F(i, j) = max F(i – 1, j) – d F(i, j – 1) – d F(i – 1, j – 1) + s(x i, y j ) Slide from Serafim Batzoglou

47 Dan Jurafsky The Smith-Waterman algorithm Termination: 1.If we want the best local alignment… F OPT = max i,j F(i, j) Find F OPT and trace back 2.If we want all local alignments scoring > t ??For all i, j find F(i, j) > t, and trace back? Complicated by overlapping local alignments Slide from Serafim Batzoglou

48 Dan Jurafsky Local alignment example ATTATC 0000000 A0 T0 C0 A0 T0 X = ATCAT Y = ATTATC Let: m = 1 (1 point for match) d = 1 (-1 point for del/ins/sub)

49 Dan Jurafsky Local alignment example ATTATC 0000000 A0100100 T0021020 C0011013 A0100212 T0020132 X = ATCAT Y = ATTATC

50 Dan Jurafsky Local alignment example ATTATC 0000000 A0100100 T0021020 C0011013 A0100212 T0020132 X = ATCAT Y = ATTATC

51 Dan Jurafsky Local alignment example ATTATC 0000000 A0100100 T0021020 C0011013 A0100212 T0020132 X = ATCAT Y = ATTATC

52 Minimum Edit Distance Minimum Edit Distance in Computational Biology

Download ppt "Minimum Edit Distance Definition of Minimum Edit Distance."

Similar presentations

Longest Common Subsequence

Longest Common Subsequence
DYNAMIC PROGRAMMING ALGORITHMS VINAY ABHISHEK MANCHIRAJU.

DYNAMIC PROGRAMMING ALGORITHMS VINAY ABHISHEK MANCHIRAJU.
CS 5263 Bioinformatics Lecture 3: Dynamic Programming and Global Sequence Alignment.

CS 5263 Bioinformatics Lecture 3: Dynamic Programming and Global Sequence Alignment.
Presented By Dr. Shazzad Hosain Asst. Prof. EECS, NSU

Presented By Dr. Shazzad Hosain Asst. Prof. EECS, NSU
Sequence allignement 1 Chitta Baral. Sequences and Sequence allignment Two main kind of sequences –Sequence of base pairs in DNA molecules (A+T+C+G)*

Sequence allignement 1 Chitta Baral. Sequences and Sequence allignment Two main kind of sequences –Sequence of base pairs in DNA molecules (A+T+C+G)*
Inexact Matching of Strings General Problem –Input Strings S and T –Questions How distant is S from T? How similar is S to T? Solution Technique –Dynamic.

Inexact Matching of Strings General Problem –Input Strings S and T –Questions How distant is S from T? How similar is S to T? Solution Technique –Dynamic.
C E N T R F O R I N T E G R A T I V E B I O I N F O R M A T I C S V U E 02-11-2006 Alignments 1 Sequence Analysis.

C E N T R F O R I N T E G R A T I V E B I O I N F O R M A T I C S V U E Alignments 1 Sequence Analysis.
6/11/2015 © Bud Mishra, 2001 L7-1 Lecture #7: Local Alignment Computational Biology Lecture #7: Local Alignment Bud Mishra Professor of Computer Science.

6/11/2015 © Bud Mishra, 2001 L7-1 Lecture #7: Local Alignment Computational Biology Lecture #7: Local Alignment Bud Mishra Professor of Computer Science.
DYNAMIC PROGRAMMING. 2 Algorithmic Paradigms Greedy. Build up a solution incrementally, myopically optimizing some local criterion. Divide-and-conquer.

DYNAMIC PROGRAMMING. 2 Algorithmic Paradigms Greedy. Build up a solution incrementally, myopically optimizing some local criterion. Divide-and-conquer.
Welcome to CS262!. Goals of this course Introduction to Computational Biology  Basic biology for computer scientists  Breadth: mention many topics &

Welcome to CS262!. Goals of this course Introduction to Computational Biology  Basic biology for computer scientists  Breadth: mention many topics &
Computational Genomics Lecture 1, Tuesday April 1, 2003.

Computational Genomics Lecture 1, Tuesday April 1, 2003.
Sequence Alignment Algorithms in Computational Biology Spring 2006 Edited by Itai Sharon Most slides have been created and edited by Nir Friedman, Dan.

Sequence Alignment Algorithms in Computational Biology Spring 2006 Edited by Itai Sharon Most slides have been created and edited by Nir Friedman, Dan.
Sequence Alignment. CS262 Lecture 2, Win06, Batzoglou Complete DNA Sequences More than 300 complete genomes have been sequenced.

Sequence Alignment. CS262 Lecture 2, Win06, Batzoglou Complete DNA Sequences More than 300 complete genomes have been sequenced.
Inexact Matching General Problem –Input Strings S and T –Questions How distant is S from T? How similar is S to T? Solution Technique –Dynamic programming.

Inexact Matching General Problem –Input Strings S and T –Questions How distant is S from T? How similar is S to T? Solution Technique –Dynamic programming.
Sequence Alignment Cont’d. Sequence Alignment -AGGCTATCACCTGACCTCCAGGCCGA--TGCCC--- TAG-CTATCAC--GACCGC--GGTCGATTTGCCCGAC Definition Given two strings.

Sequence Alignment Cont’d. Sequence Alignment -AGGCTATCACCTGACCTCCAGGCCGA--TGCCC--- TAG-CTATCAC--GACCGC--GGTCGATTTGCCCGAC Definition Given two strings.
Reminder -Structure of a genome Human 3x10 9 bp Genome: ~30,000 genes ~200,000 exons ~23 Mb coding ~15 Mb noncoding pre-mRNA transcription splicing translation.

Reminder -Structure of a genome Human 3x10 9 bp Genome: ~30,000 genes ~200,000 exons ~23 Mb coding ~15 Mb noncoding pre-mRNA transcription splicing translation.
Sequence Alignment.

Sequence Alignment.
Sequence Alignment Bioinformatics. Sequence Comparison Problem: Given two sequences S & T, are S and T similar? Need to establish some notion of similarity.

Sequence Alignment Bioinformatics. Sequence Comparison Problem: Given two sequences S & T, are S and T similar? Need to establish some notion of similarity.
Sequence Alignment. Before we start, administrivia Instructor: Serafim Batzoglou, CS x3-3334 Office hours: Monday 2:00-3:30 TA:

Sequence Alignment. Before we start, administrivia Instructor: Serafim Batzoglou, CS x Office hours: Monday 2:00-3:30 TA:
Sequence Alignment Cont’d. Evolution Scoring Function Sequence edits: AGGCCTC  Mutations AGGACTC  Insertions AGGGCCTC  Deletions AGG.CTC Scoring Function:

Sequence Alignment Cont’d. Evolution Scoring Function Sequence edits: AGGCCTC  Mutations AGGACTC  Insertions AGGGCCTC  Deletions AGG.CTC Scoring Function:

Similar presentations

Ads by Google