Alignment, Part II Vasileios Hatzivassiloglou University of Texas at Dallas

Finding the optimal alignment Simple algorithm: Construct all possible alignments, score them, and pick the best How many alignments are there for two strings of length n and m?

or C(p, q): Number of combinations of q objects out of p, p≥q Arrange the p objects in line from 1 to p Pick the first (p choices), second (p-1 choices), up to q-th (p-q+1 choices) This will generate all permutations of each set of q objects, so divide by q! Combinatorics basics

Asymptotic analysis How large is C(n+m, n)? Asymptotic analysis: we look at how fast C(n+m, n) grows in comparison to functions like (n+m) 2 or 2 n+m Ultimately, constants have a small effect on the growth rate – they are dominated by terms involving the variables n and m

Example function growth n2n2 100n 2 2n2n n=5252,50032 n= ,0001,024 n=10010,0001,000, e30

Asymptotic analysis of C(p,q)

Upper bound for C(p,q) There are 2 p subsets of a set with p elements, some of which will have q elements Therefore, C(p, q) ≤ 2 p In particular, C(2n, n) ≤ 2 2n = 4 n

Alignments versus scoring We have explored the potential alignments of two strings The set of such alignments (and their number) remains constant and independent of the scoring function The scoring function may make it hard or easy to locate the optimal alignment(s)

Dynamic programming A general strategy for addressing computationally intractable problems by solving all smaller subproblems and remembering the results “Programming” in the mathematical sense Developed in the 1950’s and applied to many problems

Calculating the Fibonacci sequence f(n) = f(n-1)+f(n-2) if n>1; f(0)=0; f(1)=1 int fib(int n) { if (n==0) {return 0}; if (n==1) {return 1}; return fib(n-1)+fib(n-2); }

Recursive computation of Fibonacci

How long does the recursion take? Calculating fib(n) takes at least f(n) addition steps –because we ultimately add 0 or 1 each time How large is f(n)?

Dynamic programming for the Fibonacci sequence int MAX_N = 10,000; int known_fib[MAX_N]; known_fib[0]=0; known_fib[1]=1; known_n=1; int dynamic_fib(int n) { if (n==MAX_N) {printf(“N is too large!”); exit;} if (n<=known_n) {return known_fib[n];} new_fib = dynamic_fib(n-1) + dynamic_fib(n-2); known_n++; /* Necessarily, n==known_n */ known_fib[known_n] = new_fib; return new_fib; }

Complexity of alternative solution Linear in n This is not necessarily the best algorithm for calculating the Fibonacci sequence –Can be calculated iteratively in O(logn) time –Can be calculated directly

Dynamic programming for optimizing choices We have three departments in an organization, each of which comes up with a number of proposals Each proposal has a cost and an estimated revenue Each department can execute at most one proposal Find the optimal set of proposals (maximizing expected revenue) with a fixed total budget

Proposal data Total budget = 5 Dept ADept BDept C ProposalCostRevCostRevCostRev I II III IV

Recursive solution The optimal solution with n+1 departments must include a combination of proposals for n departments Is this included combination optimal for the n departments when the (n+1)th department is not considered? –Not necessarily!

Dynamic programming basics Choices correspond to stages Each stage has a number of states, each of which encodes relevant information about choices in previous stages The principle of optimality: We need to know only the state of a stage to make an optimal choice for that stage We only need to deal with viable states, that can be reached from a previous stage

Proposal example Three stages, one for each department State in each stage: Total money left before reaching that stage Principle of optimality: To choose the optimal project for department X we only need to know how much money is left for X

Solving the proposal problem Construct table with states as rows, stages as columns Each cell contains the best value we can obtain for the quantity we optimize Fill it in from left to right (by stages) Within a given stage, consider each viable prior state and all transitions that can happen from that state in the current stage Update the value for the resulting state from a transition; keep the best result

DP for the proposal problem Money leftDept ADept BDept C

DP for the proposal problem Money leftDept ADept BDept C

DP for the proposal problem Money leftDept ADept BDept C

DP for the proposal problem Money leftDept ADept BDept C

DP for the proposal problem Money leftDept ADept BDept C

DP for the proposal problem Money leftDept ADept BDept C

DP for the proposal problem Money leftDept ADept BDept C

DP for the proposal problem Money leftDept ADept BDept C

DP for the proposal problem Money leftDept ADept BDept C

DP for the proposal problem Money leftDept ADept BDept C

Money leftDept ADept BDept C DP for the proposal problem

Money leftDept ADept BDept C DP for the proposal problem

Money leftDept ADept BDept C DP for the proposal problem

Money leftDept ADept BDept C DP for the proposal problem

Money leftDept ADept BDept C DP for the proposal problem

Money leftDept ADept BDept C DP for the proposal problem

Money leftDept ADept BDept C DP for the proposal problem

Money leftDept ADept BDept C DP for the proposal problem

Money leftDept ADept BDept C DP for the proposal problem

Money leftDept ADept BDept C DP for the proposal problem

Money leftDept ADept BDept C DP for the proposal problem

Money leftDept ADept BDept C DP for the proposal problem

Money leftDept ADept BDept C DP for the proposal problem

Money leftDept ADept BDept C DP for the proposal problem

Money leftDept ADept BDept C DP for the proposal problem

Money leftDept ADept BDept C DP for the proposal problem

Money leftDept ADept BDept C DP for the proposal problem

Money leftDept ADept BDept C DP for the proposal problem

Money leftDept ADept BDept C DP for the proposal problem

Money leftDept ADept BDept C DP for the proposal problem

Money leftDept ADept BDept C DP for the proposal problem

Money leftDept ADept BDept C DP for the proposal problem

Money leftDept ADept BDept C DP for the proposal problem

Money leftDept ADept BDept C DP for the proposal problem

Money leftDept ADept BDept C DP for the proposal problem

Complexity If we have n stages, m possible states, and k options at each stage –Time complexity O(nmk) –Space complexity O(nm) Straightforward recursion would take –O(k n ) time and space