1. Algorithms and data structures
Divide and conquer
dynamic programming
greedy (naive) algorithms

2. Divide and conquer
Split problem into subproblems (preferably subproblems are same or close size);
Solve subproblems (repeating the split procedure until we get unit problems);
Merge partial results.
Examples – most of recursive algorthms e.g.
merge sort;
factorial (recursive);
Fibonacci sequence (recursive) *
* Less effective in comparison to other aproaches.

3. Divide and conquer Bad cases:
High cost of merging subproblems solutions;
Many similar (potentially identical) subproblems.
Fib(n) = Fib(n-1) + Fib(n-2) =
Fib(n-2) + Fib(n-3) + Fib(n-3) + Fib(n-4) =
Fib(n-3) + Fib(n-4) + Fib(n-3) + Fib(n-3) + Fib(n-4) ….

4. Dynamic programming How to apply: Recursive definition of solution;
Bottom-up construction of optimal solution (i.e. From small problems towards bigger ones)

5. Knapsack problem – discrette v.
Formulation:
Given a set of items, each with a weight and a value, determine the number of each item to include in a collection so that the total weight is less than or equal to a given limit K and the total value is as large as possible. The amounts of items are (or are not) limited by mi.
Items cannot be split.
I.e. for Z = {(p1,w1, m1), (p2,w2,m2), (p3,w3,m3), …, (pn,wn,mn)}
find L = (l1, l2, …, ln), that
å li*wi £ K, li Î C+, li £ mi
and
å li*pi = max

6. Knapsack problem – discrette v.
Example (no limits):
Items (weight, prize):
Z = { (1, 1), (2, 1), (3, 11), (4, 16), (5, 24) }
Knapsack:
K = 7
Optimal solution:
KN(K,Z) = 27
L = (0, 0, 1, 1, 0)

7. Knapsack problem – discrette v.
Solution:
Subproblem – if we split knapsack into two smaller the solution for both has to be optimal.
Fulfill helper array KN by determining cells in order K, in the following way
KN(k) = max { ci + KN(k-wi); 1 £ i < n } .
where K – knapsack size, n – number of item types
Beside knapasack values the structures of particular solutions could be stored (in separate array?). This is obligatory for limits of items checking.

8. Knapsack problem – discrette v.
Items 1
1,0,0,0,0 2
2,0,0,0,0 3 11
0,0,1,0,0 4 16
0,0,0,1,0 5 24
0,0,0,0,1 6 25
1,0,0,0,1 7 27
0,0,1,1,0 8 35
0,0,1,0,1
Unlimmited items
(1, 1) (2, 1) (3, 11) (4, 16) (5, 24)

9. Knapsack problem algorithm
class GOOD: weight=0 price=0
def Knapsack (goods,KSize) : KTmp = Array(KSize+1,0) for i in range(1,KSize): KTmp[i] = KTmp[i-1] for j in range(1, n+1): if Z[j].weight >= i and\
Z[j].price + KTmp[i-Z[j].weight] > KTmp[i]:
KTmp[i] = Z[j].weight + KTmp[i-Z[j].weight]
return KTmp[k]

10. Matrix chain multiplication
Description:
Given a sequence of matrices <A1, A2, … An> , the goal is to find the most efficient way to multiply these matrices.
The problem is not actually to perform the multiplications, but merely to decide the sequence of the matrix multiplications involved.

11. Two matrix multiplication - sample implementation
def MulMatrix(A, B): if Columns(A)!=Rows(B): ERROR
else:
for i in range(0, Rows(A)): for j in range(0, Columns(B)): C[i][j] = 0;
for k in range(0, Columns(A)): C[i][j] += A[i][k]*B[k][j]
return C

12. Matrix chain multiplication
Example:
Given a sequence of matrices <A1, A2, … An> , the goal is to find the most efficient way to multiply these matrices.
[p, q] ´ [q, r] = [p, r]
O([p, q] ´ [q, r]) = p*q*r
A1= [10, 100] A2= [100, 3] A3= [3, 50]
O((A1´A2) ´ A3) = 10*100*3 + 10*3*50 = 4500
O(A1 ´ (A2´A3)) = 100*3* *100*50 = 65000

13. Matrix chain multiplication
Solution:
Subproblem – if we split expression into two the parenthesis should be optimal for each part.
m[i, j] = { min (m[i, k]+m[k+1, j]+pi-1pkpj; i<=k<j }
Fulfill m[i, j] starting from m[i,1] then m[i,2] etc until m[n,n].
Besides optimal values there could be stored (in separate table) the structure of particular sub-solution

14. Matrix chain multiplicationj m
Solution:
A1= [30, 35]
A2= [35, 15]
A3= [15, 5]
A4= [5, 10]
A5= [10, 20]
A6= [20, 25] 6 5 4 3 2 1 1
5000
1000
3500
750
2500
5375
2625
4375
7125
10500
15750
7875
9375
11875
15125 2 i 3 4 5 6
optdiv j 6 5 4 3 2 1 1 5 4 3 2 1 2 i 3 4 5 6

15. Matrix chain multiplication - implementation
def MatrixChain(p, len):
for i in range(1,len): m[i][j] = 0
for h in range(2,len):
for i in range(1,len-h-1):
j = i+h m[i][j] = for k in range (i,j+1): tmp = m[i][k]+m[k+1][j] + p[i-1]*p[k]*p[j]
if m[i][j] < 0 or tmp < m[i][j]: m[i][j] = tmp optdiv[i][j] = k

16. Matrix chain multiplication – recursive implementation
def RecursiveMatrixChain(p, i, j):
if (i == j) return 0
m[i][j] = -1;
for k in range(i,j) q = RecursiveMatrixChain(p,i,k)+\ RecursiveMatrixChain(p,k+1,j) +\ p[i-1]*p[k]*p[j] if m[i][j]<0 or q <= m[i][j]: m[i][j] = q
return m[i][j]
RecursiveMatrixChain(p[], 1, len-1);

17. Memoization
Although related to caching, memoization refers to a specific case of this optimization, distinguishing it from forms of caching such as buffering or page replacement. In the context of some logic programming languages, memoization is also known as tabling (->lookup table).

18. Matrix chain multiplication – memoized ver.
def MemoizedMatrixChain(p, len): for i in range(1,len):
for j in range(1,len): m[i,j] = -1 return LookupMatrixChain(p,1,len-1)

19. LookupMatrixChain – implem.
def LookupMatrixChain(p, i, j):
if m[i][j] >= 0: return m[i][j]
if i == j: m[i][j] = 0
else:
for k in range(i,j)
q = LookupMatrixChain(p,i,k) +\
LookupMatrixChain(p,k+1,j) + p[i-1]*p[k]*p[j] if (q <= m[i][j]): m[i][j] = q
return m[i][j]

20. Dynamic programming – sample applications
Longest common subsequence of two sequences
Longest montonically increasing subsequence
Polygon triangulation with minimal total length of sides
RNA structure prediction and protein-DNA binding

21. Greedy (naive) algorithms
Algorithm could be considered as a chain of diecisions (optimisations);
Each step is considered separatelly – i.e. Best local solution is picked;
This strategy does not always give best global solution.

22. Knapsack Problem – cont. v.
Formulation:
Given a set of items, each with a weight and a value, determine the number of each item to include in a collection so that the total weight is less than or equal to a given limit K and the total value is as large as possible. The amounts of items are (or are not) limited by mi.
Items CAN be split.
I.e. for Z = {(p1,w1, m1), (p2,w2,m2), (p3,w3,m3), …, (pn,wn,mn)}
find L = (l1, l2, …, ln), that
å li*wi £ K, li Î R+, li £ mi
and
å li*pi = max

23. Knapsack Problem – cont. v.
Example (without limits):
Items:
Z = { (3, 1), (60, 10), (80, 15), (210, 30), (270, 45) }
Knapsack:
K = 45
Optimal selection:
KN(K, Z) = 315
Item amounts:
L = (0, 0, 0, 1.5, 0)

24. Knapsack Problem – cont. v.
Exampe (with limits):
Items:
Z = { (3, 1, 1), (60, 10, 1), (80, 15, 1), (210, 30, 1), (270, 45, 1) }
Knapsack:
K = 45
Optimal selection:
KN(K, Z) = 300
Item amounts:
L = (0, 0, 0, 1, 1/3) or (0, 1, 0, 1, 1/9)

25. Contineous Knapsack Problem – algorithm
// unlimited items
deef Knapsack (goods, KSize):
i = GetBestPrizeToValueItem(goods, n)
KValue = KSize / goods[i].weight * goods[i].price
return KValue

26. Contineous Knapsack Problem – algorithm
Knapsack (goods, KSize)
KValue = 0 cnt=0
while KSize >= 0 :
i = GetMaxPrizeToValueItem(goods, n)
if KSize < goods[i].weight * goods[i].max:
cnt = KSize / goods[i].weight else: Z[i].max KSize = KSize - cnt
KValue += KValue + cnt* goods[i].price goods[i].max = 0
return KValue

27. Greedy algorithms - sample applications
Task assignments to the resources (with mutual exclusion);
Spanning tree in graph.
Hufman’s coding
Greedy choice property;
Matroid theory.

28. Huffman’s Coding
Fixed-length coding – codes of all symbols have same length,
i.e.: A = , B = , C = , …
Variable-length codes –codes symbols can have various lengths, (more frequent symbols can have shorter representation)
Prefix code – the code that represents any symbol cannot be a a prefix of another code

29. Huffman’s coding (1952)
Each time two trees with smallest total weight are picked and meged
NIE PIEPRZ PIETRZE WIEPRZA PIEPRZEM I II 2
Frequencies: 2 1 1
N: 1
T: 1
W: 1
A: 1
M: 1
R: 4
Z: 4
: 4
I: 6
P: 6
E: 7 A 1 W 1 T N
III 3 2 1 2 1 1 M A W 1 1 N T

30. Huffman’s coding (1952) NIE PIEPRZ PIETRZE WIEPRZA PIEPRZEM : 100
Ascii coding: 35 x 8 = 280 b
After compression: 110 bits
3, 14 bits ber char on average 8 7 9 11 T
tree 35 15 20 4 5 6 2 3 1 E R Z  I P W A M N