1. Chapter 3 The Efficiency of Algorithms 國立雲林科技大學 資訊工程研究所 張傳育 (Chuan-Yu Chang ) 博士 Office: ES 709 TEL: 05-5342601 ext. 4337 E-mail: chuanyu@yuntech.edu.tw

2. 2 Properties of Algorithms  Necessary properties  A well-ordered collection of unambiguous and effectively computable operations that, when, executed, produces a result and halts in a finite amount of time.  What about desirable properties?  Correctness

3. 3 Attributes of Algorithms  Correctness ：  First, make it correct!  Correct results may not be as straightforward as it seems. May be giving correct results to the wrong problem  Second, the algorithm must give correct results for all possible input values  Third, Issue of accuracy of the result we are willing to accept as correct.  Attribute of an algorithm: --Ease of understanding. --Clarity --Ease of handling --Elegance (style) --Efficiency

4. 4 Example: adding 1+2+3+ … +100 Solution 1:  Step 1: Set the value of sum to 0  Step 2: Set the value of x to 1  Step 3: While x is less than or equal to 100 do step 4 and 5  Step 4: Add x to sum  Step 5: Add 1 to the value of x  Step 6: Print the value of sum  Step 7: Stop Solution 2: Gauss’s Method  1+100=101, 2+99=101,…,50+51=101  50*101=5050 Solution 3: 梯形公式  (100/2)(100+1)=5050 聰明但不容易理解 不同的方法 (elegant)

5. 5 Attributes of Algorithms (cont.)  一個演算法最好能同時具備容易理解 (ease of understanding) 及優雅的形式 (elegant) 。  The Limited Resources  電腦科學家必須留意演算法所使用的資源 (Time & Space ) 。  Time and Space are not unlimited resources  “ Efficiency ” is the term used to describe an algorithm’s careful use of resources.

6. 6 Measure Efficient Algorithm  How can we measure the time efficiency of an algorithm?  Kinds of computer to be used ?  Data from ?  Data organization ?  Benchmark  Use the same input data and running an algorithm on different machines gives a comparison of machine speeds on identical tasks.

7. 7 Formal Definition  Algorithm ’ s time efficiency  An indication of amount of “ work ” required by the nature of the algorithm itself and the approach it uses.  Count the number of instruction executions  It is the number of steps each algorithm requires, not the time the algorithm takes on a particular machine, that is important for comparing two algorithms that do the same task.

8. 8 Data Cleanup Problem  We want to perform a “data cleanup” and remove the 0 entries from the list before the average is computed.  0 24 16 0 36 42 23 21 0 27  24 16 36 42 23 21 27  Legit: legitimate element The value 0 will be removed

9. 9 A Choice of Algorithms  The shuffle- left algorithm (Figure 3.1)  左手保存目前位置 ，且忽略非零值，當遇到 0 值時，以右 手邊第一個非零值取代之，且 0 右邊的所有值均左移一位。  The copy-over algorithm (Figure 3.2)  將每個 legitimate 值 ，複製到一個新的串列。  The converging-pointers algorithm (Figure 3.3)  左手指到串列的最左端，右手指到串列的最右端，當左手遇到 非零值時，則往右移位；當遇到零值時，則以右手所指到的非 零值來取代，右手往左移一位。

10. 10 Figure 3.1 The shuffle- left algorithm 一開始 legit 的值設定成串 列長度，當遇到一次 0 值， 則 legit 減 1 。 0 右邊的所有項都 被往左複製一次

11. 11 Figure 3.2 The copy-over algorithm

12. 12 Figure 3.3 The converging-pointers algorithm

13. 13 Comparisons 可用 time- 和 space-efficient 來衡量？  Case by case

14. 14 Measuring Efficiency  Sequential search (Figure 3.1) --Best Case ： 1 --Worst Case ： n --Average Case ： n/2  Usually interested not in the behavior of an algorithm on little problems but in its behavior as the size of a problem (n) gets vary large.  In the New York City telephone directory, n may be large as 20000000. If the sequential search algorithm were executed on a computer that could do 50000 comparisons per seconds, it would require on the average about (20000000/2)*(1/50000)=200 sec

15. 15 Figure 3.4 Sequential Search Algorithm Peripheral task

16. 16 Order of Magnitude  The worst case behavior of the sequential search algorithm on a list n names requires n comparisons. If c is a constant factor representing the peripheral work, it requires cxn total work.  對於大的 n ，常數項 c 可忽略。  Order of magnitude n is written Θ(n)  Sequential search is therefore an Θ(n) algorithm in both worst case and average case.

17. 17 Work = 2n for Various Values of c (figure 3.6) 以 sequential search 為例：  n 筆資料，最多需要 n 次的比較。  假設其週邊工作為 c  則需要 cn 的工作量。  右圖為 c=2 時，資料筆數 n 和 總工作量之間的關係圖。

18. 18 Work = cn for Various Values of c (figure 3.7)

19. 19 Work = cn for Various Values of c (figure 3.8)

20. 20 Calling Information To write the calling information table out. Pseudocode  For each of rows 1 through 4 do the following For each of columns 1 through 4 do the following Write the entry in this row and column  The number of write operations is 4x4=16  If there are n districts, the total works are nxn=n 2. 1234 1243187314244 2215420345172 3197352385261 4340135217344

21. 21 A Comparison of n and n 2 (Figure 3.10)

22. 22 A comparison of n and n 2 (figure 3.11)

23. 23 Figure 3.12 For large enough n, 0.25n 2 has larger values than 10n

24. 24 Figure 3.13 A Comparison of two extreme O(n 2 ) and O(n) Algorithm

25. 25 The tortoise and the hare  Pentium Pro 200 具有 75 megaflops ，成本 $2000  Cray T3E 900 具有 670 gigaflops ，成本 $31 millions nPentium Pro O(n)Cray O(n 2 ) 7500.000010.00000084 75000.00010.000084 750000.0010.0084 7500000.010.84 75000000.184 7500000018400sec =2.3hr 75000000010840000sec=9.7day

26. 26 Analysis of Algorithms (data cleanup) Shuffle- leftCopy- overConverging pointer TimeSpaceTimeSpaceTimeSpace Best CaseO(n)n n n Worst Case O(n 2 )nO(n)2nO(n)n Average Case O(n 2 )nO(n) n ≦ x ≦ 2n O(n)n Analysis of three data clean up algorithm

27. 27 Analysis of Algorithms (Cont.)  Sort  Selection Sort  best case: n 2, worst case: n 2, average case: n 2

28. 28 Analysis of Algorithms (Cont.)  Selection Sort  Ex: Pass 0: 5 7 2 8 3 Pass 1: 5 7 2 3 8 Pass 2: 5 3 2 7 8 Pass 3: 2 3 5 7 8 Pass 4: 2 3 5 7 8 43214321 比較次數 n-1 n-2 : 1 比較總次數 =n(n-1)/2 比較總次數 =10

29. 29 Exchange the value of X and Y 1. Copy the current value at position Y into position X 2. Copy the current value at position X into position Y Step 1 Step 2

30. 30 Exchange the value of X and Y (cont.) 1. Copy the current value at position X into position T 2. Copy the value at position Y into position X 3. Copy the current value at position T into position Y Step 2 Step 3 Step 1

31. 31 Binary Search  Binary Search  The list being searched is already sorted.  Search the midpoint of the list.  best case: 1, worst case: (log n), average case: (log n)

32. 32 Binary Search Tree The basic shapes of n and log n log n grows much more slowly than n

33. 33 Pattern Matching  Finding all occurrences of a pattern of the form P 1 P 2 …P m within text of the form T 1 T 2 …T n.  Forward match At each position beginning an attempt to match each pattern character against the text characters. The process stops only after text position n-m+1 (n the length of the text string, m the length of the pattern string) The best case: if the first character of the pattern is nowhere in the text. ( require n-m+1 comparisons) The worst case 1: if the pattern almost occurs everywhere in the text. (require nxm comparisons=> O(mxn)) The worst cases 2: the pattern is found at each location in the text. (require nxm comparisons => O(mxn))

34. 34 When Things Get Out Of Hand  Polynomially bounded  Order of magnitude determines how quickly the values grow as n increases.  The work done by any of these algorithms is no worse than a constant multiple of n 2, which is polynomial in n.  Non-polynomially bounded  Is it possible to start at city A, go through every city exactly once, and end up at A?  A collection of nodes and connecting edges is called a graph

35. 35 When Things Get Out Of Hand (cont.) Hamiltonian circuit  A path through a graph that begins and ends at the same node and goes through all other nodes exactly once.  If there are n node in the graph, then a Hamiltonian circuit, if it exists, must have exactly n links. AB CD

36. 36 When Things Get Out Of Hand (cont.)  The number of paths that must be examined is the number of nodes at the bottom level of the tree.  The bottom of the corresponding tree would be at level n, and there would be 2 n paths to examine.  An O(2 n ) algorithm is called an exponential algorithm

37. 37 When Things Get Out Of Hand (cont.)  Exponential algorithm  Comparisons of log n, n, n 2 and 2 n

38. 38 When Things Get Out Of Hand (cont.)  Comparisons of four orders of magnitude  假設單一指令的執行時間為 0.0001 sec ，則不同 order 所 需執行時間，如下表所示： Order10501001000 log n0.0003s0.0006 s0.0007 s0.001 s n 0.005 s0.01 s0.1 s n2n2 0.01 s0.25 s1 s1.67min 2n2n 0.1024s3570 years4*10 16 centuries Too big to compute

39. 39 Intractable problem  No polynomially bounded algorithm to solve.  They are solvable, but the solution algorithms all require so much work as to be virtually useless. bin-packing problem  Given an unlimited number of bins of volume 1 unit, and given n objects, all of volume between 0.0 and 1.0, find the minimal number of bins needs to store the n objects.  ( 只能找出 Approximation algorithm)