Chapter 3 The Efficiency of Algorithms 國立雲林科技大學 資訊工程研究所 張傳育 (Chuan-Yu Chang ) 博士 Office: ES 709 TEL: ext

2 Properties of Algorithms  Necessary properties  A well-ordered collection of unambiguous and effectively computable operations that, when, executed, produces a result and halts in a finite amount of time.  What about desirable properties?  Correctness

3 Attributes of Algorithms  Correctness ：  First, make it correct!  Correct results may not be as straightforward as it seems. May be giving correct results to the wrong problem  Second, the algorithm must give correct results for all possible input values  Third, Issue of accuracy of the result we are willing to accept as correct.  Attribute of an algorithm: --Ease of understanding. --Clarity --Ease of handling --Elegance (style) --Efficiency

4 Example: adding … +100 Solution 1:  Step 1: Set the value of sum to 0  Step 2: Set the value of x to 1  Step 3: While x is less than or equal to 100 do step 4 and 5  Step 4: Add x to sum  Step 5: Add 1 to the value of x  Step 6: Print the value of sum  Step 7: Stop Solution 2: Gauss’s Method  1+100=101, 2+99=101,…,50+51=101  50*101=5050 Solution 3: 梯形公式  (100/2)(100+1)=5050 聰明但不容易理解 不同的方法 (elegant)

5 Attributes of Algorithms (cont.)  一個演算法最好能同時具備容易理解 (ease of understanding) 及優雅的形式 (elegant) 。  The Limited Resources  電腦科學家必須留意演算法所使用的資源 (Time & Space ) 。  Time and Space are not unlimited resources  “ Efficiency ” is the term used to describe an algorithm’s careful use of resources.

6 Measure Efficient Algorithm  How can we measure the time efficiency of an algorithm?  Kinds of computer to be used ?  Data from ?  Data organization ?  Benchmark  Use the same input data and running an algorithm on different machines gives a comparison of machine speeds on identical tasks.

7 Formal Definition  Algorithm ’ s time efficiency  An indication of amount of “ work ” required by the nature of the algorithm itself and the approach it uses.  Count the number of instruction executions  It is the number of steps each algorithm requires, not the time the algorithm takes on a particular machine, that is important for comparing two algorithms that do the same task.

8 Data Cleanup Problem  We want to perform a “data cleanup” and remove the 0 entries from the list before the average is computed.    Legit: legitimate element The value 0 will be removed

9 A Choice of Algorithms  The shuffle- left algorithm (Figure 3.1)  左手保存目前位置 ，且忽略非零值，當遇到 0 值時，以右 手邊第一個非零值取代之，且 0 右邊的所有值均左移一位。  The copy-over algorithm (Figure 3.2)  將每個 legitimate 值 ，複製到一個新的串列。  The converging-pointers algorithm (Figure 3.3)  左手指到串列的最左端，右手指到串列的最右端，當左手遇到 非零值時，則往右移位；當遇到零值時，則以右手所指到的非 零值來取代，右手往左移一位。

10 Figure 3.1 The shuffle- left algorithm 一開始 legit 的值設定成串 列長度，當遇到一次 0 值， 則 legit 減 1 。 0 右邊的所有項都 被往左複製一次

11 Figure 3.2 The copy-over algorithm

12 Figure 3.3 The converging-pointers algorithm

13 Comparisons 可用 time- 和 space-efficient 來衡量？  Case by case

14 Measuring Efficiency  Sequential search (Figure 3.1) --Best Case ： 1 --Worst Case ： n --Average Case ： n/2  Usually interested not in the behavior of an algorithm on little problems but in its behavior as the size of a problem (n) gets vary large.  In the New York City telephone directory, n may be large as If the sequential search algorithm were executed on a computer that could do comparisons per seconds, it would require on the average about ( /2)*(1/50000)=200 sec

15 Figure 3.4 Sequential Search Algorithm Peripheral task

16 Order of Magnitude  The worst case behavior of the sequential search algorithm on a list n names requires n comparisons. If c is a constant factor representing the peripheral work, it requires cxn total work.  對於大的 n ，常數項 c 可忽略。  Order of magnitude n is written Θ(n)  Sequential search is therefore an Θ(n) algorithm in both worst case and average case.

17 Work = 2n for Various Values of c (figure 3.6) 以 sequential search 為例：  n 筆資料，最多需要 n 次的比較。  假設其週邊工作為 c  則需要 cn 的工作量。  右圖為 c=2 時，資料筆數 n 和 總工作量之間的關係圖。

18 Work = cn for Various Values of c (figure 3.7)

19 Work = cn for Various Values of c (figure 3.8)

20 Calling Information To write the calling information table out. Pseudocode  For each of rows 1 through 4 do the following For each of columns 1 through 4 do the following Write the entry in this row and column  The number of write operations is 4x4=16  If there are n districts, the total works are nxn=n

21 A Comparison of n and n 2 (Figure 3.10)

22 A comparison of n and n 2 (figure 3.11)

23 Figure 3.12 For large enough n, 0.25n 2 has larger values than 10n

24 Figure 3.13 A Comparison of two extreme O(n 2 ) and O(n) Algorithm

25 The tortoise and the hare  Pentium Pro 200 具有 75 megaflops ，成本 $2000  Cray T3E 900 具有 670 gigaflops ，成本 $31 millions nPentium Pro O(n)Cray O(n 2 ) sec =2.3hr sec=9.7day

26 Analysis of Algorithms (data cleanup) Shuffle- leftCopy- overConverging pointer TimeSpaceTimeSpaceTimeSpace Best CaseO(n)n n n Worst Case O(n 2 )nO(n)2nO(n)n Average Case O(n 2 )nO(n) n ≦ x ≦ 2n O(n)n Analysis of three data clean up algorithm

27 Analysis of Algorithms (Cont.)  Sort  Selection Sort  best case: n 2, worst case: n 2, average case: n 2

28 Analysis of Algorithms (Cont.)  Selection Sort  Ex: Pass 0: Pass 1: Pass 2: Pass 3: Pass 4: 比較次數 n-1 n-2 : 1 比較總次數 =n(n-1)/2 比較總次數 =10

29 Exchange the value of X and Y 1. Copy the current value at position Y into position X 2. Copy the current value at position X into position Y Step 1 Step 2

30 Exchange the value of X and Y (cont.) 1. Copy the current value at position X into position T 2. Copy the value at position Y into position X 3. Copy the current value at position T into position Y Step 2 Step 3 Step 1

31 Binary Search  Binary Search  The list being searched is already sorted.  Search the midpoint of the list.  best case: 1, worst case: (log n), average case: (log n)

32 Binary Search Tree The basic shapes of n and log n log n grows much more slowly than n

33 Pattern Matching  Finding all occurrences of a pattern of the form P 1 P 2 …P m within text of the form T 1 T 2 …T n.  Forward match At each position beginning an attempt to match each pattern character against the text characters. The process stops only after text position n-m+1 (n the length of the text string, m the length of the pattern string) The best case: if the first character of the pattern is nowhere in the text. ( require n-m+1 comparisons) The worst case 1: if the pattern almost occurs everywhere in the text. (require nxm comparisons=> O(mxn)) The worst cases 2: the pattern is found at each location in the text. (require nxm comparisons => O(mxn))

34 When Things Get Out Of Hand  Polynomially bounded  Order of magnitude determines how quickly the values grow as n increases.  The work done by any of these algorithms is no worse than a constant multiple of n 2, which is polynomial in n.  Non-polynomially bounded  Is it possible to start at city A, go through every city exactly once, and end up at A?  A collection of nodes and connecting edges is called a graph

35 When Things Get Out Of Hand (cont.) Hamiltonian circuit  A path through a graph that begins and ends at the same node and goes through all other nodes exactly once.  If there are n node in the graph, then a Hamiltonian circuit, if it exists, must have exactly n links. AB CD

36 When Things Get Out Of Hand (cont.)  The number of paths that must be examined is the number of nodes at the bottom level of the tree.  The bottom of the corresponding tree would be at level n, and there would be 2 n paths to examine.  An O(2 n ) algorithm is called an exponential algorithm

37 When Things Get Out Of Hand (cont.)  Exponential algorithm  Comparisons of log n, n, n 2 and 2 n

38 When Things Get Out Of Hand (cont.)  Comparisons of four orders of magnitude  假設單一指令的執行時間為 sec ，則不同 order 所 需執行時間，如下表所示： Order log n0.0003s s s0.001 s n s0.01 s0.1 s n2n s0.25 s1 s1.67min 2n2n s3570 years4*10 16 centuries Too big to compute

39 Intractable problem  No polynomially bounded algorithm to solve.  They are solvable, but the solution algorithms all require so much work as to be virtually useless. bin-packing problem  Given an unlimited number of bins of volume 1 unit, and given n objects, all of volume between 0.0 and 1.0, find the minimal number of bins needs to store the n objects.  ( 只能找出 Approximation algorithm)