Faculty of Communications, Health and Science School of Computer and Information Science CSP1250 Data Structures with Java Java Collections by D Watt & D Brown (2001)Edited and Modified by Dr W Guo (2003)1 2. Algorithms Analysis Fundamental Principles of Algorithms Time and Space Efficiency of Algorithms Complexity of Algorithms O -notation. Recursion. Guide.

Faculty of Communications, Health and Science School of Computer and Information Science CSP1250 Data Structures with Java Java Collections by D Watt & D Brown (2001)Edited and Modified by Dr W Guo (2003)2 Principles (1) An algorithm is a step-by-step procedure for solving a stated problem. The algorithm will be performed by a processor (which may be human, mechanical, or electronic). The algorithm must be expressed in steps that the processor is capable of performing. The algorithm must eventually terminate.

Faculty of Communications, Health and Science School of Computer and Information Science CSP1250 Data Structures with Java Java Collections by D Watt & D Brown (2001)Edited and Modified by Dr W Guo (2003)3 Principles (2) The algorithm must be expressed in some language that the processor understands. (But the underlying procedure is independent of the particular language chosen.) The stated problem must be solvable, i.e., capable of solution by a step-by-step procedure.

Faculty of Communications, Health and Science School of Computer and Information Science CSP1250 Data Structures with Java Java Collections by D Watt & D Brown (2001)Edited and Modified by Dr W Guo (2003)4 Efficiency Given several algorithms to solve the same problem, which algorithm is best? Given an algorithm, is it feasible to use it at all? In other words, is it efficient enough to be usable in practice? How much time does the algorithm require? How much space (memory) does the algorithm require? In general, both time and space requirements depend on the algorithms input (typically the size of the input).

Faculty of Communications, Health and Science School of Computer and Information Science CSP1250 Data Structures with Java Java Collections by D Watt & D Brown (2001)Edited and Modified by Dr W Guo (2003)5 Example: efficiency Hypothetical profile of two sorting algorithms: Algorithm Bs time grows more slowly than As. items to be sorted, n time (ms) Key: Algorithm A Algorithm B

Faculty of Communications, Health and Science School of Computer and Information Science CSP1250 Data Structures with Java Java Collections by D Watt & D Brown (2001)Edited and Modified by Dr W Guo (2003)6 Efficiency: measuring time Measure time in seconds? + is useful in practice – depends on language, compiler, and processor. Count algorithm steps? + does not depend on compiler or processor – depends on granularity of steps. Count characteristic operations ? (e.g., arithmetic ops in math algorithms, comparisons in searching algorithms) + depends only on the algorithm itself + measures the algorithms intrinsic efficiency.

Faculty of Communications, Health and Science School of Computer and Information Science CSP1250 Data Structures with Java Java Collections by D Watt & D Brown (2001)Edited and Modified by Dr W Guo (2003)7 Example: power algorithms (1) Simple power algorithm : To compute b n : 1.Set p to 1. 2.For i = 1, …, n, repeat: 2.1.Multiply p by b. 3.Terminate with answer p.

Faculty of Communications, Health and Science School of Computer and Information Science CSP1250 Data Structures with Java Java Collections by D Watt & D Brown (2001)Edited and Modified by Dr W Guo (2003)8 Example: power algorithms (2) Analysis (counting multiplications): Step 2.1 performs a multiplication. This step is repeated n times. No. of multiplications = n

Faculty of Communications, Health and Science School of Computer and Information Science CSP1250 Data Structures with Java Java Collections by D Watt & D Brown (2001)Edited and Modified by Dr W Guo (2003)9 Example: power algorithms (3) Implementation in Java: public class power {static int power (int b, int n){ // Return b^n (where n is non-negative int p = 1; for (int i=1; i<=n;i++) p *= b; return p;} public static void main (String [] args){ int p = Integer.parseInt (args [0]); System.out.println ("The power of b^n is " + p); }

Faculty of Communications, Health and Science School of Computer and Information Science CSP1250 Data Structures with Java Java Collections by D Watt & D Brown (2001)Edited and Modified by Dr W Guo (2003)10 Example: power algorithms (4) Idea: b 1000 = b 500 b 500. If we know b 500, we can compute b 1000 with only 1 more multiplication! Smart power algorithm : To compute b n : 1.Set p to 1, set q to b, and set m to n. 2.While m > 0, repeat: 2.1.If m is odd, multiply p by q. 2.2.Halve m (discarding any remainder). 2.3.Multiply q by itself. 3.Terminate with answer p.

Faculty of Communications, Health and Science School of Computer and Information Science CSP1250 Data Structures with Java Java Collections by D Watt & D Brown (2001)Edited and Modified by Dr W Guo (2003)11 Example: power algorithms (5) Analysis (counting multiplications): Steps 2.1–3 together perform at most 2 multiplications. They are repeated as often as we must halve the value of n (discarding any remainder) until it reaches 0, i.e., floor(log 2 n ) + 1 times. Max. no. of multiplications= 2(floor(log 2 n ) + 1) = 2 floor(log 2 n ) + 2

Faculty of Communications, Health and Science School of Computer and Information Science CSP1250 Data Structures with Java Java Collections by D Watt & D Brown (2001)Edited and Modified by Dr W Guo (2003)12 Example: power algorithms (6) public class power1 { static int power (int b, int n){ // Return b^n (where n is non-negative) int p=1, q=b, m=n; while (m>0) { if (m%2 !=0) p *=q; m /=2; q *= q; } return p; } public static void main (String [] args){ int p = Integer.parseInt (args [0]); System.out.println ("The power of b^n is " + p); }

Faculty of Communications, Health and Science School of Computer and Information Science CSP1250 Data Structures with Java Java Collections by D Watt & D Brown (2001)Edited and Modified by Dr W Guo (2003) multiplications n Example: power algorithms (7) Comparison: simple power algorithm smart power algorithm

Faculty of Communications, Health and Science School of Computer and Information Science CSP1250 Data Structures with Java Java Collections by D Watt & D Brown (2001)Edited and Modified by Dr W Guo (2003)14 Complexity For many interesting algorithms, the exact number of operations is too difficult to analyse mathematically. To simplify the analysis: –identify the fastest-growing term –neglect slower-growing terms –neglect the constant factor in the fastest-growing term. The resulting formula is the algorithms time complexity. It focuses on the growth rate of the algorithms time requirement. Similarly for space complexity.

Faculty of Communications, Health and Science School of Computer and Information Science CSP1250 Data Structures with Java Java Collections by D Watt & D Brown (2001)Edited and Modified by Dr W Guo (2003)15 Example : analysis of power algorithms (1) Analysis of simple power algorithm (counting multiplications): No. of multiplications = n Time taken is approximately proportional to n. Time complexity is of order n. This is written O ( n ).

Faculty of Communications, Health and Science School of Computer and Information Science CSP1250 Data Structures with Java Java Collections by D Watt & D Brown (2001)Edited and Modified by Dr W Guo (2003)16 Analysis of smart power algorithm (counting multiplicns): Max. no. of multiplications = 2 floor(log 2 n ) + 2 then tolog 2 n Time complexity is of order log n. This is written O(log n). then tofloor(log 2 n) Simplify to2 floor(log 2 n) Neglect slow-growing term, +2. Neglect constant factor, 2. Neglect floor(), which on average subtracts 0.5, a constant term. Example : analysis of power algorithms (2)

Faculty of Communications, Health and Science School of Computer and Information Science CSP1250 Data Structures with Java n Comparison: n log n Example : analysis of power algorithms (3)

Faculty of Communications, Health and Science School of Computer and Information Science CSP1250 Data Structures with Java Java Collections by D Watt & D Brown (2001)Edited and Modified by Dr W Guo (2003)18 O -notation (1) We have seen that an O (log n ) algorithm is inherently better than an O ( n ) algorithm for large values of n. O (log n ) signifies a slower growth rate than O ( n ). Complexity O ( X ) means of order X, i.e., growing proportionally to X. Here X signifies the growth rate, neglecting slower-growing terms and constant factors.

Faculty of Communications, Health and Science School of Computer and Information Science CSP1250 Data Structures with Java Java Collections by D Watt & D Brown (2001)Edited and Modified by Dr W Guo (2003)19 O -notation (2) Common time complexities: O (1) constant time(feasible) O (log n ) logarithmic time(feasible) O ( n ) linear time(feasible) O ( n log n ) log linear time(feasible) O ( n 2 ) quadratic time(sometimes feasible) O ( n 3 ) cubic time(sometimes feasible) O (2 n ) exponential time (rarely feasible)

Faculty of Communications, Health and Science School of Computer and Information Science CSP1250 Data Structures with Java Java Collections by D Watt & D Brown (2001)Edited and Modified by Dr W Guo (2003)20 Growth rates (1) Comparison of growth rates: 1log nNnlog nn2n2 n3n3 n 10 2n2n E E E E+161.1E E E E E E E+30

Faculty of Communications, Health and Science School of Computer and Information Science CSP1250 Data Structures with Java n Growth rates (2) Graphically: log n n n log nn2n2 2n2n

Faculty of Communications, Health and Science School of Computer and Information Science CSP1250 Data Structures with Java Example: growth rates (1) Consider a problem that requires n data items to be processed. Consider several competing algorithms to solve this problem. Suppose that their time requirements on a particular processor are as follows: Algorithm Log:0.3 log 2 n seconds Algorithm Lin:0.1 n seconds Algorithm LogLin:0.03 n log 2 n seconds Algorithm Quad:0.01 n 2 seconds Algorithm Cub:0.001 n 3 seconds Algorithm Exp: n seconds

Faculty of Communications, Health and Science School of Computer and Information Science CSP1250 Data Structures with Java Java Collections by D Watt & D Brown (2001)Edited and Modified by Dr W Guo (2003)23 Log Lin LogLin Quad Cub Exp :00 n 0:01 Example: growth rates (2) Compare how many data items ( n ) each algorithm can process in 1, 2, …, 10 seconds: 0:020:030:040:050:060:070:080:090:10

Faculty of Communications, Health and Science School of Computer and Information Science CSP1250 Data Structures with Java Java Collections by D Watt & D Brown (2001)Edited and Modified by Dr W Guo (2003)24 Recursion A recursive algorithm is one expressed in terms of itself. In other words, at least one step of a recursive algorithm is a call to itself. In Java, a recursive method is one that calls itself.

Faculty of Communications, Health and Science School of Computer and Information Science CSP1250 Data Structures with Java Java Collections by D Watt & D Brown (2001)Edited and Modified by Dr W Guo (2003)25 When should recursion be used? Sometimes an algorithm can be expressed using either iteration or recursion. The recursive version tends to be: +more elegant and easier to understand – less efficient (extra calls consume time and space). Sometimes an algorithm can be expressed only using recursion.

Faculty of Communications, Health and Science School of Computer and Information Science CSP1250 Data Structures with Java Java Collections by D Watt & D Brown (2001)Edited and Modified by Dr W Guo (2003)26 When does recursion work? Given a recursive algorithm, how can we sure that it terminates? The algorithm must have: –one or more easy cases –one or more hard cases. In an easy case, the algorithm must give a direct answer without calling itself. In a hard case, the algorithm may call itself, but only to deal with an easier case.

Faculty of Communications, Health and Science School of Computer and Information Science CSP1250 Data Structures with Java Java Collections by D Watt & D Brown (2001)Edited and Modified by Dr W Guo (2003)27 Example: recursive power algorithms (1) Recursive definition of b n : b n = 1if n = 0 b n = b b n –1 if n > 0 Simple recursive power algorithm : To compute b n : 1.If n = 0: 1.1.Terminate with answer 1. 2.If n > 0: 2.1.Terminate with answer b b n –1. Easy case: solved directly. Hard case: solved by comput- ing b n–1, which is easier since n–1 is closer than n to 0.

Faculty of Communications, Health and Science School of Computer and Information Science CSP1250 Data Structures with Java Java Collections by D Watt & D Brown (2001)Edited and Modified by Dr W Guo (2003)28 Example: recursive power algorithms (2) Implementation in Java: }

Faculty of Communications, Health and Science School of Computer and Information Science CSP1250 Data Structures with Java Java Collections by D Watt & D Brown (2001)Edited and Modified by Dr W Guo (2003)29 Example: recursive power algorithms (3) Idea: b 1000 = b 500 b 500, and b 1001 = b b 500 b 500. Alternative recursive definition of b n : b n = 1if n = 0 b n = b n /2 b n /2 if n > 0 and n is even b n = b b n /2 b n /2 if n > 0 and n is odd (Recall: n /2 discards the remainder if n is odd.)

Faculty of Communications, Health and Science School of Computer and Information Science CSP1250 Data Structures with Java Java Collections by D Watt & D Brown (2001)Edited and Modified by Dr W Guo (2003)30 Example: recursive power algorithms (4) Smart recursive power algorithm : To compute b n : 1.If n = 0: 1.1.Terminate with answer 1. 2.If n > 0: 2.1.Let p be b n / If n is even: Terminate with answer p p. 2.3.If n is odd: Terminate with answer b p p. Easy case: solved directly. Hard case: solved by comput- ing b n/2, which is easier since n/2 is closer than n to 0.

Faculty of Communications, Health and Science School of Computer and Information Science CSP1250 Data Structures with Java Java Collections by D Watt & D Brown (2001)Edited and Modified by Dr W Guo (2003)31 Example: recursive power algorithms (5) Implementation in Java: static int power (int b, int n) { // Return b n (where n is non-negative). if (n == 0) return 1; else { int p = power(b, n/2); if (n % 2 == 0) return p * p; else return b * p * p; } }

Faculty of Communications, Health and Science School of Computer and Information Science CSP1250 Data Structures with Java Java Collections by D Watt & D Brown (2001)Edited and Modified by Dr W Guo (2003)32 Example: recursive power algorithms (6) Analysis (counting multiplications): Each recursive power algorithm performs the same number of multiplications as the corresponding non-recursive algorithm. So their time complexities are the same: non-recursiverecursive Simple power algorithm O(n)O(n) O(n)O(n) Smart power algorithm O (log n )

Faculty of Communications, Health and Science School of Computer and Information Science CSP1250 Data Structures with Java Java Collections by D Watt & D Brown (2001)Edited and Modified by Dr W Guo (2003)33 Example: recursive power algorithms (7) Analysis (space): The non-recursive power algorithms use constant space, i.e., O (1). A recursive algorithm uses extra space for each recursive call. The simple recursive power algorithm calls itself n times before returning, whereas the smart recursive power algorithm calls itself floor(log 2 n ) times. non-recursiverecursive Simple power algorithm O (1) O(n)O(n) Smart power algorithm O (1) O (log n )

Faculty of Communications, Health and Science School of Computer and Information Science CSP1250 Data Structures with Java Java Collections by D Watt & D Brown (2001)Edited and Modified by Dr W Guo (2003)34 Example: Towers of Hanoi (1) Three vertical poles (1, 2, 3) are mounted on a platform. A number of differently-sized disks are threaded on to pole 1, forming a tower with the largest disk at the bottom and the smallest disk at the top. We may move one disk at a time, from any pole to any other pole, but we must never place a larger disk on top of a smaller disk. Problem: Move the tower of disks from pole 1 to pole 2.

Faculty of Communications, Health and Science School of Computer and Information Science CSP1250 Data Structures with Java Java Collections by D Watt & D Brown (2001)Edited and Modified by Dr W Guo (2003)35 Example: Towers of Hanoi (2) Animation (with 2 disks):

Faculty of Communications, Health and Science School of Computer and Information Science CSP1250 Data Structures with Java Java Collections by D Watt & D Brown (2001)Edited and Modified by Dr W Guo (2003)36 Example: Towers of Hanoi (3) Towers of Hanoi algorithm : To move a tower of n disks from pole source to pole dest : 1.If n = 1: 1.1.Move a single disk from source to dest. 2.If n > 1: 2.1.Let spare be the remaining pole, other than source and dest. 2.2.Move a tower of ( n –1) disks from source to spare. 2.3.Move a single disk from source to dest. 2.4.Move a tower of ( n –1) disks from spare to dest. 3.Terminate.

Faculty of Communications, Health and Science School of Computer and Information Science CSP1250 Data Structures with Java 1.If n = 1: 1.1.Move a single disk from source to dest. 2.If n > 1: 2.1.Let spare be the remaining pole, other than source and dest. 2.2.Move a tower of (n–1) disks from source to spare. 2.3.Move a single disk from source to dest. 2.4.Move a tower of (n–1) disks from spare to dest. 3.Terminate. sourcedest 1.If n = 1: 1.1.Move a single disk from source to dest. 2.If n > 1: 2.1.Let spare be the remaining pole, other than source and dest. 2.2.Move a tower of (n–1) disks from source to spare. 2.3.Move a single disk from source to dest. 2.4.Move a tower of (n–1) disks from spare to dest. 3.Terminate. sourcedestspare Example: Towers of Hanoi (4) Animation (with 6 disks): 1.If n = 1: 1.1.Move a single disk from source to dest. 2.If n > 1: 2.1.Let spare be the remaining pole, other than source and dest. 2.2.Move a tower of (n–1) disks from source to spare. 2.3.Move a single disk from source to dest. 2.4.Move a tower of (n–1) disks from spare to dest. 3.Terminate. sourcedestspare 1.If n = 1: 1.1.Move a single disk from source to dest. 2.If n > 1: 2.1.Let spare be the remaining pole, other than source and dest. 2.2.Move a tower of (n–1) disks from source to spare. 2.3.Move a single disk from source to dest. 2.4.Move a tower of (n–1) disks from spare to dest. 3.Terminate. sourcedestspare 1.If n = 1: 1.1.Move a single disk from source to dest. 2.If n > 1: 2.1.Let spare be the remaining pole, other than source and dest. 2.2.Move a tower of (n–1) disks from source to spare. 2.3.Move a single disk from source to dest. 2.4.Move a tower of (n–1) disks from spare to dest. 3.Terminate. sourcedestspare 1.If n = 1: 1.1.Move a single disk from source to dest. 2.If n > 1: 2.1.Let spare be the remaining pole, other than source and dest. 2.2.Move a tower of (n–1) disks from source to spare. 2.3.Move a single disk from source to dest. 2.4.Move a tower of (n–1) disks from spare to dest. 3.Terminate. sourcedestspare

Faculty of Communications, Health and Science School of Computer and Information Science CSP1250 Data Structures with Java Java Collections by D Watt & D Brown (2001)Edited and Modified by Dr W Guo (2003)38 Example: Towers of Hanoi (5) Analysis (counting moves): Let the total no. of moves required to move a tower of n disks be moves ( n ). Then: moves ( n ) = 1if n = 1 moves ( n ) = moves ( n –1)if n > 1 Solution: moves ( n ) = 2 n – 1 Time complexity is O (2 n ).

Faculty of Communications, Health and Science School of Computer and Information Science CSP1250 Data Structures with Java Java Collections by D Watt & D Brown (2001)Edited and Modified by Dr W Guo (2003)39 Week 2 Study Guide Reading: –Chapter 2: Java Collections by D.A. Watt and D.F. Brown (2001) –Appendix A: Java Collections by D.A. Watt and D.F. Brown (2001) Workshop 2 –Algorithm Analysis Next week: –First Test on Algorithm Analysis –Lecture: Array Data Structures Chapter 3: Java Collections by D.A. Watt and D.F. Brown (2001)