1. LeongHW, SoC, NUS (UIT2201: Algorithms) Page 1 Algorithms  Readings: [SG] Ch. 2 & 3  Chapter Outline: 1.Chapter Goals 2.What are Algorithms 1.Real Life Examples (origami, recipes) 2.Definition 3.Example: A = B + C 3.Expressing Algorithms – Pseudo-Code 4.Simple Algorithms 5.Recursive Algorithms 6.Time Complexity of Algorithms

2. LeongHW, SoC, NUS (UIT2201: Algorithms) Page 2 Algorithms  Computing devices are dumb oHow to explain to a dumb mechanical / computing device how to solve a problem  How to solve a problem using “a small, basic set of primitive instructions”.  Complexity of Solving Problems.

3. LeongHW, SoC, NUS (UIT2201: Algorithms) Page 3 1. Goals of Algorithm Study  To develop framework for instructing computer to perform tasks  To introduce notion of algorithm as means of specifying how to solve a problem  To introduce and appreciate approaches for defining and solving very complex tasks in terms of simpler tasks;

4. LeongHW, SoC, NUS (UIT2201: Algorithms) Page 4  Chapter Outline: 1.Chapter Goals 2.What are Algorithms 1.Real Life Examples (origami, recipes) 2.Definition of Algorithm 3.Example: A = B + C 3.Expressing Algorithms – Pseudo-Code 4.Simple Algorithms 5.Recursive Algorithms 6.Time Complexity of Algorithms

5. LeongHW, SoC, NUS (UIT2201: Algorithms) Page 5 2. Computer Science and Algorithms…  Computer Science can be considered… as study of algorithms including utheir formal properties uTheir hardware and software realisations uTheir applications to diverse areas

6. LeongHW, SoC, NUS (UIT2201: Algorithms) Page 6 Algorithms: Real Life Examples  Many Real-Life Examples oCooking: Recipe for preparing a dish oOrigami: The Art of Paper Folding oDirections: How to go to Changi Airport  Remember: oFramework: How to give instructions; oAlg: The actual step-by-step instructions; oAbstraction: Decomposing / Simplifying

7. LeongHW, SoC, NUS (UIT2201: Algorithms) Page 7 Real Life Example: Cooking  Framework: “Cooking or Recipe language”  Algorithm: “Recipe for a Dish” (step by step instructions on how to cook a dish)  Problem Decomposition oPreparing Ingredients; oPreparing Sauce;

8. LeongHW, SoC, NUS (UIT2201: Algorithms) Page 8 Real Life Example: Origami  Framework: “Origami or Paper-Folding language”  Algorithm: “Sequence of Paper-Folding Instructions” (step by step instructions for each fold)  Problem Decomposition oStart with a Bird Base; … oFinish the Head; oFinish the Legs; Finish the Tail;

9. LeongHW, SoC, NUS (UIT2201: Algorithms) Page 9 Real Life Examples: Issues  Problems/Difficulties: oImprecise Instructions; oJob can often be done even if instructions are not followed precisely oModifications may be done by the person following the instructions;  But, NOT for a Computer oNeeds to told PRECISELY what to do; uInstructions must be PRECISE; uCannot be vague or ambiguous

10. LeongHW, SoC, NUS (UIT2201: Algorithms) Page 10 Definition of Algorithm:  An algorithm for solving a problem “a finite sequence of unambiguous, executable steps or instructions, which, if followed would ultimately terminate and give the solution of the problem”.  Note the keywords: oFinite set of steps; oUnambiguous; oExecutable; oTerminates;

11. LeongHW, SoC, NUS (UIT2201: Algorithms) Page 11 Algorithm Examples?  Problem 1: What is the largest integer INPUT: All the integers { … -2, -1, 0, 1, 2, … } OUTPUT: The largest integer Algorithm: uArrange all the integers in a list in decreasing order; uMAX = first number in the list; uPrint out MAX; oWHY is the above NOT an Algorithm? u (Hint: How many integers are there?)  Problem 2: Who is the tallest women in the world? oAlgorithm: Tutorial...

12. LeongHW, SoC, NUS (UIT2201: Algorithms) Page 12 Example: Adding two (n-digit) numbers oInput: Two positive m-digit decimal numbers (a and b) a m, a m-1, …., a 1 b m, b m-1, …., b 1 oOutput: The sum c = a + b c m+1, c m, c m-1, …., c 1 (Note: In the textbook, it was a m-1,…,a 1,a 0 …)

13. LeongHW, SoC, NUS (UIT2201: Algorithms) Page 13 How to “derive” the algorithm  Adding is something we all know odone it a thousand times, know it “by heart”  How do we give the algorithm? oA step-by-step instruction oto a dumb machine  Try an example: 3 4 9 2 8 1 5 7 “Imagine you looking at yourself solving it”

14. LeongHW, SoC, NUS (UIT2201: Algorithms) Page 14 Step 1: Set the value of carry to 0 Step 2: Set the value of i to 1. Step 3: Repeat steps 4 through 6 until the value of i is > m. Step 4: Add a i and b i to the current value of carry, to get x i. Step 5: If x i < 10 then let c i =x i and reset carry to 0. Else (* i.e. x i  10 *) let c i =x i - 10 and reset carry to 1. Step 6: Increase the value of i by 1. Step 7: Set c m+1 to the value of carry. Step 8: Print the final answer c m+1, c m, …., c 1 Step 9: Stop. Algorithm: Finding sum of A & B

15. LeongHW, SoC, NUS (UIT2201: Algorithms) Page 15  Chapter Outline: 1.Chapter Goals 2.What are Algorithms 3.Expressing Algorithms – Pseudo-Code 1.Communicating Alg to computer 2.Pseudo-Code 3.Primitive Operations and examples 4.Variables and Arrays 5.Algorithm C=A+B in pseudo-code 4.Simple Algorithms 5.Recursive Algorithms 6.Time Complexity of Algorithms

16. LeongHW, SoC, NUS (UIT2201: Algorithms) Page 16 3. Expressing Algorithms to Computer  To communicate algorithm to computer oNeed way to “represent” the algorithm oCannot use English  Can use computer language omachine language and oprogramming languages (Java, Pascal, C) oBut, these are too tedious (&technical)  Use Pseudo-Code instead…

17. LeongHW, SoC, NUS (UIT2201: Algorithms) Page 17 Pseudo-Code to express Algorithms  Pseudo-Code oMixture of computer language and English uSomewhere in between uprecise enough to describe what is meant without being too tediuos oExamples: u Let c be 0; u Sort the list of numbers in increasing order;  Need to know both osyntax – representation osemantics – meaning

18. LeongHW, SoC, NUS (UIT2201: Algorithms) Page 18 Primitive Operations  To describe an algorithm, we need some well-defined programming primitives oAssignment primitive: u assignment statement, input/output statements oConditional primitive: u if statement u case statement oLooping (iterative) primitive: u for loop, u while loop,  Statements are executed one-by-one

19. LeongHW, SoC, NUS (UIT2201: Algorithms) Page 19 Conditional Primitives…  if statement oto take different actions based on condition  Syntax if (condition) then (Step A) else (Step B) endif if (condition) then (Step A) endif  Semantics condition? Step B truefalse Step A

20. LeongHW, SoC, NUS (UIT2201: Algorithms) Page 20 Examples -- (if statement)… Let mark be the total-mark obtained if (mark < 40) then (print “Student fail”) else (print “Student pass”) endif … read in mark (*from the terminal*) if (mark < 40) then (Grade  “F”) else if (mark < 50) then (Grade  “D”) else if (mark < 60) then (Grade  “C”) else if (mark < 70) then (Grade  “B”) else if (mark < 80) then (Grade  “A”); endif print “Student grade is”, Grade …

21. LeongHW, SoC, NUS (UIT2201: Algorithms) Page 21  the while-loop oloop a “variable” number of times  Syntax while (condition) do (some sequence of statements) endwhile  Semantics… Looping Primitive – while-loop condition? Some sequence of statements; true false

22. LeongHW, SoC, NUS (UIT2201: Algorithms) Page 22  First, the for-loop oloop a “fixed” or (pre- determined) number of times  Syntax for j  a to b do (some sequence of statements) endfor  Semantics… Looping Primitive – for-loop j  a; (j <= b)? Some sequence of statements; j  j+1; false true

23. LeongHW, SoC, NUS (UIT2201: Algorithms) Page 23 “Exercising the alg”: for and while for j  1 to 4 do print 2*j; endfor print “--- Done ---” Output: 2 4 6 8 --- Done --- j  1; while (j <= 4) do print 2*j; j  j + 1; endwhile print “--- Done ---” Output: 2 4 6 8 --- Done ---

24. LeongHW, SoC, NUS (UIT2201: Algorithms) Page 24 Variables and Arrays…  In the computer, each variable is assigned a storage “box” ocan store one number at any time oeg: sum, j, carry  Arrays: oOften deal with many numbers oSuch as A 1, A 2, A 3, …, A 100 oStore as an “array” A[1], A[2], …, A[100] uwe treat each of them as a variable, ueach is assigned a storage “box”

25. LeongHW, SoC, NUS (UIT2201: Algorithms) Page 25 Algorithm: A = B + C (in pseudo-code) We can re-write the C=A+B algorithm as follows: Alg. to Compute C = A + B: (*sum two big numbers*) carry  0; for i  1 to m do x[i]  a[i] + b[i] + carry ; if (x[i] < 10) then ( c[i]  x[i]; carry  0; ) else ( c[i]  x[i] – 10; carry  1; ) endfor; c[m+1]  carry; Print c[m+1], c[m], …., c[1]

26. LeongHW, SoC, NUS (UIT2201: Algorithms) Page 26  Chapter Outline: 1.Chapter Goals 2.What are Algorithms 3.Expressing Algorithms – Pseudo-Code 4.Simple Algorithms 1.Simple iterative algorithms Computing Sum, Find Max/Min 2.Modular Program Design 3.Divisibility and Prime Numbers 5.Recursive Algorithms 6.Time Complexity of Algorithms

27. LeongHW, SoC, NUS (UIT2201: Algorithms) Page 27 Simple iterative algorithm: Sum  Given:  Given: List of numbers: A 1, A 2, A 3, …., A n  Output: To compute the sum of the numbers Note: Store numbers in array A[1], A[2], …, A[n] Sum(A, n); begin Sum_sf  0; k  1; while (k <= n) do Sum_sf  Sum_sf + A[k]; k  k + 1; endwhile Sum  Sum_sf; Print “Sum is”, Sum end;

28. LeongHW, SoC, NUS (UIT2201: Algorithms) Page 28 Exercising Algorithm Sum: A[1] A[2] A[3] A[4] A[5] A[6] n=6 2 5 10 3 12 24 k Sum-sf Sum ? 0 ? 1 2 ? 2 7 ? 3 17 ? 4 20 ? 5 32 ? 6 56 ? 6 56 56 Sum is 56 Input: Processing: Output:

29. LeongHW, SoC, NUS (UIT2201: Algorithms) Page 29 Algorithm for Sum (with for-loop)  We can also use a while-loop instead of a for loop.  HW:(a) Note the differences… (b) Modify it to compute the average? Sum(A, n); (* Find the sum of A1, A2,…, An. *) begin Sum_sf  0; for k  1 to n do Sum_sf  Sum_sf + A[k]; endfor Sum  Sum_sf; Print “Sum is”, Sum end;

30. LeongHW, SoC, NUS (UIT2201: Algorithms) Page 30 Remarks about the iterative algorithm…  Note the three stages: 1.Initialization uSet some values at the beginning 2.Iteration uThis is the KEY STEP uWhere most of work is done 3.Post-Processing or Cleanup  Can use this setup for other problems oCalculating average, sum-of-squares oFinding max, min; Searching for a number,

31. LeongHW, SoC, NUS (UIT2201: Algorithms) Page 31 Modular Program Design  Software are complex oHUGE (millions of lines of code) eg: Linux, Outlook oCOMPLEX; eg: Flight simulator  Idea: Divide-and-Conquer Method oComplex tasks can be divided and each part solved separately and combined later.  Modular Program Design oDivide big programs into smaller modules oThe smaller parts are ucalled modules, subroutines, or procedures uDesign, implement, and test separately oModularity, Abstraction, Division of Labour oSimplifies process of writing alg/programs

32. LeongHW, SoC, NUS (UIT2201: Algorithms) Page 32 Simple Algorithms: Prime Numbers  Algorithm to determine if n is prime Given: A positive integer n Question: Is n a prime number?  What do we know? “A number n is prime iff n has no positive factors except 1 and n”  Idea: Express it “algorithmically” “n is not divisible by any positive number k such that 1 < k < n.” or k=2,3,4,…,(n-1)  What we already have: oA module Divisible(m,n) to check divisibility oWe can use it as a primitive

33. LeongHW, SoC, NUS (UIT2201: Algorithms) Page 33 Pseudo-Code for Prime:  Note: Prime uses the module Divisible(k,n)  Exercise it with: oPrime (5); Prime (4); oPrime (55); Prime (41); Prime(n) (* To determine if n is prime *) begin for k  2 to (n-1) do if Divisible(k,n) then (Print “False” & exit) else (* Do nothing *) endfor Print “True” & Stop end;

34. LeongHW, SoC, NUS (UIT2201: Algorithms) Page 34 A Simple Module: Divisibility  A module (procedure) for divisibility Given: Two positive integers m and n Question: Is n divisible by m? or Algorithm to compute Divisible(m,n)  Algorithm Idea: “A positive integer n is divisible by m iff for some positive integer k  n, m*k = n.”

35. LeongHW, SoC, NUS (UIT2201: Algorithms) Page 35 Module for Divisibility:  Algorithm (in pseudo-code)  Exercise it with: oDivisible (3, 9); Divisible (3, 5);Divisible (3,101); Divisible(m,n) (* To compute if n is divisible by m *) begin D  false; (* assume false, first *) for k  1 to n do if (m*k = n) then (D  true; exit-loop) else (* Do nothing *) endfor Divisible  D; (* true or false *) end;

36. LeongHW, SoC, NUS (UIT2201: Algorithms) Page 36  Chapter Outline: 1.Chapter Goals 2.What are Algorithms 3.Expressing Algorithms – Pseudo-Code 4.Simple Algorithms 5.Recursive Algorithms 6.Time Complexity of Algorithms 1.Sequential search algorithm 2.Binary search algorithm 3.Analysis of Time Complexity 7.Summary…

37. LeongHW, SoC, NUS (UIT2201: Algorithms) Page 37 Search: sequential search algorithm  Given:  Given: List of numbers: A 1, A 2, A 3, …., A n and a query number x;  Question: Search for x in the list; Sequential-Search(A, n, x); (* Search for x in A1, A2,…, An. *) begin for k  1 to n do if (x = A[k]) then (Print “Yes”; Stop) else (* do nothing *) endfor Print “No”; Stop end;

38. LeongHW, SoC, NUS (UIT2201: Algorithms) Page 38 Remarks on Sequential Search Alg  Analogy: House-to-house search…  How fast is it to search for x? oHow many comparisons do we need? oExample: 13, 38, 19, 74, 76, 14, 12, 38, 22, 55 uWhen x=14, need 6 comparisons uWhen x=13, need only 1 comparison  BEST CASE uWhen x=55, need 10 comparisons  WORST CASE uWhen x=5, need 10 comparisons  WORST CASE  In general, given n numbers, A 1,…,A n oBest Case: 1 comparison oWorst Case: n comparisons oAverage Case: (n+1)/2 comparisons

39. LeongHW, SoC, NUS (UIT2201: Algorithms) Page 39 Binary Search  If the List is sorted, that is A 1  A 2  A 3  ….  A n  Then, we can do better, o actually a lot better….

40. LeongHW, SoC, NUS (UIT2201: Algorithms) Page 40 Binary Search 1, 4, 9, 11, 14, 43, 78

41. LeongHW, SoC, NUS (UIT2201: Algorithms) Page 41 Binary Search (How fast is it?)  Imagine n=100 o After 1 step, size is  50 o After 2 steps, size is  25 o After 3 steps, size is  12 o After 4 steps, size is  6 o After 5 steps, size is  3 o After 6 steps, size is  1 o DONE!!  What if n=1000? 1,000,000?

42. LeongHW, SoC, NUS (UIT2201: Algorithms) Page 42 Binary Search Algorithm Input: Sorted List A 1  A 2  A 3  ….  A n A number x. Question: Is x in the list? Binary-Search(A,n,x); 1. First  1 Last  n 2. While ( First  Last ) do mid   (first+last) / 2  If ( x = A[mid] ) then (output Yes and Stop) else If ( x < A[mid] ) then Last  mid-1 else If ( x > A[mid] ) then First  mid+1 EndWhile 3. Output False and Stop 4. End

43. LeongHW, SoC, NUS (UIT2201: Algorithms) Page 43 Time Complexity Analysis  Sequential Search (Alg): o Worst Case: n comparisons o Best Case: 1 comparison o Avg Case: n/2 comparisons  Binary Search (Alg): oWorst Case: log 2 n comparisons o Best Case: 1 comparison o Avg Case: about log 2 n comparisons  How to get the Average Case? o using mathematical analysis…

44. LeongHW, SoC, NUS (UIT2201: Algorithms) Page 44 Complexity of Algorithm…  Logarithmic Time Algorithm oBinary Search  A Linear Time Algorithm oAlgorithm Sum(A,n) -- O(n) time oAlgorithm Sequential-Search(A,n,x) – O(n) time  A Quadratic Time Algorithm oSimple Median-Find (T2-Q3)  An Exponential Time Algorithm oAll-Subsets(A,n)

45. LeongHW, SoC, NUS (UIT2201: Algorithms) Page 45  Chapter Outline: 1.Chapter Goals 2.What are Algorithms 3.Expressing Algorithms – Pseudo-Code 4.Simple Algorithms 5.Recursive Algorithms 1.Recursion – the idea 2.Fibonacci sequence 3.Tower of Hanoi 6.Time Complexity of Algorithms

46. LeongHW, SoC, NUS (UIT2201: Algorithms) Page 46 5. Recursion  A problem solving method of “decomposing bigger problems into smaller sub-problems that are identical to itself.”  General Idea: oSolve simplest (smallest) cases DIRECTLY uusually these are very easy to solve oSolve bigger problems using smaller sub-problems uthat are identical to itself (but smaller and simpler)  Abstraction: oTo solve a given problem, we first assume that we ALREADY know how to solve it for smaller instances!!  Dictionary definition: recursion see recursion

47. LeongHW, SoC, NUS (UIT2201: Algorithms) Page 47 Example: Fibonacci Numbers…  Definition of Fibonacci numbers 1.F 1 = 1, 2.F 2 = 1, 3.for n>2, F n = F n-1 + F n-2  Problem: Compute F n for any n.  The above is a recursive definition. oF n is computed in-terms of itself oactually, smaller copies of itself – F n-1 and F n-2  Actually, Not difficult: F 3 = 1 + 1 = 2F 6 = 5 + 3 = 8F 9 = 21 + 13 = 34 F 4 = 2 + 1 = 3F 7 = 8 + 5 = 13 F 10 = 34 + 21 = 55 F 5 = 3 + 2 = 5F 8 = 13 + 8 = 21F 11 = 55 + 34 = 89 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89, 144, …

48. LeongHW, SoC, NUS (UIT2201: Algorithms) Page 48 Fibonacci Numbers: Recursive alg  The above is a recursive algorithm  It is simple to understand and elegant!  But, very SLOW Fibonacci(n) (* Recursive, SLOW *) begin if (n=1) or (n=2) then Fibonacci(n)  1 (*simple case*) else Fibonacci(n)  Fibonacci(n-1) + Fibonacci(n-2) endif end;

49. LeongHW, SoC, NUS (UIT2201: Algorithms) Page 49 Recursive Fibonacci Alg -- Remarks  How slow is it? oEg: To compute F(6)… F(5) F(4)F(3) F(2) F(1) F(2) F(1) F(4) F(3)F(2) F(1) F(6) J HW: Can we compute it faster?

50. LeongHW, SoC, NUS (UIT2201: Algorithms) Page 50 Given: Three Pegs A, B and C Peg A initially has n disks, different size, stacked up, larger disks are below smaller disks Problem: to move the n disks to Peg C, subject to 1.Can move only one disk at a time 2.Smaller disk should be above larger disk 3.Can use other peg as intermediate Example: Tower of Hanoi ABCABC

51. LeongHW, SoC, NUS (UIT2201: Algorithms) Page 51 Tower of Hanoi  How to Solve: Strategy… oGeneralize first: Consider n disks for all n  1 oOur example is only the case when n=4  Look at small instances… oHow about n=1 uOf course, just “Move disk 1 from A to C” oHow about n=2? 1.“Move disk 1 from A to B” 2.“Move disk 2 from A to C” 3.“Move disk 1 from B to C”

52. LeongHW, SoC, NUS (UIT2201: Algorithms) Page 52 Tower of Hanoi (Solution!)  General Method: oFirst, move first (n-1) disks from A to B oNow, can move largest disk from A to C oThen, move first (n-1) disks from B to C  Try this method for n=3 1.“Move disk 1 from A to C” 2.“Move disk 2 from A to B” 3.“Move disk 1 from C to B” 4.“Move disk 3 from A to C” 5.“Move disk 1 from B to A” 6.“Move disk 1 from B to C” 7.“Move disk 1 from A to C”

53. LeongHW, SoC, NUS (UIT2201: Algorithms) Page 53 Algorithm for Towel of Hanoi (recursive)  Recursive Algorithm owhen (n=1), we have simple case oElse (decompose problem and make recursive-calls) Hanoi(n, A, B, C); (* Move n disks from A to C via B *) begin if (n=1) then “Move top disk from A to C” else (* when n>1 *) Hanoi (n-1, A, C, B); “Move top disk from A to C” Hanoi (n-1, B, C, A); endif end;

54. LeongHW, SoC, NUS (UIT2201: Algorithms) Page 54 Characteristics of Algorithm  Correctness  Complexity --- time, space (memory), etc  Ease of understanding  Ease of coding  Ease of maintainence

55. LeongHW, SoC, NUS (UIT2201: Algorithms) Page 55  If you are new to algorithms oread the textbook otry out the algorithms odo the exercises … The End …