1. Algorithms 國立清華大學資訊工程學系 CS1356 資訊工程導論 2015/6/24

2. 2 Josephus Problem Flavius Josephus is a Jewish historian living in the 1st century. According to his account, he and his 40 comrade soldiers were trapped in a cave, surrounded by Romans. They chose suicide over capture and decided that they would form a circle and start killing one by skipping every two others. By luck, or maybe by the hand of God, Josephus and another man remained the last and gave up to the Romans.

3. 3 Can You Find the Safe Place? 12 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 202122 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 4041 Safe place Can you find the safe place FASTER?

4. 4 Algorithm An effective method for solving a problem using a finite sequence of instructions. –It need be able to solve the problem. (correctness) –It can be represented by a finite number of (computer) instructions. Each instruction must be achievable (by computer) –The more effective, the better algorithm is. How to measure the “efficiency”?

5. 5 Outline The first algorithm –Model, simulation, algorithm primitive The second algorithm –Algorithm discovery, recursion The third algorithm –Solving recursion, mathematical induction The central role of computer science –Why study math?

6. 6 The First Algorithm of the Josephus Problem

7. 7 A Simpler Version Let’s consider a similar problem –There are n person in a circle, numbered from 1 to n sequentially. –Starting from the number 1 person, every 2 nd person will be killed. –What is the safe place? The input is n, the output f(n) is a number between 1 and n. –Ex: f(8) = ? 1 2 3 4 5 6 7 8

8. 8 1st Algorithm: Simulation We can find f(n) using simulation. –Simulation is a process to imitate the real objects, states of affairs, or process. –We do not need to “kill” anyone to find f(n). The simulation needs –(1) a model to represents “n people in a circle” –(2) a way to simulate “kill every 2 nd person” –(3) knowing when to stop

9. 9 Simulation Using Pebbles 1.Put n pebbles numbered 1,…,n in a circle clockwise sequentially 2.From the pebble numbered 1, count pebbles, and remove every 2 nd pebble, until there is only 1 pebble left 3.The number of the remaining pebble is f(n) 1 2 3 4 5 6 7 8

10. 10 Abstraction and Model “put pebbles numbered 1,…,n in a circle” is a model to represent n people in a circle “remove every 2 nd pebble” is a simulation for killing the every 2 nd person. A model is an abstraction of objects, systems, or concepts that capture important characters of the problem. The definition of important characters is varied for different problems.

11. 11 Computer Simulation How to perform simulations on computers? –We need to use what computer can do to represent the model and perform actions. Algorithm primitives –A set of well-defined building blocks that computers can perform and human can understand easily. “Put pebbles in a circle”, “remove pebbles”,… A finite sequence of instruction

12. 12 Data Abstraction There is nothing “circular” inside computer, and nothing can be “removed” physically. –But we can arrange memory cells to “model” and “simulate” them  data structure Some common data structures

13. 13 Model n People in a Circle We can use “data structure” to model it. This is called a “circular linked list”. –Each node is of some “struct” data type –Each link is a “pointer” 1 2 3 5 6 7 8 4

14. 14 Kill Every 2 nd Person Remove every 2 nd node in the circular linked list. –You need to maintain the circular linked structure after removing node 2 –The process can continue until … 1 2 3 5 6 7 8 4

15. 15 Knowing When to Stop Stop when there is only one node left –How to know that? –When the next is pointing to itself –It’s ID is f(n) f(8) = 1 1 3 5 6 7 8 4

16. 16 Efficiency of an Algorithm The efficiency of an algorithm is usually measured by the number of operations –assignment, comparison, arithmetic operation –Eventually the number of instructions in chap 2 Expressed in a function of problem size n. –For example, 5n 3 +2n 2 +10nlogn (1) Only interested in the case of large n –decided by the order of the dominant term. –n 3 is the order of the dominant term of Eq. (1)

17. 17 How Fast to Compute f(n)? Since the first algorithm removes one node at a time, it needs n-1 steps to compute f(n) The cost of each step (removing one node) is a constant time  (independent of n) So the total number of operations to find f(n) is  (n-1) –We can just represent it as O(n).

18. 18 The Second Algorithm of the Josephus Problem

19. 19 Can We Do Better? Simulation is a brute-force method. Faster algorithms are usually expected. The scientific approach 1.Observing some cases 2.Making some hypotheses (generalization) 3.Testing the hypotheses 4.Repeat 1-3 until success

20. 20 Observing a Case Let’s check out the case n = 8. What have you observed? –All even numbered people die This is true for all kinds of n. –The starting point is back to 1 This is only true when n is even –Let’s first consider the case when n is even 1 2 3 4 5 6 7 8

21. 21 n Is Even For n=8, after the first “round”, there are only 4 people left. If we can solve f(4), can we use it to solve f(8)? If we renumber the remaining people, 1  1, 3  2, 5  3, 7  4, it becomes the n=4 problem. So, if f(4)=x, f(8) =2x – 1 1 2 3 4 1 2 3 4 5 6 7 8

22. 22 n Is Odd Let’s checkout the case n=9 The starting point becomes 3 If we can solve f(4), can we use it to solve f(9)? If we renumber the remaining people, 3  1, 5  2, 7  3, 9  4, it becomes the n=4 problem. So, if f(4)=x, f(9) =2x+1 1 2 3 4 5 6 7 8 9 1 2 3 4

23. 23 Recursive Relation Hypothesis: f(2n)=2f(n)–1. f(2n+1)=2f(n)+1. –How to prove or disprove it? This is called a recursive relation. –You can design a recursive algorithm Compute f(8) uses f(4); Compute f(4) uses f(2); Compute f(2) uses f(1); f(1) =1. Why?

24. 24 Recursive Relation  Algorithm With recursive relations, you can design a recursive algorithm Create a subroutine Inside the subroutine, include –The base case. For the Josephus problem, the base case is n=1 –The recurrence part Make procedure calls to the same subroutine with smaller problem size Compose the result based on the recursive relation

25. 25 Recursive Algorithm Procedure Josephus (n, fn) /* n is the input, fn is the output (= f(n)) */ 1. if (n = 1) then (fn  1) else 2. ( if (n is even) then 3. ( call Josephus (n/2, fn) 4. fn  2*fn – 1) else 5. ( call Josephus ((n–1)/2, fn) 6. fn  2*fn+1 ) ) Header Base case Recursive part n is even f(n)=2f(n/2)-1 n is odd f(n)=2f((n-1)/2)+1

26. 26 How Fast to Compute f(n)? n reduces half at each step. –Need log 2 (n) steps to reach n=1. Each step needs a constant number of operations . The total number of operations is  log 2 (n)

27. 27 The Third Algorithm of the Josephus Problem

28. 28 Can We Do Better? Can we solve the recursion? –If we can, we may have a better algorithm to find f(n) Remember: –observation, hypotheses, verification

29. 29 n = 6, f(6) = ? n = 7, f(7) = ? n = 8, f(8) = ? n = 9, f(9) = ? Let’s Make More Observations n = 2, f(2) = ? n = 3, f(3) = ? n = 4, f(4) = ? n = 5, f(5) = ? 1 3 1 3 5 7 1 3 Have you observed the pattern of f(n)? 05101520253035 0 5 10 15 20 25 30 35

30. 30 What Are the Patterns? 1.f(1)=f(2)=f(4)=f(8)=f(16)=1. What are they in common? 2.If we group the sequence [1], [2,3], [4,7],[8,15], f(n) in each group is a sequence of consecutive odd numbers starting from 1. 3.Let k = n – the first number in n’s group What is the pattern of k? n12345678910111213141516 f(n)1131357135791113151 k0010123012345670 f(n)=2k+1 They are power of 2, 2 m.

31. 31 Guess the Solution f(n) = 2k+1 –k = n – the first number in n’s group –The first number in n’s group is 2 m. 2 m  n < 2 m+1 How fast can you find m? –(1) use “binary search” on the bit pattern of n Since n has log 2 (n) bits, it takes log 2 (log 2 (n)) steps. –(2) use “log2” function and take its integer part It takes constant time only. (independent of n.)

32. 32 Verify the Solution If f(n)=2k+1, k=n–2 m is the solution, it must satisfy the recursion How to prove it? –Hint: using mathematical induction

33. 33 Comparison of 3 Algorithms Running time n

34. 34 The Central Role of Computer Science Why study math?

35. 35 Algorithms Studied so Far In chap 1, binary and decimal conversion, 2’s complement calculation, data compression, error correction, encryption… In chap 2, we studied how to use computer to implement algorithms –In homework 4, we have problems for pipeline, prefix sum (parallel algorithm), and virtual memory page replacement (online algorithm)

36. 36 In chap 3, we saw examples of using algorithms to help manage resources –Virtual memory mapping, scheduling, concurrent processes execution, etc. In chap 4, we learned protocols (distributed, randomized algorithms) –CSMA/CD, CSMA/CA, routing, handshaking, flow control, etc. In chap 5, we learned some basic algorithmic strategies, such as simulation, recursion, and how to analyze them

37. 37 Learning Algorithms Every class in CS has some relations with algorithms –But some classes are not obviously related to algorithms, such as math classes. – 微積分, 離散數學, 線性代數, 機率, 工程數學,… Why should we study them? –They are difficult, and boring, and difficult…

38. 38 Why Study Math? Train the logical thinking and reasoning –Algorithm is a result of logical thinking: correctness, efficiency, … –Good programming needs logical thinking and reasoning. For example, debugging. Learn some basic tricks –Many beautiful properties of problems can be revealed through the study of math –Standing on the shoulders of giants

39. 39 Try to Solve These Problems 1.Given a network of thousands nodes, find the shortest path from node A to node B –The shortest path problem ( 離散數學 ) 2.Given a circuit of million transistors, find out the current on each wire –Kirchhoff's current law ( 線性代數 ) 3.In CSMA/CD, what is the probability of collisions? –Network analysis ( 機率 )

40. 40 微積分 Limits, derivatives, and integrals of continuous functions. Used in almost every field – 網路分析, 效能分析 – 訊號處理, 影像處理, 類比電路設計 – 科學計算, 人工智慧, 電腦視覺, 電腦圖學 –… Also the foundation of many other math – 機率, 工程數學

41. 41 離散數學 Discrete structure, graph, integer, logic, abstract algebra, combinatorics The foundation of computer science –Every field in computer science needs it –Particularly, 演算法, 數位邏輯設計, 密碼學, 編 碼理論, 計算機網路, CAD, 計算理論 Extended courses –Special topics on discrete structure, graph theory, concrete math

42. 42 線性代數 Vectors, matrices, tensors, vector spaces, linear transformation, system of equations Used in almost everywhere when dealing with more than one number – 網路分析, 效能分析 – 訊號處理, 影像處理 – 科學計算, 人工智慧, 電腦視覺, 電腦圖學 –CAD, 電路設計

43. 43 機率 Probability models, random variables, probability functions, stochastic processes Every field uses it –Particularly, 網路分析, 效能分析, 訊號處理, 影 像處理, 科學計算, 人工智慧, 電腦視覺 … –Other examples: randomized algorithm, queue theory, computational finance, performance analysis

44. 44 工程數學 A condensed course containing essential math tools for most engineering disciplines –Partial differential equations, Fourier analysis, Vector calculus and analysis Used in the fields that need to handle continuous functions – 網路分析, 效能分析, 訊號處理, 影像處理, 科學計 算, 人工智慧, 電腦視覺, CAD, 電路設計

45. 45 Reference The Josephus problem: –Graham, Knuth, and Patashnik, “Concrete Mathematics”, section 1.3 –http://en.wikipedia.org/wiki/Josephus_problemhttp://en.wikipedia.org/wiki/Josephus_problem –http://mathworld.wolfram.com/JosephusProblem.htmlhttp://mathworld.wolfram.com/JosephusProblem.html Algorithm representation –Textbook: 5.2 The related courses are those listed in the last part of slides, and of course, 演算法設計, 高等程式設計實作