1. CS135601 Introduction to Information Engineering
Algorithms
12/9/2018
Che-Rung Lee
12/9/2018
CS Introduction to Information Engineering

2. CS135601 Introduction to Information Engineering
Josephus problem
Flavius Josephus is a Jewish historian living in the 1st century. According to his account, he and his 40 comrade soldiers were trapped in a cave, surrounded by Romans. They chose suicide over capture and decided that they would form a circle and start killing themselves using a step of three. As Josephus did not want to die, he was able to find the safe place, and stayed alive with his comrade, later joining the Romans who captured them.
12/9/2018
CS Introduction to Information Engineering

3. Can you find the safe place?40 41 1 2 39 3 38 37 4 5 36 6 35 7 34 8 33 9 32
Safe place 10 31 11 30
Can you find the safe place FASTER? 12 29 13 28 14 27 15 26 16 25 17 24 18 23 19 22 21 20
12/9/2018
CS Introduction to Information Engineering

4. CS135601 Introduction to Information Engineering
Algorithm
An effective method for solving a problem using a finite sequence of instructions.
It need be able to solve the problem. (correctness)
It can be represented by a finite number of (computer) instructions.
Each instruction must be achievable (by computer)
The more effective, the better algorithm is.
How to measure the “efficiency”?
12/9/2018
CS Introduction to Information Engineering

5. CS135601 Introduction to Information Engineering
Outline
The first algorithm
Model, simulation, algorithm primitive
The second algorithm
Algorithm discovery, recursion
The third algorithm
Solving recursion, mathematical induction
The central role of computer science
Why study math?
12/9/2018
CS Introduction to Information Engineering

6. CS135601 Introduction to Information Engineering
The first algorithm
12/9/2018
CS Introduction to Information Engineering

7. CS135601 Introduction to Information Engineering
A simpler version
Let’s consider a similar problem
There are n person in a circle, numbered from 1 to n sequentially.
Starting from the number 1 person, every 2nd person will be killed.
What is the safe place?
The input is n, the output f(n) is a number between 1 and n.
Ex: f(8) = ? 1 2 8 3 7 4 6 5
12/9/2018
CS Introduction to Information Engineering

8. The first algorithm: simulation
We can find f(n) using simulation.
Simulation is a process to imitate the real objects, states of affairs, or process.
We do not need to “kill” anyone to find f(n).
The simulation needs
(1) a model to represents “n people in a circle”
(2) a way to simulate “kill every 2nd person”
(3) knowing when to stop
12/9/2018
CS Introduction to Information Engineering

9. Model n people in a circle
We can use “data structure” to model it.
This is called a “circular linked list”.
Each node is of some “struct” data type
Each link is a “pointer” 1 2 3 5 6 7 8 4
struct PIC {
int ID;
struct PIC *next; }
12/9/2018
CS Introduction to Information Engineering

10. CS135601 Introduction to Information Engineering
Kill every 2nd person
Remove every 2nd node in the circular liked list.
You need to maintain the circular linked structure after removing node 2
The process can continue until … 1 8 2 7 3 6 4 5
12/9/2018
CS Introduction to Information Engineering

11. CS135601 Introduction to Information Engineering
Knowing when to stop
Stop when there is only one node left
How to know that?
When the *next is pointing to itself
It’s ID is f(n)
f(8) = 1 1 8 7 3 6 4 5
12/9/2018
CS Introduction to Information Engineering

12. How fast can it compute f(n)?
Since the first algorithm removes one node at a time, it needs n-1 steps to compute f(n)
The cost of each step (removing one node) is a constant time  (independent of n)
So the total number of operations to find f(n) is (n-1)
We can just represent it as O(n).
12/9/2018
CS Introduction to Information Engineering

13. CS135601 Introduction to Information Engineering
The Second Algorithm
12/9/2018
CS Introduction to Information Engineering

14. CS135601 Introduction to Information Engineering
Can we do better?
Simulation is a brute-force method. Faster algorithms are usually expected.
The scientific approach
Observing some cases
Making some hypotheses (generalization)
Testing the hypotheses
Repeat 1-3 until success
12/9/2018
CS Introduction to Information Engineering

15. CS135601 Introduction to Information Engineering
Observing a case
Let’s checkout the case n = 8.
What have you observed?
All even numbered people die
This is true for all kinds of n.
The starting point is back to 1
This is only true when n is even
Let’s first consider the case when n is even 1 2 8 3 7 4 6 5
12/9/2018
CS Introduction to Information Engineering

16. CS135601 Introduction to Information Engineering
n is even
For n=8, after the first “round”, there are only 4 people left.
If we can solve f(4), can we use it to solve f(8)?
If we renumber the remaining people, 11, 32, 53, 74, it becomes the n=4 problem.
So, if f(4)=x, f(8) =2x – 1 1 2 8 3 7 4 6 5 1 2 3 4
12/9/2018
CS Introduction to Information Engineering

17. CS135601 Introduction to Information Engineering
n is odd
Let’s checkout the case n=9
The starting point becomes 3
If we can solve f(4), can we use it to solve f(9)?
If we renumber the remaining people, 31, 52, 73, 94, it becomes the n=4 problem.
So, if f(4)=x, f(9) =2x+1 1 9 2 8 3 7 4 6 5 1 2 3 4
12/9/2018
CS Introduction to Information Engineering

18. CS135601 Introduction to Information Engineering
Recursive relation
Hypothesis: If f(n)=x, f(2n)=2x– If f(n)=x, f(2n+1)=2x+1.
How to prove or disprove it?
This is called a recursive relation.
You can design a recursive algorithm
Compute f(8) uses f(4);
Compute f(4) uses f(2);
Compute f(2) uses f(1);
f(1) =1. Why?
12/9/2018
CS Introduction to Information Engineering

19. CS135601 Introduction to Information Engineering
Recursive function
call Josephus(9)
int Josephus (n) {
/* base case */
if (n == 1) return 1;
if (n%2 == 0) /*n is even*/
return 2*Josephus (n/2)-1;
else /*n is odd*/
return 2*Josephus((n–1)/2)+1; }
output fn
fn=3
call Josephus(4)
fn  2*fn+1
fn=1
call Josephus(2)
fn  2*fn–1
fn=1
call Josephus(1)
fn  2*fn–1
fn=1
if (n = 1) return 1
12/9/2018
CS Introduction to Information Engineering

20. How fast can it compute f(n)?
n reduces at least half at each step.
Need log2(n) steps to reach n=1.
Each step needs a constant number of operations .
The total number of operations is log2(n)
12/9/2018
CS Introduction to Information Engineering

21. CS135601 Introduction to Information Engineering
The Third Algorithm
12/9/2018
CS Introduction to Information Engineering 21

22. CS135601 Introduction to Information Engineering
Can we do better?
Can we solve the recursion?
If we can, we may have a better algorithm to find f(n)
Remember:
observation, hypotheses, verification
12/9/2018
CS Introduction to Information Engineering

23. Let’s make more observations
n = 2, f(2) = ?
n = 3, f(3) = ?
n = 4, f(4) = ?
n = 5, f(5) = ?
n = 6, f(6) = ?
n = 7, f(7) = ?
n = 8, f(8) = ?
n = 9, f(9) = ? 5 1 3 7 1 1 3 3 5 10 15 20 25 30 35
Have you observed the pattern of f(n)?
12/9/2018
CS Introduction to Information Engineering

24. CS135601 Introduction to Information Engineering
What are the patterns? n 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
f(n) k 1 2 3 4 5 6 7
f(1)=f(2)=f(4)=f(8)=f(16)=1. What are they in common?
If we group the sequence [1], [2,3], [4,7],[8,15], f(n) in each group is a sequence of consecutive odd numbers starting from 1.
Let k = n – the first number in n’s group What is the pattern of k?
They are power of 2, 2m.
f(n)=2k+1
12/9/2018
CS Introduction to Information Engineering

25. CS135601 Introduction to Information Engineering
Guess the solution
f(n) = 2k+1
k = n – the first number in n’s group
The first number in n’s group is 2m.
2m  n < 2m+1
How fast can you find m?
(1) use “binary search” on the bit pattern of n
Since n has log2(n) bits, it takes log2(log2(n)) steps.
(2) use “log2” function and take its integer part
It takes constant time only. (independent of n.)
12/9/2018
CS Introduction to Information Engineering

26. Running time of three algorithms10 3 4 5 6 7 8 -8 -6 -4 -2 2
Algorithm 1
Algorithm 2
Algorithm 3
Running time n
12/9/2018
CS Introduction to Information Engineering

27. The Central Role of Computer Science
Why study math?
12/9/2018
CS Introduction to Information Engineering 27

28. Algorithms we studied so far
In chap 1, binary and decimal conversion, 2’s complement calculation, data compression, error correction, encryption…
In chap 2, we studied how to use computer to implement algorithms
In homework 4, we have problems for pipeline, prefix sum (parallel algorithm), and virtual memory page replacement (online algorithm)
12/9/2018
CS Introduction to Information Engineering

29. CS135601 Introduction to Information Engineering
In chap 3, we see some examples of using algorithms to help manage resources
Virtual memory mapping, scheduling, concurrent processes execution, etc.
In chap 4, we learned protocols(distributed algorithms, randomized algorithms)
CSMA/CD, CSMA/CA, routing, handshaking, flow control, etc.
In chap 5, we learned some basic algorithmic strategies, such as simulation, recursion, and how to analyze them
12/9/2018
CS Introduction to Information Engineering

30. CS135601 Introduction to Information Engineering
Learning algorithms
Every class in CS has some relations with algorithms
But some classes are not obviously related to algorithms, such as math classes.
微積分,離散數學,線性代數,機率,工程數學,…
Why should we study them?
They are difficult, and boring, and difficult…
12/9/2018
CS Introduction to Information Engineering

31. CS135601 Introduction to Information Engineering
Why study math?
Train the logical thinking and reasoning
Algorithm is a result of logical thinking: correctness, efficiency, …
Good programming needs logical thinking and reasoning. For example, debugging.
Learn some basic tricks
Many beautiful properties of problems can be revealed through the study of math
Standing on the shoulders of giants
12/9/2018
CS Introduction to Information Engineering

32. Try to solve those problems
Given a network of thousands nodes, find the shortest path from node A to node B
The shortest path problem (離散數學)
Given a circuit of million transistors, find out the current on each wire
Kirchhoff's current law (線性代數)
In CSMA/CD, what is the probability of collisions?
Network analysis (機率)
12/9/2018
CS Introduction to Information Engineering

33. CS135601 Introduction to Information Engineering
微積分
Limits, derivatives, and integrals of continuous functions.
Used in almost every field
網路分析, 效能分析
訊號處理, 影像處理, 類比電路設計
科學計算, 人工智慧, 電腦視覺, 電腦圖學 …
Also the foundation of many other math
機率, 工程數學
12/9/2018
CS Introduction to Information Engineering

34. CS135601 Introduction to Information Engineering
離散數學
Discrete structure, graph, integer, logic, abstract algebra, combinatorics
The foundation of computer science
Every field in computer science needs it
Particularly, 演算法, 數位邏輯設計, 密碼學, 編碼理論, 計算機網路, CAD, 計算理論
Extended courses
Special topics on discrete structure, graph theory, concrete math
12/9/2018
CS Introduction to Information Engineering

35. CS135601 Introduction to Information Engineering
線性代數
Vectors, matrices, vector spaces, linear transformation, system of equations
Used in almost everywhere when dealing with more than one variables
網路分析, 效能分析
訊號處理, 影像處理
科學計算, 人工智慧, 電腦視覺, 電腦圖學
CAD, 電路設計
12/9/2018
CS Introduction to Information Engineering

36. CS135601 Introduction to Information Engineering機率
Probability models, random variables, probability functions, stochastic processes
Every field uses it
Particularly, 網路分析, 效能分析, 訊號處理, 影像處理, 科學計算, 人工智慧, 電腦視覺…
Other examples: randomized algorithm, queue theory, computational finance, performance analysis
12/9/2018
CS Introduction to Information Engineering

37. CS135601 Introduction to Information Engineering
工程數學
A condensed course containing essential math tools for most engineering disciplines
Partial differential equations, Fourier analysis, Vector calculus and analysis
Used in the fields that need to handle continuous functions
網路分析, 效能分析, 訊號處理, 影像處理, 科學計算, 人工智慧, 電腦視覺, CAD, 電路設計…
12/9/2018
CS Introduction to Information Engineering

38. CS135601 Introduction to Information Engineering
Reference
The Josephus problem:
Graham, Knuth, and Patashnik, “Concrete Mathematics”, section 1.3
Algorithm representation
Textbook: 5.2
The related courses are those listed in the last part of slides, and of course, 演算法設計,高等程式設計實作
12/9/2018
CS Introduction to Information Engineering