1. CS4335: Design and Analysis of Algorithms
Who we are:
Instructors:
Shuai Cheng LI, Y6634, ,
Lusheng WANG, B6422, ,
Dept. of Computer Science
Course web site:
2019/1/11
CS4335 Design and Analysis of Algorithms /Shuai Cheng Li

2. CS4335 Design and Analysis of Algorithms /Shuai Cheng Li
Text Book:
J. Kleinberg and E. Tardos, Algorithm design, Addison-Wesley, 2005.
We will add more material in the handout.
References:
T. H. Cormen, C. E. Leiserson, R. L. Rivest, Introduction to Algorithms, The MIT Press.
R. Sedgewick, Algorithms in C++, Addison-Wesley, 2002.
A. Levitin, Introduction to the design and analysis of algorithms, Addison-Wesley, 2003.
M.R. Garry and D. S. Johnson, Computers and intractability, a guide to the theory of NP-completeness, W.H. Freeman and company, 1979
2019/1/11
CS4335 Design and Analysis of Algorithms /Shuai Cheng Li

3. CS4335 Design and Analysis of Algorithms /Shuai Cheng Li
2019/1/11
CS4335 Design and Analysis of Algorithms /Shuai Cheng Li

4. CS4335 Design and Analysis of Algorithms /Shuai Cheng Li
Step-by-step procedure for calculation.
Any well-defined computational procedure that takes some value, or set of values, as input and produces some value, or set of values, as output.
A sequence of computational steps that transform the input into output.
A sequence of computational steps for solving a well-specified computational problem.
2019/1/11
CS4335 Design and Analysis of Algorithms /Shuai Cheng Li

5. Example of well-specified problem: Sorting
Input: a sequence of numbers
1, 100, 8, 25, 11, 9, 2, 1, 200.
Output: a sorted (increasing order or decreasing order) sequence of numbers
1, 2, 8, 9, 11, 25, 100, 200.
Another example:
Create web page that contains a list of papers using HTML.
-everybody can do it.
Not need to design computational steps.
2019/1/11
CS4335 Design and Analysis of Algorithms /Shuai Cheng Li

6. CS4335 Design and Analysis of Algorithms /Shuai Cheng LiB A
Find a shortest path from station A to station B.
-need serious thinking to get a correct algorithm.
2019/1/11
CS4335 Design and Analysis of Algorithms /Shuai Cheng Li

7. A Real-Time Driver’s Direction System
Given an electronic map (stored on a computer), the position of your car (provided by GPS), and the destination,
the system can tell you the way to go to the destination.
Tell you tern left or right 40 meters before according to the shortest path.
If you did not follow the direction, re-calculate the shortest path.
US$ 400 each.
2019/1/11
CS4335 Design and Analysis of Algorithms /Shuai Cheng Li

8. Dijkstra’s Algorithm: (Recall)
Dijkstra’s algorithm assumes that w(e)0 for each e in the graph.
maintain a set S of vertices such that
Every vertex v S, d[v]=(s, v), i.e., the shortest-path from s to v has been found. (Intial values: S=empty, d[s]=0 and d[v]=)
(a) select the vertex uV-S such that
d[u]=min {d[x]|x V-S}. Set S=S{u}
(b) for each node v adjacent to u do RELAX(u, v, w).
Repeat step (a) and (b) until S=V.
2019/1/11
CS4335 Design and Analysis of Algorithms /Shuai Cheng Li

9. Descriptions of Algorithms
Flow chart
Pseudo-code
Programs
Natural languages
The purpose:
Allow a well-trained programmer to write a program to solve the computational problem.
Any body who can talk about algorithm MUST have basic programming skills
Also CS3334: Data structures.
2019/1/11
CS4335 Design and Analysis of Algorithms /Shuai Cheng Li

10. CS4335 Design and Analysis of Algorithms /Shuai Cheng Li
What We Cover:
Some classic algorithms in various domains
Graph algorithms
Euler path, shortest path, minimum spanning trees, maximum flow, Hamilton Cycle, traveling salesman,
String Algorithms
Exact string matching, Approximate string matching, Applications in web searching engines
Scheduling problems
Computational Geometry
Techniques for designing efficient algorithms
divide-and-conquer approach, greedy approach, dynamic programming
2019/1/11
CS4335 Design and Analysis of Algorithms /Shuai Cheng Li

11. What We Cover(continued):
Introduction to computational complexity
NP-complete problems
Approximation algorithms
Vertex cover
Steiner trees
Traveling sales man problem
2019/1/11
CS4335 Design and Analysis of Algorithms /Shuai Cheng Li

12. CS4335 Design and Analysis of Algorithms /Shuai Cheng Li
2019/1/11
CS4335 Design and Analysis of Algorithms /Shuai Cheng Li

13. CS4335 Design and Analysis of Algorithms /Shuai Cheng Li
2019/1/11
CS4335 Design and Analysis of Algorithms /Shuai Cheng Li

14. CS4335 Design and Analysis of Algorithms /Shuai Cheng Li
2019/1/11
CS4335 Design and Analysis of Algorithms /Shuai Cheng Li

15. Why You have to take this course
You can apply learned techniques to solve various problems
Have a sense of complexities of various problems in different domains
College graduates vs. University graduates
Supervisor v.s. low level working force
2019/1/11
CS4335 Design and Analysis of Algorithms /Shuai Cheng Li

16. CS4335 Design and Analysis of Algorithms /Shuai Cheng Li
A joke:
The boss wants to produce programs to solve the following two problems
Euler circuit problem:
given a graph G, find a way to go through each edge exactly once.
Hamilton circuit problem:
given a graph G, find a way to go through each vertex exactly once.
The two problems seem to be very similar.
Person A takes the first problem and person B takes the second.
Outcome: Person A quickly completes the program, whereas person B works 24 hours per day and is fired after a few months.
2019/1/11
CS4335 Design and Analysis of Algorithms /Shuai Cheng Li

17. Euler Circuit: The original Konigsberg bridge
2019/1/11
CS4335 Design and Analysis of Algorithms /Shuai Cheng Li

18. CS4335 Design and Analysis of Algorithms /Shuai Cheng Li
Hamilton Circuit
2019/1/11
CS4335 Design and Analysis of Algorithms /Shuai Cheng Li

19. CS4335 Design and Analysis of Algorithms /Shuai Cheng Li
A joke (continued):
Why? no body in the company has taken CS4335.
Explanation:
Euler circuit problem can be easily solved in polynomial time.
Hamilton circuit problem is proved to be NP-hard.
So far, no body in the world can give a polynomial time algorithm for a NP-hard problem.
Conjecture: there does not exist polynomial time algorithm for this problem.
2019/1/11
CS4335 Design and Analysis of Algorithms /Shuai Cheng Li

20. Evaluation of the Course
Course work: 30%
Four assignments (25%)
5 points for each of the first three assignments
10 points for the last assignment
One term paper (5%)
Find an open problem from internet.
State the problem definition in English.
Write the definition mathematically.
Summarize the current status
No more than 1 page
A final exam: 70%
2019/1/11
CS4335 Design and Analysis of Algorithms /Shuai Cheng Li

21. Evaluation of the Course (OBTL)
CILOs .
prove the correctness and analyze the running time of   the basic algorithms for those classic problems in various domains; 
apply the algorithms and design techniques to solve problems; 
analyze the complexities of various problems in different domains.
Develop an attitude and an ability to propose new solutions for problems through independent study. 
For CILO 1- CILO 4, we have assignments to evaluate.
You have to pass all CILOs in order to pass the course.
2019/1/11
CS4335 Design and Analysis of Algorithms /Shuai Cheng Li

22. CS4335 Design and Analysis of Algorithms /Shuai Cheng Li
How to Teach
Contents are divided into four classes
1. Basic part -- every body must understand in order to pass
2. Moderate part -- most of students should understand.
Aim at B or above.
3. Hard part -- used to distinguish students.
Aim at A or above.
4. Fun part -- just illustrate that some interesting things
can be done if one works very hard.
-- useful knowledge that will not be tested.
-- I will give some challenging problems for fun.
2019/1/11
CS4335 Design and Analysis of Algorithms /Shuai Cheng Li

23. CS4335 Design and Analysis of Algorithms /Shuai Cheng Li
Challenging Problems
For those who have extra energy, we have challenging problems.
Challenging problems will be distributed at the end of some lectures.
No mark will be given for those challenge problems.
I will record who completely solved the problem. (The records will be used to decide the boundary cases.)
Good Training for solving new problems
Have high chance to solve the hard problems in the final exam.
2019/1/11
CS4335 Design and Analysis of Algorithms /Shuai Cheng Li

24. CS4335 Design and Analysis of Algorithms /Shuai Cheng Li
How to Learn
1. Attend every lecture (2 hours per week) and
tutorial (1 hour per week)
Assignment: Each student has a different version.
2. Try to go with me when I am talking
3. Ask questions immediately
4. Try to fix all problems during the 1 hour tutorial
5. Ask others.
2019/1/11
CS4335 Design and Analysis of Algorithms /Shuai Cheng Li

25. The Process of Design an Algorithm
Formulating the problem
with enough mathematical precision
we can ask a concrete question
start to solve it.
Design the algorithm
list the “precise” steps. (an expert can translate the algorithm into a computer program.)
Analyze the algorithm
prove that it is correct
establish the efficiency
the running time or sometimes space
2019/1/11
CS4335 Design and Analysis of Algorithms /Shuai Cheng Li

26. CS4335 Design and Analysis of Algorithms /Shuai Cheng Li
Terminologies
A Graph G=(V,E): V: set of vertices and E: set of edges.  
Path in G: sequence v1, v2, ..., vk of vertices in V such that (vi, vi+1) is in E.
vi and vj could be the same
Circuit: A path v1, v2, ..., vk such that v1 = vk .
Degree of a vertex: number of edges incident to the vertex.
2019/1/11
CS4335 Design and Analysis of Algorithms /Shuai Cheng Li

27. CS4335 Design and Analysis of Algorithms /Shuai Cheng Li
Euler circuit
Input: a connected graph G=(V, E)
Problem: is there a circuit in G that uses each edge exactly once?
Note: G can have multiple edges, .i.e., two or more edges connect vertices u and v.
2019/1/11
CS4335 Design and Analysis of Algorithms /Shuai Cheng Li

28. The Bridges of Koenigsberg: Euler 1736A 1 2 3 B 4 C 5 6 7 D
Is it possible to start in A, cross over each bridge exactly once, and end up back in A?
2019/1/11
CS4335 Design and Analysis of Algorithms /Shuai Cheng Li

29. The Bridges of Koenigsberg: Euler 1736A 1 2 3 B 4 C 5 6 7 D
Conceptualization: Land masses are “vertices”.
2019/1/11
CS4335 Design and Analysis of Algorithms /Shuai Cheng Li

30. The Bridges of Koenigsberg: Euler 1736A 1 2 3 B 4 C 5 6 7 D
Conceptualization: Bridges are “edges”.
2019/1/11
CS4335 Design and Analysis of Algorithms /Shuai Cheng Li

31. The Bridges of Koenigsberg: Euler 1736A 1 2 3 B 4 C 5 6 7 D
Is there a “walk” starting at A and ending at A and passing through each arc exactly once?
2019/1/11
CS4335 Design and Analysis of Algorithms /Shuai Cheng Li

32. The Bridges of Koenigsberg: Euler 1736A 1 2 3 B 4 C 5 6 7 D
Is there a “walk” starting at A and ending at A and passing through each arc exactly once? Such a walk is called an Euler circuit.
2019/1/11
CS4335 Design and Analysis of Algorithms /Shuai Cheng Li

33. Adding two bridges creates such a walk3 8 1 2 B 4 C 5 6 9 D 7
Here is the walk.
A, 1, B, 5, D, 6, B, 4, C, 8, A, 3, C, 7, D, 9, B, 2, A
Note: the number of edges incident to B is twice the number of times that B appears on the walk.
2019/1/11
CS4335 Design and Analysis of Algorithms /Shuai Cheng Li

34. CS4335 Design and Analysis of Algorithms /Shuai Cheng Li
On Euler Circuit 4 1 2 4 3 7 6 5 A C D B 8 9
The degree of a vertex in an undirected graph is the number of incident arcs 6 4 4
Theorem. An undirected graph has an eulerian circuit if and only if (1) every node degree is even and (2) the graph is connected (that is, there is a path from each vertex to each other vertex).
2019/1/11
CS4335 Design and Analysis of Algorithms /Shuai Cheng Li

35. CS4335 Design and Analysis of Algorithms /Shuai Cheng Li
Euler Circuits
The name of the game is to trace each drawing
without lifting the pencil or retracing any of the lines.
These kinds of tracings are called unicursal tracings.
2019/1/11
CS4335 Design and Analysis of Algorithms /Shuai Cheng Li

36. CS4335 Design and Analysis of Algorithms /Shuai Cheng Li
Euler Circuits
When we end in the same place we started, we call it
a closed unicursal tracing; when we start and end in
different places, we call it an open unicursal tracing. .
2019/1/11
CS4335 Design and Analysis of Algorithms /Shuai Cheng Li

37. Which of the following graphs has Euler Circuit?
2019/1/11
CS4335 Design and Analysis of Algorithms /Shuai Cheng Li

38. How to solve it? An Example
Start with the trivial circuit (1). Then the greedy algorithm yields the partial circuit (1,2,4,3,1).
2019/1/11
CS4335 Design and Analysis of Algorithms /Shuai Cheng Li

39. Remove Edges and Continue
Start with the partial circuit (1,2,4,3,1). First vertex incident with an edge remaining is A greedy approach yields (2,5,8,2). Expanding, we get the new partial circuit (1,2,5,8,2,4,3,1)
2019/1/11
CS4335 Design and Analysis of Algorithms /Shuai Cheng Li

40. Remove Edges and Continue
Start with the partial circuit (1,2,5,8,2,4,3,1). First vertex incident with an edge remaining is A greedy approach yields (4,6,7,4,9,6,10,4). Expanding, we get the new partial circuit (1,2,5,8,2,4,6,7,4,9,6,10,4,3,1)
2019/1/11
CS4335 Design and Analysis of Algorithms /Shuai Cheng Li

41. Remove Edges and Continue
Start with the partial circuit (1,2,5,8,2,4,6,7,4,9,6,10,4,3,1) First vertex incident with an edge remaining is A greedy approach yields (7,9,11,7). Expanding, we get the new partial circuit (1,2,5,8,2,4,6,7,9,11,7,4,9,6,10,4,3,1). This exhausts the edges and we have an euler circuit.
2019/1/11
CS4335 Design and Analysis of Algorithms /Shuai Cheng Li

42. Theorem for Euler circuit
Theorem 1 (Euler’s Theorem) A connected graph has an Euler circuit if and only if all the vertices in the graph have even degrees.
Proof:
Necessity of two conditions:
Any Euler circuit “visits” each node an even number of times
Any Euler circuit shows the network is connected
caveat: nodes of degree 0
2019/1/11
CS4335 Design and Analysis of Algorithms /Shuai Cheng Li

43. CS4335 Design and Analysis of Algorithms /Shuai Cheng Li
Sufficiency of the condition: if the degree of every node is even, then there is an Euler circuit.
Prove by providing an algorithm:
Since each node v has even degree, when we first enter v, there is an unused edge that can be used to get out v.
The only exception is when v is a starting node.
Then we get a circuit (may not contain all edges in G)
If every node in the circuit has no unused edge, all the edges in G have been used since G is connected.
Otherwise, we can construct another circuit, merge the two circuits and get a larger circuit.
In this way, every edge in G can be used.  
2019/1/11
CS4335 Design and Analysis of Algorithms /Shuai Cheng Li

44. An efficient algorithm for Euler circuit
1. Starting with any vertex u in G, take an unused
edge (u,v) (if there is any) incident to u
2. Do Step 1 for v and continue the process until v has no unused edge. (a circuit C is obtained)
 3. If every node in C has no unused edge, stop.
 4. Otherwise, select a vertex, say, u in C, with some
unused edge incident to u and do Steps 1 and 2 until another circuit is obtained.
5. Merge the two circuits obtained to form one circuit
 6. Goto Step 3.
2019/1/11
CS4335 Design and Analysis of Algorithms /Shuai Cheng Li

45. CS4335 Design and Analysis of Algorithms /Shuai Cheng Li
C1:abca C2:bdefb a b e 1 f a b f d b e 4 5 c 3 2 7 6 c g i
merge h j
C3:abdefbca. f d b e 2 3 1 a 4 5 7 6
C5: abdeijhcgefbca c
C4:eijhcge
merge d 2 1 3 e 11 a b 10 e 13 f 13 12 c 12 c 8 4 9 g 8 11 g i 7 10 i h j 9 h j 6 5
2019/1/11
CS4335 Design and Analysis of Algorithms /Shuai Cheng Li

46. Representations of Graphs (review)
Two standard ways
Adjacency-list representation
Space required O(|E|)
Adjacency-matrix representation 
Space required O(n2).
Depending on problems, both representations are useful.
2019/1/11
CS4335 Design and Analysis of Algorithms /Shuai Cheng Li

47. Adjacency-list representation
Let G=(V, E) be a graph.
V– set of nodes (vertices)
E– set of edges.
For each uV, the adjacency list Adj[u] contains all nodes in V that are adjacent to u. 2 5 / 1 2 1 5 4 3 1 5 3 4 / 2 2 4 / 3 3 / 2 5 4 4 5 1 2 /
(a)
(b)
2019/1/11
CS4335 Design and Analysis of Algorithms /Shuai Cheng Li

48. Adjacency-list representation for directed graph2 4 / 1 5 / 6 2 1 4 5 3 2 6 5 / 3 2 / 4 4 / 5 6 / 6
(a)
(b)
2019/1/11
CS4335 Design and Analysis of Algorithms /Shuai Cheng Li

49. Adjacency-matrix representation
Assume that the nodes are numbered 1, 2, …, n.
The adjacency-matrix consists of a |V||V| matrix A=(aij) such that
aij= 1 if (i,j) E, otherwise aij= 0. 2 1 5 4 3
(a)
(c)
2019/1/11
CS4335 Design and Analysis of Algorithms /Shuai Cheng Li

50. CS4335 Design and Analysis of Algorithms /Shuai Cheng Li
Adjacency-matrix representation for directed graph It is NOT symmetric.
(c) 6 2 1 4 5 3
(a)
2019/1/11
CS4335 Design and Analysis of Algorithms /Shuai Cheng Li

51. Implementation of Euler circuit algorithm (Not required)
Data structures:
Adjacency-list representation
Each node in V has an adjacency list
Also, we have two lists to store the circuits
One for the circuit produced in Steps 1-2.
One for the circuit produced in Step 4
In Step 1: when we take an unused edge (u, v), this edge is deleted from the adjacency-list of node u.
2019/1/11
CS4335 Design and Analysis of Algorithms /Shuai Cheng Li

52. Implementation of Euler circuit algorithm
In Step 2: if the adjacency list of v is empty, v has no unused edge.
Testing whether adjacency-list v is empty.
A circuit (may not contain all edges) is obtained if the above condition is true.
If the adjacency-list for v is empty, DELETE the list.
In Step 3: if all the adjacency-list is empty, stop. (Note we did 3).
In Step 4: if some adjacency-list is not empty, use it in step 4.
2019/1/11
CS4335 Design and Analysis of Algorithms /Shuai Cheng Li

53. C1:a-b-c-a, b a d c e h f j i g d a b e f c g i h j
Figure1: The adjacency-list representation of Example 1(Slide 45)
2019/1/11
CS4335 Design and Analysis of Algorithms /Shuai Cheng Li

54. CS4335 Design and Analysis of Algorithms /Shuai Cheng Lib e a f d b f c c g g h i d b e h j e d f g i f b e
C1:a-b-c-a, C2: bdefb,
=>C3:abdefbca g c e h c g i e j j h i
After edge (a,b), (b, c) and (c, d) are used.
2019/1/11
CS4335 Design and Analysis of Algorithms /Shuai Cheng Li

55. CS4335 Design and Analysis of Algorithms /Shuai Cheng Lib c c g g h i d h j e g i
C1:abca, C2:bdefb, =>C3:abdefbca. C4:eijhcge. C3+C4=>abdeijhcgefbca f g c e h c j i e j j h i
After edges (b, d) (d, e), (e, f), (f, b) are used.
2019/1/11
CS4335 Design and Analysis of Algorithms /Shuai Cheng Li

56. Time complexity (Fun Part)
If it takes O(|V|) time to add an edge to a circuit, then the time complexity is O(|E||V|).
This can be done by using an adjacency list.
The implementation is tricky
Even O(|V||E|) algorithm is not trivial.
Do not be disappointment if you do not completely understand the implementation
The best algorithm takes O(|E|) time.
Euler Circuit: A complete example for Designing Algorithms.
-formulation, design algorithm, analysis.
2019/1/11
CS4335 Design and Analysis of Algorithms /Shuai Cheng Li

57. CS4335 Design and Analysis of Algorithms /Shuai Cheng Li
Euler Path
A path which contains all edges in a graph G is called an Euler path of G.
Deleting an edge in a graph having an Euler Circuit, we obtain an Euler path in the new graph.
Corollary: A graph G=(V,E) which has an Euler path has 2 vertices of odd degree.
Deleting the edge
2019/1/11
CS4335 Design and Analysis of Algorithms /Shuai Cheng Li

58. CS4335 Design and Analysis of Algorithms /Shuai Cheng Li
Proof of the Corollary
 Suppose that a graph which has an Euler path starting at u and ending at v, where uv.
 Creating a new edge e joining u and v, we have an Euler circuit for the new graph G’=(V, E{e}).
 From Theorem 1, all the vertices in G’ have even
degree. Remove e.
 Then u and v are the only vertices of odd degree in G.
(Nice argument, not required for exam.) 
2019/1/11
CS4335 Design and Analysis of Algorithms /Shuai Cheng Li

59. CS4335 Design and Analysis of Algorithms /Shuai Cheng Li
Application 1:
In a map, two countries may share a common border. Given a map, we want to know if it is possible to find a tour starting from a country, going across each shared border once and come back to the starting country. A B A B D C D C
2019/1/11
CS4335 Design and Analysis of Algorithms /Shuai Cheng Li

60. CS4335 Design and Analysis of Algorithms /Shuai Cheng Li
Application 2:
Formulating a graph problem: Given a map containing Guizhou, Hunan, Jiangxi, Fujian, Guangxi, Guangdong, construct a graph such that if two provinces share some common boarder, then there is an edge connecting the two corresponding vertices.
If we want to find a tour to visit each province once, then we need to find a Hamilton circuit in the graph.
2019/1/11
CS4335 Design and Analysis of Algorithms /Shuai Cheng Li

61. Challenge Problem : (only for those who are interested)
Prove that given a graph G=(V, E), one can always add some edges to the graph such that the new graph (multiple edges are allowed) has an Euler circuit.
Design an algorithm to add minimum number of edges to an input graph such that the resulting graph has an Euler circuit.
Prove the correctness of your algorithm.
2019/1/11
CS4335 Design and Analysis of Algorithms /Shuai Cheng Li

62. Summary of Euler Circuit Algorithm
Design a good algorithm needs two parts
Theorem, high level part
Implementation: low level part. Data structures are important.
Our course emphasizes the first part and demonstrates the second part whenever possible.
We will not emphasize too much about data structures.
2019/1/11
CS4335 Design and Analysis of Algorithms /Shuai Cheng Li