1. CS3335: Design and Analysis of Algorithms
Who we are:
Dr. Lusheng WANG
Dept. of Computer Science
office: Y6429
phone:
web site:
Professor Frances Yao (Dept. Head) will do tutorial for one group.
2019/4/12
CS3335 Design and Analysis of Algorithms /WANG Lusheng

2. CS3335 Design and Analysis of Algorithms /WANG Lusheng
Text Book:
J. Kleinberg and E. Tardos, Algorithm design, Addison-Wesley, 2005.
We will add more material in the handout.
References:
T. H. Cormen, C. E. Leiserson, R. L. Rivest, Introduction to Algorithms, The MIT Press.
R. Sedgewick, Algorithms in C++, Addison-Wesley, 2002.
A. Levitin, Introduction to the design and analysis of algorithms, Addison-Wesley, 2003.
M.R. Garry and D. S. Johnson, Computers and intractability, a guide to the theory of NP-completeness, W.H. Freeman and company, 1979
2019/4/12
CS3335 Design and Analysis of Algorithms /WANG Lusheng

3. CS3335 Design and Analysis of Algorithms /WANG Lusheng
Any well-defined computational procedure that takes some value, or set of values, as input and produces some value, or set of values, as output.
A sequence of computational steps that transform the input into output.
A sequence of computational steps for solving a well-specified computational problem.
2019/4/12
CS3335 Design and Analysis of Algorithms /WANG Lusheng

4. Example of well-specified problem: Sorting
Input: a sequence of numbers: 1, 100, 8, 25, 11, 9, 2, 1, 200.
Output: a sorted (increasing order or decreasing order) sequence of numbers
1, 2, 8, 9, 11, 25, 100, 200.
2019/4/12
CS3335 Design and Analysis of Algorithms /WANG Lusheng

5. Descriptions of Algorithms
Flow chart
Pseudo-code
Programs
Natural languages
The purpose:
Allow a well-trained programmer to write a program to solve the computational problem.
Any body who can talk about algorithm MUST have basic programming skills
2019/4/12
CS3335 Design and Analysis of Algorithms /WANG Lusheng

6. CS3335 Design and Analysis of Algorithms /WANG Lusheng
What We Cover:
Some classic algorithms in various domains
Graph algorithms
Euler path, shortest path, minimum spanning trees, maximum flow, Hamilton Cycle, traveling salesman,
String Algorithms
Exact string matching, Approximate string matching, Applications in web searching engines
Scheduling problems
Computational Geometry
Techniques for designing efficient algorithms
divide-and-conquer approach, greedy approach, dynamic programming
2019/4/12
CS3335 Design and Analysis of Algorithms /WANG Lusheng

7. What We Cover(continued):
Introduction to computational complexity
NP-complete problems
Approximation algorithms
Vertex cover
Steiner trees
Traveling sales man problem
2019/4/12
CS3335 Design and Analysis of Algorithms /WANG Lusheng

8. Why You have to take this course
You can apply learned techniques to solve various problems
Have a sense of complexities of various problems in different domains
College graduates vs. University graduates
Supervisor vs. low level working force
2019/4/12
CS3335 Design and Analysis of Algorithms /WANG Lusheng

9. CS3335 Design and Analysis of Algorithms /WANG Lusheng
A joke:
The boss wants to produce programs to solve the following two problems
Euler circuit problem:
given a graph G, find a way to go through each edge exactly once.
Hamilton circuit problem:
given a graph G, find a way to go through each vertex exactly once.
The two problems seem to be very similar.
Person A takes the first problem and person B takes the second.
Outcome: Person A quickly completes the program, whereas person B works 24 hours per day and is fired after a few months.
2019/4/12
CS3335 Design and Analysis of Algorithms /WANG Lusheng

10. Euler Circuit and Hamilton Circuit
2019/4/12
CS3335 Design and Analysis of Algorithms /WANG Lusheng

11. CS3335 Design and Analysis of Algorithms /WANG Lusheng
A joke (continued):
Why? no body in the company has taken CS3335.
Explanation:
Euler circuit problem can be easily solve in polynomial time.
Hamilton circuit problem is proved to be NP-hard.
So far, no body in the world can give a polynomial time algorithm for a NP-hard problem.
Conjecture: there does not exist polynomial time algorithm for this problem.
2019/4/12
CS3335 Design and Analysis of Algorithms /WANG Lusheng

12. Evaluation of the Course
Course work: 30%
Four assignments
6 points for each of the first three assignments
12 points for the last assignment
Working load is Not heavy
A final exam: 70%;
2019/4/12
CS3335 Design and Analysis of Algorithms /WANG Lusheng

13. CS3335 Design and Analysis of Algorithms /WANG Lusheng
How to Teach
Contents are divided into four classes
1. Basic part -- every body must understand in order to pass
2. Moderate part -- most of students should understand.
Aim at B or above.
3. Hard part -- used to distinguish students.
Aim at A or above.
4. Fun part -- just illustrate that some interesting things
can be done if one works very hard.
-- useful knowledge that will not be tested.
-- I will give some challenging problems for fun.
2019/4/12
CS3335 Design and Analysis of Algorithms /WANG Lusheng

14. CS3335 Design and Analysis of Algorithms /WANG Lusheng
Challenging Problems
For those who have extra energy, we have challenging problems.
Challenging problems will be distributed at the end of some lectures.
No mark will be given for those challenge problems.
I will record who completely solved the problem. (The records will be used to decide the boundary cases.)
2019/4/12
CS3335 Design and Analysis of Algorithms /WANG Lusheng

15. CS3335 Design and Analysis of Algorithms /WANG Lusheng
How to Learn
1. Attend every lecture (2 hours per week) and
tutorial (1 hour per week)
2. Try to go with me when I am talking
3. Ask questions immediately
4. Try to fix all problems during the 1 hour tutorial
5. Ask others.
2019/4/12
CS3335 Design and Analysis of Algorithms /WANG Lusheng

16. CS3335 Design and Analysis of Algorithms /WANG Lusheng
Terminologies
A Graph G=(V,E): V---set of vertices and E--set of edges.  
Path in G: sequence v1, v2, ..., vk of vertices in V such that (vi, vi+1) is in E.
vi and vj could be the same
Simple path in G: a sequence v1, v2, ..., vk of distinct vertices in V such that (vi, vi+1) is in E.
vi and vj can not be the same
2019/4/12
CS3335 Design and Analysis of Algorithms /WANG Lusheng

17. CS3335 Design and Analysis of Algorithms /WANG Lusheng
Example:
Simple path
A path, but not simple
2019/4/12
CS3335 Design and Analysis of Algorithms /WANG Lusheng

18. Terminologies (continued)
Circuit: A path v1, v2, ..., vk such that v1 = vk .
Simple circuit: a circuit v1, v2, ..., vk,where v1=vk and vivj for any 1<i, j<k.
2019/4/12
CS3335 Design and Analysis of Algorithms /WANG Lusheng

19. CS3335 Design and Analysis of Algorithms /WANG Lusheng
Euler circuit
Input: a graph G=(V, E)
Problem: is there a circuit in G that uses each edge exactly once.
Note: G can have multiple edges, .i.e., two or more edges connect vertices u and v.
2019/4/12
CS3335 Design and Analysis of Algorithms /WANG Lusheng

20. CS3335 Design and Analysis of Algorithms /WANG Lusheng
Story:
The problem is called Konigsberg bridge problem
it asks if it is possible to take a walk in the town shown in Figure 1 (a) crossing each bridge exactly once and returning home.
solved by Leonhard Euler [pronounced OIL-er] (1736)
The first problem solved by using graph theory
A graph is constructed to describe the town.
(See Figure 1 (b).)
2019/4/12
CS3335 Design and Analysis of Algorithms /WANG Lusheng

21. The original Konigsberg bridge (Figure 1)
2019/4/12
CS3335 Design and Analysis of Algorithms /WANG Lusheng

22. CS3335 Design and Analysis of Algorithms /WANG Lusheng
Hamilton circuit
Hamilton circuit : a circuit uses every vertex of the graph exactly once except for the last vertex, which duplicates the first vertex.
The Irish mathematician Sir William Hamilton was the first to study this problem.
It is NP-hard to decide if a graph has a Hamilton circuit.
2019/4/12
CS3335 Design and Analysis of Algorithms /WANG Lusheng

23. Figure for Hamilton Circuit
2019/4/12
CS3335 Design and Analysis of Algorithms /WANG Lusheng

24. Traveling salesman problem(TSP)
Input: V={v1, v2, ..., vn} be a set of cities and
d(vi, vj) the distance between vi and vj.
Problem: find a shortest circuit that visits each city exactly once.
(weighted version of Hamilton circuit)
Assumption:
there is an edge between any pair of nodes and the graph is a complete graph.
For those edges not existing, assign a big cost, e.g., .
The problem is NP-hard.
2019/4/12
CS3335 Design and Analysis of Algorithms /WANG Lusheng

25. Minimum spanning trees
Input: a connected weighted graph (each edge e has a weight w(e))
Problem: find a connected subgraph using all vertices and having minimal total weight.
the total weight of a subgraph is the sum of the weights of its edges
Such a subgraph must be a tree, i.e., there is no circle
can be solved using a gready algorithm
2019/4/12
CS3335 Design and Analysis of Algorithms /WANG Lusheng

26. Application of MST: a example
In the design of electronic circuitry, it is often necessary to make a set of pins electrically equivalent by wiring them together.
To interconnect n pins, we can use n-1 wires, each connecting two pins.
We want to minimize the total length of the wires.
Minimun Spanning Trees can be used to model this problem.
2019/4/12
CS3335 Design and Analysis of Algorithms /WANG Lusheng

27. Theorem: A minimum spanning tree is also a bottleneck spanning tree.d e f g i 4 8 7 9 10 14 2 6 1 11
Bottleneck spanning tree: A spanning tree of G whose largest edge weight is minimum over all spanning trees of G. The value of the bottleneck spanning tree is the weight of the maximum-weight edge in T.
Theorem: A minimum spanning tree is also a bottleneck spanning tree.
2019/4/12
CS3335 Design and Analysis of Algorithms /WANG Lusheng

28. CS3335 Design and Analysis of Algorithms /WANG Lusheng
Steiner trees
Input: a graph G=(V,E), a weight w(e) for each eE, and a subset R V.
Problem: find a a tree with minimum weight which contains all the nodes in R.
nodes in R --- regular points (must be in tree)
the tree could contain some nodes in V-R
the nodes in V-R --- Steiner points.
When R = V, the problem becomes minimum
spanning tree problem.
2019/4/12
CS3335 Design and Analysis of Algorithms /WANG Lusheng

29. Example (Steiner trees)
Let G be shown in the following figure. R={a,b,c}. The Steiner minimum tree T={(a,d), (b,d), (c,d)} which is shown in Figure 3. 2 a c b d 1
2019/4/12
CS3335 Design and Analysis of Algorithms /WANG Lusheng

30. Theorem for Euler circuit
Theorem 1 (Euler’s Theorem) The graph has an Euler circuit if and only if all the vertices of a connected graph have even degree.
Proof: (if)
Going through the circuit, each time a vertex is visited, the degree is increased by 2. Thus, the degree of each vertex is even.
2019/4/12
CS3335 Design and Analysis of Algorithms /WANG Lusheng

31. Proof of Theorem 1: (only if)
We give way to find an Euler circuit for a graph in which every vertex has an even degree.
 Since each node v has even degree, when we first enter v, there is an unused edge that can be used to get out v.
 The only exception is when v is a starting node.
 Then we get a circuit (may not contain all edges in G)
 If every node in the circuit has no unused edge, all the
edges in G have been used since G is connected.
 Otherwise, we can construct another circuit, merge
the two circuits and get a larger circuit.
 In this way, every edge in G can be used.  
2019/4/12
CS3335 Design and Analysis of Algorithms /WANG Lusheng

32. An example for Theorem 1:
after merge f a d b g h j i c e 1 4 3 2 8 7 6 5 10 9 12 11 13
2019/4/12
CS3335 Design and Analysis of Algorithms /WANG Lusheng

33. An efficient algorithm for Euler circuit
1. Starting with any vertex u in G, take an unused
edge (u,v) (if there is any) incident to u
2. Do Step 1 for v and continue the process until v has no unused edge. (a circuit C is obtained)
 3. If every node in C has no unused edge, stop.
 4. Otherwise, select a vertex, say, u in C, with some
unused edge incident to u and do Steps 1 and 2 until another circuit is obtained.
5. Merge the two circuits obtained to form one circuit
 6. Continue the above process until every edge in G is
used.
2019/4/12
CS3335 Design and Analysis of Algorithms /WANG Lusheng

34. CS3335 Design and Analysis of Algorithms /WANG Lusheng
Euler Path
 A path which contains all edges in a graph G is called an Euler path of G.
Corollary: A graph G=(V,E) which has an Euler path has 2 vertices of odd degree.
2019/4/12
CS3335 Design and Analysis of Algorithms /WANG Lusheng

35. CS3335 Design and Analysis of Algorithms /WANG Lusheng
Proof of the Corollary
 Suppose that a graph which has an Euler path starting at u and ending at v, where uv.
 Creating a new edge e joining u and v, we have an Euler circuit for the new graph G’=(V, E{e}).
 From Theorem 1, all the vertices in G’ have even
degree. Remove e.
 Then u and v are the only vertices of odd degree in G.
(Nice argument, not required for exam.) 
2019/4/12
CS3335 Design and Analysis of Algorithms /WANG Lusheng

36. Representations of Graphs
Two standard ways
Adjacency-list representation
Space required O(|E|)
Adjacency-matrix representation 
Space required O(n2).
Depending on problems, both representations are useful.
2019/4/12
CS3335 Design and Analysis of Algorithms /WANG Lusheng

37. Adjacency-list representation
Let G=(V, E) be a graph.
V– set of nodes (vertices)
E– set of edges.
For each uV, the adjacency list Adj[u] contains all nodes in V that are adjacent to u. 2 1 5 4 3 /
(a)
(b)
2019/4/12
CS3335 Design and Analysis of Algorithms /WANG Lusheng

38. Adjacency-list representation for directed graph1 2 3 4 5 6 /
(b)
(a)
2019/4/12
CS3335 Design and Analysis of Algorithms /WANG Lusheng

39. Adjacency-matrix representation
Assume that the nodes are numbered 1, 2, …, n.
The adjacency-matrix consists of a |V||V| matrix A=(aij) such that
aij= 1 if (i,j) E, otherwise aij= 0. 2 1 5 4 3
(a)
(c)
2019/4/12
CS3335 Design and Analysis of Algorithms /WANG Lusheng

40. CS3335 Design and Analysis of Algorithms /WANG Lusheng
Adjacency-matrix representation for directed graph It is NOT symmetric.
(c) 6 2 1 4 5 3
(a)
2019/4/12
CS3335 Design and Analysis of Algorithms /WANG Lusheng

41. Implementation of Euler circuit algorithm (Not required)
Data structures:
Adjacency-list representation
Each node in V has an adjacency list
Also, we have two lists to store the circuits
One for the circuit produced in Steps 1-2.
One for the circuit produced in Step 4
In Step 1: when we take an unused edge (u, v), this edge is deleted from the adjacency-list of node u.
2019/4/12
CS3335 Design and Analysis of Algorithms /WANG Lusheng

42. Implementation of Euler circuit algorithm
In Step 2: if the adjacency list of v is empty, v has no unused edge.
Testing whether adjacency-list v is empty.
A circuit (may not contain all edges) is obtained if the above condition is true.
If the adjacency-list for v is empty, DELETE the list.
In Step 3: if all the adjacency-list is empty, stop. (Note we did 3).
In step 4: if some adjacency-list is not empty, use it in step 4.
2019/4/12
CS3335 Design and Analysis of Algorithms /WANG Lusheng

43. b a g d f c j e h i Figure1: The adjacency-list representation of (a)
2019/4/12
CS3335 Design and Analysis of Algorithms /WANG Lusheng

44. CS3335 Design and Analysis of Algorithms /WANG Lushengb f c g h d b e e d f g i f b e g c e h c g i e j j h i
2019/4/12
CS3335 Design and Analysis of Algorithms /WANG Lusheng

45. CS3335 Design and Analysis of Algorithms /WANG Lushengb c g h d e g i f g c e h c j i e j j h i
2019/4/12
CS3335 Design and Analysis of Algorithms /WANG Lusheng

46. CS3335 Design and Analysis of Algorithms /WANG Lusheng
Time complexity
If it takes O(|V|) time to add an edge to a circuit, then the time complexity is O(|E||V|).
This can be done by using an adjacency list.
The implementation is tricky
Even O(|V||E|) algorithm is not trivial.
Do not be disappointment if you do not completely understand the implementation
The best algorithm takes O(|E|) time.
2019/4/12
CS3335 Design and Analysis of Algorithms /WANG Lusheng

47. Challenging Problem 1 (Bonus: 2 points in the final examination)
Prove that given a graph G=(V, E), one can always add some edges to the graph such that the new graph (multiple edges are allowed) has an Euler circuit.
Design an algorithm to add minimum number of edges to an input graph such that the resulting graph has an Euler circuit.
Prove the correctness of your algorithm.
The problems MUST be solved on or before Sept 11 at 12:00pm. After the deadline, no one can get any bonus. Submit your solution to (TA) with subject “Challenge Problem 1”. The submission time is based on the I receive.
The first 10 students who correctly solve the problems will get full marks for the bonus.
The next 20 students who correctly solve the problem can get half of the bonus.
No. 31 and after cannot get any bonus.
Theoretically, you can still get A+ even if you do not work on ANY of the challenging problem.
2019/4/12
CS3335 Design and Analysis of Algorithms /WANG Lusheng

48. Summary of Euler circuit algorithm
Design a good algorithm needs two parts
Theorem, high level part
Implementation: low level part. Data structures are important.
Our course emphasizes the first part and demonstrates the second part whenever possible.
We will not emphasize too much about data structures.
2019/4/12
CS3335 Design and Analysis of Algorithms /WANG Lusheng