S. J. Shyu Chap. 1 Introduction 1 The Design and Analysis of Algorithms Chapter 1 Introduction S. J. Shyu

Chap. 1 Introduction 2 Why do we need to study algorithms?  We study algorithms so that we can learn strategies to design efficient algorithms.  Besides, we study algorithms to understand the difficulty of designing good algorithms for some problems, namely NP-complete problems.

S. J. Shyu Chap. 1 Introduction 3 Consider the sorting problem.  Sorting problem: To sort a set of elements into increasing or decreasing order. 11, 7, 14, 1, 5, 9, 10 ↓sort 1, 5, 7, 9, 10, 11, 14  Insertion sort  Quick sort

S. J. Shyu Chap. 1 Introduction 4 Comparison of these two algorithms implemented on two computers :  A bad algorithm implemented on a fast computer does not perform as well as a good algorithm implemented on a slow computer.  Insertion Sort on IBM SP2  Quick Sort on PC486  When n >400, PC performs better than SP2.

S. J. Shyu Chap. 1 Introduction 5 Insertion Sort on IBM SP2 vs. Quick Sort on PC

S. J. Shyu Chap. 1 Introduction 6  High Speed Computer  High Speed Computation (if no idea about algorithms)  A good knowledge of algorithms makes your computer effective and efficient.

S. J. Shyu Chap. 1 Introduction 7 Analysis of algorithms  Measure the goodness of algorithms  efficiency  asymptotic notations: e.g. O(n 2 )  worst case  average case  amortized  Measure the difficulty of problems  NP-complete  undecidable  lower bound  Is an algorithm optimal?

S. J. Shyu Chap. 1 Introduction 8 Algorithms vs. Problems  How fast can an algorithm be? (in solving a problem) Complexity Theory  How hard can a problem be? (for any problem solving it) Computability Theory

S. J. Shyu Chap. 1 Introduction 9 0/1 Knapsack Problem  Given a set of n items where each item P i has a value V i, weight W i and a limit M of the total weights, we want to select a subset of items such that the total weight does not exceed M and the total value is maximized.

S. J. Shyu Chap. 1 Introduction 10 0/1 Knapsack problem M (weight limit)=14 best solution: P 1, P 2, P 3, P 5 (optimal) = 19 This problem is NP-complete. P1P1 P2P2 P3P3 P4P4 P5P5 P6P6 P7P7 P8P8 Value Weight

S. J. Shyu Chap. 1 Introduction 11 Traveling salesperson problem  Given: A set of n planar points Find: A closed tour which includes all points exactly once such that its total length is minimized.  This problem is NP-complete.

S. J. Shyu Chap. 1 Introduction 12 Partition problem  Given: A set of positive integers S Find: S 1 and S 2 such that S 1  S 2 = , S 1  S 2 = S, and (Partition into S 1 and S 2 such that the sum of S 1 is equal to that of S 2 )  e.g. S={1, 7, 10, 4, 6, 8, 13}  S 1 ={1, 10, 4, 8, 3}  S 2 ={7, 6, 13}  This problem is NP-complete.

S. J. Shyu Chap. 1 Introduction 13  Given: an art gallery Determine: min # of guards and their placements such that the entire art gallery can be monitored.  NP-complete Art gallery problem

S. J. Shyu Chap. 1 Introduction 14 Set Cover Problem  Given: some subsets from a set S Determine: min # of subsets whose union covers S  NP-complete ABCDABCD a b c d e f

S. J. Shyu Chap. 1 Introduction 15 Knapsack problem P 5, P 2, P 1, P 8, W = > 14 (= M ) X /15 V (3/15)  17 = 21.4 P1P1 P2P2 P3P3 P4P4 P5P5 P6P6 P7P7 P8P8 Value Weight V/WV/W 10/75/31/39/103/14/911/2217/15 P5P5 P2P2 P1P1 P8P8 P4P4 P7P7 P6P6 P3P3 Value Weight V/WV/W

S. J. Shyu Chap. 1 Introduction 16 Minimal Spanning Trees  Given a weighted graph G, a spanning tree T is a tree where all vertices of G are vertices of T and if an edge of T connects V i and V j, its weight is the weight of e ( V i, V j ) in G.  A minimal spanning tree of G is a spanning tree of G whose total weight is minimized.

S. J. Shyu Chap. 1 Introduction 17 Minimum Spanning Trees  solved by greedy method  # of possible spanning trees for n points: n n -2  n =10→10 8, n =100→10 196

S. J. Shyu Chap. 1 Introduction 18 Convex hull  Given a set of planar points, find a smallest convex polygon which contains all points.  It is not obvious to find a convex hull by examining all possible solutions  divide-and-conquer

S. J. Shyu Chap. 1 Introduction 19 One-center problem Given a set of planar points, find a smallest circle which contains all points. Prune-and-search

S. J. Shyu Chap. 1 Introduction 20 Easy Problems vs. Difficult Problems NP problems NP-complete problems P problems Sorting, Minimal Spanning Tree, Shortest Path, Convex Hall, Knapsack, Longest Common Subsequence, … Traveling salesperson problem, Gallery Guards, 0/1 Knapsack, Multiple Sequence Alignment, …

S. J. Shyu Chap. 1 Introduction 21  Many strategies, such as the  greedy approach  divide-and-conquer approach  branch and bound  prune and search  dynamic programming  and so on will be introduced in this course.

S. J. Shyu Chap. 1 Introduction 22 Strategies for P problems  Greedy  Dynamic Programming  Divide and Conquer  Prune and Search  …

S. J. Shyu Chap. 1 Introduction 23 Strategies for NP-hard problems  Optimization  Branch and bound  Dynamic programming  Exhausted search  …  Approximation  Simple heuristics with a guaranteed error ratio (e.g. Greedy, Random, …)  Sophisticated heuristics (e.g. Greedy, 2-opt, k-opt, …)  Meta-heuristics