Anany Levitin ACM SIGCSE 1999SIG

Outline Introduction Four General Design Techniques A Test of Generality Further Refinements Conclusion

Introduction-1 According to many textbooks, a consensus seems to have evolved as to which approaches qualify as major techniques for designing algorithms. This is includes: divide-and-conquer, greedy approach, dynamic programming, backtracking, and branch-and- bound.

Introduction-2 However, this widely accepted taxonomy has serious shortcoming. First, it includes techniques of different levels of generality. For example, it seems obvious that divide- and-conquer is more general than greedy approach and branch-and-bound. Second, it fails to distinguish divide-and-conquer and decrease-and-conquer. Third, it fails to include brute force and transform-and- conquer.

Introduction-3 Fourth, its linear, as opposed to hierarchical, structure fails to reflect important special cases of techniques. Finally, it fails to classify many classical algorithms (e.g., Euclid’s algorithm, heapsort, search trees, hashing, etc.) This paper seeks to rectify these shortcomings and presents new taxonomy.

Four General Design Techniques-1 Brute Force It usually based on the problem’s statement and definitions of concepts involved This technique can not be overlooked by the following reasons Applicable to a very wide variety of problems, e.g., computing the sum of n numbers, adding two matrices... Useful for solving small-size instances of a problem Serving an important theoretical or educational purpose, e.g., NP-hard problem, as a yardstick for more efficient alternatives for solving a problem

Four General Design Techniques-2 Divide-and-conquer It based on partitioning a problem into a number of smaller subproblems, usually of the same kind and ideally of about the same size The subproblems are then solved and their solutions combined to get a solution to the original problem. Divide-before-processing: the bulk of the work is done while combining solutions to smaller subproblems, e.g., mergesort. Process-before-dividing: processing before a partition into subproblems, e.g., quicksort.

Four General Design Techniques-3 Decrease-and-conquer This technique is solving a problem by reducing its instance to a smaller one, solving the latter, and then extending the obtained solution to get a solution to the original instance decrease by a constant: insertion sort decrease by a constant factor(a.k.a. prune-and-search): binary search variable size decrease: Euclid’s algorithm

Four General Design Techniques-4 Transform-and-conquer This technique is based on the idea of transformation simplification: solves a problem by first transforming its instance to another instance of the same problem which makes the problem easier to solve, e.g., Gaussian elimination, heapsort... representation change: it is based on a transformation of a problem’s input to a different representation, e.g., hashing, heapsort …

Four General Design Techniques-4 cont. preprossing: The idea is to process a part of the input or the entire input to get some auxiliary information which speeds up solving the problem, e.g., KMP algorithms. reduction: An instance of a problem is transformed to an instance of a different problem altogether, e.g, NP-hard problems.

A Test of Generality-1 We partition design techniques into two categories: more general and less general techniques. How to make such a determinate ? We would like to suggest the following test. In order to qualify for inclusion in the category of most general approaches, a technique must yield reasonable algorithms for the two problems: sorting and searching.

A Test of Generality-2 SortingSearching Brute forceSelection sortSequential search Divide-and-conquerMergesortApplicable Decrease-and-conquerInsertion sortApplicable Transform-and-conquerHeapsortSearch trees, hashing The others - greedy approach, dynamic programming, backtracking, and branch-and-bound - fail to qualify as the most general design techniques.

Further Refinements-1 Local search techniques Greedy methods: it builds solutions piece by piece … the choice selected is that which produces the largest immediate gain while maintaining feasibility, e.g., Prim’s algorithm. Iterative methods: it start with any feasible solution and proceed to improve upon the solution by repeated applications of a simple step, e.g., Ford-Fulkerson algorithm.

Further Refinements-2 Dynamic programming Bottom-up: a table of solutions to subproblems is filled starting with the problem’s smallest subproblems. A solution to the original instance of the problem is then obtained from the table constructed. Top-down: memory function

Further Refinements-3 State-space-tree techniques Backtracking: take coloring problem as an example: Use three colors to color the vertices of this graph. How many different ways ?

Further Refinements-3 cont. S V1V1 V2V2 V3V3

Branch-and-Bound: take TSP problem as an example. If there are four cities, all feasible solutions are

Further Refinements-3 cont. Consider the traveling costs between any two cities: =

Further Refinements-3 cont. Start to branch Choose 1  2 Otherwise

Further Refinements-3 cont. The current branch:

Further Refinements-3 cont. The difference between them lies in that backtracking is not limited to optimization problems, while branch- and-bound is not restricted to a specific way of traversing the problem’s space tree.

Conclusion This paper gives a new taxonomy of algorithm design techniques. No matter how many general design techniques are recognized, there will always be algorithms that cannot be naturally interpreted as an application of those techniques. Some algorithms can be interpreted as an application of different techniques, e.g., selection sort, as a brute-force algorithm and as a decrease-and-conquer method. Some algorithms may incorporate ideas of several techniques, e.g. Fourier transform takes advantage of both the transform and divide-and-conquer ideas.