CSCE350 Algorithms and Data Structure Lecture 21 Jianjun Hu Department of Computer Science and Engineering University of South Carolina

Chapter 11: How to tackle NP-hard problems... There are two principal approaches to tackling NP-hard problems or other “intractable” problems: Use a strategy that guarantees solving the problem exactly but doesn’t guarantee to find a solution in polynomial time Use an approximation algorithm that can find an approximate (sub-optimal) solution in polynomial time E.g.Genetic Algorithm----for large optimization problem Newton gradient descent method

Exact solutions The exact solution approach includes the strategies: 1 exhaustive search (brute force) useful only for small instances 2 backtracking eliminates some cases from consideration 3 branch-and-bound further cuts down on the search fast solutions for most instances worst case is still exponential

Backtracking Construct the state space tree: nodes: partial solutions edges: choices in completing solutions Explore the state space tree using depth-first search (DFS) “Prune” non-promising nodes DFS stops exploring subtree rooted at nodes leading to no solutions and... “backtracks” to its parent node

Example: The n-Queen problem Place n queens on an n by n chess board so that no two of them are on the same row, column, or diagonal

State-space of the four-queens problem

Example: Hamiltonian Circuit Problem

Subset-Sum Problem Find a subset of a given set S={s 1,s 2,…,s n } of n positive integers whose sum is equal to a given positive integer d For example: S={3,5,6,7} and d=15  solutions {3,5,7}

Branch and Bound An enhancement of backtracking. Applicable to optimization problems Uses a lower bound for the value of the objective function for each node (partial solution) so as to: guide the search through state-space rule out certain branches as “unpromising” large subsets of fruitless candidates are discarded en masse, by using upper and lower estimated bounds of the quantity being optimized. The key idea of the BB algorithm is: if the lower bound for some tree node (set of candidates) A is greater than the upper bound for some other node B, then A may be safely discarded from the search. This step is called pruning, and is usually implemented by maintaining a global variable m (shared among all nodes of the tree) that records the minimum upper bound seen among all subregions examined so far. Any node whose lower bound is greater than m can be discarded.

Select one element in each row of the cost matrix C so that: no two selected elements are in the same column; and the sum is minimized For example: Job 1Job 2Job 3Job 4 Person a Person b Person c Person d Lower bound: Any solution to this problem will have total cost of at least: Example: The assignment problem

Assignment problem: lower bounds

State-space levels 0, 1, 2

Complete state-space

Traveling salesman example:

Approximation algorithm approximation algorithms are algorithms used to find approximate solutions to optimization problems Approximation algorithms are often associated with NP- hardproblems Unlike heuristics, which usually only find reasonably good solutions reasonably fast, Approximation algorithm wants provable solution quality and provable run time bounds. Ideally, the approximation is optimal up to a small constant factor (for instance within 5% of the optimal solution)