Study Guide for ES205 Yu-Chi Ho Jonathan T. Lee Nov. 7, 2000 Tabu Search Study Guide for ES205 Yu-Chi Ho Jonathan T. Lee Nov. 7, 2000

Outline Overview Example Notations Algorithm Applications Pros and Cons

Overview An iterative procedure for solving discrete combinatorial optimization problem, suggested by Glover. To explore the search space (S) by a sequence of moves. A move transform current solution into a solution that is in the neighborhood of the current solution. In the case of traveling salesman problem, a possible move is to drop 2 existing edges and reconnect the 4 cities involved. In other words, the neighborhood of the current solution consist of all the tours that differ from the current tour by 2 edges. Note, this neighborhood is constructed artificially in the case of discrete combinatorial optimization problem. Unlikely in the continuous optimization, the neighborhood structure arise naturally from the distance between points. N

Overview (cont.) To avoid local minimum, a tabu list is constructed where forbidden moves are listed. Tabu moves are based on the short- and long-term history of the search process. Aspiration criteria is the condition which allows the tabu status of a tabu move to be overwritten so that the move can be considered at the iteration. In the simple case, the reverse of the most recent move is listed as tabu since it would bring the search to the previously visited solution, potentially creating cycling. This is an example of a short-term history based tabu move. A long term tabu move might depends on the frequency of the occurrence of a move in the long run. An aspiration criteria is a way to override the tabu status of a particular move. An example of the aspiration criteria includes the case which, by taking a tabu move which is a move from the current solution, leads to a solution which is the best obtained so far. N

Overview (cont.) The next move is the best move among the feasible moves from the neighborhood of the current solution. A tabu move is taken if it satisfies the aspiration criteria.

Example Minimum spanning tree x1 x2 x3 x4 x6 x7 x5 6 2 8 12 18 9 8 12 18 9 Constraint: x1 + x2 + x6  1 x1  x3 Penalty: 50 The goal of the minimum spanning tree is to find the set of edges that connects all the nodes and gives the minimum cost. The x’s label the edges, 1 through 7. The numerical values besides the edge labels are the cost for that particular edge, i.e., edge 1 cost 6 units. If xi = 1, then the edge is in the tree which is indicated by the solid line. If xi = 0, then edge i is not in the tree which is indicated by dashed line. Here, we have two constraints. The first constraint states that at most one of the three edges, x1, x2, and x6 is permitted to be in the tree. The second constraint states that the edge x1 is allowed in the tree only if edge x2 is also in the tree. Violation of either of the constraint would incur a penalty of 50 units. N

Example (cont.) The move considered is the standard “edge swap” move. An edge is listed as tabu if it was added within the last two iteration. The aspiration criteria is satisfied if the tabu move would create a tree that is better than the best tree so far. The “edge swap” move consist of adding an edge and dropping another edge to transform the current tree into a new tree. If the edge is listed as tabu then it is not allowed to be dropped. N

Example (cont.) Iteration 1 x1 x2 x3 x4 x6 x7 x5 6 2 8 12 18 9 8 12 18 9 J = 16 + 100 Drop Add This initial tree is the minimum cost tree without considering penalty. Since both of the constraints are violated, 100 unit penalty is added to the cost. Among all the possible moves, adding x3 and dropping x1 produces the best tree by eliminating both of the violations. Note, none of the edges are listed as tabu since there are no prior moves. N

Example (cont.) Iteration 2 J = 28 x1 x2 x3 x4 x6 x7 x5 6 2 8 12 18 9 8 12 18 9 Tabu After just adding x3 to the tree, that edge is listed as tabu. In this case, the tabu status of x3 does not effect our search process since it would not have yield a better tree. This shows that the tabu restriction does not always affect the preferred choice. Add Drop N

Example (cont.) Iteration 3 J = 32 x1 x2 x3 x4 x6 x7 x5 6 2 8 12 18 9 8 12 18 9 Add Tabu Drop The edge x7 joins x3 to be listed as tabu. The move of dropping edge x3 is usually not allowed since it is listed as tabu. Due to the fact that it would produce a tree that satisfies the aspiration criteria, a new tree that is better the best tree so far, this move is allowed. Tabu N

Example (cont.) Iteration 4 J = 23 x1 x2 x3 x4 x6 x7 x5 6 2 8 12 18 9 8 12 18 9 Tabu The search continues till the desired iteration cutoff is reached. Tabu N

Notations s: the current solution N(s): the neighborhood set of s where N(s)  S T(s): the tabu set of s where T(s)  N(s) A(s): the aspiration set of s where A(s)  T(s)

Algorithm k = 1 Generate initial solution s WHILE not finished Identify N(s), T(s), A(s) Choose s’(N(s) - T(s))  A(s), for which J(s’), the performance, is maximum s = s’ k = k + 1 END WHILE

Applications Traveling Salesman Problem Scheduling problems, e.g., machine scheduling Character recognition Telecommunication path assignment Probabilistic logic problem Network topology design Here is a partial list of problems where tabu search have been successfully applied to. The result from tabu search is better or comparable to the results from other algorithms. Note the wide range of problems that have been tried. N

Pros and Cons Pros: Cons: Have shown to provide comparable or superior solution, in various test problems, to other techniques. Cons: The construction of the tabu list is heuristic. Assume fast performance evaluation. The assumption of fast performance evaluation is important. Without it, we cannot evaluate all the possible moves. Thus, selecting the best move from the neighborhood. But this assumption is not always possible to obtain. N

References: Glover, F., “Heuristics for Integer Programming using Surrogate Constraints,” Decision Sciences, Vol. 8, pp. 156-166, 1977. Glover, F., “Tabu Search: A Tutorial,” Interfaces, 20(4):pp.74-94, 1990. Thiel, S. U., “Tabu Search,” http://www.cs.cf.ac.uk/User/S.U.Thiel/ra/subsection3_7_2.html, 1995.