Finite Convergence of Tabu Search - Semantic Scholar

MIC’2001 - 4th Metaheuristics International Conference

333

Finite Convergence of Tabu Search Fred Glover∗ ∗

Sa¨ıd Hanafi†

Hearin Center for Enterprise Science School of Business Administration University of Mississippi University, MS 38677 USA Email: [email protected] †

LAMIH - UMR CNRS n◦ 8530 Unité de Recherche Opérationnelle et d’Aide ` a la Décision Université de Valenciennes et du Hainaut-Cambrésis Le Mont Houy - B.P. 311 - 59304 Valenciennes Cedex - France Email: [email protected]

1

Introduction

Many optimization techniques (both heuristic and exact) for solving combinatorial and nonlinear problems are iterative neighborhood search procedures – i.e., they start with an initial solution (feasible or infeasible) and repeatedly construct new solutions from current solutions by moves defined by reference to a neighborhood structure. The process continues to generate a trajectory of “neighboring solutions” until a certain stopping criterion is satisfied. The adaptive memory approach of Tabu Search (TS) generates a neighborhood trajectory by including a mechanism that forbids the search to revisit solutions already encountered – unless the intervening trajectory is modified (see [7]). The main goal of memory structures in TS is not simply to forbid cycling, however. In fact, the choice of a given neighborhood and a decision criterion for selecting moves with TS can force some solutions to be revisited before exploring other new ones. Within this context, a proposal of Glover [4] identifies a simple rule for revisiting solutions that is conjectured to have implications for finiteness in zero-one integer programming and optimal set membership problems. Hanafi [8] proves Glover’s conjecture under the assumption that the graph of the neighborhood space is connected and symmetric. In this paper, we provide new proofs that yield specific bounds establishing the finite convergence of tabu search, specifically for certain TS algorithms based on recency memory or frequency memory. Our results distinguish between symmetric and asymmetric neighborhood structures and provide insights into the sequences of solutions generated by the search. The outcomes disclose interesting contrasts between TS trajectories and the those generated by the more rigid rules underlying tree search methods. Based on these findings, we also give designs for more efficient forms of convergent tabu search, and provide special rules that create a new type of tree search. This work is the first to provide explicit convergence bounds for methods based on such forms of memory. The finiteness of these methods suggests an important distinction between their underlying ideas and the rationale that gives rise to “infinite time” convergence results for certain randomized procedures such as annealing. Porto, Portugal, July 16-20, 2001

334

2


A Convergent Tabu Search

Let T ime(x) = the most recent time (iteration) that solution x was visited by a search process, whose form is determined as follows. Convergent Tabu Search (CTS) Step 0 Initialization, the values T ime(x), x ∈ X, begin as arbitrary nonnegative integers, and the starting solution x∗ for the search is assigned a value so that T ime(x∗ ) > T ime(x) for all x other than x∗ . (This includes the case where we begin with T ime(x) = 0 for all x ∈ X except x∗ . Select a starting solution x ∈ X and set Iteration = 0; Step 1 Set T ime(x) = Iteration. Select an unvisited neighbor x ∈ N (x) such that T ime(x ) = 0, if one exists, and otherwise choose to visit a solution x = argmin{T ime(y) : y ∈ N (x)}; T ime(x) = Iteration. Step 2 Stop if all solutions in X are visited i.e., T ime(x) > 0 ∀x ∈ X. Otherwise set Iteration = Iteration + 1; x = x (move from x to x ); go to Step 1. Theorem A Denote the cardinality of X by n = |X|, and consider a value Un for n ≥ 2 which is given recursively by U1 = 0 and define Un+1 = 2Un + 1, for n ≥ 1. Beginning with any solution x∗ ∈ X, the CTS method will visit every solution in X in at most Un steps if X is finite and there exists a neighborhood path from every solution in X to every other solution in X. The “min{T ime(x)} rule” is the one called the Aspiration by Default rule in the TS literature. Since frequency-based memory is also useful in TS, it is natural to speculate that a “frequency version” of Theorem A is valid. We apply the natural definition, F requency(x) = the number of times x bas been visited, and replace T ime(x) by F requency(x) and set F requency(x) = F requency(x) + 1 after visiting solution x. The conclusion of Theorem A holds when CTS is based on frequency memory. Additional enhancements that are possible using this TS methodology include the use of an associated Reverse Elimination Memory for streamlining the new tree search procedure associated with the TS process (which differs from the convergent TS approach that uses more flexible memory). We also propose an approach for accelerating the classical tabu search Aspiration by Default rule in this setting, which may transform an exponential search into a much faster polynomial search. Finally, we give designs for more efficient forms of convergent tabu search in general.

3

Tabu Tree Search

In contrast to our previous use of the label T ime(x) for each solution x, however, we add the stipulation that as soon as T ime(x) is assigned a value (i.e., as soon as x is visited), we do not permit its value to be further changed. Accompanying this, we now reverse the Aspiration by Default rule, to require that, whenever all elements of N (x) have previously been visited, the method moves from x to the node x ∈ N (x) that has the largest (rather than smallest) value of T ime(x ), subject to the limitation that this value must be smaller than that of T ime(x) itself. The resulting method is as follows. Tabu Tree Search (TTS) 1. From a given solution x, move to an unvisited neighbor x ∈ N (x) whenever possible, and stop if the label thus assigned to x is T ime(x ) = |X|. Otherwise, Porto, Portugal, July 16-20, 2001


335

2. Move to the visited neighbor x with the largest value of T ime(x ) ≤ T ime(x). We establish the relevant properties of the method as follows, under the assumption that the graph of the neighborhood space is connected. Theorem B The TTS method generates a tree, rooted at the initial solution, that spans the nodes of the neighborhood graph. Each edge of the tree is crossed exactly once in the direction away from the root, and at most once in the direction toward the root. (No edges outside of the tree are crossed.) In addition: (a) The unique path from any solution to the root is generated by repeatedly executing the rule of Step 2 of the TTS method. (b) Each time any solution x is visited, each labeled neighbor x of x is either an ancestor or descendant of x in the tree currently constructed (i.e., either x lies on the path from the root to x, or else x lies on the path from the root to x ). (c) Each time step 2 is executed to reach a visited node x , all nodes of the graph that are neighbors of visited nodes x”, where T ime(x”) > T ime(x ), are also visited nodes. (d) Each time step 1 successfully identifies an unvisited neighbor of x, then node x satisfies the condition x = Argmax{T ime(y) : y is a node of the current tree and y has an unvisited neighbor}. In common with the Aspiration by Default rule, the TTS approach in some cases may visit all solutions by only visiting each solution a single time, hence effectively generating a Hamiltonian path through the neighborhood space, in contrast to the type of trajectory created by usual forms of tree search. However, more importantly, the TTS approach allows substantially greater flexibility of choice than customary types of tree search, as embodied in branch and bound approaches. On the other hand, the TTS structure differs according to the choices made – that is, different choices may produce different numbers of revisited solutions (and, as previously remarked, some may produce no revisited solutions), thus producing trees of different topologies.

4

Conclusion

Theorems A and B can both readily be extended to apply to enhanced forms of the underlying TS designs. In summary, the key observations of our paper are: (1) strategic flexibility is compatible with assured finite convergence, by special types of memory introduced in certain variants of tabu search; (2) the resulting search traverses the nodes of a graph in a significantly different way than provided by tree search; (3) a simple tree search variant of the approach produces a type of tree search that offers novel contrasts with branch and bound, and also differs notably from other tree searches such as reverse search and the Tarry Traverse.

References [1] M. Andramonov. Some conditions of convergence of tabu search. Research report, Institute of Mathematics and Mechanics, Russia, 1999. [2] F. Dammeyer and S. Voss. Dynamic tabu list management using the reverse elimination method. Ann. Oper. Res., 41:31–46, 1993. [3] H. Farreny. Completeness and admissibility for general heuristic search algorithms - a theoretical study: Basic concepts and proofs. to appear in J. Heuristics. Porto, Portugal, July 16-20, 2001

336


[4] F. Glover. Tabu search, part 2. ORSA J. Comput., 2:4–32, 1990. [5] F. Glover and S. Hanafi. Composite tree searches for global optimization. Research report, Graduate School of Business and Administration, University of Colorado, Boulder, 1998. [6] F. Glover and S. Hanafi. Tabu search and finite convergence. to appear in Special Issue of Discr. Appl. Math. on ”Foundations of heuristics in Combinatorial Optimization”, 2000. [7] F. Glover and M. Laguna. Tabu Search. Kluwer Academic Publishers, 1997. [8] S. Hanafi. On the convergence of tabu search. J. Heuristics, 7:47–58, 2000. [9] S. Hanafi and A. Fréville. Extension of reverse elimination method through a dynamic management of the tabu list. to appear in R.A.I.R.O., 1999. [10] G. Tarry. Le problème des labyrinthes. Nouvelles Annales de Mathématiques, 14(3):187–190, 1895. [11] G.L. Thompson. The tarry traverse, 1998. Class notes, Carnegie-Mellon University, Graduate School of Industrial Administration.

Porto, Portugal, July 16-20, 2001