Evaluating some lesser used features of the tabu search algorithm

Evaluating some lesser used features of the tabu search algorithm Ian M. Whittley George D. Smith October 29, 2004 Abstract Techniques based on the use of long term, frequency memories, advanced forms of aspiration criteria and elite solution stores often take a back-seat in the Tabu Search literature to those based on the use of short term, recency memories. In this paper we attempt to ascertain whether or not this is justified. Guided Local Search is introduced to represent an advanced use of frequency memory; the Attribute Based Hill Climber as a concrete implementation of the rarely used, regional aspiration by objective; and the Attribute Based Tree Search as an example of the use of elite solution stores. Evaluated on Traveling Salesman and Quadratic Assignment problem instances the three algorithms achieve roughly the same levels of performance. The fact that Guided Local Search is known to be one of the most powerful general purpose techniques on these problems suggests that perhaps the focus taken by Tabu Search practitioners is too narrow.

1

Introduction

The Tabu Search (TS) literature describes a wide range of techniques that can be combined to construct effective, efficient optimisation algorithms. Even a cursory examination of the literature, however, shows a tendency of researchers to concentrate on what is often viewed as the defining characteristic of the algorithm, its use of short term, recency memory. A survey of 90 papers in [9] detailing the application of the Tabu Search metaheuristic to a wide range of both real world and academic problems revealed the following: • every implementation used some form of recency memory, 37 exclusively, 21 in combination with only the most basic form of aspiration criteria, • whilst 40 explicitly mentioned the use of aspiration criteria, only one moved beyond the most basic approaches, • 20 made use of longer term, frequency, memories, and 1

• only 14 considered the use of elite solutions. The only conclusion that can realistically be drawn from these results is that Tabu Search is viewed as being synonymous with the use of recency memory, additional techniques from the repertoire simply providing aids for “fine tuning” its performance. In this paper we aim to challenge this conclusion by evaluating algorithms that rely exclusively on these “supplementary” techniques. The use of frequency memory is to be demonstrated by Guided Local Search [8]; the use of aspiration criteria by the Attribute Based Hill Climber [11]; and the use of elite solutions by the Attribute Based Tree Search [12]. In Section 2 we give a brief description of each technique and justify its selection; this is followed in Section 3 by details of their application to instances of two well known problems, the Traveling Salesman Problem and the Quadratic Assignment Problem; finally in Section 4 we present our conclusions. For a full description of Tabu Search as a whole we refer the reader to the standard literature ([2, 3, 4], for example).

2

Techniques

All algorithms in this section will be modelled with reference to the attribute model for optimisation. Under this scheme a problem, P , is represented by the tuple P = (A, S, f ) where A represents the set of attributes used in the construction of the solutions, S represents the set of solutions in which we are interested (where S ⊆ 2A ), and f represents the objective function to be minimised. Under this representation a solution is viewed merely as a set of attributes. Some additional definitions that will prove useful include V, the set of solutions that have been visited by the search, E, the set of solutions examined by the search (where V ⊆ E), and S a = {s ∈ S : a ∈ s}, the set of solutions containing the attribute, a.

2.1

Frequency

In Tabu Search, frequency memory is generally used in a modified objective function to bias the search away from frequently encountered attributes. A standard form for such a function is as follows:
f  (s) = f (s) + p. M (a) a∈s

where p is a penalty term, and M is an attribute memory. M is updated as the search progresses to hold the number of solutions visited that contain each attribute, M (a) = |V ∩ S a | ∀a ∈ A Guided Local Search (GLS) uses precisely the same form of modified objective function but differs in the frequency data it tracks. GLS aims to identify

2

and penalise those attributes that contribute the most to the cost of each locally optimal solution encountered, provided there is no evidence that the presence of such attributes cannot otherwise be excused. Formally, at every local optima, s, as defined by f  , we update a memory, M , initialised to 0 at the start of the search, as follows: M (a) ← M (a) + 1, ∀a ∈ w(s), where:

w(s) = {a ∈ s : c (a, s) ≥ c (b, s) ∀b ∈ s}, c (a, s) = c(a, s)/(1 + M (a)),

with c(a, s) providing a problem specific measure of the contribution to the cost of the solution s brought about by the presence of the attribute, a. c (a, s) is used, rather than c(a, s), in the selection of the ‘worst’ attributes to prevent the search from repeatedly penalising the same set of attributes. GLS uses the memory M , in combination with f  , to construct a trajectory through the solution space from an initial solution, s0 , as follows:  s = best (N (si )), if f  (s) < f  (si ) si+1 = otherwise, and updates the memory. si , where best (si ) = (s ∈ N (si ) : f  (s) ≤ f  (t) ∀t ∈ N (si )). Note that the particular form of update described above is only generally suitable when the attribute costs are fixed irrespective of the solution being considered. See [8] for details on alternative approaches when this is not the case and for a more complete description of the algorithm.

2.2

Aspiration

Let Π = {π1 , π2 , ..., πn } represent a covering set of subsets of S. Regional aspiration by objective defines an acceptance function acceptable : S → B as follows: acceptable(s) ≡ (∃π ∈ Π : s ∈ π and f (s) < f (t), ∀t ∈ π ∩ V) Informally, a solution is acceptable if and only if it is better than all visited solutions in at least one of the subsets that contain it. One standard approach recommended in the Tabu Search literature defines a partition Π = {π1 , ..., πn } where: πi = {s ∈ S : h(s) mod n = i − 1} for a suitably defined hash function, h : S → N. The Attribute Based Hill Climber (ABHC) takes an alternative approach and can be modelled as a form of regional aspiration by specifying the set of subsets as follows: Π = {S a : a ∈ A} 3

The ABHC uses the above partition to construct a path through the solution space, from an initial solution s0 , as follows: si+1 = best({t ∈ N (si ) : acceptable(t)}) terminating if no such solution is available. Alternate descriptions of the ABHC can be found in [11] whilst additional approaches to the use of aspiration criteria in Tabu Search can be found in [6].

2.3

Elite

Elite solution stores are used to retain a set of high quality solutions encountered during a search. The Attribute Based Tree Search (ABTS) uses a novel form of elite solution store to retain interesting unexplored branches away from the search trajectory followed by an underlying local search algorithm. An implicit solution memory, M , is maintained that can be defined as follows: M = {s ∈ E : ∃a ∈ s s.t. f (s) = potential(a) < value(a)} where value(a) = min({f (s) : s ∈ V ∩ S a } ∪ {∞}) defines the value of an attribute to be equal to the value of the best visited solution that contains it, and potential(a) = min({f (s) : s ∈ E ∩ S a } ∪ {∞}) provides a bound on the value it could have taken if a different trajectory through E had been followed. The algorithm then uses M to guide the search, as follows, from an initial solution, s0 :  best(C(si )) if C(si ) = ∅ and best(C(si )) ∈ / V, si+1 = best(M ) otherwise, where C(si ) = {s ∈ N (si ) : f (s) < f (si )} represents the set of improving solutions in the neighbourhood of si . Informally, the ABTS takes the best improving solution at each iteration provided that such a solution is available and does not return the search to a previously explored trajectory. Otherwise it takes the best previously unexplored branch that realises the potential of at least one ‘unoptimised’ attribute in the search space. For a more precise description of the algorithm and its implementation see [12].

3

Evaluation

The ABHC and ABTS were both implemented in C++ as part of the Heuristic Template Library[10] and applied to two well known optimisation problems, the Traveling Salesman Problem (TSP) and the Quadratic Assignment Problem (QAP). All 58 TSP instances found in TSPLIB [7] with a size < 400 were tackled along with the entire suite of QAPLIB [1] instances. Standard 2-opt 4

neighbourhoods were used for both problems, alongside basic, permutation vector, solution representations (see [11] for more details). 10 runs were performed on each problem instance with 1000n iterations allowed for the QAP (where n is the problem size) and 200,000 iterations allowed for the TSP. The experimental conditions were selected to exactly mirror those given in [8] from which the GLS results are quoted. Table 1 displays the average percent deviation from optimality1 achieved for each problem/problem set. A count of the number of runs that failed to reach an optimal solution is also recorded for each of the TSP instances. Note also that a - in the table represents an unavailable result (not provided in original GLS paper) and a † represents an incomplete set of results (time constraints prevented the running of the ABTS on the very largest QAP instances). Finally, it should be noted that all 28 instances in TSPLIB with a size ≤ 100 were solved on every run by all three algorithms, as were 14 of the 30 with sizes between 101 and 318; these results have been excluded from the table. QAP bur* chr* els esc* had* kra* lipa* nug* rou* scr* sko* ste* tai*a tai*b tai*c tho* wil*

Sizes 26 12-25 19 16-128 12-20 30 20-90 12-30 12-20 12-20 42-100 36 10-100 10-100 256 30-150 50-100

GLS 0.001 2.350 3.416 0.016 0.000 0.631 0.118 0.005 0.013 0.003 0.139 0.907 0.271 1.196 1.347 0.132 0.093

ABHC 0.000 0.829 0.000 0.000 0.000 0.008 0.060 0.000 0.001 0.000 0.063 0.024 0.477 0.589 0.153 0.104 0.030

ABTS 0.000 0.333 0.000 0.000 0.000 0.201 0.132 0.000 0.001 0.000 0.164 0.000 0.454 0.824† † 0.010† 0.098

TSP gr120 pr136 kroA150 kroB150 rat195 d198 kroA200 kroB200 gr202 ts225 pr226 gr229 gil262 a280 pr299 lin318

GLS 0.001 (1) 0.000 (0) 0.000 (0) 0.01 (2) 0.08 (10) 0.000 (0) 0.000 (0) 0.01 (2)

ABHC 0.007 (1) 0.008 (7) 0.001 (3) 0.002 (3) 0.052 (2) 0.025 (8) 0.016 (4) 0.018 (9) 0.001 (3) 0.017 (3) 0.002 (3) 0.133 (10) 0.055 (7) 0.016 (2) 0.068 (10) 0.348 (10)

ABTS 0.000 (0) 0.000 (0) 0.000 (0) 0.000 (0) 0.000 (0) 0.000 (0) 0.000 (0) 0.000 (0) 0.000 (0) 0.000 (0) 0.000 (0) 0.000 (0) 0.000 (0) 0.000 (0) 0.000 (0) 0.000 (0)

Table 1: Frequency v Aspiration v Elite

4

Conclusions

The results of experimentation on QAP and TSP problem instances soundly contradict the standard assumption that the main strength of Tabu Search lies in its use of short term, recency memories. Algorithms based purely on the use of aspiration criteria (ABHC) or elite solution stores (ABTS) are found to 1 100*(best-opt)/opt

5

perform comparably with Guided Local Search, which was introduced in its basic form (see [5] for improvements) to represent a Tabu-esque approach to the use of frequency information, and is known to be one of the most powerful general purpose meta-heuristics when applied to these problem instances. Preliminary results from the application of algorithms based solely on the use of recency considerations have only served to corroborate these findings.

References [1] R.E. Burkard, S.E. Karisch, and Franz Rendl. QAPLIB - a quadratic assignment problem library. European Journal of Operational Research, 55:115–119, 1991. [2] F. Glover and M. Laguna. Tabu Search. Kluwer Academic Publishers, Boston, 1997. [3] Fred Glover. Tabu search – part 1. ORSA Journal on Computing, 1(3):190– 206, 1989. [4] Fred Glover. Tabu search – part 2. ORSA Journal on Computing, 2(1):4– 32, 1990. [5] P. Mills, E.P.K. Tsang, and J. Ford. Applying an extended guided local search on the quadratic assignment problem. Annals of Operations Research, 118:121–135, 2003. [6] Colin R. Reeves, editor. Modern Heuristic Techniques for Combinatorial Optimization. Mc-Graw Hill International, New York, 1995. [7] Gerhard Reinelt. Tsplib – a traveling salesman problem library. ORSA Journal on Computing, 3(4):376–384, 1991. [8] V. Voudouris. Guided Local Search for Combinatorial Optimisation Problems. PhD thesis, Department of Computer Science,University of Essex, 1997. [9] I.M. Whittley. Tabu Search – Revisited. PhD thesis, School of Information Systems, University of East Anglia, 2002. [10] I.M. Whittley and A. Reynolds. The heuristic template library. In preparation., 2004. [11] I.M. Whittley and G.D. Smith. The attribute based hill climber. Journal Of Mathematical Modelling and Algorithms, 3 (2):167–178, 2004. [12] I.M. Whittley and G.D. Smith. An attribute based tree search. Submitted to European Journal of Operations Research, 2004.

6