Karp algorithm for the symmetric traveling

Mathematical Programming 7 (1974) 87-96. North-Holland Publishing Company

IMPROVEMENTS OF THE HELD-KARP ALGORITHM FOR THE SYMMETRIC TRAVELING-SALESMAN PROBLEM Keld HELBIG HANSEN and Jakob KRARUP Spadille, Inc., and Institute o f Datalogy, University o f Copenhagen, Denmark Received 8 August 1973 Revised manuscript received 26 April 1974 A highly efficient algorithm (HK) devised by Held and Karp for solving the symmetric traveling-salesman problem was presented at the 7th Mathematical Programming Symposium in 1970 and published in Mathematical Programming in 1971. Its outstanding performance is due to a clever exploitation of the relationship between the traveling-salesman problem and minimum spanning trees. However, various improvements of their method have led to a version (IHK) which tends to be some 25 times faster than the original one. Experiments with data selected at random, ranging in size up to 80 cities, show that the computing time for IHK is roughly doubled as the number of cities is increased by 10.

1. Introduction Recently, M. Held and R.M. Karp published two papers [ 4, 5 ] describing a highly efficient algorithm for solving the symmetric traveling-salesman problem. The last section of [5], however, deals with various suggestions for possible improvements in computing time. We have implemented the algorithm on an IBM 360/75 and carried out a number of numerical experiments in order to investigate the effect of the aforementioned suggestions. Furthermore, some additional improvements have been incorporated. The improved algorithm, in the sequel, referred to as IHK for simplicity, has been tested with those classical examples reported on in [5], a n d shows a tendency to be 25 times faster on the average than the original version (HK). No estimate of the computing time as a function of the number of cities involved has been given in [ 5 ]. However, a sequence of tests with data selected at random, ranging in size up to 80 cities, indicates that the computing time with IHK is roughly doubled as the number of cities is increased by 10.

88

K. HelbigHansen, J. Krarup, Improvements oft he HeM-Karp algorithm

Due to its nature, the present paper is intimately related to certain sections of [5]. In the following description as to how IHK deviates from HK, we have therefore found it practical to adopt the terminology and notation from [5]. It should be stressed, that readers not familiar with [5] are strongly recommended to start with [5], Sections 0 through 3 (as to Section 2, the first ten lines will do). Otherwise, they will find the present paper unreadable. Our computational experience with IHK is portrayed in Section 3. Finally, some questions yet unanswered are discussed briefly in Section 4.

2. The improved algorithm Like HK, the IHK is a branch-and-bound procedure combined with an ascent method for calculating lower bounds on C*, the weight of an optimum tour. Three parameters influence the computation: t m governs the step size in the ascent method, p determines when the ascent method is terminated, C is an upper bound on C*. In [5], there is no principal distinction between initial ascent (i.e., the first application of the ascent method where no branchings have yet been performed) and general ascent (i.e., eventual supplementary applications of the ascent methody. The phrase "initial ascent" is introduced by Held and Karp mostly for descriptional purposes, cf. [5, Table 2] exhibiting amongst others w(zr) after initial ascent. The situation is, however, somewhat different for IHK where we have to distinguish between initial and general ascent, because it affects the determination of t m as well as the choice of a subroutine for computing 1-trees of minimum weight. 2.1. The parameters t m and p

The central part of the iterative method for computing or approximating max~ {w(~r)} is the expression ~rm+l = ~rm + ~ v k ( ~ . ) ,

(1)

where, cf. [5, Section 2]: for any rr, k(Tr) is the index of a minimumweight 1-tree at the point 7r, and {t m} is a sequence of scalars.

K. HelbigHansen, J. Krarup, Improvements of the Held-Karp algorithm

89

Held and Karp have, except for a single example, fixed tm at 7, where 7 is a constant, the value of which appears from [5, Tables 1 and 2], However, the structure of (1) makes it intuitively reasonable to select a tm which depends on the elements of the distance matrix c. But as we shall see, tm is not changed concurrently with 7rm: while 7rm is subject to changes in each iteration, tm is kept at a fixed level during a sequence of iterations. In order to avoid confusions, (1) is therefore replaced by 7rm+l = 7rm + tStJk(~rm)

(2)

D e t e r m i n a t i o n o f t °. Due to its definition, w(0) is the minimum weight of a 1-tree provided that 7r = 0, i.e., w(0) depends solely on the original distance matrix. Let v 0 be the n-vector corresponding to the particular 1-tree. We shall now endeavor to select t 0, the initial value for t s, such that w(Trl), where rr1 = rr° + t ° v °,

ir ° = 0

becomes as large as possible. Experience has shown that w(lrl), except for extraordinary cases, tends to increase for increasing values of t o until a m a x i m u m is attained, and thereafter drops. The most likely explanation is that relatively small mutual movements of the n cities caused by small changes of ~r have only a limited effect on the weight of a minimum 1-tree and, in turn, on the magnitude of the lower bound. The following procedure for calculating t o has proved to be appropriate:

/if w(0)> w(v°) to = 1 2i

[or w ( v ° ) > w(0) and w ( v °) >I w(2v °) otherwise,

(3)

where 2 i is the smallest positive power of 2 for which the inequalities w ( 2 i - l v O) < W(2ivO),

w(2iv O) ~> w(2i+luO),

i = 1,2; ...

are satisfied. D e t e r m i n a t i o n o f p. The parameter p determines the n u m b e r of repetitions of (2) for each application of the ascent method, p is, more precisely, an upper bound for the number of successive iterations yielding no improvement on the lower bound.

90

K. HelbigHansen, Z Krarup, Improvements of the Held-Karp algorithm

Held and Karp do not state explicitly how p is to be determined. However, its actual value when solving various examples is shown in their tables. It appears that p varies from 25 to 100 and the experience gained by Held and Karp seems to coincide with ours, expressible as the following rule of thumb: p should be increased with increasing values of n; "ill-conditioned" problems (e.g. like Tutte's 46-city p r o b l e m ) r e q u i r e presumably even larger values of p than those normally used. For 1HK, we have determined p empirically by p = 4 + [n/10],

(4)

where n is the number of cities and where [n/10] denotes the largest integer smaller than or equal to n / l O . Initial ascent, s := 0, rr° := 0. Calculate t s = t o in accordance with (3), and p = 4 + [n/10]. Perform a sequence of iterations by means of (2) until no improvement occurs for a block o f p iterations, and let w(Tr') denote the largest lower bound thus obtained. If s < 4 and t s > 1, increase s b y one and repeat the sequence of iterations controlled by p n o w with t s = t s - 1 / 2 and 7r° = rr'. Otherwise, i f s = 4 or t s = 1, the initial ascent is terminated. Possible sequences o f t °, t 1 .... could be: 4, 2, 1 or perhaps 64, 32, 16, 8, 4. The obvious effect is, that the lower bounds in general are augmented more rapidly at the beginning of the initial ascent. G e n e r a l a s c e n t . While p remains unaltered throughout the entire computation, t s may assume two different values, t o and t 1, whenever the general ascent is applied. Here, t o is set equal to twice the final value of t s o f the initial ascent. When no improvements occur for p iterations, t o is replaced by t I = t ° / 2 , and a new sequence of iterations controlled b y p is performed. Again, numerical experiments underlie this strategy.

2.2. C o m p u t a t i o n

o f 1-trees o f m i n i m u m

weight

As observed by Held and Karp [5, Section 4, §c], there are t w o competing ways to compute minimum 1-trees: The method o f Dijkstra [ 1 ] 1 which was employed in HK and the method of Kruskal [6]. Held and Karp have further noted that the latter method is much faster on

1 Actually, credit for this method should be given to Prim [7].

K. Helbig Hansen, J. Krarup, Improvements o f the Held-Karp algorithm

91

average, but presupposes that the ci! are sorted into ascending numerical order. Our test results have certainly confirmed the truth of the above statement and led to the following compromise for IHK: Dijkstra'smethod for the initial ascent and Kruskal's method for the general ascent. The crucial point for the choice between the two methods in question is an estimate of the time consumption incurred by sorting the cij. Initial ascent is characterized by substantial changes of the c-matrix in each step, which, in turn, imply an extensive sorting before a new sequence of ci] into ascending numerical order is produced. The situation is somewhat different as far as the general ascent is concerned. The vast majority of vertices will n o w be of degree 2, which means that only a very limited number of the ci] are subject to changes, t s is furthermore relatively small, which is known to amplify the limiting effect on the changes. Thus, only few elements are likely to change from one sorted sequence to the next. This speaks in favor for applying a sorting procedure prior to Kruskal's method based on interchanging of consecutive elements, since the labor involved will be rather modest as compared to other methods. The sorting procedure described in [2] has been incorporated in IHK. Besides that, the method of Kruskal has another advantage from which we can benefit during the general ascent: it enables us, as we shall see, to calculate in a simple way the lower bounds which govern the branching. 2.3. The branching strategy To generate 1-trees of minimum weight constitutes an essential part of the branching procedure. Actually, we shall be dealing with a sequence of 1-trees where each one is only slightly different from the previous one. The following lemma indicates a way in which a sequential generation of 1-trees can be effectively achieved. Lemma 1. L e t T 1 be a 1-tree o f m i n i m u m weight and let (u, v) be any edge o f T 1. I f Cur and Cvu are set to infinity, thus excluding (u, v) f r o m appearing in any 1-tree o f finite weight, a new 1-tree, T 2, o f m i n i m u m weight can be obtained by T 2 = T 1 - (u, v) + (u', v'),

92


provided there exists a 1-tree o f finite weight in which (u, v) does not appear. The crucial point is that only a single edge has to be replaced. The new edge (u', v') will henceforth be designated as the replacement edge or, more briefly, the r-edge. For the p r o o f o f Lemma 1, see [ 3], where, in addition, a constructive method for finding (u', v') by means o f Kruskal's method is given. We shall n o w turn to the branch-and-bound procedure as described by Held and Karp [5, Section 3]. When a branching is performed, those edges which have not yet been included or excluded are ordered according to the amount by which the lower bound would increase if the corresponding edge was excluded. In IHK, this increment is easily calculated by means of Lemma 1. Held and Karp create new entries until an R i is met and return t h e n to the ascent method. In most cases, however, it is possible to create more entries. Let the 1-tree in Fig. 1 be the one from which we select edges. Furthermore, let X = 0, Y = 0, e 1 = (1,2) and e 2 = (2,4). Consequently, R i = R 2 and q = 3. At this point, Held and Karp would stop and return to the ascent method. But if the .1-tree - as in the present example neither consists of edges from R 2 nor of the edge (1,4) (which would create a cycle), it is clearly possible to pick out more edges. The same bounds as were calculated for the ordering of the ei's can be used for selecting more el's, but if we have used r-edges belonging to R 2 u (1,4) for the calculations, it will be reasonable to search for new r-edges and thus eventually increase the lower bound.

1

&

05

Fig. 1.

2.4. Selection of a new entry When the general ascent is to be applied, there are essentially two possibilities for choosing the next entry.


93

(a) Branching out from that entry with the smallest lower bound; (b) branching "to-the-right", both having their individual advantages and drawbacks. Held and Karp have applied (a) for HK. Numerical experiments, however, have led us to a combination of (a) and (b) as we realized that the vast majority of the solutions ended up with a C*, deviating less than 1% from the lower bound obtained in the initial ascent. For that reason, we branch to-theright until this l%-limit is exceeded; thereafter, that entry with the smallest lower bound is selected. In addition, we have in this way overcome some of the more serious computational disadvantages related to (a). Test runs have indicated that this particular combination o f selection rules makes IHK twice as fast as HK, on average. 2.5. Checking f o r a solution It has proved to be worthwhile to check for every 1-tree of minimum weight whether or not it is a feasible solution. This is easily done by means o f the vector v, Ivl = 0 indicating that the actual 1-tree constitutes a tour, since all vertices then are o f degree 2. If Lvl = 0 during the initial ascent, we are faced with an optimal solution to the given problem and no branching is required. When an r-edge is substituted for an edge excluded from a 1-tree (cf. Section 2.3), the same test is performed by IHK in order to limit the number of iterations. If a former 1-tree with lul > 0 by replacement of a single edge is converted to a tour, then the length of that tour must be an upper b o u n d on C*.

3. Computational experience In this section, our computational results with IHK are reported. First, Table 1 shows results corresponding to a sequence of problems, ranging in size from 20 up to 64 cities. Most of the problems have appeared in the open literature; for adequate references, the reader is referred to [5]. Furthermore, all problems have been solved by Held and Karp too, cf. [5, Tables 1 and 2]. Their results are, for the sake of comparison, repeated here in parentheses. The computing times corresponding to IHK and HK refer to IBM 360/75 and IBM 360/91, respectively, where the latter is claimed by IBM to be about 3 times faster than the first. The computing times are

K. Helbig Hansen, J. Krarup, Improvements of the Held-Karp algorithm

94

Table 1 Problems solved by HK and IHK Problem

n

t

IHK Hg IHK Croes

8 × 3 Knight's tour Held and Karp 10× 3 Knight's tour Tutte Held and Karp 8 × 6 Knight's tour Karp and Thompson 8 X 8 Knight's tour

w(~r) after initial ascent

C*

p

HK (20) (25) (20)

IHK HK 246 (246) > 0 (> 0) 1711 (1711)

Time (sec.)

IHK c HKd

20 24 25 30 46 48 48

4-I 1 8-1 1 1 32-2 1

(1) 6 (i) 25 a (1) 6 (1) 7 (1) 75 a (1) 8 (1) 8

(100) >0 b (15) 11461 (25) 0

0 (0) 11439(11439) 0 (0)

0.4 (4) 0.3 (0) 0.8 (12) 1.1 (20) 591 (900) 9.8 (84) 10 (160)

57 64

32-2 1

(1) 20 a (1) 10

(30)'12955 (50) 0

12905 (12906) 0 (0)

134 38

(25)

246 > 0b 1711

o

o

(0)

(780) (416)

a The "automatic" determination of p has been replaced by a manually selected value. b No solution exists, the underlying graph is not Hamiltonian. c IBM 360/75. d IBM 360/91.

not adjusted by a factor 3 in Table 1, but are those originally observed. Of course, the difference should be taken into account when the performance of IHK is appraised; otherwise we cannot assert that IHK tends to be some 25 times faster, as was done in the abstract. Where the value of the parameter t appears as, e.g., 3 2 - 2 it means that t has successively taken on the values 32, 16, 8, 4 and 2, in accordance with Section 2.1. Table 2 lists the results obtained for examples where the elements in the symmetric distance matrix have been chosen at random from a discrete uniform distribution over the interval (1, 1000). The number of cities goes up to 70; besides, a fev~ problems with 80 cities have been solved. The column headed by "w(Tr) after initial ascent ..." exhibits a significant minimum for n = 30. We have found no convincing explanation for this interesting phenomenon, but conjecture that the minimizing value of n depends on the choice of the parameters p arid t. Also noteworthy is the remarkable number of examples with w(rr) within the 1%-range. This should justify our l%-rule as was described in Section 2.4. We observe finally that IHK, like other branch-and-bound methods, manifests the well-known behaviour: The computing time increases ex-

K. HelbigHansen, £ Krarup, Improvements of the HeM-Karp algorithm

95

Table 2 Random examples

10 20 30 40 50 60 70 80

Number of examples

Solution in initial ascent (perc.)

wOO after initial ascent deviates less Time (sec.) than 1% from C* IBM 360/75 (perc.)

100 100 78 41 30 15 15 2

87 50 31 32 13 7 0 0

96 82 79 95 98 100 100 100

0.08 0.91 3.5 6.9 19 39 78 28.5 a

Standard deviation

0.07 1.07 3.7 7.6 16 29 44

a Only three test runs were performed in a batch with an upper bound on the accumulated computing time (3 minutes). Only two out of the three examples were solved within that amount of time.

ponentially with the dimension of the underlying problem. In this particular case, the computing time is seen to be doubled (approximately) as the number of cities is increased by 10. More details on various computational aspects of IHK can be found in [3], including in their entirety the search trees associated with the examples o f Table 1. Also a number of "ill-conditioned", Tutte-like examples are presented.

4. Concluding remarks Although the effect of several modifications has been elucidated, a number of questions still remain to be answered. The potential effect of solving the assignment problem for the matrix (cij), and applying the dual variables of the solution as starting point for the initial ascent was suggested by Held and Karp but has not been investigated. Also the numbering of the vertices (cities) has been done - in IHK - quite arbitrarily, disregarding the fact that vertex no. 1 takes up an exceptional position (1-trees). Whether or not a clever numbering will lead to some improvement is an open question. We are, on the other hand, convinced that the first part of IHK (the initial ascent) could be a useful alternative tool to those practitioners who can accept near-optimal solutions to large-scale problems. The time-

96

K. Helbig Hansen, J. Krarup, Improvements o f the Held-Karp algorithm

consuming part of IHK is the branch-and-bound procedure, whereas the computations in the initial ascent are performed rapidly, even for large problems. Furthermore, the initial ascent algorithm is easily programmed, and finally - as we have seen in the preceding section - the lower bound generated by the initial ascent appears in general to be very close to the length o f an optimal tour. Being part-time practitioners of Operations Research ourselves, we shall, as a suggestion for future research, express our hope in seeing the elegant approach by Held and Karp extended to cover more realistic situations than those which can be formulated in terms of symmetric traveling-salesman problems. By c o m m o n assent: The variety of real-world problems is enormous!

References [ 1 ] E.W. Dijkstra, "A note on two problems in connexion with graphs", Numerische Mathematik 1 (1959) 269-271. [2] M.H. van Emden, "Increasing the efficiency of quicksort", Algorithm 402, Communications o f the Association for Computing Machinery 13 (1970) 693-695. [3] K. Helbig Hansen, "Sekvensproblemer med vaegt p/l modifikation af Held og Karp's algoritme", M.Sc. thesis, Institute of Datalogy, University of Copenhagen, Denmark (1971). [4] M. Held and R.M. Karp, "The traveling-salesman problem and minimum spanning trees", Operations Research 18 (1970) 1138-1162. [5] M. Held and R.M. Karp, "The traveling-salesman problem and minimum spanning trees: Part II", Mathematical Programming 1 (1971) 6-25. [6] J.B. Kruskal, "On the shortest spanning subtree of a graph and the traveling-salesman problem", Proceedings of the American Mathematical Society 2 (1956) 48-50. [7] R.C. Prim, "Shortest connection networks and some generalizations", Bell System Technical Journal 36 (1957) 1389-1401.