J Heuristics (2008) 14: 571–585 DOI 10.1007/s10732-007-9054-y
Simple and fast surrogate constraint heuristics for the maximum independent set problem Bahram Alidaee · Gary Kochenberger · Haibo Wang
Received: 2 September 2006 / Revised: 17 January 2007 / Accepted: 19 January 2007 / Published online: 24 October 2007 © Springer Science+Business Media, LLC 2007
Abstract In a recent paper Glover (J. Heuristics 9:175–227, 2003) discussed a variety of surrogate constraint-based heuristics for solving optimization problems in graphs. The key ideas put forth in the paper were illustrated by giving specializations designed for certain covering and coloring problems. In particular, a family of methods designed for the maximum cardinality independent set problem was presented. In this paper we report on the efficiency and effectiveness of these methods based on considerable computational testing carried out on test problems from the literature as well as some new test problems. Keywords Maximum independent set · Surrogate constraints · Heuristics 1 Introduction Surrogate constraint strategies have been employed in a variety of ways in an effort to solve combinatorial optimization problems. Research involving surrogate constraints has produced important theoretical developments (see for instance, Greenberg and Pierskalla 1970, 1973; Glover 1975; Fréville and Plateau 1993) as well as practical computational results (as reported by Kochenberger et al. 1974; Glover and Kochenberger 1996; Lokketangen and Glover 1997; and Osorio et al. 2002). The recent paper B. Alidaee School of Business, University of Mississippi, Oxford, USA e-mail:
[email protected] G. Kochenberger () School of Business, University of Colorado at Denver, Denver, USA e-mail:
[email protected] H. Wang College of Business Administration, Texas A&M International University, Laredo, USA e-mail:
[email protected]
572
B. Alidaee et al.
by Glover (2003) presented a family of surrogate-based heuristics intended to quickly generate solutions to the maximum independent set problem (MIS). In this paper we report on the testing of these proposed methods on medium to large sized MIS test problems. We note that the literature contains accounts of sophisticated methods for solving MIS as illustrated by the work of Busygin et al. (2002) and the recent paper by Butenko and Trukhanov (2006). Our intent here is not to provide an extensive comparison with other methods but rather to test and report on the performance of the very simple and fast methods put forth in Glover’s paper. Extensive comparisons with other methods will be reported in future papers. In the sections below we review the key notions that underlie the methods tested followed by our computational results. We then conclude the paper with some summary remarks.
2 Definitions and basic notions The surrogate based methods tested here are slight variations of basic surrogate constructs specialized for the unique structure found in MIS problems. In the interest of continuity, we’ll adopt the same notation utilized by Glover (2003). We assume a graph G = (N, E), where N = {1, . . . , n} denotes the set of nodes of the graph and E denotes the set of edges. For each node i ∈ N , define EdgeStar(i) = {{i, j } ∈ E}, NodeStar(i) = {j : {i, j } ∈ E}, SizeStar(i) = |NodeStar(i)|(equivalently, |Edgestar(i)|). We also apply these definitions relative to an induced sub-graph G = (N , E ) of G, where N is a subset of N and E is the (largest) subset of E determined by N . Given a graph G = (N, E), a set of nodes with no edges between them is called independent set. The MIS problem seeks to find an independent set with largest number of nodes. The standard mathematical programming formulation for the maximum independent set problem associates a 0–1 integer variable xi with each node i ∈ N , where xi = 1 denotes that node i is chosen as an element of the independent set. This problem can be expressed as an integer program as follows. MIS:
Maximize xo = subject to
(xi : i ∈ N )
xi + xj ≤ 1, {i, j } ∈ E, xi binary, i ∈ N.
The basic surrogate-based approximation to MIS is obtained by summing the constraints to produce a surrogate constraint and then defining the surrogate optimization
Simple and fast surrogate constraint heuristics
573
problem: SUR:
Maximize xo = (xi : i ∈ N ) subject to (ai xi : i ∈ N ) ≤ ao , xi binary i ∈ N, where ao = |E| and ai = SizeStar(i).
The heuristic solution process proceeds as follows. The surrogate problem, SUR, is used to identify a variable to be set to one and the problem is then reduced by eliminating variables whose values are forced and by removing redundant constraints. A new (reduced) version of MIS is then formed, yielding an updated surrogate problem, and the process repeats until no additional nodes can be selected. Since all objective function coefficients are 1, the greedy choice rule reduces to selecting xr corresponding to ar = Min(ai ). In terms of the graph G the rule can be stated as follows. Choose r ∈ N , for setting xr = 1 by identifying Sizestar(r) = Min(SizeStar(i) : i ∈ N ). Many variations of this simple choice rule are possible and will be explored in the computational work reported later in the paper. The rule given above, however, is widely used in a variety of methods (as detailed in Glover’s paper 2003) and serves as a basis for the first method we test in this paper. Once a variable choice is made, the graph and the corresponding MIS model is reduced as follows: Setting xr = 1 in MIS forces xj = 0 for all variables xj appearing in constraints xr + xj ≤ 1. Such constraints are now redundant and can be excluded from further consideration. In the graph G this corresponds to dropping all the associated nodes j and their incident edges, thereby defining a new node set N and a new edge set E by N = N − NodeStar(r) − r, E = E − EdgeStar(EdgeStar(r)). The new (reduced) problem has exactly the same form as the original. By replacing G = (N, E) with the reduced graph G = (N , E ) and defining SizeStar (i) relative to G , for each node i ∈ NodeStar(NodeStar(r)) we have: NodeStar (i) = NodeStar(i) − NodeStar(NodeStar(r)) − r, EdgeStar (i) = EdgeStar(i) − EdgeStar(NodeStar(r)). These outcomes result from examining each j ∈ NodeStar(r) and then, for each i ∈ NodeStar(j ), removing node j from NodeStar(i) and removing the edge {i, j } from EdgeStar(i) (reducing SizeStar(i) by 1 for each such step). Thus in the surrogate constraint problem SC the coefficients are changed so that ai = ai − |NodeStar(i) ∩ NodeStar(r)|, ao = ao − SizeStar(NodeStar(r)).
574
B. Alidaee et al.
The problem can be further reduced by setting xj = 1 for all isolated nodes j ∈ N (all of whose constraints reduce to the form xj ≤ 1), and removing these nodes from N . 2.1 Methods tested in this study For the testing reported below, the simple surrogate-based method outlined above is referred to as greedy method #1 (or simply G1). Ten additional surrogate-based methods derived from Glover’s paper, slight variations of G1, were part of the testing reported here. The additional methods, referred to as G2–G11, differ from each other in one or more simple ways. By complementing variables, a MIS instance is transformed into an equivalent set covering (SC) problem. Some methods, like G1, work directly on the problem in the original MIS form. Others, like G2, apply to the equivalent model in SC form. Generally, a method’s number denotes its level of evolution from the base heuristic G1 with higher numbered methods typically adopting and enhancing various features of lowered numbered methods. In this sense, as we move from low numbered methods to higher number methods the level of sophistication (and expectation for performance) slightly rises, although all 11 are straightforward heuristics designed to quickly produce solutions. In some methods, like G1, greedy decisions are made setting node selection variables to 1. In others, like G2, decisions are made setting node selection variables to 0. Yet others make assignments of 1 or 0 depending on closeness to a threshold value, while other methods employ differing reduction methods. Finally, some methods make selection/assignment rules utilizing node degree information and adopting features of lower numbered methods. Table 1 gives a very brief description of the key features of the various methods. For complete details, the reader is referred to Glover’s paper. In order to be succinct, the table makes statements like “assigning 1 to nodes.” This should be interpreted as “assigning the value of 1 to the node selection variables.”
3 Computational experience To assess their relative effectiveness and efficiency, the 11 heuristics outlined in the previous section were applied to a set of 84 well-known test problems taken from the DIMACS testbed. These problems are given as graphs intended for the maximum clique problem. To test our heuristic rules we used the complement of these graphs and solved the equivalent MIS problem. The 11 methods were coded in FORTRAN and all runs made on a SGI Origin 2000 supercomputer at the University of Mississippi. To provide a means of comparison with a well-established heuristic for solving MIS problems, these 84 test problems were also solved using the GRASP code of Feo et al. (1994). This code is written in Fortran and is available via http://www.research.att.com/~mgcr/src/index.html. Results from this code were obtained using the recommended default parameters. While the GRASP procedure is much more sophisticated than the simple methods tested here, it is well established in the literature and serves our purpose of providing a benchmark of comparison.
Simple and fast surrogate constraint heuristics
575
Table 1 A description of rules Rule
Description
G1
Greedy rule assigning 1 to nodes: Considers the remaining MIS problem at each step and assigns a value of 1 to most promising node. That is, in each step after updating SUR, choose a variable xr with ar = Min(ai ) and assign value of 1; continue the process. (See Sect. 3.2.1 of Glover’s paper).
G2
Greedy rule assigning 0 to nodes: This rule concentrates on the set covering (SC) equivalent of the MIS problem. At each step it considers the remaining SC problem and assigns a value of 0 to the most promising node. That is, in each step after updating SUR (of MIS problem,) choose a variable xr with ar = Max(ai ) and assign value of 0; continue the process. (See Sect. 3.2.2 of Glover’s paper).
G3
A greedy rule assigning 1 or 0 to nodes: This rule combines G1 and G2. It considers the updated MIS and SC problems at the same time and at each step assigns a value of 1 or 0 to the most promising node depending on the most promising node is chosen from MIS or from SC problems, respectively. (See Sect. 4 of Glover’s paper).
G4
A greedy rule assigning 1 or 0 to nodes: Similar to G3 except at each step all calculations and graph updates are carried out with respect to the MIS formulation. (See Sect. 4 of Glover’s paper).
G5
Look-Ahead greedy rule assigning 1 to nodes: Considers the MIS problem and identifies a subset of nodes (based on node degree) from which the most promising node is chosen to receive a value of 1. The subset considered at each step includes all nodes based on Min(SizeStar(i)). (See Appendix 1 of Glover’s paper).
G6
Look-Ahead greedy rule assigning 0 to nodes: Similar to G5 except that the set of potential choices is made based on the remaining SC representation and the most promising node from this set is assigned a value of 0. (See Appendix 1 of Glover’s paper).
G7
A threshold greedy rule: Considers the MIS problem and at each step defines a threshold value for the upper bound of the surrogate weights. A subset of remaining nodes is defined and the most promising node from this set is picked and given a value of 1. The threshold value is chosen based on N ∗ = {i ∈ N : ai ≤ a ∗ } where a ∗ = Max(ai : a1 + · · · + ah−1 + ai ≤ ao ), and the ai coefficients are arranged in ascending order; ao is the updated value of right hand side for SUR. (See Appendix 1 of Glover’s paper).
G8
A simple reduction greedy rule: At each step the MIS problem is reduced to yield a new, smaller surrogate constraint problem. Additional constraints are added to the updated SUR problem and the method optimally solves a nested set of reduced problems. In the process nodes are successively identified and assigned a value of 1. (See Sect. 5.2 of Glover’s paper).
G9
A repeated reduction greedy rule: The repeated use of rule G8 by creating a set of candidate nodes from the nodes of surrogate problem and more fully exploiting them. In this rule feasibility is sometimes destroyed. (See Sect. 6 of Glover’s paper).
G10
A repeated reduction greedy rule: Similar to G9 except that a repair procedure is used in the algorithm to restore feasibility as needed. (See Sect. 6 of Glover’s paper).
G11
A greedy rule giving different weights to edges taking into account small and large degree nodes, then assigning value of 1 to nodes: G11 enhances G1 via a weighting scheme that assigns weights to each edge and then applies the criterion of G1 rule to choose the next node to receive a value of 1. Specifically, it considers the MIS problem and at any point a subset of nodes, S, e.g., p% of the remaining nodes with smallest SizeStar (i.e., degree) is chosen. If an edge is connected to a node in S a constant weight of wi = α is assigned to that edge and the rest of edges will be assigned a weight of wi = 1. The weights are used to create a surrogate constraint and then the criterion of Max(ai ) is used to choose the next node to be given a value of 1. In our testing we used values of p and α from the sets {2, 6, 10, 14, 24}, and {4, 8, 12, 20, 25} respectively. All combinations of the two parameters, a total of 25 combinations, were used in our testing. (See Appendix 2 of Glover’s paper).
576
B. Alidaee et al.
As indicated in Table 1, heuristic G11 involves two key parameters p and α. To test the application of this method, we initially experimented on some sample problems with a variety of combinations of p and α, each taken in the range from 2 up to 100. Our preliminary tested did not identify a “best” set of values for p and α. However, the testing did indicate that values greater than 25 generally did poorer than smaller values. In order to allow for diverse combinations of these parameters for the testing reported in this paper, we employed all values of p and α in the ranges {2, 6, 10, 14, 24}, and {4, 8, 12, 20, 25}, respectively. (i.e., 25 combinations). Thus, each problem was solved 25 times and the best solution along with the cumulative CPU time was recorded for each. The results of our testing are reported in Tables 2 and 3. Table 2 gives the objective function value each method produced on each problem and Table 3 reports the corresponding solution times. A review of the tables indicates that all 11 methods quickly produced high quality solutions across the collection of test problems. Considering all 84 problems, the average worst objective function value produced was 75% of optimal and the average best objective function value was 92% of optimal. For 45% of the 84 problems, one or more of the 11 heuristics quickly produced the optimal solution. Tables 2 and 3 also reveal noteworthy differences in the performances of the heuristics. Generally those methods based on assigning a value of 1 to nodes are preferred, both in terms of solution time and solution quality, to those that proceed by assigning a value of 0 to nodes. In large part, the lower CPU times result because setting a node selection variable to 1 allows edges and consequently more nodes will be dropped after each assignment. As expected, the most basic methods, G1-G4, are relatively fast and produce fairly high quality solutions. However, for the testing reported here, G5 (a simple look ahead rule) is generally the fastest of all the 11 methods and typically produces objective function values as good as or better than those given by methods G1–G4. Methods G6, G7, and G8 have larger run times than methods G1–G5 and seldom produce preferred objective function values. Similarly, methods G9 and G10 have run times orders of magnitude greater than those of G1–G5 yet rarely produce preferred objective function values. Overall the results shown in Tables 2 and 3 suggest that Methods G6–G10 are generally less attractive than methods G1–G5 in terms of quickly producing high quality solutions. Considering both solution quality and solution time, Tables 2 and 3 strongly suggest that the single best method (of the 11 heuristics tested) is G11. Method 11 gave the best overall performance, quickly matching the optimal solution on 38 of the 84 problems and producing on average an objective function value that was 92% of the optimal value. Method 11 also performed nicely compared to the benchmark offered by the wellestablished GRASP procedure. On 10 of the 84 test problems, G11 produced better solutions than those given by GRASP and for 41 problems the two procedures gave the same result. GRASP produced better quality solutions than G11 on the remaining problems. In all cases, G11 produced results in a fraction of the time required by GRASP. Moreover, for three of the 84 problems, GRASP was unable to give a solution within a limit of 30 hours of CPU time and on another problem was unable
Simple and fast surrogate constraint heuristics
577
Table 2 Results for DIMACS MIS problems: Objective function values G1
G2
G3
G4
G5
G6
G7
G8
G9
G10 G11 GRASP Optimal
Brock200_1.clq 18 18 18 18 19 18 19 16 19 18 21 Brock200_2.clq 9 8 11 9 9 8 8 7 9 8 10 Brock200_3.clq 11 11 11 11 13 11 11 10 12 12 14 Brock200_4.clq 14 14 14 14 14 14 13 13 15 15 16 Brock400_1.clq 21 21 22 21 22 21 22 19 22 19 24 Brock400_2.clq 22 19 22 22 22 19 22 20 23 20 24 Brock400_3.clq 22 19 23 22 23 19 21 20 20 16 24 Brock400_4.clq 22 20 22 22 22 20 22 18 23 19 24 Brock800_1.clq 18 17 18 18 18 17 17 15 18 19 19 Brock800_2.clq 17 15 17 17 17 15 18 14 19 14 20 Brock800_3.clq 17 16 18 17 17 16 17 14 18 16 20 Brock800_4.clq 17 14 18 17 18 14 17 14 17 14 19 MANN_a9.clq 16 16 16 16 16 16 16 16 15 15 16 MANN_a27.clq 125 125 125 125 125 125 125 125 125 125 125 MANN_a45clq 342 342 342 342 342 342 342 342 342 342 342 MANN_a81.clq 1096 1096 1096 1096 1096 1096 1096 1096 1096 1096 1096 c-fat200-1.clq 12 12 12 12 12 12 12 12 12 12 12 c-fat200-2.clq 24 24 24 24 24 24 24 24 24 24 24 c-fat200-5.clq 58 58 58 58 58 58 58 58 58 58 58 c-fat500-1.clq 14 14 14 14 14 14 14 14 14 14 14 c-fat500-2.clq 26 c-fat500-5.clq 64 c-fat500-10.clq 126 hamming10-2.clq 512 hamming10-4.clq 36 hamming6-2.clq 32 hamming6-4.clq 4 hamming8-2.clq 128 hamming8-4.clq 16 johnson16-2-4.clq 8 johnson32-2-4.clq 16 johnson8-2-4.clq 4 johnson8-4-4.clq 14 keller4.clq 11 keller5.clq 22 keller6.clq 48 p_hat300-1.clq 7 p_hat300-2.clq 25 p_hat300-3.clq 33 p_hat500-1.clq 8 p_hat500-2.clq 34 p_hat500-3.clq p_hat700-1.clq
44 8
26 64 126 512 32 32 4 128 16 8 16 4 14 8 15 33 7 24 30 7 34
26 64 126 512 38 32 4 128 16 8 16 4 14 11 22 49 7 25 33 8 34
26 64 126 512 36 32 4 128 16 8 16 4 14 11 22 48 7 25 33 8 34
26 64 126 512 36 32 4 128 16 8 16 4 14 7 15 31 7 25 34 8 34
26 64 126 512 32 32 4 128 16 8 16 4 14 8 15 33 7 24 30 7 34
26 64 126 512 36 32 4 128 16 8 16 4 14 8 22 46 7 25 34 8 34
26 64 126 512 32 32 4 128 16 8 16 4 14 8 17 37 7 23 30 6 29
26 64 126 512 36 32 4 128 16 8 16 4 14 11 21 44 8 25 33 9 33
26 64 126 512 30 32 4 128 16 8 16 4 14 9 20 39 7 23 30 9 29
26 64 126 512 35 32 4 128 16 8 16 4 14 11 25 45 8 25 36 9 36
44 9
44 10
44 8
46 8
44 9
46 8
42 7
46 9
44 7
49 9
21 10 14 15 25 25 25 25 21 20 21 21 16 126 335 1082 12 24 58 14
21 12 15 17 27 29 31 33 23 24 25 26 16 126 ≥345 ≥1100 12 24 58 14
26 64
8 25 36 9 36
26 64 126 512 ≥40 32 4 128 16 8 ≥16 4 14 11 27 ≥59 8 25 36 9 36
50 11
≥50 ≥11
a a
40 32 4 128 16 8 16 4 14 11 27 a
578
B. Alidaee et al.
Table 2 (Continued) G1 G2 G3 G4 G5 G6 G7 G8 G9 G10 G11 GRASP Optimal p_hat700-2.clq p_hat700-3.clq p_hat1000-1.clq p_hat1000-2.clq p_hat1000-3.clq p_hat1500-1.clq p_hat1500-2.clq p_hat1500-3.clq r100.5.clq r200.5.clq r300.5.clq r400.5.clq r500.5.clq san200_0.7_1.clq san200_0.7_2.clq san200_0.9_1.clq san200_0.9_2.clq san200_0.9_3.clq san400_0.5_1.clq
43 60 9 45 61 9 61 88 7 9 10 11 12 16 15 47 40 34 9
41 59 9 43 60 9 58 85 8 10 10 10 10 15 12 45 35 25 7
44 60 10 45 62 9 62 90 8 9 11 11 10 16 15 47 40 34 7
43 60 9 45 61 10 61 88 7 9 10 11 12 16 15 47 40 34 9
43 59 9 45 61 10 62 86 8 11 11 10 11 17 14 48 41 34 9
41 59 9 43 60 9 58 85 8 10 10 10 10 15 12 45 35 25 7
43 58 9 43 62 10 62 85 7 9 9 10 11 16 15 48 39 34 9
38 55 7 39 57 8 54 75 8 9 9 10 9 16 12 45 37 31 7
43 61 10 44 64 11 64 86 8 10 10 12 10 16 12 46 40 33 8
37 53 9 38 58 8 51 72 7 11 8 10 9 16 12 47 40 32 7
44 62 10 46 65 11 64 91 9 11 11 13 13 17 15 48 41 37 13
44 62 10 46 68 11 65 94 9 11 12 13 13 29 15 70 60 44 9
44 ≥62 10 ≥46 ≥65 12 ≥63 ≥94 9 11 12 13 13 30 ≥18 70 60 44 13
san400_0.7_1.clq san400_0.7_2.clq san400_0.7_3.clq san400_0.9_1.clq san1000 sanr200_0.7.clq sanr200_0.9.clq sanr400_0.5.clq sanr400_0.7.clq C125.9.cliq C250.9.cliq C500.9.cliq C2000.5.cliq C2000.9.cliq C4000.5.cliq gen200_p0.9_44.cliq gen200_p0.9_55.cliq gen400_p0.9_55.cliq gen400_p0.9_65.cliq gen400_p0.9_75.cliq DSJC500.5.cliq DSJC1000.5.cliq
21 18 15 75 10 17 39 12 19 33 39 52 12 66 14 35 39 47 45 47 11 12
20 15 12 50 8 14 38 10 17 33 35 46 11 61 11 29 38 33 33 40 9 11
20 18 15 75 8 17 39 12 19 33 39 52 13 66 14 35 39 47 45 47 11 14
21 18 15 75 10 17 39 12 19 33 39 52 12 66 14 35 39 47 45 47 11 12
21 17 16 82 9 15 40 12 18 33 39 49 13 67 14 37 38 49 45 47 11 13
20 15 12 50 8 14 38 10 17 33 35 46 11 61 11 29 38 33 33 40 9 11
21 16 15 83 10 15 40 11 18 33 42 49 13 67 14 37 39 48 44 47 11 13
21 15 12 42 8 14 36 10 16 29 35 43 10 56 12 32 36 44 40 45 8 10
21 17 16 52 8 18 38 12 19 31 40 48 13 62 16 35 39 47 44 48 12 12
17 17 15 46 8 15 39 12 17 31 40 49 10 57 11 35 39 44 42 48 12 11
22 30 17 93 10 18 41 12 21 34 42 54 15 71 16 38 48 50 46 50 12 13
21 30 18 100 9 18 42 13 21 34 44 56 16 75
40 30 22 100 15 18 ≥42 13 ≥21 34 ≥44 ≥57 ≥16 ≥75 ≥17 44 55 55 65 75 ≥13 ≥15
a Given 30 hours of super computer, no solution returned b Problem required more memory than available
b
40 55 53 64 74 12 14
200
200
200
200
400
400
400
400
800
800
800
800
45
378
1035
3321
200
200
200
500
500
500
500
Brock200_1.clq
Brock200_2.clq
Brock200_3.clq
Brock200_4.clq
Brock400_1.clq
Brock400_2.clq
Brock400_3.clq
Brock400_4.clq
Brock800_1.clq
Brock800_2.clq
Brock800_3.clq
Brock800_4.clq
MANN_a9.clq
MANN_a27.clq
MANN_a45clq
MANN_a81.clq
c-fat200-1.clq
c-fat200-2.clq
c-fat200-5.clq
c-fat500-1.clq
c-fat500-2.clq
c-fat500-5.clq
c-fat500-10.clq
Node
46627
23191
9139
4459
8473
3235
1534
5506380
533115
70551
918
207643
207333
208166
207505
59765
59681
59786
59723
13089
12048
9876
14834
Edge
0.06
0.06
0.06
0.06
0.01
0.01
0.01
3.62
0.33
0.05
0
0.16
0.16
0.16
0.16
0.04
0.04
0.04
0.04
0.01
0.01
0.01
0.01
G1
0.08
0.08
0.08
0.08
0.01
0.01
0.01
3.13
0.31
0.04
0
0.23
0.23
0.23
0.23
0.06
0.06
0.06
0.06
0.01
0.01
0.01
0.01
G2
0.08
0.09
0.09
0.09
0.01
0.01
0.02
3.89
0.38
0.05
0
0.18
0.18
0.18
0.18
0.05
0.05
0.05
0.05
0.01
0.01
0.01
0.01
G3
Table 3 Results for DIMACS MIS problems: CPU times (in seconds)
0.08
0.08
0.09
0.09
0.01
0.01
0.01
3.89
0.39
0.05
0
0.18
0.18
0.18
0.18
0.05
0.05
0.05
0.05
0.01
0.01
0.01
0.01
G4 0
0
0
0
0.05
0.03
0.06
0.04
0.01
0
0.01
76.54
4.27
0.35
0
0.04
0.03
0.04
0.04
0.01
0.01
0.01
0.01
G5
0.41
0.25
0.16
0.13
0.03
0.03
0.02
278.37
8.99
0.51
0
0.33
0.33
0.33
0.33
0.08
0.08
0.08
0.09
0.02
0.02
0.02
0.02
G6
0.13
0.12
0.13
0.12
0.02
0.02
0.02
21.25
1.41
0.13
0
0.29
0.66
0.28
0.68
0.07
0.07
0.07
0.07
0.02
0.02
0.02
0.02
G7
0.09
0.09
0.06
0.08
0.02
0.02
0.01
0.31
0.04
0.01
0
0.25
0.24
0.24
0.25
0.06
0.06
0.06
0.06
0.02
0.01
0.01
0.01
G8 1.42
8.26
0.01
58.26
58.78
60.72
59.27
11.1
10.08
10.25
11.09
1.05
1.03
0.78
54.39
14.87
8.28
7.7
2.52
0.71
0.55
11794.95
217.81
G9
126.3
34.63
13.18
10.63
5.65
1.37
0.83
713
1763.93
46.34
0.03
92.05
90.77
94.03
87.42
15.94
14.86
16.28
16.64
1.8
1.61
1.14
2.43
G10
1.5
1.5
1.5
1.5
0.25
0.25
0.25
544.75
25
2.75
0
4.5
4.5
4.5
4.5
1.25
1.25
1.25
1.25
0.25
0.25
0.25
0.25
G11
a
39824.324
13699.123
9619.696
8507.068
2997.773
1672.509
5158.483
2399.003
1264.165
47.179
7932.228
8072.638
8079.736
7997.388
1861.186
1720.095
1877.757
1819.941
65.907
656.465
694.215
469.351
GRASP
Simple and fast surrogate constraint heuristics 579
300
300
500
500
500
700
700
700
1000
p_hat300-2.clq
p_hat300-3.clq
p_hat500-1.clq
p_hat500-2.clq
p_hat500-3.clq
p_hat700-1.clq
p_hat700-2.clq
p_hat700-3.clq
p_hat1000-1.clq
300
p_hat300-1.clq
3361
70
johnson8-4-4.clq
keller6.clq
28
johnson8-2-4.clq
776
496
johnson32-2-4.clq
171
120
johnson16-2-4.clq
keller5.clq
256
hamming8-4.clq
keller4.clq
64
256
64
hamming6-2.clq
hamming8-2.clq
1024
hamming6-4.clq
1024
hamming10-4.clq
Node
hamming10-2.clq
Table 3 (Continued)
122253
183010
121728
60999
93800
62946
31569
33390
21928
10933
4619898
225990
9435
1855
210
107880
5460
20864
31616
704
1824
434176
518656
Edge
0.23
0.12
0.12
0.11
0.06
0.06
0.06
0.02
0.02
0.02
2.57
0.14
0.01
0
0
0.06
0
0.02
0.02
0
0
0.27
0.32
G1
0.34
0.16
0.16
0.16
0.08
0.08
0.08
0.03
0.03
0.03
3.64
0.2
0.01
0
0
0.08
0.01
0.02
0.02
0
0
0.39
0.32
G2
0.37
0.14
0.13
0.18
0.07
0.07
0.09
0.03
0.02
0.03
2.85
0.16
0.01
0
0
0.07
0
0.02
0.02
0
0
0.31
0.38
G3
0.31
0.14
0.13
0.15
0.07
0.07
0.08
0.03
0.02
0.03
2.85
0.16
0.01
0
0
0.07
0
0.02
0.02
0
0
0.31
0.37
G4
0.04
0.05
0.04
0.02
0.02
0.02
0.01
0.01
0.01
0
0.93
0.08
0.01
0
0
0.47
0.02
0.04
0.09
0
0
0.65
1.74
G5
0.45
0.22
0.21
0.21
0.11
0.1
0.11
0.04
0.04
0.04
8.19
0.44
0.02
0.01
0
1.05
0.03
0.11
0.22
0
0.01
3.85
11.19
G6
0.42
0.22
0.22
0.2
0.11
0.11
0.1
0.04
0.04
0.04
4.36
0.62
0.02
0
0
0.48
0.02
0.05
0.06
0
0
0.7
0.87
G7
0.38
0.19
0.18
0.18
0.1
0.1
0.09
0.03
0.03
0.03
3.73
0.19
0.01
0
0
0.01
0
0
0.01
0
0
0.05
0.11
G8
70.62
84.63
50.8
24.64
22.87
20.69
9.22
6.03
4.38
2.03
8647.56
72.62
0.57
0.03
0
4.29
0.09
1.25
17.01
0.02
0.07
196.07
4496.84
G9
102.55
151.71
84.74
35.43
48.55
33.28
13.27
11.17
7.08
2.99
10395.38
102.81
0.95
0.06
0
13.12
0.18
1.97
27.64
0.03
0.15
257.68
6486.5
G10
6.25
4
3.5
3
2.25
2
1.5
0.75
0.75
0.5
73.25
4
0.25
0
0
0.75
0
0.5
1.25
0
0
8.25
31.5
G11
24181.509
8536.552
12535.506
12099.65
4024.715
6398.371
6178.502
1538.544
2133.733
2286.241
a
6016.179
426.129
119.893
20.25
1314.157
156.443
1255.052
3447.065
102.897
202.14
8352.997
a
GRASP
580 B. Alidaee et al.
200
200
200
200
200
400
400
400
san200_0.7_1.clq
san200_0.7_2.clq
san200_0.9_1.clq
san200_0.9_2.clq
san200_0.9_3.clq
san400_0.5_1.clq
san400_0.7_1.clq
san400_0.7_2.clq
200
500
r500.5.clq
sanr200_0.9.clq
400
r400.5.clq
200
300
r300.5.clq
1000
200
r200.5.clq
sanr200_0.7.clq
100
r100.5.clq
san1000
1500
p_hat1500-3.clq
400
1500
p_hat1500-2.clq
400
1500
p_hat1500-1.clq
san400_0.9_1.clq
1000
san400_0.7_3.clq
1000
p_hat1000-3.clq
Node
p_hat1000-2.clq
Table 3 (Continued)
17863
13868
250500
71820
55860
55860
55860
39900
17910
17910
17910
13930
13930
62589
39739
22489
9864
2442
847244
568960
284923
371746
244799
Edge
0.01
0.01
0.23
0.04
0.04
0.04
0.04
0.04
0.01
0.01
0.01
0.01
0.01
0.06
0.04
0.02
0.01
0
0.53
0.53
0.51
0.24
0.24
G1
0.01
0.01
0.33
0.05
0.05
0.05
0.05
0.05
0.01
0.01
0.01
0.01
0.01
0.08
0.06
0.03
0.01
0
0.74
0.74
0.75
0.33
0.34
G2
0.01
0.01
0.27
0.05
0.04
0.05
0.05
0.05
0.01
0.01
0.01
0.01
0.01
0.07
0.04
0.02
0.01
0
0.61
0.6
0.81
0.28
0.27
G3
0.01
0.01
0.26
0.05
0.05
0.05
0.05
0.04
0.01
0.01
0.01
0.01
0.01
0.07
0.04
0.02
0.01
0
0.6
0.6
0.69
0.28
0.27
G4
0.01
0
0.06
0.05
0.01
0.01
0.02
0.01
0.01
0.01
0.02
0
0
0.01
0.01
0
0
0
0.2
0.15
0.08
0.09
0.07
G5
0.02
0.02
2.99
0.25
0.28
0.25
0.24
0.29
0.04
0.04
0.05
0.06
0.05
0.11
0.08
0.04
0.02
0
1.03
0.96
0.98
0.47
0.43
G6
0.02
0.02
0.41
0.08
0.07
0.07
0.08
0.07
0.02
0.02
0.02
0.02
0.02
0.26
0.07
0.09
0.02
0
1
0.98
0.93
0.43
0.45
G7
0.01
0.02
0.37
0.06
0.06
0.06
0.06
0.06
0.01
0.02
0.01
0.02
0.02
0.09
0.06
0.03
0.02
0
0.86
0.85
0.85
0.38
0.39
G8
3.52
1.33
38.07
27.56
8.94
8.94
10.78
8.52
3.48
3.6
3.93
0.71
1.69
10.96
5.64
2.37
0.77
0.11
558.08
456.46
220.45
190.03
141.16
G9
6.05
2.13
115.65
40.88
13.44
13.97
15.35
13.66
5.48
5.94
6.28
2.09
2.7
16.42
8.01
3.52
1.17
0.18
1166.39
816.19
336.72
434.83
196.73
G10
8
7
0.5
0.25
6.25
2
1.25
1.25
1.25
1
0.5
0.5
0.5
0.25
0.25
1.5
1
0.5
0.25
0
18
16
13.75
G11
484.065
545.544
15541.5
3835.371
17376.14
1574.768
1574.768
25075.52
468.045
463.091
522.619
411.633
493.73
4566.028
2894.921
1580.379
782.107
199.024
41344.599
55430.044
52939.693
16507.981
22481.761
GRASP
Simple and fast surrogate constraint heuristics 581
1000
DSJC1000.5.cliq
244755
62126
8778
7416
7182
1992
1831
3997732
199902
979510
25418
2801
787
55869
39984
Edge
0.23
0.06
0.04
0.04
0.04
0.01
0.01
3.82
0.95
0.93
0.06
0.02
0
0.04
0.04
G1
b Problem required more memory than available
a Given 30 hours of super computer, no solution returned
400
500
400
gen400_p0.9_65.cliq
DSJC500.5.cliq
400
gen400_p0.9_55.cliq
gen400_p0.9_75.cliq
200
gen200_p0.9_55.cliq
4000
C4000.5.cliq
200
2000
C2000.9.cliq
gen200_p0.9_44.cliq
500
250
C250.9.cliq
2000
125
C125.9.cliq
C2000.5.cliq
400
C500.9.cliq
400
sanr400_0.7.clq
Node
sanr400_0.5.clq
Table 3 (Continued)
0.34
0.08
0.05
0.05
0.05
0.01
0.01
5.58
1.35
1.36
0.08
0.02
0.01
0.06
0.06
G2
0.26
0.07
0.05
0.05
0.05
0.01
0.01
4.39
1.09
1.07
0.07
0.02
0.01
0.05
0.04
G3
0.26
0.07
0.05
0.05
0.05
0.01
0.01
4.39
1.08
1.06
0.07
0.02
0.01
0.05
0.05
G4
0.04
0.01
0.02
0.02
0.03
0.01
0.01
0.7
0.23
0.15
0.03
0.01
0
0.01
0.01
G5
0.46
0.11
0.15
0.14
0.13
0.02
0.03
7.75
2.9
1.86
0.15
0.04
0.01
0.08
0.08
G6
0.41
0.25
0.08
0.08
0.08
0.02
0.02
6.87
1.77
1.65
0.12
0.03
0.01
0.07
0.06
G7
0.37
0.09
0.06
0.06
0.06
0.01
0.02
6.17
1.55
1.5
0.1
0.02
0.01
0.06
0.06
G8
5.76
1.14
9.84
6.54
90.94
11.16
22.29
21.18
22.1
3.42
3.07
5797.61
2210.8
696.58
37.76
G9
128.2
16.02
43.04
38.2
41.77
6.31
5.7
8150.25
4016.99
973.51
67.96
9.5
1.88
14.08
9.89
G10
1
6.25
1.5
1.75
1.5
1.75
0.5
0.5
102
31.25
25.5
2.25
0.75
0.25
1.25
G11
16033.628
4193.963
1626.48
1762.65
1692.85
440.362
442.723
b
22445.422
65829.697
2717.97
698.208
175.967
2124.729
2751.203
GRASP
582 B. Alidaee et al.
Simple and fast surrogate constraint heuristics
583
to return a solution due to an out of memory error. These four problems were quickly and easily solved by G11. Across all 80 problems for which GRASP was able to terminate naturally with a solution, the solution value given by G11 was an average of 96% of that given by GRASP and on average the G11 result was obtained in 0.2% of the time required by GRASP. To provide additional computational evidence of the performance of these surrogate-based heuristics on larger test problems, three of the most basic methods that performed well in Tables 2 and 3 (G1, G2, and G5) and the best performing method, G11, were applied to 22 new test problems. These additional problems are generally too large to be solved by GRASP. Using a problem generator (available from the authors) and the portable random number generator of Pardalos and Rodgers (1990), new test problems with varying characteristics were generated and solved. The results of this additional testing are shown below in Table 4. Note that these new problems ranged in size from 2000 to 12500 nodes with edges counts ranging from roughly 200,000 to more than 70 million. Densities for these problems ranged from 10% to 90%. Note also that Table 4 gives the seed used for each problem so that these exact problems can be reproduced by others. The relative pattern of performance displayed in Table 4 is similar to that of Tables 2 and 3. Method G2 generally lagged behind the others in performance as G1 and G5 quickly produced preferred solutions. As we saw previously in Table 3, G5 is extraordinarily fast requiring less that 7 seconds of CPU time for its most time consuming problem of Table 4. Heuristic G11 dominated the other methods in terms of solution quality. Considering the size of these problems, it is fair to say that G11 was also reasonably efficient. All told, G11 stands out once again as the top performing method. Note that all these methods are quite robust and scale up nicely with increasing node, edge and density count. Over the range of densities, for instance, G11 takes roughly 25 seconds for 2000 node problems, 600 seconds for 10,000 node problems, and 950 seconds for 12,500 node problems.
4 Summary and conclusions In this paper we reported on the preliminary computational testing of 11 new surrogate—based heuristics designed to quickly generate solutions for the maximum independent set problem. While all 11 methods performed reasonably well, two methods stand out as being particularly attractive based on the 106 test problems considered in this study. G5, a simple look ahead method, very quickly produced good solutions to even the largest of test problems. G11, while not as fast as G5, is nonetheless very efficient and produces, across the board, the highest quality solutions. Considering both solution quality and CPU time, G11, a greedy rule influenced by differential edge weights and node degrees, delivers the best overall performance. Its performance relative to that of GRASP is particularly noteworthy.
584
B. Alidaee et al.
Table 4 Results for randomly generated MIS problems. CPU times are in seconds n
Seed
Density %
No.edges
G1
G2
G5
G11
Ind.Set (cpu)
Ind.Set (cpu)
Ind.Set (cpu)
Ind.Set (cpu)
2000
12345
10
200093
66 (.96)
54 (1.35)
67 (.25)
72 (26.60)
2000
12345
20
398789
35 (.94)
31 (1.36)
35 (.18)
39 (27.75)
2000
12345
30
598540
24 (.94)
21 (1.36)
24 (.17)
26 (24.75)
2000
12345
35
698897
20 (.94)
17 (1.36)
21 (.16)
23 (24.42)
2000
12345
50
998668
13 (.93)
11 (1.36)
13 (.16)
15 (24.15)
2000
12345
70
1399174
8 (.93)
8 (1.36)
9 (.15)
10 (24.10)
2000
12345
90
1799196
5 (.93)
5 (1.36)
5 (.15)
6 (23.95)
10000
123456
15
7495907
57 (23.27)
48 (33.91)
58 (3.85)
62 (605.25)
10000
123456
25
12496911
37 (23.15)
30 (33.79)
35 (3.71)
37 (597.25)
10000
345678
25
12502489
36 (23.18)
31 (33.85)
35 (3.69)
37 (598.25)
10000
12
50
24997023
16 (23.24)
13 (33.99)
16 (3.62)
17 (598.25)
10000
123456
50
25000916
15 (23.16)
14 (33.90)
16 (3.62)
18 (597.25)
10000
123456
70
34999161
10 (23.16)
9 (33.81)
10 (3.59)
11 (593.00)
10000
123456
90
44995286
6 (23.11)
5 (33.85)
6 (3.59)
7 (591.25)
10000
345678
90
44996679
5 (23.18)
5 (33.96)
5 (3.57)
6 (593.25)
12500
124
5
3906256
159 (35.63)
140 (51.48)
167 (6.40)
169 (980)
12500
6543
10
7811391
85 (35.43)
78 (51.54)
85 (5.23)
92 (930.05)
12500
23456
20
15619763
45 (35.43)
38 (51.72)
44 (4.95)
47 (914.25)
12500
56789
25
19530502
37 (35.43)
32 (51.78)
35 (4.91)
38 (913.75)
12500
321
50
39060869
17 (35.97)
14 (51.62)
17 (4.74)
17 (905)
12500
654321
70
54682092
10 (35.52)
9 (51.62)
10 (4.74)
11 (904.75)
12500
21
90
70305877
6 (35.48)
5 (51.85)
6 (4.76)
7 (907.5)
The results shown in Tables 2, 3, and 4 make a strong case for the effectiveness of simple heuristics based on surrogate constructs for quickly producing high quality solutions. Solutions to problems with more than 12 thousand nodes and 70 million edges were found in less than 5 seconds by G5 and 910 seconds by G11. Such results can be taken as solutions to be implemented and/or high quality starting points for more sophisticated methods. In our continuing work, we intend to carry out further testing of the most promising methods in an effort to gain additional insight into their performance capabilities and to identify opportunities for enhancing performance. In the context of method 11, we are particularly interested in learning more about the most appropriate settings for p and α. In addition, we intend to investigate the impact of jump starting more advanced methods with starting solutions generated by the fast heuristics tested here. Finally, we intend to extend our testing to include weighted instances of MIS. Acknowledgement The authors would like to thank Mauricio Resende for making the GRASP code for MIS available to the public.
Simple and fast surrogate constraint heuristics
585
References Busygin, S., Butenko, S., Pardalos, P.M.: A heuristic for the maximum independent set problem. J. Comb. Optim. 6, 287–297 (2002) Butenko, S., Trukhanov, S.: Using critical set for the maximum independent set problem. Working paper, Texas A&M University (2006) Feo, T., Resende, A., Mauricio, G.C., Smith, S.H.: A greedy randomized adaptive search procedure for maximum independent set. Oper. Res. 42(5), 860–878 (1994) Fréville, A., Plateau, G.: An exact search for the solution of the surrogate dual of the 0–1 bidimensional knapsack problem. Eur. J. Oper. Res. 68, 413–421 (1993) Glover, F.: Surrogate constraint duality in mathematical programming. Oper. Res. 23(3), 434–451 (1975) Glover, F.: Tutorial on surrogate constraint approaches for optimization in graphs. J. Heuristics 9, 175–227 (2003) Glover, F., Kochenberger, G.: Critical event tabu search for multidimensional knapsack problems. In: Osman, J.H., Kelly, J.P. (eds.) Meta-Heuristics: Theory & Applications, pp. 407–427. Kluwer Academic, Dordrecht (1996) Greenberg, H.J., Pierskalla, W.P.: Surrogate mathematical programs. Oper. Res. 18, 924–939 (1970) Greenberg, H.J., Pierskalla, W.P.: Quasi-conjugate functions and surrogate duality. Cah. Cent. Etudes Rech. Oper. 15, 437–448 (1973) Kochenberger, G.A., McCarl, B.A., Wyman, F.P.: A heuristic for general integer programming. Decis. Sci. 5(1), 36 (1974) Lokketangen, A., Glover, F.: Surrogate constraint analysis—new heuristics and learning schemes for satisfiability problems. DIMACS Ser. Discrete Math. Theor. Comput. Sci. 35, 537–572 (1997) Osorio, M., Glover, F., Hammer, P.: Cutting and surrogate constraint analysis for improved multidimensional knapsack solutions. Ann. Oper. Res. 117, 71–93 (2002) Pardalos, P., Rodgers, G.: Computational aspects of a branch and bound algorithm for quadratic zero-one programming. Computing 45, 131–144 (1990)
