Implementation of an Ant Colony Optimization Algorithm with Constraint Handling for Continuous and Mixed Variable Domains Ricardo M. Bernardo, Jr. and Prospero C. Naval, Jr. Computer Vision & Machine Intelligence Group University of the Philippines Diliman Diliman, Quezon City, Philippines
[email protected]
ABSTRACT
2.
The Ant Colony Optimization algorithm was originally conceptualized as an algorithm for solving combinatorial optimization problems. In recent years, it has shown that the ACO metaheuristic can be extended such that it can handle both continuous and mixed variable optimization problems. In this paper we present an constraint-capable extension of Socha and Dorigo’s ACOR and ACOM V .
Ant colony optimization is an algorithm to solve computational problems inspired by the characteristic of ant colonies to be a highly distributed system with a structured social organization. The communication of individual ants in a colony is aided by depositing pheromones, chemicals used to mark their paths in the ground (e.g., path from food source to the nest). Other ants can then follow paths to food sources found by other ants by sensing the pheromone trails. This indirect communication of individual ants mediated by modifications in their environment (stigmergy) is the concept that has inspired the design of ACO to solve problems [2]. ACO is a general algorithmic framework to solve different optimization problems wherein a colony of artificial ants cooperate to find good solutions to difficult discrete optimization problems.
1.
INTRODUCTION
The Ant Colony Optimization metaheuristic is commonly applied to handle combinatorial optimization problems. The motivation for the conceptualization of ACO is to find a method for solving the traveling salesman problem, one of the more frequently studied combinatorial optimization problems. This makes the structure of the algorithm easily adapted for and reformulated to handle combinatorial optimization problems. However, this characteristic of the ACO metaheuristic also makes it somewhat inflexible when it is extended to handle other types of optimization problems (i.e., continuous optimization problems). Compared to the applications of ACO to combinatorial optimization problems, there are fewer proposals on how to extend ACO in tackling continuous optimization problems. Some of the work in extending ACO to continuous optimization differ from the original ACO metaheuristic, while, only a few extend ACO without major conceptual changes in the structure of the metaheuristic [4]. Nevertheless, some implementations of ant colony optimization for continuous domains can compare to other algorithms in the same field. In addition, given the robustness of ACO applied to both discrete and continuous optimization, ACO may be extended further to handle both continuous and discrete variables simultaneously. In this paper, the concepts that are the basis for the implementation of an ant colony algorithm for solving problems defined over continuous and mixed-variable domains are presented. A discussion is also included on how ACO is extended to handle variables defined in the continuous and mixed domains and how the metaheuristic is used to solve multiobjective optimization problems that are discrete. Results to experiments performed after implementing ACOR and ACOM V as proposed by Socha and Dorigo [4, 3] are shown.
THE ACO METAHEURISTIC
With this formulation, ants build solutions by performing randomized walks on a completely connected graph Gc = (C, L), whose nodes are the components C and the set L of connections fully connects the components C The ACO metaheuristic consists of the three main procedures ConstructAntSolutions, UpdatePheromones, and DaemonActions [2]. In ConstructAntsSolutions, each ant in the colony visit adjacent states of the considered problem by moving through neighbor nodes of the problem’s construction graph GC . The ants move by applying a stochastic local decision policy that utilize pheromone trails and heuristic information in order to incrementally build solutions to the optimization problem. Once a solution is built, each ant will evaluate the (partial) solution that will be used by UpdatePheromones to determine how much pheromone to deposit. Each ant starts with an empty partial solution sp = {}, then, at each construction step, sp is extended by adding feasible components from the set of feasible neighbors N (sp ) ⊆ C. The next solution component to be added to the partial solution is done probabilistically at each construction step using the following rule
p(cij |sp ) = P
β α τij · ηij
β α cil ∈N (sp ) τil · ηil
, ∀cij ∈ N (sp )
(1)
Where τij is the amount of pheromone on the ij th component, ηij is the heuristic value associated with the ij th component and α and β determine the importance of the pheromone and heuristic values respectively. UpdatePheromones is the procedure that handles the modification and update of pheromone trails. Trail values increase as ants deposit pheromone on the components or connections they use, hence, increasing the probability that other ants will use these same components or connections in the future. On the other hand, trail values may also decrease by implementing a concept of pheromone evaporation. The implementation of pheromone evaporation helps to avoid premature convergence towards a suboptimal region and aids in exploring new areas in the search space.
τij ← (1 − ρ) · τij + ρ ·
X
F (s)
(2)
s∈Supd |cij ∈s
Where Supd is the set of chosen good solutions used for the update. ρ ∈ (0, 1] is a parameter evaporation rate. F (s) is commonly called a fitness function. Different ACO algorithms have their own ways of updating pheromone. DaemonActions is an optional procedure where one can centralize actions which cannot be performed individually by ants(e.g., observe the path found by each ant in the colony and select one or few ants which are allowed to deposit additional pheromone on components or connections they used).
3.
ACO FOR CONTINUOUS DOMAINS
Combinatorial optimization involves looking for optimal combinations or permutations of available components. This requires the problem to be partitioned into a finite set of components. In contrast, there exists another class of optimization problems that requires finding values for variables with a continuous domain. This class of optimization problems is also known as Continuous optimization. ACO has originally been designed to solve combinatorial optimization problems, however, there are also approaches proposed to apply ACO in tackling continuous optimization problems as well. Socha and Dorigo presented an extension of ACO applied to continuous domains. Their approach applied ACO to continuous optimization without major conceptual changes in the original metaheuristic structure. They have taken into account the central idea which is the incremental solution construction based on the biased (by pheromone) probabilistic choice of solution component [4]. In ACO applied to combinatorial optimization, the set of available solution components is dependent on how the problem was formulated. At each construction step of the solution construction process, the ants choose solution components based on the probabilistic rule in Equation 1 . The probabilities associated with the solution components that an ant samples make up a discrete probability distribution. In contrast, the proposed ACO for continuous optimization, ACOR , uses a continuous distribution for the probabilities associated with the solution components to be sampled. In-
stead of choosing solution components from a discrete set, we sample a Probability Density function (PDF) instead. Pheromones are represented in ACOR by keeping track of a certain number of solutions found by ants and storing them in a solution archive, T of size k. At the start of the algorithm, T initially contains k solutions generated by uniform random sampling. We denote the ith variable of the lth solution by sil . The components of the solutions in the archive will be used to generate the probability density functions which will determine what solution components will be chosen next in the solution construction process. ACOR uses a Gaussian kernel PDF, a weighted sum of several onedimensional Gaussian functions gli (x) denoted by
Gi (x) =
k X
ωl gli (x) =
l=1
k X l=1
ωl
− 1 √ e i σl 2π
2 (x−µi l) (2σ i )2 l
(3)
The Gaussian kernel Gi is parameterized by the vectors ω, µi , and σ i , all of which have cardinality k. ω is the vector of weights associated with each Gaussian function in the Gaussian kernel. µi is the vector of means, and σ i is the vector of standard deviations. The values of these parameters are based on the solutions stored in the archive T, and, therefore, determines the Gaussian kernel PDF that will be used to guide the ants in the solution construction process. For each dimension i = 1, ..., n of the problem, a different Gaussian kernel Gi is defined. The vector of means µi will be different for each Gaussian kernel PDF Gi . The elements of µi are the ith variable value of all solutions in the archive:
µi = {µi1 , ..., µik } = {si1 , ..., sik } The vector of weights ω = {ω1 , ..., ωl , ..., ωk } can be calculated after evaluating and ranking the solutions stored in T . The solutions are then sorted by rank with solution sl having rank l. The weight ωl is calculated based on
2
ωl =
(l−1) 1 − √ e 2q2 k2 qk 2π
(4)
The vector of standard deviations σ i = {σ i1 , ..., σ il , ..., σ ik } is dynamically calculated at each construction step of the solution construction process. To find the standard deviation σ il at construction step i, the average distance from the chosen solution sl to other solutions in the archive is determined and multiplied to the parameter ξ.
σ il = ξ
k X |sie | − sil k−1 e=1
(5)
ACOR requires the parameters k, q and ξ to be set. The first parameter, k, is the number of solutions that will be stored
in the solution archive T . Hence, k determines the complexity of the Gaussian kernel PDF to be used by the algorithm since there will be k separate Gaussian functions that will make up the Gaussian kernel PDF Gi . Another parameter, q, pertains to the locality of the search process done by ACOR . When q approaches 0, the best ranked solutions are strongly preferred in the iterative solution construction process. In other words, the Gaussian function associated with the best solution found so far is the only one being used to generate further solutions by the ants. When q is large, the probability of choosing solutions becomes more uniform and the algorithm samples the search space based on a larger number of reasonably good solutions. The parameter ξ > 0 determines the convergence speed of the algorithm. The algorithm will converge more slowly toward an area in the search space when ξ is assigned a higher value. This parameter has an effect similar to the pheromone evaporation rate in ACO. Following the ACO metaheuristic pseudocode in Figure 1, described next is the ACOR version of each major procedure. In AntBasedSolutionConstruction, an ant constructs a solution for the decision variable xi , i = 1, ..., n by performing n construction steps. At the ith construction step, the ant chooses a value for the decision variable xi by sampling the Gaussian kernel PDF Gi as described in Equation 3. Gi can be defined by solving for the values of the vectors µi , σ i , and ω as discussed previously. The sampling process can be performed in a more practical way. First, choose one of the Gaussian functions of the Gaussian kernel PDF. The probability pl of choosing the lth Gaussian function is
ωl p l = Pk
r=1
ωr
be extended in order to handle both continuous and discrete variables. ACOM V is an extension of ACOR with the ability to declare each variable of a considered problem as continuous, ordered discrete and categorical discrete. The use of a solution archive T to represent pheromone information, as well as the overall flow of ACOR remains unchanged. The only difference is the addition of a new routine to handle discrete variables and for constructing the solution for these variables. For optimization problems with defined ordered discrete variables, ACOM V employs a continuous-relaxation approach. Ordered discrete variables are discrete variables whose values’ natural ordering have nothing to do with the actual numerical values. Therefore, the same procedure of ACOR is applied, only, instead of operating on the actual numerical values of the ordered discrete variables, ACOM V operates on their indeces. In the solution construction process the values of the indeces for the new solutions are generated as real numbers. This is in contrast with ACOR where the actual real number values within the domain of the variable are generated. Therefore, before the objective function is evaluated, the continuous values of the variables are rounded to the nearest valid index, and the value at that index is then used to evaluate the objective function. When there categorical discrete variables are defined, values for these type of discrete variables need to be generated using a different method because the information about the ordering of the values in the domain may not be taken into consideration. As in ACOR , for each i = 1, ..., n discrete variable, probabilistically choose one of ci available values vli ∈ D = {v1i , ..., vci i } with the probability of choosing the lth value given by:
(6)
ωl oil = Pc
r=1
The process of sampling the chosen Gaussian function is the same as using a random number generator that is able to generate random numbers according to a normal distribution parametrized by the lt h elements of the mean vector µi and standard deviation vector σ i l. The whole process is performed for each dimension i = 1, ..., n. The PheromoneUpdate procedure is accomplished by performing an update on the solution archive T , the representation of pheromone information in ACOR . The set of newly generated solutions are added to the solution archive and the same number of worse ones removed. This ensures that the best solutions guide the ants in the search process. In DaemonActions, the best solution found is updated and returned when the termination condition is met.
3.1
Mixed Variable ACO for Continuous Domains
ACO has been successfully used to deal with discrete, as well as, continuous optimization problems. It should also follow, that, similar to some continuous optimization algorithms, it may be used to tackle mixed-variable optimization problems. According to Socha and Dorigo [3], ACOR may
ωr
(7)
Where ωl is the weight associated with the lth available value, and is defined by:
ωl =
ωjl q + η uil
(8)
Above, ωjl is calculated in the same way as calculating the weight of a solution in ACOR , with jl being the index of the highest quality solution that uses vli for the ith variable. uil is the number of solutions using vli for the ith variable in the archive. η is the number of values vli from the available ones that are unused by any of the solutions in the archive. Some available values vl may not be used by an ith decision variable in all the solutions in the archive leaving their initial weight to be zero. In such case, the final weights w are modified further by adding to all of them the second component. This enhances the exploration of other areas of the search space and preventing premature convergence.
3.2
Constraint Handling in ACO
The penalty-parameterless constraint handling approach proposed by Deb [1] for single objective optimization is adapted
in the implementation of the single objective ant colony optimization algorithm. The method uses a tournament selection based algorithm where two solutions are compared at a time. With the existence of constraints, either can be feasible or infeasible, therefore, yielding these three cases:
4.1.2
PheromoneUpdate
1. both solutions are feasible
Pheromone update corresponds to performing an update on the solutions stored in the solution archive T shown in figure 2. After each ant has found a solution, the set of newly generated solutions are added to the solution archive T and the same number of worse solutions are removed. This is done to ensure that the search process of ants are guided by the best solutions in the archive.
2. one is feasible and the other is not
4.1.3
3. both are infeasible
In this procedure, the algorithm updates the best solution found so far. When the termination condition is met, the best solution found so far is returned.
For single objective optimization, the following rules govern the above cases: 1. Choose the solution with a better objective value 2. Choose the feasible solution 3. Choose the solution with smaller overall constraint violation
4.
ACO IMPLEMENTATION
The ant colony optimization metaheuristic was initially implemented following the algorithm ACOR described by [4] and then extended based on ACOM V [3]. This gives the algorithm the capability to solve single objective problems with variables defined over continuous and mixed domains. Constraint handling is implemented following the method proposed by Deb [1].
4.1
ACO for Continuous Domains
The ant colony optimization metaheuristic was extended to solve single objective continuous optimization problems. The method proposed by Socha and Dorigo [4], ACOR was implemented. The original ACO metaheuristic outline in figure 1 was followed in implementing ACOR .
4.1.1
AntBasedSolutionConstruction
Each ant in the colony constructs a solution for the decision variables xi , i = 1, ..., n by performing n construction steps. At the ith construction step, an ant chooses a value for xi by sampling the selected PDF from the Gaussian Kernel. The selected PDF Gi from the Gaussian kernel can be defined by solving for the values of the vectors µi , σ i , and ω. A practical way to sample the selected PDF Gi from the Gaussian kernel was used. Probabilistically choose the lth Gaussian function from the Gaussian kernel using the rule described in Equation 6 After selecting which PDF Gi to select from the Gaussian kernel, the next step is to sample it. The process of sampling the chosen Gaussian function is the same as using a random number generator that is able to generate random numbers according to a normal distribution parametrized by the lth elements of the mean vector µi and standard deviation vector σ i . The whole process is performed for each dimension i = 1, ..., n.
4.2
DaemonActions
Mixed Variable ACO for Continuous Domains
The proposed method by Socha and Dorigo [3], ACOM V , is implemented to extend the single objective ACO for continuous domains in order to handle both continuous and discrete variables. This requires creating an additional routine to declare each variable of the problem as continuous, ordered discrete or categorical discrete and to construct the discrete part of the solutions.
4.2.1
Solution Construction for Ordered Discrete Variables
The Continuous Relaxation method was used when there are ordered discrete variables. The same solution construction procedure for ACOR is followed but instead of operating on the actual numerical values of the variables, ACOM V operates on their indeces. Hence, before evaluating the objective function, the continuous values of the variables are rounded to the nearest valid index, and the value at that index is then used to evaluate the objective function.
4.2.2
Solution Construction for Categorical Discrete Variables
When there exists categorical discrete variables in the problem, a different method for generating values for this type of variable will be used. For each i = 1, ..., n discrete variable, choose one of the ci available values vi ∈ Di = {v i1 , ..., v ici } probabilistically. The probability of choosing the lth value is given by the same equation in section 3.1.
4.3
ACO Constraint Handling for Continuous Domains
To give the algorithm a way to handle constraints, the constraint handling method for single objective optimization proposed by Deb [1] is adapted. Following the original formulation of ACO, the probabilistic solution construction process and the pheromone update routines are still used. In the case of ACO for continuous domains, the solutions are stored in the solution archive T . Iteratively, two solutions are picked from the archive of solutions and compared with each other, and in the presence of constraints, the cases mentioned in section 4.3 may be observed about the two solutions
Function 1 2 3 4 5
No. of ants 10 50 50 10 20
k 100 50 100 50 50
q 0.5 0.001 0.5 0.5 0.001
ξ 0.85 10 0.85 0.85 0.85
Table 1: Parameter values for testing ACOR
5.
EXPERIMENTS AND RESULTS
The implementation of the ACOR algorithm was tested on several single objective test functions. The following are the functions used to test the implementation of ACOR and the algorithm parameters used to solve the test functions are shown in table 1.
1. De Jong’s P Function 2 f (x) = n i=1 xi −5.12 ≤ xi ≤ 5.12 Global minimum f (x) = 0 2. Schwefel’s P Function p f (x) = n |xi |] i=1 [−xi sin −500 ≤ xi ≤ 500 Global minimum f (x) = −418.9829n 3. Rosenbrock’s Valley P 2 2 f (x) = n−1 i=1 [100(xi+1 − xi ) + (1 − xi ) ] −2.048 ≤ xi ≤ 2.048 Global minimum f (x) = 0 4. Rastrigin’s Function P √ 2 xi ] f (x) = 10n + n i=1 [xi − 10 cos −5.12 ≤ xi ≤ 5.12 Global minimun f (x) = 0 5. Easom’s Function f (x1 , x2 ) = − cos(x1 ) cos(x2 )exp(−(x1 − π)2 − (x2 − π)2 )) −100 ≤ x1 ≤ 100, −100 ≤ x2 ≤ 100 Global Minimum f (x) = −1.0
ACOR was run 25 times for each test function and the function error value (f (x) − f (x∗)) was computed. The best and worst function error values among the 25 runs after 1, 10, 25, 50 and 100 iterations of the algorithm were recorded. The mean and standard deviation of the function error values were also noted. Based on the results, we can verify that ACOR , indeed, performs well. Around the 10th iteration, ACOR has already found the optimal objective function value for the test functions of De Jong, Rastrigin and Easom (refer to tables 2, 5, and, 6). By the 25th iteration, it has already searched for the optimal objective function value for the test function for the test function of Rosenbrock (refer to table 4). It typically takes 100 iterations to find the objective function value for the Schwefel test function (refer to table 3). The implementation of ACOM V was also tested on the mixed variable test problem proposed by Socha and Dorigo [3]:
Function 1
No. of ants 2
k 200
q 0.5
ξ 0.85
Table 7: Parameter values for testing ACOM V t 2 4 8 11 13 15 16 18 20 22 32 38 50
Mean 4.49E-25 2.69923E-23 5.83209E-20 3.406E-18 2.39168E-20 2.37992E-19 2.27216E-19 6.09131E-21 4.66899E-20 1.12315E-22 3.89552E-20 1.72329E-18 7.04837E-19
SD 0 0 0 0 0 0 1.60663E-19 0 0 0 0 0 7.07703E-19
Table 8: Performance of ACOM V for problems with Continuous and Ordered Discrete Variables f (x1 , x2 ) = x21 + β · x22 , where x1 ∈ T, x2 ∈ (−3, 7) For different number of intervals t ∈ {2, 4, 8, 11, 13, 15, 16, 18, 20, 22, 32, 38, 50}, the performance of ACOM V was tested in the cases where, 1)variable x1 is declared as an ordered discrete variable, and, 2) x1 is declared as a categorical discrete variable. ACOM V was run for 3 times and the objective function value f (x) after 10 iterations was recorded. The performance of ACOM V in problems that have both continuous and ordered discrete variables and in problems that have both continuous and categorical discrete variables are comparable to each other when t < 13. When more elements of T are defined (t ≥ 13), the performance of ACOM V for problems with categorical discrete variables decreases.
t 2 4 8 11 13 15 16 18 20 22 32 38 50
Mean 9.35E-24 1.77412E-21 6.73129E-22 9.14995E-19 1.64578E-17 2.31123E-16 5.12326E-17 1.10509E-18 1.5675E-16 5.23581E-16 5.42081E-14 1.26191E-15 1.01413E-08
SD 1.29E-23 2.48373E-21 6.59336E-22 1.28606E-18 2.32719E-17 3.26846E-16 7.24512E-17 1.55573E-18 2.21646E-16 7.40425E-16 7.66617E-14 1.78404E-15 1.4342E-08
Table 9: Performance of ACOM V for problems with Continuous and Categorical Discrete Variables
Iteration 1 10 25 50 100
Best -6.00186E-24 0 0 0 0
Worst -2.36773E-22 0 0 0 0
Mean -7.02802E-23 0 0 0 0
SD 7.53631E-23 0 0 0 0
Table 2: Function Error Values(De Jong)
Iteration 1 10 25 50 100
Best -14.69872549 -4.459259506 -4.459259506 -0.345021082 -0.00094009
Worst -206.535482 -92.66965383 -51.02949741 -32.4171988 -0.077165977
Mean -131.0810746 -32.40762159 -20.51072959 -9.599555528 -0.020999899
SD 50.29643966 22.57802708 13.12844044 7.114358581 0.024998238
Table 3: Function Error Values(Schwefel)
Iteration 1 10 25 50 100
Best -0.006621728 -9.97143E-15 0 0 0
Worst -0.174146623 -2.63137E-08 -8.15666E-23 -9E-30 -9E-30
Mean -0.043910792 -2.04398E-09 -5.75358E-24 -1E-30 -1E-30
SD 0.0424059 5.58032E-09 1.74643E-23 2E-30 2E-30
Table 4: Function Error Values(Rosenbrock)
Iteration 1 10 25 50 100
Best -3.81618979 0 0 0 0
Worst -8.171711307 0 0 0 0
Mean -5.384177536 0 0 0 0
SD 2.090650328 0 0 0 0
Table 5: Function Error Values(Rastrigin)
Iteration 1 10 25 50 100
Best -2.1024E-08 0 0 0 0
Worst -3.43858E-05 0 0 0 0
Mean -3.24674E-06 0 0 0 0
SD 9.28094E-06 0 0 0 0
Table 6: Function Error Values(Easom)
6.
CONCLUSIONS
In this paper we presented constraint-capable extension of ACO variants that have been the basis for the implementation of an ACO algorithm for continuous and mixed variable domains. The ant colony optimization algorithm extensions ACOR and ACOM V proposed by Socha and Dorigo have been implemented and successfully tested to solve single objective continuous and mixed variable optimization problems found in literature. The results show that ACOR performs well, with the algorithm finding the optimal solution for some test functions even before the algorithm satisfies its termination condition. We can say the same for ACOM V as it also finds the optimal solutions for the mixed variable problem proposed by Socha and Dorigo.
7.
REFERENCES
[1] K. Deb. An efficient constraint handling method for genetic algorithms. Elsevier, 1998. [2] M. Dorigo and T. Stutzle. Ant Colony Optimization. The MIT Press, Cambridge, Massachusetts, 2004. [3] K. Socha. Ant colony optimization for continuous and mixed-variable domains, 2004. [4] K. Socha and M. Dorigo. Ant colony optimization for continuous domains. European Journal of Operational Research, 2006.
