An Evolutionary Hill Climbing Algorithm for Dynamic Optimisation Problems Ayad Mashaan Turky • Salwani Abdullah • Barry McCollum and Nasser R. Sabar
1
Introduction In contrast to stationary problems where the problem parameters are static, dynamic optimisation problems (DOPs) present a challenging research area to the community [1]. This is mainly because, changes can occur any time during the optimisation course. Hence, when such a change happens (for example a change in the fitness landscape), the previous local optimal solution is no longer be as it is [2,3]. It is widely known that local search algorithms usually face a serious challenge for dynamic optimisation problems because they operate on a single solution which can easily get stuck in a local optima [2]. In order to overcome this shortage, local search algorithms should be re-started from scratch. Although, it seems a reasonable solution, restarting the algorithm from scratch will be time consuming as it may lose some promising areas. A promising trend to solve this issue is to hybridise a local search with a population-based method such as a genetic algorithm [2] [3]. Literature shows that population-based methods have been widely and successfully used to tackle dynamic optimisation problems (DOPs). The work in this paper is motivated by the success of the above mentioned methods. Thus, we propose the evolutionary hill climbing algorithm which combines the population of solutions with the hill climbing local search for dynamic optimisation problems. Our aim here is to use a population of solutions (instead of a single solution) in order to keep track of any changes during the search process, since the population of solution are scattered over the entire search space and consequently will increase the diversification of the algorithm. A hill climbing algorithm is applied to refine the promising solution generated by the population method. Once the solution cannot be improved for a certain number of iterations, then the hill climbing algorithm will restart the search by selecting a new solution from the population (instead of generating a new one). This idea will not only reduce the computation time, but will also help the search to focus on a promising point, as they already been improved previously and been modified by the population-based method. The performance of the proposed method is validated over the wellknown dynamic optimisation functions i.e., OneMax, Plateau, Royal Road and Deceptive. Experimental results show that the proposed algorithm is able to achieve competitive results when compared to other available methods. The rest of this paper is outlined as follows: Section 2 describes the proposed method. The experimental results and relevant discussion are presented in Section 3. Section 4 concludes the work presented in this paper. 2
Proposed Method In this section, we present the proposed evolutionary hill climbing algorithm (coded as Pop-HC), which combine the population of solutions with the hill climbing local search for solving dynamic optimization problems, as depicted in Fig. 1. The proposed method consists Ayad Mashaan Turky Data Mining and Optimization Research Group (DMO), Center for Artificial Intelligence Technology, Universiti Kebangsaan Malaysia E-mail:
[email protected]
of two phases. In the first phase, an evolutionary process is used to perturb the generated solutions. The second phase uses a hill climbing algorithm to improve the solution generated by the evolutionary process. These two phases are successively invoked until a stopping condition is satisfied (e.g. number of iterations). The idea behind the use of the population of solutions is to provide initial solutions to the hill climbing algorithm due to the fact that the hill climbing algorithm can easily get stuck in local optima, and re-start the algorithm from scratch will be time consuming as it may lose some promising areas. Hence, by using a population of solutions, we may get a better result than re-start the solution from scratch, because the search process can focuses on the promising point, as the solutions have been improved previously and also been modified by the population-based method. a.
Solution representation, initial population generation method and the neighbourhood structure
In this work, we use a binary representation scheme where each bit in the solution is represented by either „1‟ or „0‟ with a size of 100 (see Table 1 for parameter setting). The population of solutions is generated by randomly filling up each bit by either „0‟ or „1‟. The solution is directly used in the objective function to calculate the fitness of the generated solution, i.e. calculate the number of „1s‟. The neighbourhood of the current solution is generated by randomly selecting one bit and changes it to its complement, e.g., „1‟ to „0‟ or vice versa.
Fig. 1 Proposed approach, Pop-HC b.
Phase 1: Evolutionary process. In this phase, the evolutionary process begins with a population of solutions. Then, we evaluate the population of solutions. The evolutionary process contains a population of solutions as well as a perturbation mechanism. The perturbation mechanism uses a flip-flop operator in order to help the solution to get out from the local optima (mutation operators). Thus, the role of the evolutionary process is to perturb the solution that is already improved by the hill climbing algorithm. In this work, a simple perturbation mechanism is employed to modify one solution (or solutions) that has (have) been improved by the hill climbing algorithm by randomly selecting one bit and change its value to „1‟ if the current value is „0‟ and vice versa. This helps the algorithm to jump out from the local optima if the same solution is selected again.
c.
Phase 2: Hill climbing algorithm. In this phase, the hill climbing algorithm is invoked. The algorithm starts with a single solution (S) that is randomly selected from the population of solutions. Set S as a best solution (BestSol). The algorithm iteratively generates a neighbor of the solution (S*) with an aim to obtain a better solution than the current one. The neighbor of the solution is generated by selecting one bit at random and change its value to its complement (i.e. change “1” to “0” or “0” to “1”). If the generated solution is better than the current one (S), the current solution is updated (S=S*) and will be used in the next iteration. If the current solution is better than the best solution then the best solution is updated (BestSol=S). This process is repeated for a certain number of iterations which is set by user. Once the stopping condition is satisfied (MaxIter), the solution will be added into the population if its fitness is better than the worst solution in the population.
3 Results and Discussions In this section, we evaluate the proposed method on the well-known dynamic optimisation test functions. The tested functions are: OneMax, Plateau, Royal Road and Deceptive [1]. These functions (which use a binary-encoding) have been widely used by many researchers. Originally, they are stationary functions and have been changed to be dynamic test functions. In order to generate dynamic environments from these binary-encoded stationary test functions, we use the dynamic generator that was proposed in [4,5]. To use this dynamic generator, two parameters need to be set in advance which are the speed of change, T (which represents the periodically of changes) and P (which represents the amount of change). These two parameters are set based on the work reported in [1]. The parameter setting for the proposed method is listed in Table 1.
Parameter Population Size Solution size Number of iterations T P
Table 1: Parameters settings Value 50 100 100 100 0.9
The proposed method is compared against the well-known methods in the literature. The methods in comparison are: - MIGA: Genetic algorithm with memory based immigrants [6]. - MEGA: memory-enhanced Genetic algorithm [7]. - AHMA: Memetic algorithm with the AHC operator [1]. - MRIGA: Genetic algorithm with memory and random immigrants schemes[6] The average results of Pop-HC over 30 runs (with different initial solutions and seeds) as well as other methods in comparison are presented in Table 2. The presented results indicate that Pop-HC outperforms other methods on the Plateau test function and competitive results on OneMax, Royal Road and Deceptive. Pop-HC is at the 2nd, 3rd and 4th rank on OneMax, Royal Road and Deceptive functions, respectively. The results also show that the percentage of deviation from the best result is quite competitive. Individual comparison shows that the PopHC is able to produce better results than MIGA, MEGA, AHMA and MRIGA in between 1 and 2 instances, and can be considered as competitive results across all tested functions. Note that, the methods in comparison here did not attempt on all test functions. We believe this is due to the use of the population of solutions with the hill climbing algorithm in order to preserve the diversity during the search process. The results also indicate that a mechanism to improve the diversification might lead to better results. Thus, in the future work we intend to integrate other diversification methods and tested their performance on the same instances. Table 2: Results Comparison
Function name Pop-HC % Deviation MIGA MEGA OneMax 86.35 9.94 94.0 79.3 Pleateau ** 74.21 Royal Road 51.11 2.68 Deceptive 72.56 15.38 71.1 83.1 “-“: no results reported. “**”: our algorithm better than others.
AHMA 95.89 62.88 52.52 85.75
MRIGA 80.8 68.6
. 4 Conclusions The evolutionary hill climbing algorithm (that hybridised the population-based method with the hill climbing algorithm) for dynamic optimisation problems has been presented in this paper. The role of the population-based method is to generate a population of solutions to be used as an initial solution for the hill climbing algorithm. It also maintains the diversity of solution by using a simple perturbation mechanism in modifying the solution. Whilst, the hill climbing algorithm aims to further improve the randomly selected solution from the population. The performance of the proposed algorithm is assessed on the dynamic optimisation functions and experimental results show that the proposed method obtained competitive results when compared to the methods in the literature. References
