A Study of Harmony Search Algorithms: Exploration and Convergence Ability Anupam Yadav, Neha Yadav and Joong Hoon Kim
Abstract Harmony Search Algorithm (HSA) has shown to be simple, efficient and strong optimization algorithm. The exploration ability of any optimization algorithm is one of the key points. In this article a new methodology is proposed to measure the exploration ability of the HS algorithm. To understand the searching ability potential exploration range for HS algorithm is designed. Four HS variants are selected and their searching ability is tested based on the choice of improvised harmony. An empirical analysis of the proposed method is tested along with the justification of theoretical findings and experimental results. Keywords Harmony search · Exploration · Convergence
1
Introduction
Harmony Search algorithm is one of the nature inspired optimization algorithms which is inspired from music improvisation process. Geem et. al. [1] proposed the very first idea of Harmony Search. It was an outstanding idea to design optimization algorithms which is inspired from the tuning of music over music A. Yadav() Department of Sciences and Humanities National Institute of Technology Uttarakhand, Srinagar 246-174, Uttarakhand, India e-mail:
[email protected],
[email protected] N. Yadav · J.H. Kim School of Civil, Environmental and Architectural Engineering, Korea University, Seoul 136-713, South Korea e-mail:
[email protected],
[email protected] © Springer-Verlag Berlin Heidelberg 2016 J.H. Kim and Z.W. Geem (eds.), Harmony Search Algorithm, Advances in Intelligent Systems and Computing 382, DOI: 10.1007/978-3-662-47926-1_6
53
54
A. Yadav et al.
instruments. Many developments of the HS algorithm have been recorded in literature from the very first inception of this novel method. Mahadavi et. al. [2] designed an improved version of HS algorithm for solving optimization problems, later on may more versions of HS algorithm has been proposed to improve the performance of the algorithm on real life applications as well as benchmark problems. Few of them are, Self adaptive HSA [3,4] , Global best Harmony search [5], Adaptive binary HSA [6] and Novel global HSA [7]. Some of the hybridization based HS algorithms are also recorded for various purposes, Feshanghary [8] has hybridized HSA with sequential quadratic programming for engineering optimization problems. Some of the application such as blocking permutation flow shop scheduling problem [9], total flow time on a flow shop [10] and water resource engineering problems are also solved using many variants of HS algorithm. In the mean time some the research has been carried out on the mathematical foundations and characteristics of HSA. Das et. al. [11] and Gao et. al. [12] have developed some exploratory & convergence behavior of HSA based on expected variance of the harmonies. These articles provide a significant mathematical formulation of HSA. In the current article the exploratory power of some the variants are analyzed and discussed along with their convergence. To understand the exploration power of HSA, potential exploration range of the improvised harmony is defined. This study provides the exact nature of exploration of HSA variants due to improvised harmonies. A class of competent HS algorithms has been collected from the literature and based on their designing of the improvised harmony vector; the potential exploration range is designed to analyze the possible exploration of the harmonies during a run.
2
Potential Exploration Range of Harmony Search
To understand the exploration power of HS algorithm due to improvised harmony vector a novel approach is defined. The proposed methodology provides a measure of exploitation ability of a harmony search algorithm. Let , ,… is the initially generated harmony where varies from 1: . The very first harmony search proposed by Geem et. al. [1] follows the following strategy for generation of new harmony ( , , … ). If , ,… is the harmony vector from all the initialized harmony vectors then the improvised harmony is generated based on the following rule: ′
,
,…
(1)
1
This follows the pitch adjustment 0, 1 . 1
(2)
A Study of Harmony Search Algorithms: Exploration and Convergence Ability
55
Eq. (1) and (2) provides the implementation of new improvised harmony and all the parameters are the standard parameters of the HS algorithm. This new improvised harmony is the key vector for providing a good exploration to the entire search space. In this continuation to measure the total volume searched by the harmonies due to improvised harmony, an exploration range equation is proposed for original harmony search. The following algorithm is used to design the potential exploration range equation for harmony search Initialize the harmony of size HMS Generate improvised harmony with concern strategy of the particular harmony search variant for
:
_ for
:
,: end of i
– |
,:
|;
end of In the next section a brief idea of few state-of-the-art harmony search algorithms is presented based on their strategy of improvised harmony generation.
2.1
Harmony Search Algorithm [1]
Based on the need and new ideas many variants and hybrid versions of HS algorithm has been proposed in last decade throughout the globe. State-of-the-art of HS algorithms are collected to understand their exploration and convergence power. It was the very first inception of harmony search by Geem et. al. [1]. The idea of generation of improvised harmony for this version of harmony search is explained in previous section. Based on the algorithm defined in section 2, let the following equation provides the potential exploration range for Harmony Search. , where
2.2
,…
_
is the maximum volume covered by improvised harmony at
(3) iteration.
Global Best Harmony Search [5]
Mahamed and Madhavi [5] proposed a global best harmony search algorithm which was designed on the line of Particle Swarm Optimization. It follows the following approach for the generation of improvised harmony for each 1:
56
A. Yadav et al.
do if
0,1
then /* memory consideration */
begin if 0,1 begin
, where ~ 1, 2, … , . then /* pitch adjustment */ where best is the index of the best harmony in the HM and
~ 1, endif else /* random selection */ 0,1 endif done Again, based on the algorithm defined in section 2, the potential exploration range for global best harmony search will be defined as , where
2.3
,…,
(4)
_
is the maximum volume covered by improvised harmony at
iteration.
Improved Harmony Search [2]
Mahadavi et. al. [2] proposed an improved harmony search algorithm. The major finding of the improved harmony search is to use dynamic values of and in place of their fixed values as coined in original harmony search. The dynamic choices of and are made based on the following two equations (5)
exp
.
(6)
Where : pitch adjusting rate for each generation : minimum pitch adjusting rate :maximum pitch adjusting rate : number of solution vector generations : generation number With this change of choice of and values, the proposed potential search range and potential search volume of improved harmony search will be , where
,…,
_
is the maximum volume covered by improvised harmony at
(7) iteration.
A Study of Harmony Search Algorithms: Exploration and Convergence Ability
2.4
57
Self Adaptive Global Best HS Algorithm [3]
Kuan-ke et. al. [3] proposed a self adaptive global best harmony search algorithm. The major modification was to provide a self adaptive nature to the parameters of global best harmony search algorithm [5]. The parameter values and are advised to choose statistically as well as based on harmony memory. The choice of is provided by using the following formulation 2
/2
(8)
/2 Where and are the bounds for , is the current iteration and is the total number of iterations. All the parameter values are exactly inherited as they are in the original article. Based on this approach the defined potential search range and potential search volume for self adaptive Harmony search algorithm will be ,
,…,
(9)
_
where is the maximum volume covered by improvised harmony at iteration. In the next section the potential search range and volume is calculated and discussed over various benchmark problems.
3
Experimental Results of Potential Exploration Range
In order to understand the exploration power of HS algorithms on experimental problems, the following problems are solved with the HS algorithms and detailed study of the potential exploration range is discussed based on the results. The benchmark problems and experimental setup is presented in Table 1. Table 1 Benchmark problems and experimental setup
Sr. No. 1. 2. 3. 4.
Problems Sphere Rastrigin Schewfel Greiwank
Dim
20
Algorithm HS [1] GBHS [5] IHS [2] SGHS [3]
100 100 100 100
As per the original setup
maximum iteration ( _ ) is set 10000. The values of , , , and used in corresponding algorithms is taken as suggested by the authors of their original articles. The size of is fixed for all the algorithms which is 100. The search range for all the functions is kept 100, 100 . The
58
10
10
10
10
20
10
10
R2
R1
10
Potential Exploration Range of HS algorithm over Sphere function
30
10
-10
0
10
10
R3
10
0
10
10
10
1000
2000
3000
4000
5000 Iterations
6000
7000
8000
9000
10
10000
Potential Exploration Range of IHS algorithm over Sphere function
40
Potential Exploration Range of GBHS algorithm over Sphere function
30
20
10
0
-10
0
1000
2000
3000
4000
5000 Iterations
6000
7000
8000
9000
10000
9000
10000
Potential Exploration Range of SGHS algorithm over Sphere function
40
10
20
20
10
0
R4
10
A. Yadav et al.
-20
0
10
-20
10
-40
-40
0
1000
2000
3000
4000
5000 Iterations
6000
7000
8000
9000
10
10000
0
1000
2000
3000
4000
5000 Iterations
6000
7000
8000
Fig. 1 Potential exploration behavior of HS, GBHS, IHS and SGHS over Sphere function with 20D against iterations Potential Exploration Range of HS algorithm over Rastrigin function
40
Potential Exploration Range of GBHS algorithm over Rastrigin function
30
10
10
25
30
10
10
20
10
20
10
15
10
R2
R1
10 10
10
10 0
10
5
10 -10
10
0
10
-5
-20
10
10
10
1000
2000
3000
4000
5000 Iterations
6000
7000
8000
9000
10
10000
0
1000
Potential Exploration Range of IHS algorithm over Rastrigin function
40
10
20
10
0
10
2000
3000
4000
5000 Iterations
6000
7000
8000
9000
10000
Potential Exploration Range of SGHS algorithm over Rastrigin function
40
20
0
R
3
R4
10
0
10
10
10
-20
10
-40
10
10
-60
0
1000
2000
3000
4000
5000 Iterations
6000
7000
8000
9000
10000
-20
-40
-60
0
1000
2000
3000
4000
5000 Iterations
6000
7000
8000
9000
10000
Fig. 2 Potential exploration behavior of HS, GBHS, IHS and SGHS over Rastrigin function with 20D against iterations
A Study of Harmony Search Algorithms: Exploration and Convergence Ability Potential Exploration Range of HS algorithm over Schwefel function
40
59
Potential Exploration Range of GBHS algorithm over Schewfel function
40
10
10
30
10
30
10
20
10
20
2
R
R1
10 10
10
10
10
0
10
-10
0
10
10
-20
10
0
1000
2000
3000
4000
5000 Iterations
6000
7000
8000
9000
10000
-10
10
0
1000
Potential Exploration Range of IHS algorithm over Schewfel function
40
10
10
10
20
2000
3000
4000
5000 Iterations
6000
7000
8000
9000
10000
Potential Exploration Range of SGHS algorithm over Schewfel function
40
20
10
0
R4
R3
10 0
10
10
-20
-20
10
10
-40
10
10
0
1000
2000
3000
4000
5000 Iterations
6000
7000
8000
9000
-40
-60
0
10000
1000
2000
3000
4000
5000 Iterations
6000
7000
8000
9000
10000
Fig. 3 Potential exploration behavior of HS, GBHS, IHS and SGHS over Schewfel function with 20D against iterations Potential Exploration Range of HS algorithm over Greiwank function
40
10
10 10
Potential Exploration Range of GBHS algorithm over Griewank function
30
25
30
10
10
20
20
10
15
R2
R1
10 10
10
10
10
10
5
0
10
10 -10
10
0
1000
2000
3000
4000
5000 Iterations
6000
7000
8000
9000
10
10000
Potential Exploration Range of IHS algorithm over Greiwank function
30
0
-5
0
1000
2000
3000
4000
5000 Iterations
6000
7000
8000
9000
10000
9000
10000
Potential Exploration Range of SGHS algorithm over Greiwank function
30
10
10
20
10
20
10
10
10
10
10
0
0
R4
R3
10
10
-10
10 -10
10
-20
10 -20
10
-30
10
-30
10
0
-40
1000
2000
3000
4000
5000 Iterations
6000
7000
8000
9000
10000
10
0
1000
2000
3000
4000
5000 Iterations
6000
7000
8000
Fig. 4 Potential exploration behavior of HS, GBHS, IHS and SGHS over Griewank function with 20D against iterations
4
Expected Population Variance
Das et. al. [11] provides a formula for the calculation of expected population variance. This formula is designed based on the theoretical findings and statistical measures of HS algorithm. Eq. 10 presents this formula
60
A. Yadav et al.
1 3
1
.
.
. 1
. (10)
.
.
3
. 1
Expected Variance of the New Harmony
Where is the intermediate harmony and is Harmony population. An empirical study based on this equation is performed. The expected variance of improved harmony search [2] is evaluated and a plot of expected variance of the New Harmony is presented in Fig. 5 3000 2500 2000 1500 1000 500 0 0
100
200
300
400
500 Iterations
600
700
800
900
1000
Fig. 5 Expected variance of the New Harmony against iterations
The expected variance of the new improvised harmony depicted in Fig 5, justifies the exploration ability of the improved harmony search as it was analyzed with the help of potential exploration range of the same.
5
Discussions
The exploration range of the each algorithm is plotted against iteration for all four functions with fixed number of Harmony Size. Fig. 1 provides the plot of the maximum volume covered by HS, IHS, GBHS and SGHS algorithms against the progressive iterations. It is observed that the volume covered by HS and GBHS algorithms is little more than other two algorithms but the IHS and SGHS are having better convergence approach towards a point. This analysis provides an idea that over unimodal function the behavior if HS algorithms is alike. In Fig. 2 it has been observed that the behavior of SGHS is slightly better than other three algorithms since it is constantly having a better exploration rate in comparison to others. The important point is before execution point the chaotic nature of the SGHS gives a good edge over others because this kind of behavior is helpful for the algorithms to clear the local minima. The Fig. 3 repeats the same kind of behavior for each algorithm, but on careful observation it is observed that HS and GBHS are having good exploration rate but their converging ability is poor in comparison to other two algorithms. The strong converging nature of SGHS and IHS provides a better convergence which is of the order 10 . In Fig. 4, again it
A Study of Harmony Search Algorithms: Exploration and Convergence Ability
61
has been observed that the exploration of SGHS is very excellent, the continuous better value of the exploration shows its strong ability over multimodal optimization problems. The other three algorithms are also having the satisfactory performance. On the other hand the convergence of SGHS and IHS is again much better than other two. Over all it is observed that the exploration and convergence power of original HS algorithm is good while its convergence is not up to the mark when we deal with multimodal problems. The supremacy of the SGHS is well recorded on each kind of the problems and shows an edge over other HS algorithms based on this study.
6
Conclusions
In this article a novel approach is proposed to measure the exploration ability of HS algorithms based on the new improvised harmony. The potential exploration range is defined for original HS algorithm as well as three other improved variants of the harmony search. The exploration ability and convergence is measured by calculating the volume explored by the improvised harmony in each algorithm. The idea of potential exploration range is tested numerically over four benchmark problems and the results are plotted against each iteration. The plot of expected variance of the population is also plotted against iterations which justify the availability of the designed potential exploration range. Based on this study the exploration ability of SGHS is found better than other select algorithms. In future this theory may be applied to check the exploration ability of other HS variants and the idea may be extended by incorporating theoretical finding and some more function can be tested. Acknowledgement This work was supported by SERB, Department of Science and Technology, Govt. of India, National Institute of Technology Uttarakhand and National Research Foundation (NRF) of Korea under a grant funded by the Korean government (MSIP) (NRF-2013R1A2A1A01013886).
References 1. Geem, Z.W., Kim, J.H., Loganathan, G.V.: A new heuristic optimization algorithm: harmony search. Simulation 76(2), 60–68 (2001) 2. Mahdavi, M., Fesanghary, M., Damangir, E.: An improved harmony search algorithm for solving optimization problems. Applied Mathematics and Computation 188(2), 1567–1579 (2007) 3. Pan, Q.K., Suganthan, P.N., Tasgetiren, M.F., Liang, J.J.: A self-adaptive global best harmony search algorithm for continuous optimization problems. Applied Mathematics and Computation 216(3), 830–848 (2010)
62
A. Yadav et al.
4. Wang, C.M., Huang, Y.F.: Self-adaptive harmony search algorithm for optimization. Expert Systems with Applications 37(4), 2826–2837 (2010) 5. Mahamed, G.H.O., Mahdavi, M.: Global-best harmony search. Applied Mathematics and Computation 198(2), 643–656 (2008) 6. Lee, K.S., Geem, Z.W.: A new meta-heuristic algorithm for continuous engineering optimization: harmony search theory and practice. Computer Methods in Applied Mechanics and Engineering 194(36), 3902–3933 (2005) 7. Das, S., Mukhopadhyay, A., Roy, A., Abraham, A., Panigrahi, B.K.: Exploratory power of the harmony search algorithm: analysis and improvements for global numerical optimization. IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics 41(1), 89–106 (2011) 8. Fesanghary, M., Mahdavi, M., Jolandan, M.M., Alizadeh, Y.: Hybridizing harmony search algorithm with sequential quadratic programming for engineering optimization problems. Computer Methods in Applied Mechanics and Engineering 197(33), 3080–3091 (2008) 9. Wang, L., Pan, Q.K., Tasgetiren, M.F.: A hybrid harmony search algorithm for the blocking permutation flow shop scheduling problem. Computers & Industrial Engineering 61(1), 76–83 (2011) 10. Wang, L.: An improved adaptive binary harmony search algorithm. Information Sciences 232, 58–87 (2013) 11. Wang, L., Pan, Q.K., Tasgetiren, M.F.: Minimizing the total flow time in a flow shop with blocking by using hybrid harmony search algorithms. Expert Systems with Applications 37(12), 7929–7936 (2010) 12. Zou, D., Gao, L., Wu, J., Li, S.: Novel global harmony search algorithm for unconstrained problems. Neurocomputing 73(16), 3308–3318 (2010)
