A FAST PARTICLE SWARM OPTIMIZATION ... - Semantic Scholar

International Journal of Innovative Computing, Information and Control Volume 2, Number 6, December 2006

c ICIC International °2006 ISSN 1349-4198 pp. 1365—1380

A FAST PARTICLE SWARM OPTIMIZATION Zhihua Cui1,2 , Jianchao Zeng2 and Guoji Sun1 1

State Key Laboratory for Manufacturing Systems Engineering Xi’an Jiaotong University Xi’an 710049, P. R. China [email protected]; [email protected] 2

Division of System Simulation and Computer Application Taiyuan University of Science and Technology Taiyuan 030024, P. R. China [email protected]

Received November 2005; revised March 2006 Abstract. Particle swarm optimization (PSO) is a popular and robust strategy for optimization problems. One main difficulty in applying PSO to real-world applications is that PSO usually need a large number of fitness evaluations before a satisfying result can be obtained. This paper introduces a new ”fast particle swarm optimization” (FPSO) that does not evaluate all new positions owning a fitness and associated reliability value of each particle of the swarm and the reliability value is only evaluated using the true fitness function if the reliability value is below a threshold. Moreover, applying random evaluation, reliability value update and self-adaptive threshold strategies to the FPSO further enhances the performance of the algorithm. Keywords: Particle swarm optimization, Random evaluation, Self-adaptive threshold, Reliability value update strategy, Convex combination

1. Introduction. Particle swarm optimization (PSO) is a new population-based evolutionary computation method first proposed by Kennedy and Eberhart [1,2]. Because of the ease of implementation and the fact that no gradient information is required, it has been used to solve many different optimization problems, such as neural network training [3,4], data mining [5,6],web content organizing [7], traffic incident detection [8], etc. Numerous improvements to the standard PSO have been proposed. Some are focusing the rate of convergence, while the others are diversity. One of the most widely used improvement is the introduction of the inertia weight [9] that can be seen as the equivalent of a temperature schedule in the simulated annealing algorithm. To ensure convergence, Clerc indicated a constriction factor model which is a way of choosing the values of some parameter coefficients [10]. Inspired by the power of evolutionary computation techniques, several versions of PSO with selection, reproduction and recombination and Gaussian mutation operators are introduced by Angeline [11], Løvbjerg [12], Higashi and Iba [13] respectively. For tackling the overshooting problem in the motion behavior of PSO, a novel variant of PSO named memetic particle swarm optimization algorithm is proposed [14]. When PSO with multiple swarms is applied to solve an optimization problem, many factors influence the working separately and cooperating with each other. Abd and Kamel 1365

1366

Z. H. CUI, J. C. ZENG AND G. J. SUN

[15] investigates two factors that affect this approach behavior. J. F. Chang proposed a parallel strategy for PSO [16]. Despite the great successes achieved in real-world applications, PSO has also encountered many challenges such as fitness approximation (the more details can be seen from [17]). This technique has been applied in evolutionary computation successfully mainly because of no explicit model for fitness computation, noisy environment, multi-model fitness landscape, and extremely time-consuming fitness computation. This paper considers the latter option. Usually the real time to compute the position and velocity of particles of PSO is much less than the time needed to evaluate the objective function and so the computation time to run a PSO is dominated by the ’slow-to-compute’ objective function. There has been some work on using the fitness approximation in evolutionary computation while two approaches are often used: functional approximation and evolutionary approximation. In functional approximation, a model of the objective function is building. Bull [18] used a neural network training by the example individuals with the explicit fitness and the resulting model of the fitness function is then used by the evolutionary algorithm to find a solution. The same method also used by Y. Jin [19]. A popular class of the evolutionary approximation methods is known as fitness inheritance [20,22]. In these methods, fitness inheritance instead of fitness evaluation is used to estimating the fitness value of the offspring individuals from the fitness value of their parents. In the second class of the evolutionary approximation, several groups are clustered derived from the individuals [23], and the individual representing its cluster will be evaluated using the fitness function while the others in the same cluster will be estimated based on a distance measure. The difference of the structure between PSO and other evolutionary algorithms is that the PSO maintains information regarding position and velocity (changes in position); in contrast, traditional evolutionary algorithms only keep track of position. Figure 1 gives an abbreviate description of PSO. Though there are so many works on using the fitness approximation published, few attempts have been concerned with PSO. This paper considers a version of an PSO, a fast particle swarm optimization (FPSO), that runs quickly even for large population sizes thus allowing more generations to be evaluated in a given time. To design the FPSO, particles can change the update equations or reduce the number of fitness evaluation.The latter option and fitness inheritance method are considered in this paper. For FPSO, there are some problems we need to solve: (1)PSO evaluations using position and velocity update equations, how to select the parents vectors to estimate the fitness value? (2)If the parents vectors are selected, how to estimate the fitness value of the offspring vector? (3)The threshold is used to control the true evaluation number to obtain a better behavior, how to design the threshold selection strategy? Section 2 gives a brief description of particle swarm optimization. The basic FPSO and two enhancements are proposed in Section 3 and Section 4 provides some experimental results and analysis for comparing proposed FPSO with PSO. Finally, some concluding remarks are made in Section 5. 2. Particle Swarm Optimization. Particle swarm optimization is based on the sociological behavior associated with bird flocking [24].The position and velocity update

A FAST PARTICLE SWARM OPTIMIZATION

1367

Figure 1. Structure of PSO equations of standard PSO at time t + 1 are presented in equation (1) and (2): vjk (t + 1) = wvjk (t) + c1 r1 (pjk (t) − xjk (t)) + c2 r2 (pgk (t) − xjk (t))

(1)

xjk (t + 1) = xjk (t) + vjk (t + 1)

(2)

where the k th dimensional variable of velocity vector Vj (t + 1) = (vj1 (t + 1), vj2 (t + 1), ..., vjn (t + 1)) (n denotes the dimension of problem space) limited by |vjk (t + 1)| < vmax

(3)

where vjk (t) and xjk (t) are the k th dimensional variables of velocity and position vectors of particle j at time t, pjk (t) and pgk (t) are the k th dimensional variables of historical positions found by particle j and the whole swarm at time t respectively. w is an inertia weight between 0 and 1, accelerator coefficients c1 and c2 are two random numbers generated with uniform distribution within (0, 1). The cognition component c1 r1 (pjk (t) − xjk (t)) only takes into account the particle’s own experiences. The performance of the ’cognition only’ model was inferior due to no interaction between the different particles [25]. The third term in the velocity update equation c2 r2 (pgk (t)−xjk (t)), represents the social interaction between the particles. Little is currently known about the relative importance of these two terms, although initial results seem to indicate that the social component may be more significant on some problems. The PSO is clearly related to some of the evolutionary algorithms. Firstly, the PSO maintains a population of individuals representing potential solutions, a property common to all evolutionary algorithms. Secondly, if the historical personal best position is treated as part of the population, there is clearly a weak form of selection. Thirdly,the velocity update equation resembles the arithmetic crossover operator found in real-valued genetic algorithms. Alternatively, the velocity update equation, without the wvjk (t) term, can be seen as a mutation operator. A better way of modelling the velocity is to think of each iteration not as a process of replacing the previous population with a new one (death and birth), but rather as a process of adaption. The gbest model offers a faster rate of convergence at the expense of robustness. This model maintains only a single best solution, called the global best particle, across all the particles in the swarm. This particle acts as an attractor, pulling all the particles towards it. The lbest model tries to prevent premature convergence by maintaining multiple attractors. In fact, gbest model is actually a special case of the lbest model. Experiments

1368


have shown the lbest algorithm converges somewhat more slowly than the gbest version, but it is less likely to become trapped in an inferior local minimum. 3. The Fast Particle Swarm Optimization. Particle swarm optimization is a different method without recombination operators. The next position vector of each particle of the swarm lies in its neighborhood of the current position vector limited by the threshold vmax . In the following parts, the principles of parents vectors selection principles and the estimation methods are discussed. 3.1. How to select the parents vectors? In evolutionary computation, the known fitness values of the parents provide a basis for estimating the fitness values of their children. Suppose the fitness value of position vector of particle j at time t+1 is estimated, substituting (1) into (2) results in xjk (t + 1) = wvjk (t) + ϕ1 pjk (t) + ϕ2 pgk (t) + (1 − ϕ)xjk (t)

(4)

xjk (t) = xjk (t − 1) + vjk (t)

(5)

vjk (t) = xjk (t) − xjk (t − 1) Substituting (6) into (4) and grouping the terms yield

(6)

whereϕ1 and ϕ2 denote c1 r1 and c2 r2 , respectively. ϕ is the sum of ϕ1 and ϕ2 .From equation (2) it is known that Thus

xjk (t + 1) = (1 + w − ϕ)xjk (t) − wxjk (t − 1) + ϕ1 pjk (t) + ϕ2 pgk (t)

(7)

Formula (7) implies that the position vector of particle j at time t+1 is dominated by the historical best position vectors of itself and the swarm, the previous position vectors at time t and t−1 respectively. In other words, the above four positions can be considered as the parents vectors as well as the position vector of particle j at time t + 1 is their child. From this viewpoint, the estimation method can be discussed as follows. 3.2. How to estimate the fitness value of offspring vector? In Mehrdad Salami’s paper [22], the estimated fitness value is dominated by the convex combination of their parents vectors, whether the offspring vector lies in the area determined by their parents vectors(parent area, in briefly, definition can be seen as follows) or not. If the offspring vector lies in the parent area, the convex combination of the fitness value of parents vectors is reasonable and intuitive. On the country, using convex combination technique may result in large errors between the estimated and true fitness values. Since then, a new technique is proposed to overcome this shortcoming. The following parts discuss the solutions methods of the two cases mentioned above. 3.2.1. The offspring vector lies in the parent area. dimensional variables of Definition 3.1. Suppose s1k , s2k , ... , sjk denotes the k th Q parent1 , parent2 ,..., parentj , the parent area is a hypercube ni=1 [mi , Mi ] (n denotes the dimension of problem space) where the k th dimensional interval [mk , Mk ] is defined with mk = min{s1k , s2k , ..., sjk } Mk = max{s1k , s2k , ..., sjk }


1369

In this case, the method is similar with [22] (traditional convex combination, in brief). When the fitness value of the child vector is estimated, another parameter is needed to show how reliable the assigned fitness value is thought to be. If this reliability value falls below some threshold, the true fitness value is given, otherwise the estimated fitness value is used. Thus each particle consists of four elements: a position vector(x), a velocity vector(v), a fitness value(f ), and a reliability value(r) between 0 and 1. To give the estimated fitness value of particle j at time t+1 (offspring vector, in briefly), the similarity measure is used. Definition 3.2. Let the vector1 be (c1 , c2 , ..., cn ) and the vector2 be (d1 , d2 , ..., dn ), where ch and dh are real number and lie between ±A, then the similarity S is defined as: q Pn h=1 (ch −dh )

S = 1.0 −

2

n

(8) 2A Suppose the fitness and reliability values of the historical best position vectors of particle j and the swarm, the position vectors at time t and t − 1 are f1 and r1 , f2 and r2 , f3 and r3 and f4 and r4 , respectively. Then the fitness and reliability values of offspring vector at time t + 1 is computed with: S 1 r 1 f1 + S 2 r 2 f2 + S 3 r 3 f3 + S 4 r 4 f4 (9) f= S1 r1 + S2 r2 + S3 r3 + S4 r4 and (S1 r1 )2 + (S2 r2 )2 + (S3 r3 )2 + (S4 r4 )2 r= (10) S1 r1 + S2 r2 + S3 r3 + S4 r4 Remark 3.1. Formula (9) implies the fitness value of offspring vector lies between the largest and smallest fitness values of its parents vectors. Since then, one additional limitation fj > 0.or.fj < 0 may cancel some troubles. For convenience, we suppose the fitness value is no less than zero as follows. 3.2.2. The offspring vector lies outside the parent area. In this case, since the offspring vector does not lie in the parent area, the convex combination of the four parents vectors may result a large error to affect the estimation accuracy. To overcome this shortcoming, two visual parents vectors called max− parent and min− parent are used to estimate the fitness value of the offspring vector. These two visual parents vectors max− parent and min− parent can be considered as two points within a hypercube described by four parents vectors and corresponding offspring vector. Definition 3.3. Suppose min− parentk , max− parentk , pjk (t), xjk (t + 1) are the k th dimensional variables of visual parents min− parent, historical best position vectors of particle j and position vectors at time t, t − 1 and t + 1 of itself respectively.

pgk (t), xjk (t), xjk (t − 1), vectors max− parent and the swarm at time t, the Then

min− parentk = min{pgk (t), pjk (t), xjk (t − 1), xjk (t), xjk (t + 1)} max− parentk = max{pgk (t), pjk (t), xjk (t − 1), xjk (t), xjk (t + 1)}

1370


The offspring vector lies in the hypercube constructed by visual parents vectors max− parent and min− parent, and the fitness and reliability values are estimated with f=

S 1 r 1 f1 + S 2 r 2 f2 S1 r1 + S2 r2

(11)

and (S1 r1 )2 + (S2 r2 )2 (12) S1 r1 + S2 r2 where S1 ,S2 are the similarity measure of the offspring vector with max− parent and min− parent respectively as well as r1 , f1 , r2 , f2 are the reliability and the fitness values of two visual parents vectors. This new method also uses a convex combination of parents vectors, we call it additional convex combination contrasted with traditional convex combination. Now, the problem is how to estimate the fitness and reliability values of max− parent and min− parent. The method is simple and intuitive. Firstly, one parent vector is selected as a visual offspring as well as the other five vectors are selected as parents vectors not only consisting the original four parents vectors but also including one of visual parents vectors. It is obvious, the selected parent vector lies in the parent area, and the case 1 is satisfied, using formula (9), the reliability value of max− parent or min− parent is estimated. (S1 r1 )2 + (S2 r2 )2 + (S3 r3 )2 + (S4 r4 )2 + (S5 r5 )2 (13) rj = S1 r1 + S2 r2 + S3 r3 + S4 r4 + S5 r5 where rj is the reliability value of visual offspring, r1 , r2 , r3 , r4 are the reliability values of four parents vectors while r5 is the reliability value of max− parent or min− parent, S1 , S2 , S3 , S4 are the similarity of four parents vectors with the visual offspring vector while S5 is the similarity of max− parent or min− parent with the same visual offspring vector. From (13), the following equation is true. r=

S52 r52 − rj S5 r5 + α = 0

(14)

∆ = (rj S5 )2 − 4 × S52 × α ≥ 0

(15)

rj2 − 4 × α ≥ 0

(16)

where α denotes S12 r12 + S22 r22 + S32 r32 + S42 r42 − rj (S1 r1 + S2 r2 + S3 r3 + S4 r4 ). The roots of equation (14) exist if and only if

Thus Equation (16) implies the selection of visual offspring vector needs some conditions. Theorem 3.1. The parent vector j can be selected as the visual offspring vector if and only if rj satisfies the following condition. rj = max{r1 , r2 , r3 , r4 }

(17)

Proof: For convenience, we will always suppose r1 satisfies the above condition in the following parts. The corresponding parent vector can be selected as the visual offspring vector if and only if there exists the real-valued roots in equation (16). It implies the r12 − 4 × α ≥ 0


1371

where α denotes S22 r22 + S32 r32 + S42 r42 − r1 (S2 r2 + S3 r3 + S4 r4 ) because of S1 = 1.0. Substituting α into (17) results r12 + 4r1 (S2 r2 + S3 r3 + S4 r4 ) − 4(S22 r22 + S32 r32 + S42 r42 ) = r12 + 4S2 r2 (r1 − S2 r2 ) + 4S3 r3 (r1 − S3 r3 ) + 4S4 r4 (r1 − S4 r4 ) ≥ 0

(18)

By the selection of r1 , equation (18) is true. Since then, the selection principles of condition (17) guarantees that the roots of equation (14) exist. Corollary 3.1. In equation (14), parameter α is a non-positive number. Proof: By the definition of α, we have α = S22 r22 + S32 r32 + S42 r42 − r1 (S2 r2 + S3 r3 + S4 r4 ) = S2 r2 (S2 r2 − r1 ) + S3 r3 (S3 r3 − r1 ) + S4 r4 (S4 r4 − r1 ) ≤ 0

If equation (16) is true, the roots of equation (14) are p r1 ± r12 − 4α r5 = 2S5 Lemma 3.1. The root of equation (14) satisfies

(19)

r5 > r1 Proof: If r5 ≤ r1 , r1 =

(S1 r1 )2 + (S2 r2 )2 + (S3 r3 )2 + (S4 r4 )2 + (S5 r5 )2 S1 r1 + S2 r2 + S3 r3 + S4 r4 + S5 r5 (S2 r2 )2 + (S3 r3 )2 + (S4 r4 )2 + (S5 r5 )2 = S2 r2 + S3 r3 + S4 r4 + S5 r5 (S2 r2 )2 (S3 r3 )2 (S4 r4 )2 (S5 r5 )2 ≤ max{ , , , } S2 r2 S3 r3 S4 r4 S5 r5 = max{S2 r2 , S3 r3 , S4 r4 , S5 r5 } ≤ max{r2 , r3 , r4 , r5 } ≤ r1

(20)

The equation is true if and only if S2 = S3 = S4 = S5 = 1, it means the four parents and max− parent or min− parent vectors are the same vectors. It is obviously not true. This conclusion verifies r5 > r1 is true. Theorem 3.2. The solution of r5 is r5 =

r1 +

Proof: Suppose

It means

r12 − 4α 2S5

r1 −

(21)

p

r12 − 4α 2S5 q r1 − 2S5 r5 = r12 − 4α ≥ 0 r5 =

Then

p

r12 − 4α = (r1 − 2S5 r5 )2 = r12 + 4S52 r52 − 4r1 S5 r5

(22)

1372


Grouping the related terms yields S2 r2 (S2 r2 − r1 ) + S3 r3 (S3 r3 − r1 ) + S4 r4 (S4 r4 − r1 ) + S5 r5 (S5 r5 − r1 ) = 0

The above equation is true if and only if

S5 r5 − r1 ≥ 0

It is contrast with formula (22), so the solution of r5 is p r1 + r12 − 4α r5 = 2S5 From Corollary 3.1, the following conclusion draws from equation(24) : p r1 + r12 − 4α r1 + r1 r1 ≥ = r5 = 2S5 2S5 S5 Corollary 3.2. If r1 > S5 , there does not exist the real value r5 .

(23)

(24)

(25)

Remark 3.2. If the conditions r1 > S5 and r5 < 1 are not satisfied, the reliability value of offspring vector can compute with formula (9) and (10) directly without using visual parent vectors max− parent. On the country, the reliability value of max− parent can be computed with formula (13)and the computation formula of reliability value of min− parent is the same with max− parent. Thus the reliability value of offspring vector can compute with formula (11). The fitness values of max− parent or min− parent are easily implemented with the similar method. From the above mentioned, the fitness value of offspring vector is no less than 0, how to select the visual offspring vector to providing non-negative fitness value of max− parent or min− parent? Theorem 3.3. if fj satisfies with the following condition fj = max{f1 , f2 , f3 , f4 }

(26)

the fitness value of max− parent or min− parent can obtained with fj b − a f5 = (27) S5 r5 where f5 denotes the fitness value of max− parent or min− parent, b denotes S1 r1 + S2 r2 + S3 r3 + S4 r4 + S5 r5 , and a denotes S1 r1 f1 + S2 r2 f2 + S3 r3 f3 + S4 r4 f4 . Proof: Suppose f1 satisfies the condition (26),for simplicity. Similar with formula (9), we have S1 r1 f1 + S2 r2 f2 + S3 r3 f3 + S4 r4 f4 + S5 r5 f5 f1 = (28) S1 r1 + S2 r2 + S3 r3 + S4 r4 + S5 r5 Thus f1 b − a = f1 (S1 r1 + S2 r2 + S3 r3 + S4 r4 + S5 r5 ) − (S1 r1 f1 + S2 r2 f2 + S3 r3 f3 + S4 r4 f4 ) = S2 r2 (f1 − f2 ) + S3 r3 (f1 − f3 ) + S4 r4 (f1 − f4 ) + f1 S5 r5

By the selection principle of f1 , the above equation implies that f1 b − a ≥ 0. From equation (27), we have. f1 b − a f5 = ≥0 (29) S5 r5


1373

3.2.3. Reliability value update technique. From theorem 3.1, the reliability value of visual offspring satisfies with rj = max{r1 , r2 , r3 , r4 }. however, the reliability value of historical best position of the swarm is 1, it implies rj = 1 > S5 always true, by Corollary 3.2, the new convex combination method may not be used. In other words, the new convex combination method used if and only if a modification of reliability value of historical best position of the swarm needs to be proposed so that it can less than 1. How to modify the reliability value?The most simple way is to choose the corresponding reliability value randomly with (0, 1). The algorithm mainly explores the problem space to maintaining high diversity in the early period while exploits within a small neighborhood to determine the global solution in the late period. Based on the above phenomenon, the reliability value can be selected increasing from 0.1 to 1.0 with the increased generation so that in the early stage the estimated fitness value mainly used as well as the true fitness value used in the late stage. 3.2.4. Self-adaptive threshold technique. The threshold is used to control the true fitness evaluation numbers based on the reliability value of each particle. Large threshold value results the more numbers of true fitness evaluation as well as small threshold value increases the number of estimation fitness evaluation. Because of the character of PSO, particles sample within the whole domain range in the first period uniformly while the errors of estimated fitness values and true fitness values may be large, and converge in a small neighborhood of historical best position found by the swarm in the late period where the errors are small. To obtain a better behavior, analogous with inertia weight, we select the threshold value linearly decreased from 1.0 to 0.1 such that in the early stage, the number of true fitness value is large enough as well as the number of estimated fitness value is large in the late stage. 3.2.5. Random fitness evaluation technique. Premature convergence is a major characteristic for evolutionary computation techniques. To avoid it, [22] introduces random fitness evaluation technique. This technique means a probability of evaluation (PE ) is chosen. When the reliability value of one particle is larger than the threshold in some generation, a random number is obtained. If it is less than PE , the fitness value is truly evaluated. Random fitness evaluation technique ensures a minimum number of true evaluation for each generation, since then, it is always set to 0.05 [22]. The pseudocode of FPSO is as follows: Step1: Initialization: generating initial random population and coefficients initialization. For each particle, the fitness value is calculated using the true fitness function and the reliability value is set to 1.0. Step2: For each position vector of particle j at time t + 1, the position vector and velocity vector are computed with formula (1), (2) and (3). Step3: Traditional and additional convex combination methods are used to assign fitness value of position of each particle. Step4: If the reliability value of a position vector of particle is less than a threshold value, the assigned fitness for that particle is replaced by a fitness that is calculated using the true fitness function and the reliability value is set to 1.0. On the other country, traditional or additional convex combination will be selected by different manners. Step5: Reliability value update techniques, self-adaptive threshold techniques and random fitness evaluation techniques are used to enhance the performance.

1374


Step6: If stop criteria is satisfied, the fitness value of historical best position of swarm is given. Otherwise, go to Step2. 4. Simulation Results. The benchmark functions in this section provide a balance of multi-modal with many local minima and only a few local minima as well as easy and difficult functions. Shekel’s Foxholes Function: 25 X 1 1 ]−1 + f1 (x) = [ P2 500 j=1 j + i=1 (xi − aij )6 where |xj | ≤ 65, aij is satisfied with the following matrix. µ ¶ −32 −16 0 16 32 −32 ... 0 16 32 −32 −32 −32 −32 −32 −16 ... 32 32 32 f1 (x∗ ) = f1 (−32, −32) ≈ 1.0

Six-hump Camel-Back Function:

1 f2 (x) = 4x21 − 2.1x41 + x61 + x1 x2 − 4x22 + 4x42 + 2 3 where |xj | ≤ 65,and

f2 (x∗ ) = f2 (0.08983, −0.7126) = f2 (−0.08983, 0.7126) ≈ 0.0003075

Goldstein-Price Function:

f3 = [1 + (x1 + x2 + 1)2 (19 − 14x1 + 3x21 − 14x2 + 6x1 x2 + 3x22 )] ×[30 + (2x1 − 3x2 )2 (18 − 32x1 + 12x21 + 48x2 − 36x1 x2 + 27x22 )]

where |xj | ≤ 100.0, and

f3 (x∗ ) = f3 (0, −1) = 3

(30)

Generalized Schwefel Problem 2.26:

f4 (x) = 12569.5 − where |xj | ≤ 500.0, and

30 X j=1

(xj sin(

q

|xj |))

f4 (x∗ ) = f4 (420.9687, 420.9687, ..., 420.9687) ≈ 0

(31)

Rastrigin’s Function:

f5 (x) =

30 X j=1


(x2j − 10cos(2πxj ) + 10.0)

f5 (x∗ ) = f5 (0, 0, ..., 0) = 0

Generalized Griewank Function: 30 30 1 X 2 Y xj xj − cos( √ ) + 1 f6 (x) = 4000 j=1 j j=1

(32)



f6 (x∗ ) = f6 (0, 0, ..., 0) = 0

1375

(33)

Shekel’s foxholes, six-hump camel-back and Goldstein-price are 2-dimensional multimodal functions with only a few local minima while generalized Schwefel problem 2.26, rastrigin and griewank are 30-dimensional multi-modal functions with many local minima. To give a more detailed comparison, four different versions of FPSO combining different fitness approximation techniques are designed: PSO with traditional convex combination technique (FPSO1), FPSO1 with random fitness evaluation technique(FPSO2)(FPSO1 and FPSO2 are used in [22]), FPSO2 with additional convex combination techniques using two different reliability value update methods of historical best position of the swarm: linearly increased from 0.1 to 1.0 (FPSO3) and randomly selected within (0, 1)(FPSO4). All four versions use self-adaptive threshold techniques to own better performances. For each experiment the simulation records the mean(Mean. Val.), standard deviation(Std. Val.), evaluation number(Eval. Num.), additional visual evaluation number(Addi. Num.), and traditional visual evaluation number(Trad. Num.) over all the runs. The evaluation number,additional visual evaluation number,traditional visual evaluation number represent the evaluation number computing with true fitness function, estimating fitness function using the new method of convex combination techniques, and estimating fitness function using traditional method of convex combination techniques [22], respectively. The coefficients of standard PSO (PSO) and other four versions of FPSO are set as follows. The inertia weight w is decreased linearly from 0.9 to 0.4, and two accelerator coefficients are set to 2.0. Total individuals are 100, and vmax is set to 10% of the upper bound of domain. In each experiment the simulation run 30 times while each time the largest evolutionary generation is 1000 for Shekel’s Foxholes,Six-hump Camel-Back,Goldstein-Price and 5000 for Generalized Schwefel Problem 2.26,Rastrigin and Generalized Griewank respectively. Table 1 to Table 6 are the comparison results for six test functions. Figure 2 to 7 verify the dynamic behavior and 20 sample points are selected within the same intervals. In these points, the average best fitness of historical best position of the swarm of all 30 runs are computed and plotted. For more clarity, the first points of f1 , f2 , f3 and f6 are deleted. Because the six function lines need to draw,we separate each figure into two parts to provide figures clearly. From the obtained results, the performance of FPSO1 and FPSO2 is worse or equal FPSO3 and FPSO4. So, the two separate figures are based on the following rules: (1) If performance of FPSO3 is better than FPSO4, the left figure contains FPSO1, FPSO2 and FPSO4, while the right figure contains FPSO3, FPSO4 and SPSO. All test functions except generalized Schwefel Problem 2.26 falls into the case. (2) If performance of FPSO3 is worse than FPSO4, the left figure contains FPSO1, PFSO2 and FPSO3, while the right figure contains FPSO3, FPSO4 and SPSO. The figure of generalized Schwefel Problem 2.26 is this case. Table 1 and Figure 2 are the comparison results of shekel’s foxholes function. The truly fitness evaluation numbers of FPSO1, FPSO2, FPSO3 and FPSO4 are 6.1%, 10.5%, 17.7%, and 16.25% of SPSO respectively while the mean value obtained are 18.46%, 13.15%, −0.38%, 5.09% worse than SPSO. Though the true fitness evaluation number of

1376


FPSO3 and FPSO4 are always larger than FPSO1 and FPSO2 nearly 5.75% − 11.6%, the performance of mean value is better by nearly 8.06% − 18.84%, this implies the additional estimation fitness values contributes to the performance significantly. The results of sixhump camel-back and Goldstein-price functions are the same as function f1 although it is not remarkable as f1 . Table 4 and Figure 5 are the comparison results of Schwefel problem 2.26. The Eval. Num. of FPSO1, FPSO2, FPSO3 and FPSO4 are 14.13%, 11.58%, 21.33%, and 19.81% of SPSO respectively while the mean value obtained are 11.85%, 7.8%, 0.09%, −0.75% worse than SPSO. Though the true fitness evaluation number of FPSO3 and FPSO4 are always larger than FPSO1 and FPSO2 nearly 5.68% − 9.75%, the performance of mean value was better by nearly 7.71% − 1.6%, this also implies that the additional estimation fitness values contributes to the performance significantly. The results of rastrigin and griewank functions are the same as function f1 especially f6 . Based on the above analysis, the following conclusions can draw form it. Remark 4.1. Additional convex combination technique is more fit for high-dimensional multi-modal functions with many local optima verified from the comparison results of the latter three test functions. Remark 4.2. Additional convex combination technique provides a better balance between computational efficiency and computational costs. Table 1. Simulation results of Shekel’s Foxholes Function Alg. SPSO FPSO1 FPSO2 FPSO3 FPSO4

Mean. Val. 1.233333e+000 1.460978e+000 1.395483e+000 1.228634e+000 1.296080e+000

Std. Val. Eval. Num. Addi. Num. Trad. Num. 4.301827e-001 100000 0 99800 9.610166e-001 6099 0 99800 5.594908e-001 10512 0 99800 9.227708e-001 17702 14791 85009 5.313548e-001 16253 11326 88474

Table 2. Simulation results of Six-hump Camel-Back Function Alg. Mean. Val. Std. Val. Eval. Num. Addi. Num. Trad. Num. SPSO 9.683715e-001 3.387608e-016 100000 0 99800 FPSO1 1.008381e+000 3.580833e-002 4513 0 99800 FPSO2 9.683715e-001 1.710410e-015 8342 0 99800 FPSO3 9.683715e-001 1.376362e-015 17997 15171 84629 FPSO4 9.683715e-001 1.714257e-015 15991 12042 87758 5. Conclusion. This paper introduces a fast particle swarm optimization which uses new convex combination method as well as reliability value update, self-adaptive threshold and random fitness evaluation techniques. The results show that the proposed algorithm has an approximate capability of computational efficiency though it needs a nearly 20% computational cost. The further research is the efficiency selection principles of the reliability value update technique, and more fast model combined with other methods.


Figure 2. Comparison results of Shekel’s Foxholes Function

Figure 3. Comparison results of Six-hump Camel-Back Function

Figure 4. Comparison results of Goldstein-Price Function

1377

1378


Figure 5. Comparison results of Generalized Schwefel Problem 2.26

Figure 6. Comparison results of Rastrigin’s Function

Figure 7. Comparison results of Generalized Griewank Function


1379

Table 3. Simulation results of Goldstein-Price Function Alg. SPSO FPSO1 FPSO2 FPSO3 FPSO4

Mean. Val. Std. Val. Eval. Num. Addi. Num. Trad. Num. 3.000000e+000 1.250648e-015 100000 0 99800 1.784673e+001 1.224108e+001 4206 0 99800 3.000000e+000 1.298534e-014 8274 0 99800 3.000000e+000 6.900034e-015 17190 14807 84993 3.000000e+000 5.159850e-015 15306 11508 88292 Table 4. Simulation results of Generalized Schwefel Problem 2.26

Alg. SPSO FPSO1 FPSO2 FPSO3 FPSO4

Mean. Val. 5.853812e+003 6.547768e+003 6.312348e+003 5.859118e+003 5.809730e+003

Std. Val. Eval. Num. Addi. Num. Trad. Num. 7.048758e+002 500000 0 499800 8.885068e+002 70651 0 499800 8.261003e+002 57882 0 499800 6.752567e+002 106659 98255 401545 6.156246e+002 99042 70185 429615

Table 5. Simulation results of Rastrigin’s Function Alg. SPSO FPSO1 FPSO2 FPSO3 FPSO4

Mean. Val. 3.100953e+001 1.138808e+002 4.850254e+001 2.941767e+001 2.825683e+001

Std. Val. Eval. Num. Addi. Num. Trad. Num. 8.979678e+000 500000 0 499800 1.386281e+001 58862 0 499800 3.190187e+001 51916 0 499800 7.530736e+000 105043 99165 400635 8.314577e+000 98105 70653 429147

Table 6. Simulation results of Generalized Griewank Function Alg. Mean. Val. Std. Val. Eval. Num. Addi. Num. Trad. Num. SPSO 1.669454e-002 2.150488e-002 500000 0 499800 FPSO1 2.967991e+000 2.976063e-001 57536 0 499800 FPSO2 7.570843e-001 1.257272e+000 51528 0 499800 FPSO3 2.114311e-002 1.710436e-002 104676 98363 401437 FPSO4 2.387290e-002 2.191084e-002 96813 70438 429362 Acknowledgment. This work is supported by Educational Department Key Project Science and Technology Funds under Grant No.204018. REFERENCES [1] Eberhart, R. C. and J. Kennedy, A new optimizer using particle swarm theory, Proc. of the Sixth International Symposium on Micro Machine and Human Science, Nagoya, Japan, pp.39-43, 1995. [2] Kennedy, J. and R. C. Eberhart, Particle swarm optimization, Proc. of the IEEE International Conference on Neural Networks, vol.IV, pp.1942-1948, 1995. [3] Eberhart, R. C. and Y. Shi, Evolving artificial neural networks, Proc. of the 1998 International Conference on Neural Networks and Brain, Beijing, China, pp.5-13, 1998.

1380


[4] Wei, H. Z., C. Yang, L. Gao, X. Yao, R. C. Eberhart and Y. Shi, Extracting rules from fuzzy neural network by particle swarm optimization, Proc. of the IEEE International Conference on Evolutionary Computation, Anchorage, Alaska, USA, 1998. [5] Li, Q. Y., Z. P. Shi, J. Shi and Z. Z. Shi, Swarm intelligence clustering algorithm based on attractor, Lecture Notes on Computer Science 3612, Changsha, China, pp.496-504, 2005. [6] Shen, H. Y., X. Q. Peng, J. N. Wang and Z. K. Hu, A mountain clustering based on improved PSO algorithm, Lecture Notes on Computer Science 3612, Changsha, China, pp.477-481, 2005. [7] Hassas, S., Using swarm intelligence for dynamic web content organizing, Proc. of the 2003 IEEE Swarm Intelligence Symposium, Indianapolis, USA, pp.19-25, 2003. [8] Srinivasan, C., W. H. Loo and R. L. Cheu, Traffic incident detection using particle swarm optimization, Proc. of the 2003 IEEE Swarm Intelligence Symposium, Indianapolis, USA, pp.144-151, 2003. [9] Shi, Y. and R. C. Eberhart, A modified particle swarm optimizer, Proc. of the IEEE International Conference on Evolutionary Computation, Anchorage, Alaska, USA, pp.69-73, 1998. [10] Clerc, M., The swarm and the queens: Towards a deterministic and adaptive particle swarm optimization, Proc. of the IEEE International Conference on Evolutionary Computation, Washington D.C., USA, pp.1951-1957, 1999. [11] Angeline, P. J., Using selection to improve particle swarm optimization, Proc. of the International Joint Conference on Neural Networks, Washington D.C., USA, pp.84-89, 1999. [12] Løvbjerg, M., T. K. Rasmussen and T. Krink, Hybrid particle swarm otpimizer with breeding and subpopulations, Proc. of the Genetic and Evolutionary Computation Conference, San Francisco, USA, 2001. [13] Higashi, N. and H. Iba, Particle swarm optimization with Gaussian mutation, Proc. of the 2003 IEEE Swarm Intelligence Symposium, Indianapolis, USA, pp.72-79, 2003. [14] Liu, B. F., J. H. Chen, S. F. Hwang and S. Y. Ho, MeSwarm: Memetic particle swarm optimization, Proc. of the Genetic and Evolutionary Computation Conference, Washington D.C., USA, pp.267-268, 2005. [15] Abd, M. E. and M. Kamel, Factors governing the behavior of multiple cooperating swarms, Proc. of the Genetic and Evolutionary Computation Conference, Washington D.C., USA, pp.269-270, 2005. [16] Chang, J. F., S. C. Chu, J. F. Roddick and J. S. Pan, A parallel particle swarm optimization algorithm with communication strategies, Journal of Information Science and Engineering, vol.21, no.4, pp.809-818, 2005. [17] Jin, Y. C., A comprehensive survey of fitness approximation in evolutionary computation, Soft Computing, vol.9, no.1, pp.3-12, 2005. [18] Bull, L., On model-based evolutionary computation, Soft Computing, vol.3, pp.76-82, 1999. [19] Jin, Y., M. Olhofer and B. Sendhoff, On evolutionary optimisation with approximate fitness functions, Proc. of the Genetic and Evolutionary Computation Conference, San Francisco, CA, USA, pp.786-793, 2000. [20] Salami, M. and T. Hendtlass, A fitness estimation strategy for genetic algorithms, Proc. of the 15th International Conference on Industrial and Engineering, Applications of Artificial Intelligence and Expert Systems, pp.502-513, 2002. [21] Sastry, K., D. E. Goldberg and M. Pelikan, Don’t evaluate, inherit, Proc. of the Genetic and Evolutionary Computation Conference, pp.551-558, 2001. [22] Salami, M. and T. Hendtlass, A fast evaluation strategy for evolutionary algorithms, Applied Soft Computing, pp.156-173, 2003. [23] Kim, H. S. and S. B. Cho, An efficient genetic algorithms with less fitness evaluation by clustering, Proc. of the IEEE Congress on Evolutionary Computation, pp.887-894, 2001. [24] Reynolds, C. W., Flocks, herds and schools: A distributed behavioral model, Computer Graphics, vol.21, no.4, pp.25-34, 1987. [25] Kennedy, J., The particle swarm: Social adaption of knowledge, Proc. of the International Conference on Evolutionary Computation, Indianapolis, USA, pp.303-308, 1997.