A Novel Tangent based Framework for Optimizing

4 downloads 0 Views 1MB Size Report
genetic operators constitutes a very efficient mechanism for diversification on the ... tend in general to spend most of the time exploring a re- stricted area of the ...
Vol. 4 No. 2 Feb 2013 ISSN 2079-8407 Journal of Emerging Trends in Computing and Information Sciences c

2009-2013 CIS Journal. All rights reserved.

A Novel Tangent based Framework for Optimizing Continuous Functions Bjørn Jensen, Noureddine Bouhmala, Thomas Nordli

Abstract—We present a novel framework for optimizing continuous functions based on genetic algorithms (GAs). The framework utilizes a scheme for approximating the slope of the functions in contrast to standard GA implementations which simply exploit the function values. We present experimental results which show that this adaption in general provides significantly higher accuracy compared to the standard GA implementations.

Keywords: optimization, calculus, genetic algorithms, continuous optimization I. Introduction The problem of finding numerically the global minimum (or maximum) of a continuous function of a single real variable f (x) in a simply connected search domain is of central importance in the sciences and engineering. Several methods have been proposed in order to solve this problem. These methods can generally be classified as either deterministic or stochastic. The deterministic methods can be very precise when the region where the global optimum is located is known. One of the strengths of the stochastic approaches is the ability to quickly find the region where the global maximum is located. However, they lack the ability of the deterministic approaches to more accurately locate the position of the global maximum in search space. In this paper we present a novel search approach which seeks to marry the precision of the deterministic approaches with the localization ability of the stochastic approaches. The search approach we present is situated in a genetic algorithm (GA) environment. Deterministic methods attempt to produce search trajectories in search space which hopefully converge to points which satisfy a criterion of local optimality. Central to several of the deterministic methods is that they employ the tangent to the curve which is searched. The criterion to be satisfied by a given function f (x) at the optima is prescribed by Calculus as df (x) = 0. (1) dx Deterministic methods are efficient to employ in finding the global optimum when the approximate location of it is known; situations where the starting point belongs to the region of attraction of the global optimum. This implies that a major problem with deterministic methods is the inherent inability to differentiate between local and global optimal solutions. The possibility thus exists that the algorithm gets trapped in a local minimum, e.g. This deficiency follows directly from the fundamental condition Eq.(1.1) since it is entirely of a local nature without any generic abilities to take global features of the function f (x) into account. Stochastic methods seem to be the right tools to employ when the problem at hand possesses more than one Departement of Micro and Nano Systems Technology, Vestfold University College, Norway [email protected]. Department of Maritime Technology and Innovation,Vestfold University College, Norway [email protected]. Departement of Technology,Vestfold University College, Norway [email protected].

optimum. These methods attempt to reasonably cover the whole search space so that all optima are visited. One main difference between deterministic and stochastic algorithms is that in the stochastic methods points that worsen the function f (x) can also be generated and be part in the search process. However, the performance of stochastic algorithms such as GAs deteriorates very rapidly when evolved mostly due to the following three reasons. First, the complexity of the problem usually increases with the size of the search space. Secondly, the solution space of the problem increases exponentially with the search space size and, thirdly, deterioration ensues due to the methods lack of ability of intensification. In the GA environment the combination of a population of potential solutions and genetic operators constitutes a very efficient mechanism for diversification on the search space. However, GA algorithms completely lack comparable generic mechanisms to enable quick identification of better solutions in a localized region of the search space (intensification, i.e.) and hence the lack of precision which is typical for GA-based search method results. Due to the issues above optimization search techniques tend in general to spend most of the time exploring a restricted area of the search space. This prevents the search to visit more promising areas. This naturally leads to solutions of poorer quality. Clearly, what is needed is an algorithm which manages to combine the general strengths of the deterministic and the stochastic approaches. Designing efficient optimization search techniques obviously requires a tactical balance between diversification and intensification. In [6] the authors discuss the latest design of hybrid GA based approaches in order to find an adequate balance between diversification and intensification. In this paper a novel GA based approach is developed which combines some of the best diversification and intensification properties of the two main search schemes. Tangent based deterministic approaches are efficient in tracking down the position of an optimum in a given region. This can be seen as due to the tangent to a function f (x) as a measuring device for how the curve bends. From that information it is straightforward to design an algorithm which constructs an efficient and successful search trajectory in a given domain. The approach developed in this paper also utilizes the information carried by the tangent to a curve. However, in contrast to the deterministic approaches in the literature our GA based stochastic scheme utilizes the information carried by the tangent to the integral function 239

Vol. 4 No. 2 Feb 2013 ISSN 2079-8407 Journal of Emerging Trends in Computing and Information Sciences c

2009-2013 CIS Journal. All rights reserved.

g(x) to f (x), i.e. dg(x) . (2) dx The GA algorithm is fed with the f (x)-values, but since these are calculated from the tangent of another function we infer from the above that the GA algorithm should naturally inherit the intensification ability of the deterministic approaches. This expectation is indeed confirmed by the experiments we present in this paper where it is clearly demonstrated that our approach often produces more accurate search results than comparable traditional GA based searches. Recently the work conducted in [30] also uses an approach that exploits the notion of the derivative within an evolutionary approach, but in a very different way than ours. In that work a quasi-Newton optimizer is used on the best individual at the end of each generation of a GA algorithm. The evolutionary part of the approach is simply employed as a means to quickly locate a local optimum. The approach in [30] thus deviates fundamentally from our scheme. This paper is organized as follows. In the next section we discuss meta-heuristics used for optimizing continuous functions followed by Section 3 where we introduce our novel optimization framework. In Section 4 we describe the approach. Section 5 discusses implementation issues. In Section 6 we present and analyze benchmarks from several test runs taken from several generic functions. In the final section we discuss our findings and provide a perspective for further research. f (x) =

II. M ETA - HEURISTICS - A BRIEF REVIEW Many real world problems can be formulated as continuous optimization problems when the variables at hand live in a simply connected domain. Increased interest has been shown over the past few years to solve this kind of problem using different meta-heuristics. In this section we give a very brief overview of most important stochastic approaches used applied in the continuous optimization. At the heart of Simulated annealing (SA) [18] there is an analogy between thermodynamics, specifically with the way that liquids freeze and crystallize and the solution of optimization problems. The SA algorithm is a random search technique which avoids getting trapped in local minima. This is achieved by accepting in a biased manner all possible moves including those corresponding to a worse objective function value. A probabilistic acceptance criterion is employed such that the probability of accepting a bad move decreases as the algorithm progresses. The moves corresponding to a deteriorating result make it possible to escape from local optima. The first simulated annealing algorithm applied to continuous optimization was reported in [25]. Major contributions have more recently been reported in [24] [13] [23] [29] [4] [31]. The Tabu Search [14] [15] algorithm is another popular meta-heuristic which has been applied with great success to various combinatorial optimization problems. The algorithm starts from a randomly selected initial solution S. From this solution a set of nearby solutions S ? is generated. In order to avoid visiting solutions that the algorithm has just left (cyclic behavior, i.e.), and hence in order to avoid getting stuck in a local minimum (e.g.), a so called tabu list is created and maintained. This

finite list contains the moves that the algorithm has recently utilized. These are treated as disallowed (tabu, i.e.) moves by the algorithm. The objective function to be minimized is evaluated for each nearby solution S ? . The best nearby solution becomes the new current solution even if it is worse than S. Moves generally remain tabu only during a certain number of iterations. When the tabu list is full the oldest entry is removed from the list. The algorithm has proven computationally successful when tested on several functions exhibiting several minima [8] [12] [27]. Evolutionary algorithms represent a class of meta-heuristics which has gained wide acceptance in solving complex problems. Unlike other meta-heuristics genetic algorithms operate on a population of potential solutions applying the principle of survival of the fittest to produce better and better approximations to a solution. At each generation a new set of approximations is created by the process of first selecting individuals according to a selection scheme (based on their fitness relative to some fitness function) with the consequent application of breeding operators on pairs of individuals in the pool of selected individuals. The following papers [7] [19] [28] [2] [3] present an excerpt from the literature regarding the application of evolutionary algorithms in solving continuous optimization problems. Additional application of various meteorites for solving continuous optimization problems can be found in [17] [11] [1] [22] [26]. We note that the meta heuristics above are based on the numerical values generated directly from the function to be optimized at single discrete points in the parameter space. Recently a hybrid approach combining evolutionary algorithm methods with a derivative based, quasi-newton method has been proposed to solve unconstrained optimization problems [20] In the next section we will present an optimization framework set in a GA environment which deviates considerably from the aforementioned approaches. III. T HE OPTIMIZATION FRAMEWORK Deterministic optimization algorithms often employ the principles of Calculus in both the search for, as well as in identifying, the positions of the points in search space which correspond to optima for a given function. These algorithms are, compared to the stochastic ones, able to generate very precise results. At the very core of these schemes resides the notion of the derivative of the function which is to be optimized. Monitoring the development of the tangent of a given function along a search trajectory will tell the algorithm whether it approaches or is reseeding from an optimum. The method does even carry with it a mechanism to identify the optima through the condition Eq.(1.1). Furthermore, when the region of the location of an optimum is found the search can be intensified so as to generate searches with higher precisions. The search precision is in principle only restricted by the accuracy of the math libraries used by the computer code which is employed. This ability of intensification is missing in the frameworks of the stochastic methods found in the literature. The stochastic processes will naturally counter attempts at intensifying search results. In the GA environment this is partially due to a ’premature’ convergence of the content of the genetic material in the selected population. The development of the population is aimed at a future population which 240

Vol. 4 No. 2 Feb 2013 ISSN 2079-8407 Journal of Emerging Trends in Computing and Information Sciences c

2009-2013 CIS Journal. All rights reserved.

is best fit in relation to a fitness function which calculate a value used to discriminate the individuals. This implies less genetic variation with time, something which quite logically implies a continuously decreasing ability to generate better results (intensification) with time. A degree of mutation in the population can be introduced in order to increase the genetic diversity. However, this will not alter the general picture painted above of the intensification properties of genetic algorithms to any significant degree. With or without mutation genetic algorithms display less than satisfactory intensification properties [10]. Our optimization approach is designed to bypass the natural loss of genetic variation over time in genetic algorithms. This is obtained by feeding the genetic operators with more information about the function to be optimized than the mere function values, as is usually done, by utilizing the tangent tool. However, this is not achieved in our approach by feeding the operators with the information about the tangent space of the function to be optimized. This could very well have been an option, and it does represent the natural adaption of the use of the tangent tool in the deterministic approaches. However, such an approach will not generically be able to automatically identify the global optimum. It will inherit the same challenges in this regard as the deterministic approaches. However, by optimizing instead the derivative of the integral function to the actual function to be optimized (see Eq.(1.2)) we will be able to bypass that issue in a natural manner. Such an approach will utilize the information residing in the tangent space of the integral function. This, one would expect, will make searches with higher precision possible as in the deterministic algorithms, and hopefully without compromising the speed at which GA algorithms usually manage to locate the region of the global optimum. This constitutes the core in our novel framework for optimizing continuous functions based on genetic algorithms. In order to implement and put our framework into perspective we need to address certain mathematical issues. We have found it useful to discriminate between two possible implementation schemes which we will call scheme I and scheme II. Scheme I Let us first define the search domain. Choose a partition IN of the simply connected open domain D (D ∩ ∂D = ∅) of f (x) consisting of finite open sets Ii (Ii ∩∂Ii = ∅), IN = {Ii , i ∈ {1, ..., N }} such that D0 ≡

N [

Ii | Ii ∩ Ij = ∅ ; i 6= j

(3)

i=1

is a partial covering of D. Note that D0 ⊂ D; ∃ xi ∈ D | xi ∈ / D0 . The xi ’s constitute the set Xn ≡ {xi ∈ D − D0 }. We will assume that D0 is maximal in the sense that XN = {xi ∈ D − D0 ; i ∈ {1, ..., N − 1}} .

(4)

Note that such partitions do not fix the relative geometrical sizes (relative to some measure) of the domains. Let for definiteness D ≡ (0, 1) ⊂ 0 and sufficiently small. We will return to what we precisely mean by smallness later. We will employ this latter discretization in the following. We also defy at this point from defining the numerical precision in both the x-values and the h-value. We will employ ∆M as the fitness function of choice in this paper. It is interesting to note that fitness functions in our approach in general define fitness surfaces U in (x0 , h, u)space. To each function f (x) we thus naturally associate a fitness surface Uf (x0 , h) in general. This should be contrasted with the standard GA-approach which deals with fitness functions and fitness curves. Scheme II Interestingly, we can naturally relax some of the assumptions in scheme I. There we assumed that x0 ∈ Ii , x0 + h ∈ Ii+1 . This assumption was applied since it secures that the differences appearing in the formula above are strictly forward propagating thus securing a consistent approximation of the derivative operator since we assume that h > 0. However, we could assume that x0 ∈ Ii , x0 + h ∈ Ii (x0 − h ∈ Ii ). We will call this approach Scheme II. Note that since x0 is assumed a randomly generated number this approach naturally accommodates a dynamic value on h which may vary between domains in the same search. Still, assuming h > 0 we must evaluate the absolute values of the difference formulas above if we are to retain consistency with calculus. Simple set-ups Possibly the simplest set-up of both schemes is as follows. Partition the open interval D into Ii ’s with equal geometric sizes (lengths ≡ L) relative to the canonical Euclidean measure. Then generate an initial population by choosing the center point in each Ii . In Scheme I we can thus choose h = 23 L, e.g. In Scheme II we can similarly choose h = 25 L. We employ scheme I in this work. From the above it follows that we in our approach in fact involve three different function spaces on two different point sets. We can put this on a more logical foundation. The fundamental function is the one defined by g(xi ) when g is defined on the set S = {xi }. We call this construction the g-space which we denotes by Πg . A second space is naturally defined by g 0 (xi ) when xi ∈ Sf which we denote by Πg0 . A third space is defined by the approximator ∆R/M defined on the set S 0 = {xi − ∆x, xi + ∆x} and denoted Π∆ . We call this space the secant space. It is natural to extend this collection of spaces with a fourth space defined by some approximator A acting on the set S+ defined as the direct sum of S and S 0 , S+ = S ⊕ S 0 . We are in this way able to construct more general approximation schemes using enlarged spaces by selecting even more 241

Vol. 4 No. 2 Feb 2013 ISSN 2079-8407 Journal of Emerging Trends in Computing and Information Sciences c

2009-2013 CIS Journal. All rights reserved.

refined approximators like splines. However, in this paper the meta heuristic will only make use of the secant space. IV. A SIMPLE GENETIC ALGORITHM In this section we will provide the details of an implementation of the framework presented above within a GA environment. The implementation is a straightforward modification of A Simple Genetic Algorithm (SGA) as described in [16]. In SGA no parents survive from one generation to the next. This is called a non-elitist approach as opposed to an elitist approach where the most fit individuals (’chromosomes’) are kept from one generation to the next. The new generation is exclusively populated by crossing paired individuals in the previous generation. The pairs are picked from a mating pool. We will let the size of the pool coincide with the size of the original population. The algorithm we have implemented is depicted in Algorithm 1 below. In the subsequent subsections below we describe the main technical features of our modified SGA-implementation. We emphasize in particular deviations from the SGA. Algorithm 1: A Modified Simple Genetic Algorithm begin Generate initial population ; Evaluate the fitness of each individual in the population ; while (Last generation not reached) do Select individuals for mating using roulette function ; Produce new population by crossing the selected pairs ; Evaluate the fitness of the individuals in new population ; Replace the previous population with the new population ; end Return the best individual ; end

B. Fitness function In order to compute the fitness of the individuals their chromosomes are passed to a fitness function which we choose equal ∆M (x0 , h) (Eq.(3.4)). A higher return value signifies a better fit. The population will therefore evolve towards the global maximum of the function which is considered. C. Initial population The initial population is generated by randomly selecting 100 floating point numbers in the half-open interval [0.0, 1.0). D. Selection The selection of individuals to be included in the mating pool is done by iteratively picking individuals with a roulette function. In the roulette method, the selection is stochastic and biased toward the best individuals. The first step is to calculate the cumulative fitness of the whole population through the sum of the fitness of all individuals. The normalized probability pi for selecting an individual is calculated as fi (9) pi = P j fj where fj is the fitness of individual j. The roulette function is implemented as shown in Algorithm 2. Algorithm 2: Algorithm implementing the roulette function input : X - Population ordered from least fit to best fit output: x - Selected individual r ← random(); for x ∈ X do if r < f itness(x) then break ; else r ← (r − f itness(x)) The probability for being included in the mating pool is proportional to the fitness of the individuals. In this way the selection is both stochastic and biased towards the best solution.

A. Representation

E. Crossover

The chromosomes are stored in memory as floating point numbers. This representation is used by the fitness functions. In order to do bitwise manipulation the crossover operator needs an integer representation. Therefore the chromosomes are converted to an integer representation using the function code() defined by

When the mating pool is filled up by the roulette function pairs of individuals are picked orderly out of the pool. A crossover operator is applied to each pair creating two new solutions. Thus each individual in the mating pool produces two children, and every child have two parents. The genes are ordered with increasing numerical significance. To make sure that the genes inherited from each parent will be a good mix of genes of high and low numerical significance a uniform crossover with probability 0.5 is used, as described in [9]. This means that each gene in the child have an equal chance of being selected from any of the parents irrespectively of the position in the genome. This is a deviation from SGA as it uses a two point crossover. The crossover operator is implemented as shown in the Algorithm 3. In Algorithm 3 code() is given by Eq.(7), not() returns a bitstring that is binary complement to the passed argument,

code(xf loat ) = int(2k xf loat ) ,

(7)

where xf loat is the floating point representation, and int() is a function returning the nearest integer. k is the number of significant bits in the floating point representation. The children which are generated are converted back to a floating point representation using the function decode() defined by decode(xint ) = 2−k xint , where xint is the integer representation.

(8)

242

Vol. 4 No. 2 Feb 2013 ISSN 2079-8407 Journal of Emerging Trends in Computing and Information Sciences c

2009-2013 CIS Journal. All rights reserved.

Algorithm 3: Algorithm implementing the crossover operator. input : bitstring representation of parents, p1 and p2 output: bitstring representation of children, c1 and c2 m1 ← code(random()); m2 ← not(m1); c1 ← (p1&m1)|(p2&m2); c2 ← (p1&m2)|(p2&m1);

Each experiment waw set up to run 250 times. Each run evolves the population of 100 individuals to (and including) the 500th generation. These values are based on trail and error in relation to convergence of the search results. In addition to these experiments with the modified SGAalgorithm we also made a parallel search directly on each f (x) using the modified SGA-algorithm. The experiments were all conducted on an 64-bit Intel Xenon E5420 utilizing the four cores available at 2.50 GHz using phyton version 2.5 with NumPy version 1.1.0.

& is the bitwise and operator, | is the bitwise or operator and random() returns a random bitstring. Figure 1 shows an example of the production of two children which we denote by c1 and c2 respectively by crossing the two parents p1 and p2 using Algorithm 3. In 1 individuals are represented as strings of only eight bits to make it more readable.

A. Results In the appendix we have collected the output from the experiments. We present the output both in a tabular form and in the form of plots. The plots together with the figure captions should be self explanatory. The tables, two for each function pair f (x), g 0 (x), are organized with the categories in the table below. The table caption informs about which

Fig. 1.

Illustration of the crossover operator.

function

best

worst

mean

std

excess

TABLE I T HE ORGANIZATION OF THE TABLES IN THE APPENDIX .

F. Mutation Our algorithm does not implement mutation; the mutation probability is set to zero. G. Stop criteria The evolution terminates when reaching the maximum number of generations.

function pair above that has been tested. In the function column we indicate the function which has been tested, and the value of h which has been employed. The first entry in this column is the f (x) function. This function is tested using same algorithm as the other functions. The second column best represents the best result after 250 runs. The third column worst is the worst result after 250 runs. The fourth column mean is the mean of the 250 runs. The fifth column std is the standard deviation from the mean. The sixth column excess is the difference between the best result (the best column) and the analytically exact solution.

VI. A NALYSIS The plots in Figures 2-6 exhibit some general features V. E XPERIMENTS with GA searches. All searches exhibit a rapid convergence Five experiments were conducted using our modiuntil a maximum is reached from where the searches in fied SGA-algorithm. The following pairs of functions general move away from a best estimate of the global opti0 f (x), g (x) where tested mum. It is particularly interesting to observe the relatively  1: f1 (x) = ex rapid convergence of the searches since all the searches    ex+h −ex−h 0  are based on a non-elitist approach. An elitist approach g (x) =  1 2h    would probably exhibit even faster convergence properties.    1  However, it is known that this will decrease the search 2: f (x) =  2 1+x2    arctan(x+h)−arctan(x−h) precision [21]. Since our aim is to develop a more precise 0  g2 (x) =  2h   code the non-elitist approach is probably the best choice     3: due to its relatively strong convergence properties. 2 f3 (x) = x (10) 3 3 1 1 We observe that the value on the h-parameter is crucial (x+h) − (x−h) 3  g30 (x) = 3  2h  for the success of the tangent based searches. We do also     observe that no single value on h will always generate the   4: f4 (x) = e−x   best search. In fact, the best value on h does vary over a  −x−h −x+h  −e  g40 (x) = − e  very wide range of values in the collection of experiments 2h     studied in this work. The best h-value is read directly from    f5 (x) = e−2πx cos(2πx)  the tables provided with each plot in the excess column;  5: 1 −2πx g50 (x) = ( 4π e (sin(2πx) − cos(2πx)))0 a lesser value implies a more precise search result. From −11 in Each of these pairs were all tested using several different these data we−5learn that h is in the range from 10 Table 2 to 10 in Table 6, e.g. values on h. These values were Thus far we have not addressed possible sources for h ∈ {10−3 , 10−5 , 10−7 , 10−9 , 10−11 , 10−13 } . (11) errors. The most serious possible source for errors is 243

Vol. 4 No. 2 Feb 2013 ISSN 2079-8407 Journal of Emerging Trends in Computing and Information Sciences c

2009-2013 CIS Journal. All rights reserved.

connected with representational issues. phyton uses the IEEE-754 double-precision binary floating-point format (binary64) which gives a precision of 53 bits. This implies that errors which are induced due to the representation of floating point numbers in our calculations will occur (at the latest) at log2 (1053 ) ∼ 9 · 10±15 . This implies in our case a natural lower bound on h which is thus advisable to avoid. Our experiments are performed well above this critical value. VII. C ONCLUSIONS In this paper we have presented a novel framework for optimizing continuous function of a single real variable within a genetic algorithm environment. We have also described a concrete phyton implementation of it together the results from several experiments we have conducted. Our initial intent was to create code which exhibits better intensification properties than the standard genetic algorithm search approach. The plots prove that our approach does exactly that in general. Only in the experiments related to Figure 2 does the traditional GA approach prove the most precise. The experiments presented in this paper clearly shows a typical ten-fold increase in the intensification properties of the genetic algorithm when our approach is employed compared to standard genetic algorithm searches. In this paper we have experimented on a set of very simple functions. This choice was made in order to be able to simplify the analysis and thus provide a transparent proof of concept. Clearly, further experiments must be conducted with more complicated functions. The research in this paper can be extended in various directions. We observe that the value on h is crucial. Why this is so is unclear at present. However, this feature seems to imply that an implementation of a dynamic h-parameter should prove very interesting. One could f.ex. contemplate a scenario where the code starts out with a relatively large h-value and that this value is progressively lowered as the search evolves with increasing degree of precision. If such a scheme works it would provide a novel kind of zooming mechanism . Clearly, some mechanism must be implemented which detects when the optimal h-value is attained. Hopefully, this could result in code with even better intensification properties than the implementation the current research has produced. At the core of the tangent approach in this work lies the integral function to the actual function to be optimized. Herein does also lie a challenge since the integral function to a given function is not readily available in general. In a practical context the whole optimization process should be automatic in a true genetic algorithm fashion. Even though the integral function is often difficult to derive in general, if at all, we do not, strictly speaking, need the exact result. We have been working with the exact integral functions in this paper. From a practical point of view we only need an approximation to them. One such approximation scheme would be to exactly integrate the expansion of the function to be optimized as a power series on the unit interval. Only those terms in the series need to be included such that the error made after truncating the expansion is less than the precision needed in the search results. Such a initial preprocessing stage should be quite straightforward to implement.

R EFERENCES [1] C. Ahn, R. Ramakrishna. On the scalability of real-coded bayesian optimization algorithm. IEEE Transaction of Evolutionary Computation, 12(3), 2008:307-322. [2] D. Ardia, J. Arango, N. Gomez. Jump-Diffusion Calibration using Differential Evolution. Wilmott Magazine, 55 (2011):76-79. [3] D. Ardia, K. Boudt, P. Carl, K. Mullen, B. Petersen. Differential Evolution with Deoptim: An Application to Non-Convex Portfolio Optimization. The R Journal, 3(1), 2011:27-34. [4] S. Bandyopadhyay, S. Saha, U. Maulik, Deb K: A Simulated Annealing-Based Multiobjective Optimization Algorithm: AMOSA. IEEE Transactions on Evolutionary Computation, 12(3), 2008:269283. [5] C. Blum and A. Roli. Metaheuristics in combinatorial optimization: Overview and conceptual comparison. ACM Computing Surveys, 35(3),2003:268-308. [6] C. Blum, J. Puchinger, G.R. Raidl, and A.Roli. Hybrid metaheuristics in combinatorial optimization: A survey. Applied Soft Computing,11, (2011):4135-4151. [7] R. Chelouah and P. Siarry. Genetic and Nelder-Mead algorithms hybridized for a more accurate global optimization of continous multiminima functions.European Journal of Operational Research, 148 (2003):335-348. [8] R. Chelouah and P. Siarry. Tabu Search Applied to global optimization. European Journal of Operational Research, 123, (2000):256270. [9] S.D. Cogs. Uniform crossover revisited: maximum disruption in real-coded GAs. GECCO’03 Proceedings of the international conference on Genetic and evolutionary computation: PartII, 2003:1576-1577. [10] K. Dejong. Genetic algorithms: a 30 year perspective. Perspectives on adaptation in Natural and Artificial Systems, L. Booker, S. Forrest, M. Mitchell, and R. Riolo, (2005), Eds.: Oxford University Press, [11] R. Formato. Central Force Optimization: A new Metaheuristic with Applications in Applied Electromagnetics. Progress In Electromagnetics Research, 77, 2007:425-491. [12] F. Franze, N. Speciale. A tabu-search-based algorithm for continuous multiminima problems. International Journal for Numerical Engineering, 50, 2001:665-680. [13] S.B. Gelfand and S.K. Mitter. Metropolis-type annealing algorithm for global optimization in Rd . SIAM Journal Control and Optimization, 31(1), January 1993:111-131. [14] F. Glover. Tabu Search: Part I, ORSA Journal on Computing, 1(3), 1989:190-206. [15] F. Glover. Tabu Search: Part II, ORSA Journal on Computing, 2(1), 1990:4-32. [16] D.E. Goldberg. Genetic Algorithms in Search, Optimization and Machine Learning. Addison-Wesley Longman Publishing Co., Inc. Boston, 1989, MA, USA. [17] F. Herrera, M. Lozano, D. Molina. Continuous scatter search: an analysis of the integration of some combination methods and improvement strategies. European Journal of Operational Research, 169(2), 2004:450-476. [18] S. Kirkpatrick, C.D. Gelatt, M.P. Vicchi: Optimization by simulated annealing. Science, New series, 220(4598), 1983:671-680. [19] M. Lozano, F. Herrera, F, N. Krasnogor, D. Molina. Real-coded memetic algorithms with crossover hill-climbing. Evolutionary Computation Journal, 12(3), 2004:273-302. [20] W. Mebane, S. Sekhon. Genetic Optimization Using Derivatives: The Rgenoud Package for R. Journal of Statistical Software, 42(11), 2011: 2011. [21] J. M. Renders and S.P. Flasse: Hybrid Methods Using Genetic Algorithms for Global Optimization. IEEE Transactions on Systems. MAN and CYBERNETICS-PART B. CYBERNITICS, 26 (2), 1996: 243-258 [22] N. Mladenovik, M. Drazic, V. Kovacevik-Vujcic, M. Cangalovik. General variable neighborhood search for the continuous optimization. European Journal of Operational Research, 191, (2008):753770. [23] K. Miettinen, M. Mkel, H.eikki Maaranen. Efficient hybrid methods for global continuous optimization based on simulated annealing. Computers and Operations Research, 33(4), (2006):1102-1116. [24] W.H. Press and S.A.Teukolsky: Simulated annealing optimization over continuous spaces. Comput. Phys., 5, 1991:426-429. [25] H.E. Romeijn and R.L. Smith. Simulated Annealing and adaptive Search in Global Optimization. Probability in the Engineering and Informational Sciences, 8, 1994:571-590. [26] K. Socha and M. Dorigo. Ant colony optimization for continuous domains. European Journal of Operational Research, 18(3), 2008:1155-1173.

244

Vol. 4 No. 2 Feb 2013 ISSN 2079-8407 Journal of Emerging Trends in Computing and Information Sciences c

2009-2013 CIS Journal. All rights reserved.

[27] S. Stepanenko, B. Engels. New Tabu Search based global optimization methods outline of algorithms and study of efficiency. Journal of Computational Chemistry, 29(5), 2008:768-780. [28] Y. Wang, C. Dang. An Evolutionary Algorithm for Global Optimization Based on Level-Set Evolution and Latin Squares. IEEE Transactions on Evolutionary Computation, 11(5), 2007:579-595. [29] Y. Wang and J. Zang. An efficient algorithm for large scale global optimization of continuous functions. Journal of computational and Applied Mathematics, 206(2), 2007:1015-1026. [30] W. R.M, J.S. Sekhon. Genetic Optimization Using Derivatives: The rgenoud Package for R. Journal of Statistical Software, 42(11) 2011. [31] Y. Xiang, S. Gubian. B. Suomela, J. Hoeng. Generalized Simulated Annealing for Efficient Global Optimization: the GenSA Package for R., The R Journal, Forthcoming. URL http://journal.rproject.org/(2012).

Fig. 2. Plot related to functions f1 (x) and g10 (x) in Eq.10. The red line represents the analytically exact solution. 2.7183

solution f() h=1e-03 h=1e-05 h=1e-07 h=1e-09 h=1e-11 h=1e-13

2.71828 2.71826 2.71824 2.71822 2.7182 2.71818 2.71816 2.71814 2.71812 2.7181

0

50

100

150

200

250

300

350

400

450

500

Fig. 3. Plot related to functions f2 (x) and g20 (x) in Eq.10. The red line represents the analytically exact solution. solution f() h=1e-03 h=1e-05 h=1e-07 h=1e-09 h=1e-11 h=1e-13

1

0.999998

0.999996

0.999994

0.999992

0.99999

0

50

100

150

200

250

300

350

400

450

500

Fig. 4. Plot related to functions f3 (x) and g30 (x) in Eq.10. The red line represents the analytically exact solution. 1.0003

solution f() h=1e-03 h=1e-05 h=1e-07 h=1e-09 h=1e-11 h=1e-13

1.00025 1.0002 1.00015 1.0001 1.00005 1 0.99995 0.9999 0.99985 0.9998

0

50

100

150

200

250

300

350

400

450

500

245

Vol. 4 No. 2 Feb 2013 ISSN 2079-8407 Journal of Emerging Trends in Computing and Information Sciences c

2009-2013 CIS Journal. All rights reserved.

Fig. 5. Plot related to functions f4 (x) and g40 (x) in Eq.10. The red line represents the analytically exact solution. 1.0003

solution f() h=1e-03 h=1e-05 h=1e-07 h=1e-09 h=1e-11 h=1e-13

1.00025

1.0002

function f2 (x) g20 (x) : h = 10−03 g20 (x) : h = 10−05 g20 (x) : h = 10−07 g20 (x) : h = 10−09 g20 (x) : h = 10−11 g20 (x) : h = 10−13

best 1.0000000000 0.9999996666 0.9999999997 0.9999999994 0.9999999998 1.0000000000 1.0000008634

worst 0.9999336086 0.9997362623 0.9999289350 0.9999015781 0.9998912696 0.9999226273 0.9999119588

1.00015

TABLE IV B EST VS WORST PERFORMANCE OF FUNCTIONS f2 (x) AND g20 (x) IN E Q .10.

1.0001

1.00005

1

0.99995

0.9999

0

50

100

150

200

250

300

350

400

450

500

Fig. 6. Plot related to functions f5 (x) and g50 (x) in Eq.10. The red line represents the analytically exact solution. 1

0.9999

mean 0.9999950758 0.9999938145 0.9999944485 0.9999930954 0.9999940803 0.9999946734 0.9999940392

std 0.000008 0.000018 0.000009 0.000013 0.000012 0.000009 0.000011

excess 0.0000049242 0.0000061855 0.0000055515 0.0000069046 0.0000059197 0.0000053266 0.0000059608

TABLE V BASIC DESCRIPTIVE STATISTICS OF FUNCTIONS f2 (x) AND g20 (x) IN E Q .10.

solution f() h=1e-03 h=1e-05 h=1e-07 h=1e-09 h=1e-11 h=1e-13

0.99995

function f2 (x) g20 (x) : h = 10−03 g20 (x) : h = 10−05 g20 (x) : h = 10−07 g20 (x) : h = 10−09 g20 (x) : h = 10−11 g20 (x) : h = 10−13

function f3 (x) g30 (x) : h = 10−03 g30 (x) : h = 10−05 g30 (x) : h = 10−07 g30 (x) : h = 10−09 g30 (x) : h = 10−11 g30 (x) : h = 10−13

0.99985

0.9998

0.99975

0.9997

best 0.9999998905 1.0000002026 0.9999999451 0.9999996831 0.9999998885 1.0000000827 1.0003109452

worst 0.9921798739 0.9960534565 0.9960899257 0.9989546815 0.9979835014 0.9960920977 0.9964251646

0.99965

0.9996

0

50

100

150

function f1 (x) g10 (x) : h = 10−03 g10 (x) : h = 10−05 g10 (x) : h = 10−07 g10 (x) : h = 10−09 g10 (x) : h = 10−11 g10 (x) : h = 10−13

200

250

300

best 2.7182817705 2.7182822499 2.7182816856 2.7182817708 2.7182818219 2.7182922580 2.7200464103

350

400

450

worst 2.7129736837 2.7129703330 2.7156259425 2.7076839348 2.6761104444 2.7155389049 2.7178259643

TABLE II B EST VS WORST PERFORMANCE OF FUNCTIONS f1 (x) AND g10 (x) IN E Q .10.

function f1 (x) g10 (x) : h = 10−03 g10 (x) : h = 10−05 g10 (x) : h = 10−07 g10 (x) : h = 10−09 g10 (x) : h = 10−11 g10 (x) : h = 10−13

mean 2.7181938758 2.7181250726 2.7181833500 2.7181291436 2.7180416318 2.7182095685 2.7200197650

std 0.000369 0.000653 0.000314 0.000791 0.002666 0.000288 0.000242

excess 0.0000879526 0.0001567558 0.0000984785 0.0001526848 0.0002401967 0.0000722599 -0.0017379365

TABLE III BASIC DESCRIPTIVE STATISTICS OF FUNCTIONS f1 (x) AND g10 (x) IN E Q .10.

500

TABLE VI B EST VS WORST PERFORMANCE OF FUNCTIONS f3 (x) AND g30 (x) IN E Q .10.

function f3 (x) g30 (x) : h = 10−03 g30 (x) : h = 10−05 g30 (x) : h = 10−07 g30 (x) : h = 10−09 g30 (x) : h = 10−11 g30 (x) : h = 10−13

mean 0.9999232269 0.9998985467 0.9999205628 0.9999438680 0.9999516603 0.9999218786 1.0002787487

std 0.000507 0.000456 0.000305 0.000121 0.000149 0.000360 0.000278

excess 0.0000767731 0.0001014533 0.0000794372 0.0000561320 0.0000483397 0.0000781214 -0.0002787487

TABLE VII BASIC DESCRIPTIVE STATISTICS OF FUNCTIONS f3 (x) AND g30 (x) IN E Q .10.

function f4 (x) g40 (x) : h = 10−03 g40 (x) : h = 10−05 g40 (x) : h = 10−07 g40 (x) : h = 10−09 g40 (x) : h = 10−11 g40 (x) : h = 10−13

best 0.9999996916 1.0000000014 0.9999999691 0.9999999617 0.9999999717 1.0000000827 1.0003109452

worst 0.9960424674 0.9960718632 0.9990090585 0.9989607958 0.9989936656 0.9960976488 0.9964251646

TABLE VIII B EST VS WORST PERFORMANCE OF FUNCTIONS f4 (x) AND g40 (x) IN E Q .10.

246

Vol. 4 No. 2 Feb 2013 ISSN 2079-8407 Journal of Emerging Trends in Computing and Information Sciences c

2009-2013 CIS Journal. All rights reserved.

function std f4 (x) g40 (x) : h = 10−03 g40 (x) : h = 10−05 g40 (x) : h = 10−07 g40 (x) : h = 10−09 g40 (x) : h = 10−11 g40 (x) : h = 10−13

best excess 0.9999346334 0.9999551902 0.9999565963 0.9999563883 0.9999552197 0.9999439499 1.0002598749

worst

mean

0.000286 0.000257 0.000118 0.000110 0.000107 0.000282 0.000366

0.0000653666 0.0000448098 0.0000434037 0.0000436117 0.0000447803 0.0000560501 -0.0002598749

TABLE IX BASIC DESCRIPTIVE STATISTICS OF FUNCTIONS f4 (x) AND g40 (x) IN E Q .10.

function f5 (x) g50 (x) : h = 10−03 g50 (x) : h = 10−05 g50 (x) : h = 10−07 g50 (x) : h = 10−09 g50 (x) : h = 10−11 g50 (x) : h = 10−13

best 0.9999998590 0.9999999596 0.9999999944 0.9999999666 0.9999999578 1.0000000827 1.0001027784

worst 0.9754130202 0.9877221022 0.9984294488 0.9938574134 0.9934659970 0.9509310006 0.9509754095

TABLE X B EST VS WORST PERFORMANCE OF FUNCTIONS f5 (x) AND g50 (x) IN E Q .10.

function f5 (x) g50 (x) : h = 10−03 g50 (x) : h = 10−05 g50 (x) : h = 10−07 g50 (x) : h = 10−09 g50 (x) : h = 10−11 g50 (x) : h = 10−13

mean 0.9997099837 0.9997872472 0.9999244381 0.9998799019 0.9998911359 0.9993977701 0.9996539707

std 0.001806 0.000940 0.000197 0.000465 0.000461 0.004508 0.003246

excess 0.0002900163 0.0002127528 0.0000755619 0.0001200981 0.0001088641 0.0006022299 0.0003460293

TABLE XI BASIC DESCRIPTIVE STATISTICS OF FUNCTIONS f5 (x) AND g50 (x) IN E Q .10.

247