Efficient Global Optimization Using Hybrid Genetic ... - Semantic Scholar

AIAA 2002-5429

9th AIAA/ISSMO Symposium on Multidisciplinary Analysis and Optimization 4-6 September 2002, Atlanta, Georgia

EFFICIENT GLOBAL OPTIMIZATION USING HYBRID GENETIC ALGORITHMS Kurt A. Hacker Postdoctoral Research Associate AIAA Student Member Dept. of Mechanical and Aerospace Engineering State University of New York at Buffalo [email protected] John Eddy Kemper E. Lewis Graduate Research Assistant Associate Professor, Corresponding Author AIAA Student Member AIAA Member Dept. of Mechanical and Aerospace Engineering Dept. of Mechanical and Aerospace Engineering State University of New York at Buffalo State University of New York at Buffalo [email protected] [email protected] ABSTRACT The optimization of many realistic large-scale engineering systems can be computationally expensive. The evaluation of a single design configuration can take minutes or hours, and although computing power is steadily increasing, the complexity of the analysis codes continues to keep pace. In this paper a novel hybrid optimization method is introduced to efficiently find the global optimal of complex, highly multimodal systems. The motivation lies in the fact that to optimize many realistic engineering systems often requires numerous computationally expensive analyses to be performed. Heuristic optimization algorithms such as Simulated Annealing or Genetic Algorithms often can locate near optimal solutions but can require many function evaluations. Local search algorithms, including both gradient and non-gradient based methods, are quite efficient at finding the optimal within convex areas of the design space but often fail to find the global optimal in multimodal design spaces. The hybrid optimization approach presented in this work switches between global and local search methods based on the local topography of the design space. The global and local optimizers work in concert to efficiently locate quality design points better that either could alone. To demonstrate the usefulness of the approach presented in this paper, two case studies of differing complexity are considered. INTRODUCTION In the design of today's increasingly complex engineering systems, the designer is increasingly dependent on computationally expensive computer analysis and simulation codes. Examples of such codes include finite element analysis (FEA), computational fluid dynamics (CFD), heat transfer and vehicle dynamics simulations. The execution time for

these types of analyses can be on the order of hours or days for a single function evaluation. This often prevents the application of formal optimization techniques that can require many such evaluations. Thus, the designer is restricted to examining a small subset of the feasible design space. Despite continuing increases in computing power, the complexity of these analysis codes seems to keep pace with computing advances, making multi-objective, multidisciplinary optimization and concept exploration time consuming to say the least1. In this paper we present the results of work to develop an approach that provides the designer with high quality designs with less computational cost than other optimization methods. This is important because computational cost is often directly related to the solution time. Since designers often have a limited time in which to solve a problem, long solution times may preclude the use of optimization techniques or even evaluating more than a cursory number of potential solutions. This can have a detrimental effect on the performance of the final design. In addition to being computationally expensive to solve, complex design optimization problems may have highly nonlinear design spaces with many local optima. Local optimization methods, which are very efficient in finding local optima within convex areas of the design space2, are not well suited to this type of problem. Conversely, global optimization techniques such as Genetic Algorithms and Simulated Annealing are better suited to finding global solutions in multimodal design spaces but may require many function evaluations before convergence to the global optimum is achieved. The remainder of the paper is divided up as follows. In the next section the technical foundation of this work is presented. This includes a discussion on the

1 American Institute of Aeronautics and Astronautics Copyright © 2002 by the American Institute of Aeronautics and Astronautics, Inc. All rights reserved.

use of Genetic Algorithms in optimization, hybrid methods, and regression and statistical analysis. Following this, the specific approach being presented in this work is described. Finally, the results from the case studies we are considering are discussed followed by some conclusions and recommendations. AN EFFICIENT HYBRID OPTIMIZATION APPROACH The optimization algorithm used in this work is a recently developed hybrid Genetic Algorithm3. Genetic Algorithms were originally developed to imitate the processes by which living beings evolved4. Nearly all design methodologies incorporate the concept of evolving designs. Even designers who do not subscribe to a methodology typically evolve their designs by trial and error. Genetic Algorithms have been successfully applied to a number of engineering problems in areas as diverse as optimal control, production scheduling, and gas turbine design5-8. They have also been combined with local search techniques in previous research9, 10. Genetic Algorithms are most useful for problems having multimodal design spaces. The fact that they do not consider any gradient information makes it possible for the algorithm to move between the peaks of a multimodal space. A local search algorithm would typically remain within a single mode throughout the optimization process likely resulting in a suboptimal solution. A suboptimal solution is also possible when using a GA in a multimodal space, but it is less likely. The final solution found by a GA is also likely to be closer to the true optimal than a solution found by a local optimization method. Although quite successful at locating good areas of the design space, a GA can suffer from poor convergence qualities. Local search algorithms, including both gradient and non-gradient based methods are quite efficient at finding the optimal within a convex area of the design space but often fail to find the global optimal design. In this paper a novel optimization approach is used that switches between global and local search algorithms based on the local topography of the design space. The global and local optimizers work in concert to efficiently locate quality design points more efficiently than either could alone. To determine when it is appropriate to execute a local search, some characteristics about the local area of the design space need to be determined. One good source of information is contained in the population of the GA. By calculating the relative homogeneity of the

population we can get a good idea of whether there are multiple local optima located within this local region of the design space. Figure 1 illustrates this idea. In the top portion of the figure all of the members of the population are located in a single minimum; the values of both the design variables and the objective function are similar. In the center portion of the figure is an area of the design space with two local minima, one better than the other. In this case both the design variables and objective function values are different. The bottom part of the figure also shows two local minima, but this time they are of equal magnitude. In this case the designs have similar objective function values but different design variable values. To quantify the relative homogeneity of the population, the Coefficient of Variance of the objective function and design variables is calculated. The Coefficient of Variance is a normalized measure of variation, and unlike the actual variance, is independent of the magnitude of the mean of the population. A high Coefficient of Variance could be an indication that there are multiple local optima present. Very low values could indicate that the GA has converged to a small area in the design space, warranting the use of a local search algorithm to find the best design within this region. By calculating the Coefficient of Variance of the both the design variables and the objective function as the optimization progresses, it can also be used as a criterion to switch from the global to the local optimizer. As the Coefficient of Variance of the objective function values and design variable values in the current population increase it may indicate that the optimizer is exploring new areas of the design space or hill climbing. If the variance is decreasing, the optimizer may be converging toward a local minimum, and the optimization process could be made more efficient by switching to a local search algorithm. The second method used in this work to help us determine when to switch between the global and local optimizer is regression analysis. The design information present in the current population of the GA can be used to provide information as to the local topography of the design space by attempting to fit polynomial models of various order to it.

2 American Institute of Aeronautics and Astronautics

Single Mode

15

F(x)

Sampled Designs Second Order Fit True Design Space

F(x)

10

x F(x)

Multiple Modes - Case 1

5 0 -5

-10 0 x F(x)

Multiple Modes - Case 2

x Figure 1: Distribution of the Final Population of a Genetic Algorithm The use of regression analysis to augment optimization algorithms is not new1. In problems in which the objective function or constraints are computationally expensive, approximations to the design space are created by sampling the design space and then using regression or other methods to create a simple mathematical model that closely approximates the actual design space, which may be highly nonlinear. The design space can then be explored to find regions of good designs or optimized to improve the performance of the system using the predictive surrogate approximation models instead of the computationally expensive analysis code, resulting in large computational savings. The most common regression models are linear and quadratic polynomials created by performing ordinary least squares regression on a set of analysis data.

1

2

3

4

5 x

6

7

8

9

10

Figure 2: Approximating Multiple Modes with a Second-Order Model To make clear the use of regression analysis in this work, consider Figure 2, which represents a complex design space. Our goal is to minimize this function, and as a first step the Genetic Algorithm is run. Suppose that after a certain number of generations the population consists of the sampled points shown in the figure. Since the population of the GA is spread throughout the design space, having yet to converge into one of the local minima, it seems logical to continue the GA for additional generations. Ideally, before the local optimizer is run it would be beneficial to have some confidence that its starting point is somewhere within the mode that contains the optimum. Fitting a second-order response surface to the data and noting the large error (the R2 value is 0.13), there is a clear indication that the GA is currently exploring multiple modes in the design space. In Figure 3, the same design space is shown after the GA has begun to converge into the part of the design space containing the optimal design. Once again a second-order approximation is fit to the GA’s population. The dotted line connects the points predicted by the response surface. Note how much smaller the error is in the approximation (the R2 is 0.96), which is a good indication that the GA is currently exploring a single mode within the design space. At this point, the local optimizer can be started to quickly converge to the best solution within this area of the design space, thereby avoiding the slow convergence properties of the GA.


Sampled Designs Second Order Fit True Design Space

F(x)

88 66 44 22 00 -2 -2 -4 -4 -6 -6 -8 -8 -10-10 55

The results in this paper focus specifically on how to best select the threshold levels a priori for a particular problem and how this affects efficiency and solution quality. In the next section, two case studies are considered. The first is a highly multimodal GA benchmarking problem with two design variables. The second is the same problem with the number of design variables increased to 10. CASE STUDIES AND RESULTS

5.5 5.5

66

6.5 6.5

7 7

7.5 7.5

8 8

8.5 9 8.5 9.0

x Figure 3: Approximating a Single Mode with a Second-Order Model After each generation of the GA the values of the Coefficient of Determination and the Coefficient of Variance of the entire population are compared with the designer specified threshold levels. The first threshold simply states that if Coefficient of Determination of the population exceeds a designer set value when a second order regression analysis is performed on the design data in the current GA population, then a local search is started from the current ‘best design’ in the population. The second threshold is based on the value of the Coefficient of Variance of the entire population. This threshold is also set by the designer and can range upwards from 0%. If it increases at a rate greater than the threshold level then a local search is executed from the best point in the population. The flowchart in Figure 4 illustrates the stages in the algorithm. The algorithm can switch repeatedly between the global search (Stage 1) and the local search (Stage 2) during execution. In Stage 1, the global search is initialized and then monitored. This is also where the regression and statistical analysis occur. In Stage 2 the local search is executed when the threshold levels are exceeded, and then this solution is passed back and integrated into the global search. The algorithm stops when convergence is achieved for the global optimization algorithm. Setting proper values for the threshold levels is very important to maintaining the efficiency of the hybrid algorithm. Set too low and extraneous local searches may be run, resulting in computational inefficiency. The same problem can result if they are set too high. The global search may be allowed to run too long, which can also be inefficient.

This problem considered in this paper was developed by Keane12 as a benchmarking tool for measuring the performance of Genetic Algorithms. It is similar to real world design problems in that it is multivariate, multimodal, and highly nonlinear. Although this is not an engineering design problem per se, it does have many characteristics present in complex design problems. The definitions of the objection function and two constraints for Case Study 1 appear in Equations 1 through 4. The variable n is the number of design variables and in this work is equal to 2 in the first case study and is increased to 10 in the second. This problem becomes increasingly multimodal as the number of design variables increases. Minimize: n

n

F ( x) = −

∑ cos 4 ( x i ) − 2 ∏ cos 2 ( x i )

i =1

i =1

(1)

n

∑ ix i2 i =1

Subject to: n

g1 ( x )

0.75 − ∏ x i ≤ 0

g 2 ( x)

∑ xi −

0 ≤ xi ≤ 10


i =1

n

15n

i =1

2

≤0

i = 1, n

(2)

(3) (4)

Initialize Global Search Methods

STAGE 1

Begin Global Search

Monitor Global Search Collect Statistics Perform regression analysis No

Thresholds Exceeded?

STAGE 2

Seed Population with Result from Local Search

Check Global Convergence

Yes

Trigger Local Search

Execute Local Search

No

Exit

Yes

Figure 4: Steps in Two-Stage Hybrid Optimization Approach To demonstrate exactly how multimodal the design space is, Figure 5 shows the design space with two design variables. To make it easier to view, the negative (-F(x)) of the design space has been plotted. F(x*)

The infeasible regions of the design space are the flat areas in the figure with an objective function value of zero. By inspection, the best design F(x*), has an objective function value of approximately –0.36 along the constraint boundary at [x1 ≈ 1.6, x2 ≈ 0.45]. Since there are few published results for this problem, the evaluation of the performance of the hybrid optimization method is based on the comparison with the traditional global and local searches applied individually. The goal of these two case studies is to study the performance of the hybrid algorithm for different values of the threshold levels for R2 and Coefficient of Variance and compare these results with those achieved using traditional methods. In the next section results for the case n=2 are presented followed by the results for n=10. RESULTS FOR N=2

Figure 5: Design Space for Keane Bump Test with Two Design Variables

In Table 1, results are presented for this problem solved using some traditional global and local search methods.


Method

F(x)

g1(x)

g2(x)

# Fcn. Evals

GA 1

-0.364

-0.0001

-12.9487

3643

GA 2

-0.365

-0.0002

-12.922

4652

GA 3

-0.336

-0.0005

-12.8066

2622

GA 4

-0.347

-0.017

-12.977

3312

GA 5

-0.321

-0.061

-12.802

2932

GA 6

-0.350

-0.011

-12.856

3463

Powell’s

-0.263

-3.934

-10.395

157

Conjugate -0.143 -0.0003 -10.134 92 Gradient Variable -0.277 -0.0500 -13.0391 1278 Metric Simulated -0.342 -0.0027 -13.0201 5558 Annealing Table 1: Results Using Traditional Optimization Methods for Keane Problem with 2 Design Variables Since the GA is not a deterministic algorithm the best six solutions are presented out of 15 runs. The best solution comes from the first run of the GA. The local search methods, Powell’s Conjugate Search, the Conjugate Gradient Method, and the Variable Metric Method failed to achieve good results, which is what would be expected in such a highly multimodal design space. Next, the best six solutions from 15 runs of the hybrid optimizer are presented. The settings for the hybrid optimizer are given in Table 2. The GA settings are the same as those used in obtaining the results in Table 1. The Variable Metric Method is used as the local optimizer. For these runs the settings are 0.6 for the threshold on R2. The threshold for the Coefficient of Variance is an increase in 5% over 3 generations. Global Search Algorithm: GA Local Search Algorithm: Variable Metric Method Pop Size: 500 Crossover Rate: 15% Mutation Rate: 5% R2 Threshold: 0.5 CV Increase Threshold: Increase in 5% over 3 generations Table 2: Settings of Hybrid Optimizer for Keane Problem with 2 Design Variables

The results are presented in Table 3 for these six hybrid optimizer runs. In a similar fashion to the GA, the best six runs out of 15 are taken. Method

F(x)

g1(x)

g2(x)

# Fcn. Evals

Hybrid -0.363 -0.0003 -12.901 2772 GA 1 Hybrid -0.365 -0.0002 -12.918 1911 GA 2 Hybrid -0.364 -0.0002 -12.952 3115 GA 3 Hybrid -0.364 -0.0003 -12.920 2800 GA 4 Hybrid -0.361 -0.0061 -12.945 3100 GA 5 Hybrid -0.365 -0.0001 -12.929 2266 GA 6 Table 3: Results Using Hybrid Genetic Algorithm for Keane Problem with 2 Design Variables These results along with those in Table 1 indicate that the optimum design has an objective function value in the neighborhood of –0.365. The results also indicate that the hybrid optimizer is capable of achieving that optimum more regularly and more efficiently than the traditional local and global methods used alone. The smallest number of function evaluations required by any of the traditional methods to get within 5 percent of –0.365 occurs in GA 4 where there are 3312 function evaluations. The maximum number of function evaluations taken by the hybrid method occurs in run Hybrid GA 3, where 3115 function evaluations are need. Moreover, all 6 runs of the hybrid GA were within 1 percent of the optimum. Next, the effect of the threshold values on the performance of the hybrid optimizer is considered for the same problem. The settings of the hybrid optimizer are kept the same except for the values of the threshold values, which are varied in an effort to determine the best settings of the hybrid optimizer for this problem. Three different settings were used for each of the variables. The performance for a particular set of threshold levels is taken to be the average of five consecutive runs, because the hybrid GA is not deterministic in nature. The threshold levels used are given in Table 4. The results from these runs are presented in Table 5.


R2 CV Threshold: % Increase Threshold over 3 generations 1 0.1 1 2 0.1 10 3 0.1 50 4 0.3 1 5 0.3 10 6 0.3 50 7 0.6 1 8 0.6 10 9 0.6 50 10 0.9 1 11 0.9 10 12 0.9 50 Table 4: Settings for Threshold Parameters for Keane Problem with 2 Design Variables

Run

There are a couple of interesting points that can be made from Table 5. First the best results strictly in terms of objective function value occur when the R2 threshold is set to 0.1, which would result in a local search when the population is still spread over multiple modes. This would seem to indicate that running a local search early on in the optimization process is beneficial for this problem providing good building blocks for the GA to utilize to jump to good areas of the design space. Another point is that the average number of function evaluations is similar for all the settings of the threshold values. This may be explained by the fact that when the threshold values are low multiple local searches are run which require additional function evaluations, while when the threshold values are high, few local searches are run, but more generations of the GA are executed. Finally, when compared with the results from the GA alone (Table 1), the hybrid optimizer at any of the settings used requires less function evaluations on average and delivers comparable results. A two-factor ANOVA13 analysis is performed to see if the settings for the threshold levels significantly impact the quality of the solution and the number of function evaluations. Table 6 contains the results of the ANOVA analysis with the dependent variable being the difference between F(x) and the optimal for this problem, -0.65. The analysis is performed with alpha equal to 0.10. The results show that the effect of the R2 threshold value is significant (P=0.07) while the effect of the Coefficient of Variance threshold is not (P=0.29).

Coefficient Avg. Avg. of Variance Avg. g1 Avg. g2 # Fcn. F(x) Threshold Evals 0.1 1 -0.362 0.0005 -12.9469 2536 0.1 10 -0.361 0.0002 -12.9576 2690 0.1 50 -0.338 -0.0020 -12.9815 2288 0.3 1 -0.334 -0.7901 -12.3747 2483 0.3 10 -0.331 -0.7942 -12.4374 3034 0.3 50 -0.330 -0.7873 -12.4036 2682 0.6 1 -0.302 -1.5744 -11.8424 2250 0.6 10 -0.350 -0.0052 -12.9038 2987 0.6 50 -0.334 -0.7907 -12.4670 2039 0.9 1 -0.331 -0.0547 -12.8604 2442 0.9 10 -0.329 -0.6428 -12.4955 2480 0.9 50 -0.300 -0.0489 -12.8762 2548 Table 5: Average Results for Different Threshold Settings for Keane Problem with 2 Design Variables R2

Source of P F SS df MS F Variation Value Critical R2 0.01 3 0.0031 2.50 0.07 2.20 Threshold CV 0.00 2 0.0016 1.26 0.29 2.42 Threshold Table 6: ANOVA Results for Determining Significance of Threshold Settings on Objective Function Value for Case Study 1 Table 7 contains the results of the ANOVA analysis when the dependent variable is the number of function evaluations needed to reach convergence. Here the effect of the CV threshold is statistically significant (P=0.03) but the effect of the R2 threshold is not (P=0.54). Source of P F SS df MS F Variation Value Critical R2 767049 3 255683 0.71 0.54 2.20 Threshold CV 2702007 2 1351004 3.77 0.03 2.41 Threshold Table 7: ANOVA Results for Determining Significance of Threshold Settings on Efficiency for Case Study 1


The results indicate that the CV threshold value has a significant impact on the efficiency of the solution and the R2 threshold value affects the quality of the solution. However it is dangerous to generalize the results from a single case study and only five runs for each setting. The same analysis of variance study is performed for the next, larger study. Although multimodal in nature, the small size of this design space makes it likely that the general area of the global optima can be found every run. In the next section, a much larger version of the same problem is considered to provide more insight into the issue of choosing good values for the threshold values. RESULTS FOR N=10 In this section a much larger version of the same problem is optimized. The number of design variables is increased to 10; otherwise the problem remains the same. The settings for the hybrid GA are presented in Table 8. Note that the mutation rate has been increased to 15% and the threshold on the Coefficient of Variance has been increased to 10%. This is to increase the exploration of the GA and decrease the number of local searches that are performed. Additionally, the population size has been decreased to 200 to reduce the number of function evaluations needed to reach convergence. The results from three runs of the hybrid GA and the GA by itself (using the same settings as the hybrid GA with no local search) are compared in Table 9. Since there are ten design variables, the values of the design variables are not listed. Global Search Algorithm: GA Local Search Algorithm: Conjugate Gradient Method Pop Size: 200 Crossover Rate: 15% Mutation Rate: 15% Reproduction Rate: 2 Children R2 Threshold: 0.8 CV Increase Threshold: Increase in 10% over 3 generations Table 8: Settings of Hybrid Optimizer for Keane Problem with 10 Design Variables For these runs, the Hybrid GA has performed marginally better than the GA with slightly better objective function values achieved in fewer function evaluations. One reason that the performance difference is less noticeable in this problem than with two design variables is that the design space has so many modes when there are ten design variables that

the improvement in objective function obtained by running a single local search is small. The local search portion of the hybrid GA is considerably more effective for problems where large increases in objective function can be gained with a single local search. In this case the local searches often seemed to locate local optima far from the global optimum, which only marginally enhances the overall global search. F(x*)

g1(x)

g2(x)

# Fcn. Evals.

-0.6245

-0.0167

-58.8743

4705

-0.6265

0.0007

-57.7155

3416

-0.6737

-0.0056

-59.3372

3362

GA 1

-0.6623

-0.0816

-57.5642

5110

GA 2

-0.537

-0.0604

-54.7410

3690

GA 3

-0.5828

-5.2794

-56.4285

3770

Method Hybrid GA 1 Hybrid GA 2 Hybrid GA 3

Table 9: Comparison of Hybrid GA and Normal GA for 10 Design Variables For completeness, other optimization algorithms are used to solve the same problem using random starting points. This is to demonstrate how inadequate local search methods are for design spaces of this complexity. These results are summarized in Table 10. Although these results are from only a single run, they show that the other methods are fairly unsuccessful at getting near the global optimum. The poor performance of the local searches is not surprising as these algorithms are designed only to find local optima. Method

F(x*)

g1(x)

g2(x)

# Fcn. Evals.

Powell’s

-0.2387

0.0010

-37.3571

1278

-0.1819

-46.3449

-37.4955

89

-0.1569

-0.0306

-34.8801

159

-0.4769

-0.0258

-49.0258

45585

Conjugate Gradient Variable Metric Simulated Annealing Table 10:

Comparison of Other Search Methods for 10 Design Variables

Next, the effect of the settings for the R2 and Coefficient of Variance thresholds on the efficiency


of the optimization is studied. The number of function evaluations and the final value of the objective function are examined for 12 different combinations of the settings. The settings for these combinations are listed in Table 11. CV Threshold: % Increase Run R Threshold over 3 generations 1 0.1 1 2 0.1 7 3 0.1 15 4 0.3 1 5 0.3 7 6 0.3 `5 7 0.6 1 8 0.6 7 9 0.6 15 10 0.9 1 11 0.9 7 12 0.9 15 Table 11: Settings for Local Search Threshold Parameters 2

Run

F(x*)

g1(x)

g2(x)

# Fcn. Evals.

1

-0.5478

-0.0017

-54.6250

2806

2

-0.5478

-0.0017

-54.6250

2806

3

-0.6111

-0.0117

-59.2482

4365

4

-0.5478

-0.0017

-54.6250

2806

5

-0.6056

-0.1212

-54.2482

4365

6

-0.5455

-0.1179

-54.5334

2604

7

-0.5076

-0.0240

-54.4910

2772

8

-0.6032

-0.1304

-54.9646

3374

9

-0.6567

-0.1039

-57.4004

4475

10

-0.5059

-0.0682

-54.6155

2212

11

-0.5405

-0.0032

-57.9554

1806

12

-0.5544

-0.1827

-59.2923

2040

Table 12: Results of Hybrid GA for Different Settings of Local Search Thresholds

seems to be strongly related to the number of function evaluations, suggesting that the runs that performed poorly prematurely converged before good areas of the design space were located. As in Case Study 1, a two-factor ANOVA analysis is performed to see if the settings for the threshold levels significantly impact the quality of the solution and the number of function evaluations. Table 13 contains the results of the ANOVA analysis with the dependent variable being the difference between F(x) and the optimal for this problem, -0.70. The results show that the effect of the Coefficient of Variance threshold value is significant for an alpha=0.12 while the effect of the R2 threshold is not (P=0.42). Source of P F SS df MS F Variation Value Critical R2 0.005 3 0.002 1.09 0.42 3.29 Threshold CV 0.009 2 0.004 3.08 0.12 3.46 Threshold Table 13: ANOVA Results for Determining Significance of Threshold Settings on Objective Function Value for Case Study 2 Table 14 contains the results of the ANOVA analysis when the dependent variable is the number of function evaluations needed to reach convergence. Here the effect of R2 threshold would only be statistically significant if alpha where 0.20. The trends in the ANOVA analysis for Case Study 2 are generally opposite from those of Case Study 1. In this case study, the R2 threshold has more of an influence on the number of function evaluations while the CV threshold more strongly influenced the quality of the solution. This may be explained by the fact that in a small design space like that of Case Study 1, executing a local search early in the optimization may pay off because there is a decent chance that the search will occur within a good mode of the design space. This is not true with a larger problem like Case Study 2 where many thousands of local optima exist. The many local searches that would be run if the R2 threshold were low would likely result in a poor quality local optimum being found. Only once the GA has located a good area of the design space is there a great advantage to running a local search.

The results presented in Table 12 show that the best solution comes from Run 9, which has settings of 0.6 and 15% for the two thresholds. The worst performance comes from Run 10, which has settings of 0.9 and 1%. In general, the quality of the solution 9 American Institute of Aeronautics and Astronautics

Source of P F SS df MS F Variation Value Critical R2 4263906 3 1421302 2.14 0.20 3.29 Threshold CV 1058688 2 529344 0.80 0.49 3.46 Threshold Table 14: ANOVA Results for Determining Significance of Threshold Settings on Efficiency for Case Study 2 Although the threshold parameters had varying effects on the variation in the solution quality and efficiency, it is important to recognize that there are many factors that influence both the solution quality and efficiency. These include the initial population, random seed chosen, and other GA parameters. Further studies will be conducted using a larger number of runs (~100) to better determine the influence of the various parameters and optimal settings for them. CONCLUSIONS This section contains a discussion of the results of the case studies and some recommendations. •

•

•

•

The hybrid GA performed significantly better than the GA alone for Case Study 1, the Keane Problem with two design variables. Due to the small size of the design space, the hybrid algorithm found better solutions more efficiently than the GA alone because the local search could isolate the best design within a local area of the design space much faster than the GA. The performance difference between the hybrid algorithm and GA alone is much less for the case study with ten design variables. One reason for this may be that the sheer number of local optima present in the design space prevents a single local search from making any significant progress. The hybrid algorithm did manage to slightly outperform the GA, however, both in terms of the quality of the solution and the number of function evaluations. The values chosen for Coefficient of Variance and Coefficient of Determination thresholds directly determine the number of local searches run. The following rules of thumb can be suggested for determining the optimum settings for the thresholds for R2 and Coefficient of Variance. For problems with only a few local minima a high value of the regression threshold (greater than 0.9) is suggested. By waiting until the population has moved until a single mode the

•

•

local search has only to be run a single time to isolate the optimal solution. If the problem is of larger size, then designers need to decide how aggressive they want the local search component of the algorithm to be. An aggressive search may result in quick convergence of the algorithm for some problems if the GA can quickly find a good mode in the design space. If the problem has many modes, however, the 10 design variable Keane problem for example, then more function evaluations are required to get there although an aggressive search generally results in a higher final objective function value. An aggressive search may have settings of 0.5 and 5% threshold setting while a less aggressive local search might have settings of 0.9 and 15%.

In this paper an approach is presented to perform optimization using a hybrid Genetic Algorithm It is intended to efficiently locate the optimal design for systems that have nonlinear, multimodal design spaces and require computationally expensive performance analyses. Although the results in described in this paper are promising there are some issues that will be explored in greater depth in future work. Of primary importance is to more thoroughly study of the effect of the threshold levels on the performance of the hybrid optimizer over a much greater number of trials. In addition, other case studies need to be considered before the utility of the hybrid algorithm can be conclusively shown over larger classes of problems. ACKNOWLEDGEMENTS We acknowledge the support of the National Science Foundation, grant DMII9875706, and NASALangley Research Center, grant NGT152185 in this work. REFERENCES 1.

Simpson, T. W., Mauery, T. M., Korte, J.J., and Mistree F., “Comparison of Response Surface and Kriging Models for Multidisciplinary Design Optimization,” 7th AIAA/USAF/NASA/ISSMO Symposium on Multidisciplinary Analysis & Optimization, St. Louis, MI, AIAA-98-4755, 1998.


2.

Vanderplaats, G.N., Numerical Optimization Techniques for Engineering Design, Colorado Springs, CO, Vanderplatts Research and Development Inc., 1998.

3.

Hacker, K., “Efficient Robust Systems Design through the Use of Hybrid Optimization and Distributed Computing,” Ph.D. Dissertation, University at Buffalo, 2002.

4.

Bentley, P. J., Evolutionary Design by Computers, Morgan Kaufmann Publishers Inc., San Francisco, CA, 1999.

5.

Dakev, N. V., Chipperfield, A. J. and Fleming, P. J., “A General Approach for Solving Optimal Control Problems Using Optimization Techniques”, Proc. IEEE Int. Conf. on Systems, Man and Cybernetics, Vancouver, BC, Canada, Oct 22-25, pp. 4503-4508, 1995.

6.

Shaw, K. J. and Fleming, P. J., ”An Initial Study of Practical Multi-Objective Production Scheduling Using Genetic Algorithms'”, Proceedings of International Conference on Control '96, University of Exeter, September 25, 1996.

7.

Chipperfield, A.J. and Fleming, P.J., "Gas Turbine Engine Controller Design Using Multiobjective Genetic Algorithms", Proceedings 1st IEE/IEEE International Conference on Genetic Algorithms in Engineering Systems: Innovations and Applications, Sheffield, UK, pp. 214-219, 1995.

8.

Hajela, P., Lee E. and Lin, C. Y., “Genetic Algorithms in Structural Topology Optimization”, Topology Design of Structures, eds. M.P. Bendsoe and C.A. Mota-Soares, Kluwer Academic, pp. 117-134, 1993.

9.

Trabia, M.B., “ A Hybrid Fuzzy Simplex Genetic Algorithm,” Proceedings of the ASME Design Engineering Technical Conferences, Baltimore, MD, DETC2000/DAC-1423, 2000.

11. Gen, M. and Cheng, R. Genetic Algorithms & Engineering Optimization, New York, NY, John Wiley & Sons, Inc., 2000. 12. Keane, A.J., "Experiences with optimizers in structural design," Proceedings of the 1st Conf. on Adaptive Computing in Engineering Design and Control, ISBN-0-905227-33-6, published by the University of Plymouth, UK, I.C. Parmee, ed., pp. 14-27, 1994. 13. Box, G., Hunter, W.G. and Hunter, J.S. Statistics for Experimenters: An Introduction to Design, Data Analysis, and Model Building. New York: John Wiley and Sons, 1978.

10. Nassef, A.O., Hegazi, H.A., and Metwalli, S.M., “A Hybrid Genetic-Direct Search Algorithm for the Shape Optimization of Solid C-Frame CrossSections,” Proceedings of the ASME Design Engineering Technical Conferences, Baltimore, MD, DETC2000/DAC-14295, 2000.
