Interactive Evolutionary Computation for Model based Optimization of Batch Fermentation Janos Madar, Janos Abonyi, Balazs Balasko and Ferenc Szeifert University of Veszprem, Department of Process Engineering, Veszprem, P.O. Box 158, H-8201, Hungary
[email protected], http://www.fmt.vein.hu/softcomp ABSTRACT At the optimization of temperature and feeding profiles of batch processes it is often desirable to consider several objectives and constraints into the optimization problem. During the beer fermentation a temperature profile is applied to drive the process so as to obey to certain constraints. The design of this temperature profile is an optimization problem where the objective is to minimize the operation time and optimize the quality of the beer. Similarly to other practical problems, these objectives and constraints are often non-commensurable and the objective functions are explicitly/mathematically not available. In this paper, Interactive Evolutionary Computation (IEC) is used to effectively handle such optimization problems. IEC is an evolutionary algorithm whose fitness function is provided by human users. The proposed approach has been implemented in MATLAB and applied to design temperature profile for beer fermentation process. The results show that IEC is an efficient and comfortable method to incorporate the priori knowledge of the user into the model based optimization of batch processes. A detailed description of the proposed approach helps the construction of the algorithms; still easier, the developed EAsy-IEC Toolbox and the beer fermentation model written can be downloaded from the website of the authors: http://www.fmt.vein.hu/softcomp/EAsy KEY WORDS Model Based Optimization, Interactive Evolutionary Computing, Beer Fermentation
1 Introduction Batch processes play an important role in industry. During batch- and fed-batch operation of bioreactors the system states change considerably. As a consequence of the varying process states, the best operation results can be realized by varying the input variables along optimal trajectories during the operation time. This explains why searching for efficient methods for calculating the optimal trajectories has been an important issue for bioreactor control [1]. Several methods have been discussed in literature: e.g. firstorder gradient method and dynamic programming.
During the beer fermentation a temperature profile is applied to drive the process so as to obey to certain constraints. The design of this temperature profile is an optimization problem where the objective is to minimize the operation time and optimize the quality of the beer. These objectives are frequently in conflict with one another. Trade-offs exist between some objectives, where advancement in one objective will cause deterioration in another. These multi-objective optimization problems involve the simultaneous consideration of multiple performance criteria that should be defined prior to the optimization procedure. This requires in-depth information concerning the various trade-offs and valuation of each individual objective. Such detailed model-based multi-criteria optimization of the temperature profile of beer fermentation is discussed in several articles [2, 3, 4]. Similarly to other practical problems, in the optimization of beer fermentation the objectives are often non-commensurable and are explicitly/mathematically not available. This paper proposes the application of Interactive Evolutionary Computation (IEC) to handle these problems. IEC uses human evaluation in the optimization system [5]. Simply stated, IEC is a technique from the class of evolutionary algorithms (EAs), whose fitness function is replaced by a human. As in interactive evolution, the user selects one or more individual(s) which survive(s) and reproduce(s) (with variation) to constitute a new generation, IEC uses two different spaces for its search. The human user evaluates the output of the target system according to the distance between the target goal and the system output in psychological space [6]. On the other hand, the EA searches in the parameter space. It can be said that the IEC is the optimization technology where the EA and a human search are cooperatively based on the mapping between the two spaces. The idea of using human knowledge or intuition as part of an optimization process has appeared in e.g. [7]. Most optimization techniques which work by improving a single solution step by step are not suited for this technique. Genetic Algorithms (GAs) and other population based optimization procedures are better suited for interactive optimization, because a suggestion can be put into the population to see whether it is worth further consideration or not. A good suggestion can be expected to survive
and lead the optimization process towards a good solution, while a bad suggestion will be eliminated quite soon. These techniques have already been applied in computer graphics [8] and design [9]. Although in recent years several interesting and efficient approaches based on genetic or evolutionary algorithms (GAs) have been developed for the optimization of bioprocesses [2, 3, 10, 11], according to our best knowledge, the application of IEC for this purpose has not been studied yet. The proposed approach has been implemented in MATLAB (EAsy-IEC Toolbox). With the application of the developed toolbox the users can evaluate the performances of the individuals (set of solutions) and can directly modify the suggested temperature profiles. A detailed description of the proposed approach helps the construction of the algorithms; still easier, the EAsy-IEC Toolbox and the beer fermentation model written in MATLAB can be downloaded from the website of the authors: http://www.fmt.vein.hu/softcomp/EAsy The remaining part of the paper is organized as follow. In Section 2 the algorithm of Evolutionary Strategy is presented, which is tailored taking into account the specialities of IEC. Section 3 shows an application example when the proposed approach is applied to identification of a tendency model of a heating-cooling jacket. Finally, some conclusions are drawn in Section 4.
2
Evolutionary Strategy Based IEC
Contrary to automated evolutionary optimization algorithms, the IEC cannot use many individuals and searching generations because of resulting human fatigue. Hence, there is a need to develop and tune EAs which effectively search with a few individuals within a few generations. For this purpose, this section presents an algorithm based on Evolutionary Strategy.
2.1 Evolution Strategy Evolution Strategies [12] are search procedures that mimic the natural evolution of the species in the natural systems. They work directly with the real representation of the parameter set, searching from an initial population (a set of points). Like GAs, they only require data based on the objective function and constraints, and not derivatives or other auxiliary knowledge. Evolution strategies (ESs) were developed by [13], with selection, mutation, and a population of size one. Schwefel introduced recombination and populations (P) with more than one individual, and provided a nice comparison of ESs with more traditional optimization techniques [14]. The general scheme of the algorithm is given in table 2.1. According to this general scheme, the main elements of the algorithm used in this paper are the following.
Representation Search points in ESs are ndimensional vector of x ∈ Rn of object variables. To allow for a better adaptation to the objective functions’s topology, the object variables are accompanied by a set of the so-called strategy parameters. An ES-individual aj = (xj , σ j ) consists of two components, the object variables xj and up to n different standard deviations σj,i to control the step sizes, σ j = [σj,1 , . . . , σj,n ]T . Selection In the applied (µ+λ) Evolution Strategy, at a given generation (t), there are µ parents, and λ offspring are generated by recombination and mutation. Then, the µ+λ members of then next population are sorted according to their objective function values given by the user, and he or she selects the best µ to become the parents of the next generation. Mutation Because in the nature small changes occur frequently, but large ones only rarely, as mutation operator random numbers generated by normal distributions are added to the individuals, zj,i ∼ N (0, σj,i ): xj,i = xj,i + zj,i
(1)
Before the object variables are changed, the standard deviations are mutated using a multiplicative normally distributed process. (t)
(t−1)
σj,i = σj,i
exp(τ 0 N (0, 1) + τ Ni (0, 1))
(2)
with exp (τ 0 N (0, 1)) as a global factor which allows an overall change of the mutability and exp (τ Ni (0, 1)) allowing for individual changes of the mean step sizes σj,i . The τ 0 and τ parameters can be interpreted in the sense of global learning rates. Schwefel suggests to set them as [14]: 1 1 τ0 = √ , τ = p √ 2n 2 n
(3)
Recombination Recombination in ESs can be either sexual, where only two parents are involved in the creation of an offspring, or global, where up to the whole population contributes to a new offspring. Sexual recombination of just two individuals is often called local while the contribution of all individuals is called global recombination. Traditional recombination operators are discrete recombination, intermediate recombination, and geometric recombination, all existing in a sexual and global form.
Table 1. procedure of the ES t = 0; initialize population P(t); evaluate P(t); until (done) t = t + 1; parent selection P(t); recombine P(t); mutate P(t); evaluate P(t); survive P(t);
When F and M denote two randomly selected individuals from the µ parent population, the following operators can be defined: xF,i no recombination xF,i or xM,i discrete 0 (4) xi = (x F,i + xM,i ) /2 intermediate P µ global avarage k=1 xK,i /µ σF,i no recombination discrete σF,i or xM,i (σ σi0 = (5) pF,i + σM,i ) /2 intermediate (σ σ ) geometric Pµ F,i M,i global avarage k=1 σK,i /µ
the numerical results with the plotted trajectories, etc. An example of such evaluation figure is given in fig. 4. Based on this visual inspection of the solutions and the analysis of some calculated numerical values and parameters the user can select a set of individuals that are used to formulate the next generation. The developed toolbox allows further human intervention. This means, the user can change the individuals directly, e.g. in this case the temperature profiles can be manually changed by drag and drop technique, which provides possibility to the user to incorporate her or his prior knowledge into the multi-criteria evolutionary optimization.
Throughout this work discrete recombination of the object variables and global intermediate combination of the strategy parameters were used.
3
2.2 MATLAB EAsy-IEC Toolbox IEC systems seek to interface EA to human preference in order to create systems capable of evolving artifacts that require a human expertise that has not yet succumb to computation. Hence, the interfacing of human ability with machine computation requires resolving difficult issues [15]. This section reports on one such interactive evolutionary system that seeks to combine human perception with the previously presented evolutionary strategy to evolve the parameters of MATLAB and Simulink models (see fig. 1). The EAsy-IEC Tune Toolbox, which is under development, is designed to be applicable for different types of optimization problems, (e.g., system identification, controller tuning, data mining). The number of displayed individuals, µ + λ is set to eight, which can be displayed spatially. The number of searching generations is limited to twenty generations at the most due to the fatigue of human operators. In the proposed framework human users evaluate the performances of the individuals (set of solutions. For example, the user can simultaneously analyze
Application to Model based Optimization of Beer Fermentation
The performance of the proposed IEC based optimization technique is illustrated in the model-based temperature profile optimization of beer fermentation.
3.1
Process Description
In this paper a kinetic model published by Carrillo-Ureta [3] and Andres-Toro [16] has been used to estimate the effect of the temperature profiles. This model has been developed from experimental data and shows good results in the aspect of a realistic view of the fermentation process. The model takes into account seven components: three components of the biomass (latent, active, dead), ethanol and sugar, and two important byproducts: ethyl acetate and diacetyl. The model equations and parameters are taken from the article of Carrillo-Ureta [3]. Most of the process parameters vary as Arrhenius function of temperature, expect diacetyl appearance and disappearance rate which are constant values. dxlag dt dxactive dt dxbottom dt ds dt de dt d(acet) dt d(diac) dt
=
−µlag xlag
=
−µx xactive − km xactive + µlag xlag
=
km xactive − µD xbottom
=
−mus xactive
=
−mua f xactive
=
−mueas µs xactive
=
kdc sxactive − kdm (diac)e
(6)
The reaction rates: µx = Figure 1. Scheme of the Proposed IEC System Integrated to MATLAB
µx0 s 0.5si +e ,
µs = f =1
µs0 s ks +s , e − 0.5s i
0.5si µD0 0.5si + e µa0 s µa = ks + s
µD =
(7)
15
The parameters: 11654.64
µs0 = e−41.92− T +273.15 1267.24 µa0 = e3.27− T +273.15 9501.54 µlag = e30.72− T +273.15 34203.95 ks = e−119.63− T +273.15 kdm = 0.00113864 (8)
The initial values: xlag,i = 192, xact,i = 0.08, xbottom,i = 2, si = 130, ei = 0, (acet)i = 0, (diac)i = 0. Table 2. Nomenclature used T xlag xactive xbottom s e (acet) (diac) µx µD µs µa f µeas µlag km ks kdc kdm
3.2
Temperature Suspended latent biomass Suspended active biomass Suspended dead biomass Sugar conc. Ethanol conc. Ethyl acetate conc. Diacetyl conc. Yeast growth rate Yeast settling down rate Substrate (sugar) consumption rate Ethanol production rate Fermentation inhibition factor Ethyl acetate coefficient rate Specific rate of latent formation Yeast growth inhibition parameter Sugar inhibition parameter Diacetyl appearance rate Diacetyl disappearance of reduction rate
Implementation of IEC algorithm for the Trajectory Optimization
In order to apply the IEC algorithm for this problem, it is necessary to design a suitable representation of the temperature trajectory. For this purpose a simple but effective method has been developed. The profile is divided to six segments and every chromosome contains the time and the temperature of the breakpoints for this piecewise-linear trajectory. These numbers are coded as real values in the ES algorithm. I.e. see fig. 2. As this figure shows, the breakpoints of the piecewise temperature profiles are moveable during the optimization which flexibility improves the effectiveness of algorithm but increases the search space of the optimization problem. During the optimization the user selects a set of good temperature profiles given by the ES (interactive evolution), and he or she can manually change the selected profiles (human intervention). The technique of human intervention is very simple: the human operator can move freely the breakpoints of the trajectories with drag and drop technique (see fig. 4). After the modification of the trajectory, the simulation runs automatically with this new trajectory and the evaluation figures refresh.
Temp. [C]
31934.09
µx0 = e108.31− T +273.15 , 10033.28 µD0 = e33.82− T +273.15 , 26589 µeas = e89.92− T +273.15 , 38313 km = e130.16− T +273.15 , kdc = 0.000127672,
10
5 0
50
100 time [h]
150
200
Figure 2. Representation of temperature trajectory chromosome = [50, 70, 100, 150, 170, 10, 9.5, 11, 11, 13, 7]
Certainly one can use more than five inner breakpoints, but the chromosome should not be much bigger because it is limited by human abilities and the highly increased complexity of the optimization problem. Furthermore, when a big number of segments are used the GA generates a very abrupt profiles which needs smoothing [10, 3].
3.3
Results
A good temperature profile should result in high ethanol, low sugar and ethyl acetate concentrations, a very low diacetyl and biomass concentrations, and relatively smooth temperature profile, and short operation time. This leads to a multi-objective optimization problem and the task is to find a temperature profile which can stratify all of the above objectives and some other constraints. Compared to other standard approaches, the IEC has the great advantage that the human operator can effectively balance among these objectives. In this case study, with EAsy-IEC Toolbox, the user found a set of quite good solutions only after 11 generations (while every generation consists only 8 individuals). To analyze and demonstrate the result, the resulted temperature trajectory is compared to the results obtained by the industrial trajectory taken from the article of Andres-Torro et al. [16] (see table 3 and fig. 3). (The industrial temperature profile is shown in the first column of fig. 4.) As can be seen from fig. 3, the optimal trajectory starts at 10 degrees centigrade because of safety reason [2]. Compared to the industrial profile, although the final ethanol level is a little bit smaller, the optimized temperature trajectory results in much less ethyl acetat and diacetyl concentrations, and the biomass and sugar concentration has been decreased too.
References
Table 3. Results Ethanol Sugar Ethyl acetate Diacetyl Biomass
Industrial profile 45.49 31.96 3.25 0.0054 0.0583
[1] J.A. Roubos, C.D. de Gooijer, G. van Straten, and A.J.B. van Boxtel. Comprasion of optimization methods for fed-batch cultures of hybridoma cells. Bioprocess engineering, 17:99–102, 1997.
Optimal profile 45.01 3.26 2.12 0.0052 0.0048
[2] B. de Andres-Toro, J.M. Giron-Sierra, J.A. LopezOrozco, and C. Fernandez-Conde. Evolutionary optimization of an industrial batch fermentation process. In Europan Control Conference, ECC’97. http://www.cds.caltech.edu/conferences/related/, 1997.
20 temperature 10
0 0 6
50
100
150
200 biomass etyl acetat diacetyl
4 2 0 0 150
50
100
150
200 sugar ehanol
100 50 0 0
50
100
150
200
Figure 3. Resulted optimal profile
4
[3] G.E. Carrillo-Uerta, P.D. Roberts, and V.M. Becerra. Genetic algorithms for optimal control of beer fermentation. In Proc. IEEE International Symposium on Intelligent Control, pages 391–396. Mexico City, Mexico, 2001.
Conclusions
This paper demonstrated how interactive evolution algorithms can be applied in model based multi-objective optimization problems. A MATLAB toolbox has been developed to provide a human-machine interface for the IEC. With the use of this tool, the user can easily select and modify the individuals which then are reproduced and mutated to constitute the next generation. The proposed approach has been applied to design temperature profile for beer fermentation. The results show that IEC is an efficient and comfortable method to incorporate the priori knowledge and preferences of the user into model-based optimization problems.
Acknowledgement The authors would like to acknowledge the support of the Cooperative Research Center (VIKKK) (KKK-I-7), the Hungarian Ministry of Education (FKFP-0063/2000 and FKFP-0073/2001), and the Hungarian Science Foundation (OTKA TO37600). Janos Abonyi is grateful for the financial support of the Janos Bolyai Research Fellowship of the Hungarian Academy of Science.
[4] I.C. Trelea, M. Titica, S. Landaud, E. Latrille, G. Corrieu, and A. Cheruy. Predictive modelling of brewing fermentation: from knowledge-based to balckbox models. Mathematics and Computers in Simulation, 56:405–424, 2001. [5] H. Takagi. Interactive evolutionary computation cooperation of computational intelligence and human kansei. In 5th Intl Conf. on Soft Computing (IIZUKA98), pages 41–50. World Scientic, Iizuka, Fukuoka, Japan, 1998. [6] H. Takagi. Interactive evolutionary computation: System optimization based on human subjective evaluation. In IEEE Intl Conf. on Intelligent Engineering Systems (INES98), pages 1–6. Vienna, Austria, 1998. [7] I. Parmee, D. Cvetkovic, C. Bonham, and I. Packham. Intoducing prototype interactive evolutionary systems for ill-defined multi-objective design environments. Advances in Engineering Software, 32:429– 441, 2001. [8] J. McCormack. Interactive evolution of l-system grammars for computer graphics modelling. In D.G. Green and T. Bossomaier, editors, Complex Systems: from Biology to Computation, pages 118–130. IOS Press, Amsterdam, Netherlands, 1993. [9] M. Ohsaki and H. Takagi. Application of interactive evolutionary computation to optimal tuning of digital hearing aids. In Int’l Conf. on Soft Computing (IIZUKA’98), Iizuka, Fukuoka, Japan. World Scientic, 1998. [10] B. de Andres-Toro, J.M. Giron-Sierra, J.A. LopezOrozco, and C. Fernandez-Conde. Using genetic algorithms for dynamic optimization: an inudstrial fermentation case. In Proc of the 36th IEEE CDC, pages 828–829. San Diego, USA, 1997.
Generation: 2
Parents:
Offsprings:
20
20
20
20
20
20
20
20
15
15
15
15
15
15
15
15
10
10
10
10
10
10
10
10
5
5
5
5
5
5
5
5
0
0
0
0
0
0
0
0
100
200 0
100
200 0
100
200 0
100
200 0
100
200 0
100
200 0
100
0 200 0
6
6
6
6
6
6
6
6
4
4
4
4
4
4
4
4
2
2
2
2
2
2
2
2
0
0
100
0 200 0
100
0 200 0
100
0 200 0
100
0 200 0
100
0 200 0
100
0 200 0
100
0 200 0
6
6
6
6
6
6
6
6
4
4
4
4
4
4
4
4
2
2
2
2
2
2
2
2
0
0
100
0 200 0
100
0 200 0
100
0 200 0
100
0 200 0
100
0 200 0
100
0 200 0
100
0 200 0
150
150
150
150
150
150
150
150
100
100
100
100
100
100
100
100
50
50
50
50
50
50
50
50
0
0
100
0 200 0
100
0 200 0
100
0 200 0
100
0 200 0
100
0 200 0
100
0 200 0
100
0 200 0
Figure 4. Human-machine interface of the IEC toolbox
[11] J.A. Roubos, G. van Straten, and A.J.B van Boxtel. An evolutionary strategy for fed-batch bioreactor optimization; concepts and performance. Jorunal of Biotechnology, 67:173–187, 1999. [12] H.P. Schwefel. Kybernetische evolution als strategie der experimentellen forschung in der stromungstechnik. 1965. [13] I. Rechenberg. Case studies in evolutionary experimentation and computation. Comput. Methods Appl. Mech. Engrg., 186:125–140, 2000. [14] H.P. Schwefel. Numerical Optimization of Computer Models. Wiley, Chichester, 1995. [15] R.W. Noel and S. Acchione-Noel. Objet trouv, holism, and morphogenesis in interactive evolution. CogSci, 2002, Fairfax Virginia, Aug., 2002. [16] B. de Andres-Toro, J.M. Giron-Sierra, J.A. LopezOrozco, C. Fernandez-Conde, J.M. Peinado, and F. Garcia-Ochoa. A kinetic model for beer producion under industrial operational conditions. Mathematics and Computers in Simulation, 48:65–74, 1998.
100
200
100
200
100
200
100
200
