R!S P(also called the predecessor) and S(the successor)are what we earlier called the left-hand-side and the right-handside of a production rule. L and R(the left- and right-context respectively) may be absent. Commonly, an L-system without context is called a 0L-system. If all production rules have one-sided context, it is called a 1L-system, and a 2 L-system has production rules with two-sided context. A production rule with left and right context L and R can only replace P by S if L precedes P and followed by R. If two production rules apply for a certain character, one with and one without context, the one with context is used. 2.3 Time Series Prediction The problem of time series prediction is this: given values of the past, x, one must find a function, f , which predicts values of the futures. Past values, x t , can be considered as a vectors,
(
~x(t) = (x(t); x(t
(+ ) ~( ( )) 19
)
~
1); ; x(t
))
(1)
The future values, x t , are estimated by a function of previous values, f ~x t . In this paper we consider the shortterm prediction of and , i.e., we predict the
=1
(t 1) at time t + 1 from the input ~x(t) = (x(t); x(t 1); ; x(t 19))
value x
Table 2: Translation table of codon (2)
The predictive accuracy of models are evaluated by estimating the normalized mean squared error (NMSE) as follow:
E=
1
N var
N X t=1
jx(t + 1) f~(~x(t))j2
(3)
2
N N 1 X 1 X var = x(t + 1) x(t + 1) N N t=1
(4)
t=1
This problem of time series prediction is then reduced to finding the predicator f ~x t that minimizes its NMSE value E.
~( ( ))
3 Construction of Neural Network In this research we tried to combine four methods with there origin in biology: - Genetic Algorithms - DNA coding method - L-system - Neural Networks Our goal is to design a method that searches automatically for neural networks architecture. 3.1 DNA Coding Method for Neural Network The chromosome of proposed neural network is DNA code. Its code consists of A, G, T and C. Table 2 is translation table for DNA to production rules. In first step, initial DNA code is proposed at random. The production rules made of translated amino acid in DNA code. The translation starts from start codon(ATG) in DNA code. The first codon translates to name of node. The second codon translates to connecting range(C/R). Connecting range(x,y) is ,in string(or node’s array), its connecting range is determined by the value of x,y. Current node links between xth and yth nodes. If node’s name is comma(’,’) in string of between the range, next node does not link. The third codon is bias. The weight of each node is translated 4th codon to 8th codon.
Amino Acid Leu Arg Ser Thr Ala Gly Val Pro Stop Ile Tyr Gln Phe Asp Cys Asn Glu His Lys Trp Met
# of Amino Acid 6 6 6 4 4 4 4 4 3 3 2 2 2 2 2 2 2 2 2 1 1
Node’s Name A B C D A B C D A B C D , , , , , , C D
C/R 1,1 2,2 3,3 1,2 1,3 1,4 2,3 2,4 3,4 4,4 1,1 2,2 3,3 1,2 1,3 1,4 2,3 2,4 3,4 4,4
values of bounds from -3.2 to 3.1 at 0.1 intervals.
W eight =
(DNA 42 + DNA 41 + DNA 40) 32 10 (5)
where DNA is one of the nucleotides which the values are A=0, G=1, T=2, C=3 respectively. Now, one node completes. Repeatedly, DNA code translates as for until stop codon. The first codon is predecessor and the remainder is successor. This string is a rule. The other rules find in the same way.
Figure 5: Translation of DNA code for Production Rules Figure 4: Structure of Node The weight and bias is calculated by equation 5. This has
The translated rule by referred way like production rule of context-free L-system. If there are rules have a same predecessor, only use the first rule. Repeatedly determined rewriting steps make a neural network by final string.
The neural network has one over more input and output nodes. But usable network has suitable number of in-output nodes. The input value is past data and output value is predicted value. To evolve neural network by genetic algorithms, we have to select suitable neural network. The suitable neural network is that network has a determined number of in-output nodes. And high performance of networks is low error in network’s time prediction. Error is difference between original data and output of predicator. The selection method in GA is mixed ranking selection and selection of ES. Ranking selection is that the population is sorted according to objective function value. And section is selected by individuals in parents and offspring. Through the GA, prediction performance of neural networks is higher.
P2 : B(2,3)C(1,2)A(2,2) P3 : B(2,3)C(1,2)A(2,2)B(1,1)C(2,4) Its string may have unavailable nodes. The unavailable nodes eliminate at organizing network. Figure 7 shows the neural network created from string for solving XOR problem.
(+ ) ( + )
Figure 7: Organization of Neural Network
4 Prediction results 4.1 The Mackey-Glass equation To validate our method, we use test data as MackeyGlass([mac77]) chaotic data. The equation of Mackey-Glass time series is as follows:
dx(t) dt
= 1 +axx( t(t ))
bx(t)
(6)
The variables are chosen to be a=0.2, b=0.1, =10 and equal to 30. The fitness function of neural network is equation 7:
F itness = e E
(7)
=
Where E is normalized mean squared error(NMSE) and . Figure 8 is result of Mackey-Glass time series prediction. The number of learning data is 250 and test set is 50.
2
Figure 6: Process of Evolving Neural Network
3.2 Example of Neural Network Suppose that five rules created by DNA code. But, there are two rules that have predecessor ’A’. In that case, only use the first rule. Third rule eliminates in the set of rules. p1 : A ! B(1,1)C(2,4) p2 : B ! B(2,3) p3 : A ! A(2,3)B(3) p4 : D ! A(1,1)
p5 : C ! C(1,2)A(2,2) String is created from the axiom after three rewriting steps following abovementioned rules. - Axiom : A P1 : B(1,1)C(2,4)
Figure 8: Predicted Mackey-Glass Data(— Ideal, Predict) Figure 9 is transition of fitness in our simulation. Figure 9 shows that best individual converges fast and other individuals follow it.
Table 3: Parameter of Simulation
Figure 9: Transition of Fitness 4.2 The Sunspot data Sunspot is a dark spot, some as large as 80,000km in diameter, move across the surface of the sun, contracting and expanding as they go. It is made of low temperature than around. These strange and powerful phenomena are known as sunspots. The first set of experiments was conducted on Wolf’s sunspot series acquired during 1700-1988. Only pastobserved data is in existence in sunspot data. Since in so, prediction method use only past data. The data set was partitioned into a training set in 17191918(200) and test set in 1919-1968(50). Another parameter is equal to Mackey-Glass data prediction.
Population Initial String Length Crossover method Crossover Prob. Mutation Prob. Selection Generation No. of input node No. of output node Training data Test data Range of output
Mackey-Glass 50(50+150) 300 one point 0.9 0.3 Ranking 300 5 19 1 250 50 -2 2
Sunspot data 50(50+150) 300 one point 0.9 0.3
& ( + )
500 5 19 1 200 50 -2 2 (
100)
The artificial neural networks have computational ability by interconnection of the artificial neuron that is simple component of networks. In conventional neural networks, weight between neurons is an important parameter for behavior of network, because it is the objective of learning. Accordingly, weight and architecture find through evolutionary algorithms. The DNA code of short length makes diverse production rules. And neural networks consist of string made of production rule in L-system. In the future work, we have plan to make use of contextsensitive and bracket L-system. We are studying for extends Genetic Programming. And this application is applied to problem in variety of real world problems(e.g., financial forecasting, stock forecasting).
Bibliography [yomo96] Yomohiro, T. Uchikawa, Y. ”Effect of New Mechanism of Development from Artificial DNA and discovery of Fuzzy Control Rules,” Proc. of IIZUKA ’96, pp.498-501. 1996 Figure 10: Predicted Sunspot Data(— Ideal, Predict) Table 3 shows parameters used in time predicted simulation of Mackey-Glass time series and Sunspot data. Parameter of sunspot data prediction problem is equal except for size of training data set and generation number.
5 Conclusions In this paper, we proposed a new method of constructing neural networks. These evolutionary neural networks are based on the concept of development and evolution. To make evolutionary neural networks, we use DNA coding method, Lsystem and GA. We make the diverse production rule of Lsystem from DNA coding of short length. Neural networks are made of string based on production rules.
[mac77] Mackey, M.C. and Glass, L. ”Oscillation and Chaos in Physiological Control Systems,” Science 1977, p.287 [boers93] Boers, E.J.W Kuiper, H. Happel, B.L.M. and Kuyper, S. ”Designing Modular Artificial Neural Networks,” Proc. of Computer Science in the Netherlands, pp. 87-96. 1993 [garis94] Garis, H.D. ”CAM-BRAIN: The Genetic Programming of an Artificial Brain Which Grows/Evolves at Electronic Speed in a Cellular Automata Machine,” Proc. of The First int’l Conf. on Evolutionary Computation, vol. 1, pp. 337-339, 1993
[sugi98] Sugisaka, M. ”Design of an artificial brain for robots,” Proc. of The Third Int’l Symp. on Artificial Life and Robotics. pp. (I-1)-(I-11), 1998 [gruaru93] Gruaru, F. Whitley, D. ”Adding Learning to the Cellular Development of Neural Networks: Evolution and the Baldwin Effect,” Evolutionary Computation, vol. 1-3, pp. 213-233. 1993 [wu96]
A.S Wu, R.K. Lindsay, ”A computation of the fixed and Floating Building Block Representation in Genetic Algorithms,” Evolutionary Computation, vol. 4, no.2, pp. 169-193, 1996
[goldberg89] David E. Goldberg ”Genetic Algorithms in search, Optimization, and Machine Learning,” Addison Wesley Publishing Company, 1989 [lind68] Aristid Lindenmayer, ”Mathematical Models for Cellular Interaction in Development, Part I, II”, Journal of Theoretical Biology, vol. 18, pp. 280315, 1968 [lind87] Aristid Lindenmayer, ”Developmental Models of Multicellular Organisms : A Computer Graphics Perspective,” Artificial Life VI, pp. 221-249, 1987
&
[jacob95] Jacob, C. ”Modeling Growth with L-systems Mathematica,” Mathematica in Education and Research, Vol. 4, No. 4, TELOS Springer, pp. 12-19, 1995 [lee99]
Lee, D.W. Sim, K.B. ”Evolving Chaotic Neural Systems for Time Series Prediction,” Congress on Evolutionary Computation. pp. 310-316, 1999
[casd89] Casdagli, Martin. ”Nonlinear Prediction of Chaotic Time Series,” Phisica D, vol 35, pp. 335-356, 1989