The Third International Conference on Discrete Chaotic Dynamics in Nature and Society Chuo University, Tokyo, Japan, September 9–13, 2002

IDENTIFICATION OF NONLINEAR DYNAMIC SYSTEMS WITH REDUCED DEGREE OF NONLINEARITY Eduardo M. A. M. Mendes∗ , Erivelton G. Nepomuceno‡ and Oriane M. Neto∗ ∗

Departamento de Eletricidade Fundação de Ensino Superior de São João Del Rei Praça Frei Orlando 170 - Centro - São João Del Rei, MG - 36300.000 - Brazil Tel.: +55(32)3792552, Fax: +55(32)3792306, E-mails: [email protected], [email protected] ‡ Centro de Pesquisa e Desnvolvimento em Engenharia Elétrica Av. Antônio Carlos 6627 Belo Horizonte-MG CEP 31270-010, Brazil Tel:+55(31)499-5465 Fax:+55(31)499-5480 Email: [email protected]

Abstract In this paper identification of discrete monovariable rational models with reduced degree of nonlinearity from data generated by iterating nonlinear systems is addressed. In order to find the input-output equations and fixed points of such systems, the so-called Gröbner Bases and the concepts of Term Clusters and Clusters Coefficients are reviewed. Keyword:

Chaos, Identification, Nonlinear Discrete Systems, Fixed points

1. Introduction In the evolution of the study of chaotic systems, several distinct but sometimes co-existent phases can be distinguished. In the first phase, chaos was recognized as a deterministic dynamical regime which could be responsible for fluctuations that had been regarded as noise and therefore modelled as a stochastic process [1]. In a subsequent phase, it was necessary both to develop criteria to detect chaotic dynamics and to establish dynamical invariants to quantify chaos [2, 3, 4]. Having succeeded in diagnosing chaos, the next step was to build models which would learn the dynamics from data on the strange attractor [5, 6, 7, 8, 9, 10, 11, 12]. One of the great challenges on the identification of nonlinear dynamic systems is the choice of representation and selection of model structures. As far as model structure is concerned, several techniques have been suggested in the literature which are based mainly on statistical approaches and orthogonal matrix decomposition methods [13, 14, 15]. The models identified using such approaches are frequently used for predictions. However, when the main interest is the use of models for designing controllers for nonlinear systems, it could be necessary to obtain identified models without sobreparametrization. An attempt towards this direction is the use of concepts of Term Clusters and Cluster Coefficients introduced by Aguirre and Billings[16] and further developed by Aguirre and Mendes [17]. The Term clusters are crucial when the objective is to determine the fixed points of the monovariable systems. Besides, the judicious choice of the term clusters can avoid the deleterious effects of the overparametrization. In this work the aim is to identify dynamically valid models from data generated by a multivariable nonlinear system in a chaotic motion. In this case, the calculation of the fixed points are not straightforward: Gröbner Bases may be used when an analytical solution is required. The idea behind Gröbner Bases is the transformation of a given set of polynomials F into a standard form G. This form has the property of isolating one of the variables which allows the calculation of the fixed points. The information generated by the calculation of the fixed points is used to determine whether the identified model is overparametrized. The example used in this work is the so-called Rössler Hyperchaos equations. It will be shown that it is possible to identify dynamically valid models with a reduced degree of the nonlinearity when compared to the original set of equations. It is believed that the results presented here will be helpful when the objetive is to obtain models that can describe the original dynamics and are parsimonious at the same time.

Identification of Nonlinear Dynamic Systems by E. Mendes, E. Nepomuceno and O. Neto

2

This article is divided as follows: in Section 2 basic background necessary to the understanding of this work; in Section 3 the problem of the identification of nonlinear models with reduction of the nonlinearity degree is addressed whereas and the conclusions are presented in Section 4. 2. Background Material

2.1. Rational Representation of Nonlinear Systems Consider the Rational NARMAX model proposed in [18]

y(k) =

a(y(k−1), . . . , y(k−ny ), u(k−1), . . . , u(k−nu ), ... b(y(k−1), . . . , y(k−ny ), u(k−1), . . . , u(k−nu ), a(e(k), . . . , e(k−ne )) , ... b(e(k), . . . , e(k−ne ))

(1)

where u(k) and y(k) are respectively the input and output signals sampled at intervals of Ts . e(k) accounts for uncertainties, possible noise, unmodelled dynamics, etc. ny , nu and ne are the maximum lags of the output, input and noise, respectively. Moreover, such lags need not be the same in a(. . .) and b(. . .) which are nonlinear polynomial functions. In order to estimate the parameters in (1), it is convenient to define the numerator and denominator polynomials respectively as [18] Nn

a(k − 1) =

∑ pn j θn j = ψTn (k − 1)Θn

,

(2)

∑ pd j θd j = ψTd (k − 1)Θd

,

(3)

j=1 Nd

b(k − 1) =

j=1

where θn j , θd j are the parameters of the regressors up to time k − 1. Nn + Nd is the total number of parameters to be estimated. Unfortunately, the use of (1) to perform parameter estimation is not straightforward because such a function is nonlinear in the unknowns. An alternative solution to this problem is to multiply both sides of (1) by b(k − 1) and rearranging terms in order to yield [19] Nd

Y (k) = a(k − 1) − y(k) ∑ pd j θd j + b(k − 1)e(k) j=2

Nn

=

∑

j=1

Nd

pn j θn j − y(k) ∑ pd j θd j + ζ(k) j=2

= ψTn (k − 1)Θn − ψTd1 (k − 1)Θd + ζ(k) ,

(4)

where ψTd (k − 1) = [pd1 ψTd1 (k − 1)], θd1 = 1 and Y (k) = y(k)pd1 =

a(k − 1) pd1 + pd1 e(k) , b(k − 1) !

ζ(k) = b(k − 1)e(k) =

(5)

Nd

∑ pd j θd j

e(k) .

(6)

j=1

Because e(k) (white noise) is independent of b(k − 1) and has zero mean, E[ζ(k)] = E[b(k − 1)]E[e(k)] = 0. (5) reveals that all the terms of the form y(k)ψTd (k − 1), because of y(k), implicitly include the noise e(k) which is correlated with ζ(k). This, of course, results in parameter bias even if the noise e(k) is white. The aforementioned correlation occurs as a consequence of multiplying (1) by b(k − 1) and should be interpreted as the price paid for turning a function which is nonlinear in the parameters to one which is linear in the parameters.

3

Identification of Nonlinear Dynamic Systems by E. Mendes, E. Nepomuceno and O. Neto 2.2. Forward Selection Using Orthogonalisation

To avoid an excessive number of parameters in a rational model, an efficient subset selection procedure based on the orthogonal least squares (OLS) method [20, 13] can be performed. Given the full set of candidate process terms (regressors), the OLS algorithm selects the significant terms one by one until a criterion which indicates the adequacy of the model structure is satisfied. Following the procedure devised in [21], (1) can be rewritten in the well-known form Θ = [ΨT Ψ]−1 ΨT Y ,

(7)

Nn +Nd −1 = ∑i=1 pi (k).

where Θ = [Θn Θd1 T ] and Ψ pi (k) are column-vectors which represent the process terms. T ˆ satisfies the condition The regression vector [p1 . . . pNn +Nd −1 ] forms a set of basis vectors, and the OLS solution Θ ˆ that ΨΘ will be the projection of Y onto the space spanned by these basis vectors. The OLS method [22] involves the transformation of the set of pi into a set of orthogonal basis vectors, and thus makes it possible to calculate the individual contribution to the desired output from each basis vector. The regression matrix Ψ can be decomposed into T

T

Ψ = QR ,

(8)

where R, shown below, is an (Nn + Nd − 1) × (Nn + Nd − 1) triangular matrix with 1’s on the diagonal and 0’s below, as 1 r12 r13 . . . r1(Nn +Nd −1) 0 1 r23 . . . r2(Nn +Nd −1) .. R= (9) 0 0 1 . . . . . . . . . .. .. .. .. .. 0 ... ... ... 1 and Q is an N × (Nn + Nd − 1) matrix with orthogonal columns qi such that QT Q = D, where D is a diagonal matrix. Since the space spanned by the set of orthogonal basis vectors qi is the same space spanned by the original set of vectors (pi ), the regression equation (4) can be rewritten as Y (k) = QT (k − 1)gˆ + ξ(k)

(10)

The OLS solution, g, ˆ can be calculated using gˆ = D−1 QT Y or gˆi =

qTi Y , 1 ≤ i ≤ (Nn +Nd −1). qTi qi

(11)

ˆ can be retrieved by solving the triangular system RΘ ˆ = g. The original set of parameters Θ ˆ A great advantage of the orthogonal estimator is the possibility of selecting the relevant vectors (terms) as a byproduct. To demonstrate this, consider again the orthogonal regression equation (10). In doing so, it is assumed that the orthogonal property qTi q j = 0 for i 6= j holds. Therefore, if (10) is multiplied by itself and the time average is taken, the following equation can be derived 1 T 1 (Nn +Nd −1) 2 T 1 Y Y= ∑ gi qi qi + N ξT ξ N N i=1

(12) (N +N −1)

The output mean square value (MSV) Y T Y /N consists of two terms. The first term, ∑i=1n d g2i qTi qi /N, is the part of the output MSV explained by the regressors whereas the second term, ξT ξ/N, accounts for the unexplained output MSV. Owing to the orthogonal estimator, the increment towards the overall output MSV of each regressor (term or vector) can be computed independently as g2i qTi qi . Expressing this quantity as a fraction of the overall output MSV yields the Error Reduction Error (ERR) [ERR]i =

g2i qTi qi , 1 ≤ i ≤ (Nn +Nd −1) , Y TY

(13)

4

Identification of Nonlinear Dynamic Systems by E. Mendes, E. Nepomuceno and O. Neto

that can be used as a simple and effective means of selecting the most relevant regressors in a forward-regression manner. Therefore ERR imposes a hierarchy of terms according to their contribution towards the overall output MSV. 2.3. Term clustering The deterministic part of a rational NARMAX model can be expanded as the summation of terms with degrees of nonlinearity in the range 1 ≤ m ≤ `. Each m th-order term can contain a p th-order factor in y(k − ni ) and a (m − p) thorder factor in u(k − ni ) and is multiplied by a coefficient α p,m−p (n1 , . . . , nm ) or β p,m−p (n1 , . . . , nm ) as follows nα ,nα

α

p m 1 ` m y u y(k) = × ∑ ∑ ∑ α p,m−p (nα1 , . . . , nαm ) ∏y(k−nαi ) ∏ u(k−nαi ) D m=0 p=0nα ,nα i=1 i=p+1 1

(14)

m

where `β

D= ∑

β

β

m ny ,nu

p

β

β

m

∑ ∑ β p,m−p (n1 , . . . , nβm )∏ y(k−ni ) ∏

m=0 p=0nβ ,nβ 1

m

β

u(k−ni )

(15)

i=p+1

i=1

and α nα y ,nu

∑

nα y

≡

α nα 1 ,nm β

∑

nα u

···

nα 1 =1 β

β

ny ,nu

∑

β β n1 ,nm

∑

,

β

ny

≡

,

nα m =1

∑

β n1 =1

nu

···

∑

. ,

(16)

β nm =1

and the upper limit is ny if the summation refers to factors in y(k − ni ) or nu for factors in u(k − ni ).

Definition 2.1 [23]. nα ,nα

y u ∑nα ,nαm α p,m−p (nα1 , . . . , nαm ) in (14) are the coefficients of the term clusters of numerator Ωαy p um−p , which contain 1 terms of the form y(k − i) p u(k − j)m−p for m = 0, . . . , `α and p = 0, . . . , m. Such coefficients are called cluster coefficients of numerator and are represented as Σαy p um−p . 2

Definition 2.2 [23]. β

β

ny ,nu

∑

β β n1 ,nm

β

β

β

β p,m−p (n1 , . . . , nm ) in (15) are the coefficients of the term clusters of denominator Ωy p um−p , which con-

tain terms of the form y(k − i) p u(k − j)m−p for m = 0, . . . , `β and p = 0, . . . , m. Such coefficients are called cluster β coefficients of denominator and are represented as Σy p um−p . 2

For details about the definition of term clusters and cluster coefficients the reader is referred to [16, 23]. For the purposes of this section it suffices to realize that a term cluster is a set of terms of the same type and the respective cluster coefficient is obtained by the summation of the coefficients of all the terms, of the respective cluster, which are contained in the model. In practice it will be helpful to notice that terms of the same cluster explain the same type of nonlinearity.

Identification of Nonlinear Dynamic Systems by E. Mendes, E. Nepomuceno and O. Neto

5

2.4. Gröbner Bases Gröbner Bases are used for solving problems related to a system of polynomial ideals. The idea is to find the contractions of these polynomials that, hopefully, isolate one of the variable of the system in one of the system equations. In this work, Gröbner Bases will be used as a tool in the solution of the following problems : • conversion from state space form to input-output representation; • calculation of fixed points of multivariable systems [24]. The literature on Gröbner Bases is very extensive and complex. In this work, the main concern is to present the examples that help the reader to understand the importance and versatility of this tool. More details can be found in [25]. For the sake of clarity, an example on how to use Gröbner Bases as a tool for calculating the fixed points of multivariable systems will be given next. In the case of the conversion from state space to I/O representation, the reader is referred to the excellent work by Forsman [24]. Fixed Points of Multivariable Systems One of the applications of the Gröbner Bases is the calculation of the fixed points of multivariable systems. For instance, considerer the equations originally proposed by Rössler [26]: x˙1 x˙2 x˙3

= −x2 − x3 = x1 + σx2 = α + x3 x1 − x3 ρ

(17)

Computing the Gröbner Bases using the routine gbasis of MAPLE V, the following set of equations can be found: x1 − αx3 x2 + x3 α − x3 ρ + σx32

= 0 = 0 = 0

(18)

From (18), it can be noticed that the last polynomial depends on the variable x3. only. Therefore, the fixed points for the variable x3 can be readily calculated. The remaining fixed points, that is, the fixed points for variables x1 and x2 , can be calculated by substituting the fixed points for x3 in the previous polynomials and solving the resulting equations. It can also be noticed that the last polynomial has the same form as the polynomial used for calculating the fixed points for monovariable systems. Unfortunately, this is not always the case. 3. Results In this section, identification of nonlinear systems with reduced degree of the nonlinearity is presented and discussed. For the sake of clarity, the system used as an example is the well-know Rössler Hyperchaos oscillator[27]. Rössler Hyperchaos Oscillator Consider the equations for the Rössler Hyperchaos oscillator [27]: x˙ y˙ z˙ w˙

= = = =

−y − z x + 0.25y + w 3 + xz −0.5z + 0.05w

(19)

6

Identification of Nonlinear Dynamic Systems by E. Mendes, E. Nepomuceno and O. Neto A Poincaré Section for this system is generated by considering the following set of three difference equations:

x(k + 1) = 3.8x(k)(1 − x(k)) − 0.05(y)k) + 0.35)(1 − 2z(k)) y(k + 1) = 0.1((y(k) + 0.35)(1 − 2z(k)) − 1)(1 − 1.9x(k)) z(k + 1) = 3.78z(k)(1 − z(k)) + 0.2y(k)

(20)

In order to obtain the Poincaré section (please note that this section can be considered as a discrete dynamical system), (20) was simulated with initial conditions x=0.085, y=-0.121 and z=0.075. The result can be seen in Figure 1.

Figure 1: Poincaré section of the Rössler Hyperchaos system For the calculation of the fixed points of the Poincaré section, the ideas shown in the previous section were used. Table 1 shows the results.

1 2 3 4 5 6 7 8 9

x 2.06046 0.9229-1.5026i 0.9229+1.5026i -0.7951-0.8361i -0.7951+0.8361i 0.740225 -0.00193 0.00494 0.73007

y 60.7097 48.025+35.43i 48.025-35.43i 13.34-43.548i 13.34+43.548i 0.048379 -0.1112 -0.07178 0.02417

Z 2.1973 0.9352+1.6516i 0.9352-1.6516i -0.9156+0.898i -0.9156-0.898i 0.7389 0.7274 0.0052 -0.00173

Table 1: Fixed points calculated algebraically suing Gröbner basis Note that the system exhibits nine fixed point that cannot be detected simply by inspecting (20). The presence of four complex fixed points for each variable should be also noticed. This can cause serious difficulties when the interest is the identification of global models [25]. Considering that the data range for the system under investigation is between 0.1 and 0.98, it can be seen that only two fixed points are inside of these limits and the other seven (three real and four complex values) are outside. This situation shows that there is a possibility of identifying dynamically valid models with reduced degree of the nonlinearity.

Identification of Nonlinear Dynamic Systems by E. Mendes, E. Nepomuceno and O. Neto

7

For this system in particular, only the variable x and z have an I/O relationship. From the results related to the Gröbner basis calculations, it was determined that the model structure to be used in the identification procedure is rational as pointed in [25]. In addition to that, the variable z leads a much simpler (less terms) rational model than the variable x. However, this equation will not be listed in this work due to the size, 116 terms in the numerator and 15 terms in the denominator. Using the above information and the fact of that there are fixed points located outside of the range of the data, several models with degrees of nonlinearity between l=6 and l=9, maximum lag ny=3 and maximum number of terms equals to 131 were identified for the coordinate z. The best identified model has degree of nonlinearity l=7 and 131 terms of process. The fixed points for this model are shown in Table 2.

1 2 3 4 5 6 7

z 0.09 0.7389 0.7274 0.823+0.129i 0.823-0.129i 0.058+0.207i 0.058-0.207i

Table 2: Fixed Points for the identified model It is worth noticing that the fixed points highlighted in Table 2 are in agreement with the original listed elsewhere in the text. These two fixed points are located inside of data range, which confirms previous results regarding the location of the fixed points: a good estimation of the fixed points is closely related to the fact they are located inside of the data range. Since the other fixed points, that is, four complex and one real (0.09) values, are located outside of the data range, there is no much information on the time seriesnfor identification purposes. This causes the estimation algorithm to place them inside of the data range, that is, between 0.1 and 0.98. The lack of relevant information related to these points impairs a precise identification. On the other hand, the fixed points 0.7389 and 0,7273 are accurately identified since the data contain enough information to do so. In order to provide a better understanding, Figure 2 shows tow-dimensional attractors reconstructed from the original time series and from data collected from iterating the identified model. From these attractors, it can be noticed that the identified model reproduces the dynamics of the Poincaré section of Rössler Hyperchaos system fairly. 4. Conclusions In this work it has been shown that it is possible to obtain models with reduced degree of nonlinearity that can reproduce the dynamic properties of Rössler Hyperchaos system. The influence of the location of the fixed points in relation to the data range is addressed. It has been shown that when the fixed points are located outside of the data range, their influence on the data is such that they can be eliminated and thus leading the parsimonious models. It is believed that the results presented in this paper are general and therefore can be applied to a wider range of nonlinear systems. Acknowledgments E. Mendes acknowledges the support of CNPq under the grant 301313/96-2 and 201557/91-6. E. Nepomuceno ackonowledges the financial support from CNPq and O. Neto acknowledges the financial support from CAPES. References [1] Edward N. Lorenz. Deterministic nonperiodic flow. Journal of The Atmospheric Sciences, 20:131–141, 1963. [2] J. Guckenheimer. Noise in chaotic systems. Nature, 298:358–361, 1982.

Identification of Nonlinear Dynamic Systems by E. Mendes, E. Nepomuceno and O. Neto

(a)

(b)

Figure 2: a) Attractor for the original system; b) Attractor for the identified model

8

Identification of Nonlinear Dynamic Systems by E. Mendes, E. Nepomuceno and O. Neto

9

[3] J. P. Eckmann and D. Ruelle. Ergodic theory of chaos and strange attractors. Rev. Mod. Phys., 57:617–656, 1985. [4] T. A. Denton and G. A. Diamond. Can the analytic techniques of nonlinear dynamics distinguish periodic, random and chaotic signals? Compt. Biol. Med., 21(4):243–264, 1991. [5] J. D. Farmer and J. J. Sidorowich. Predicting chaotic time series. Phys. Rev. Lett., 59(8):845–848, 1987. [6] J. P Crutchfield and B. S. McNamara. Equations of motion from a data series. Complex Systems, 1:417–452, 1987. [7] D. S. Broomhead and D. Lowe. Multivariable functional interpolation and adaptive networks. Complex Systems, 2:321–355, 1988. [8] M. Casdagli. Nonlinear prediction of chaotic time series. Physica D, 35:335–356, 1989. [9] Tiam Li Sze. Radial Basis Function Network for Nonlinear System Identification. PhD thesis, Department of Automatic Control and Systems Enginnering, University of Sheffield, Mappin Street, S1 4DU, Sheffield, UK, 1995. [10] J. B.Elsner. Predicting time series using a neural network as a method of distinguishing chaos from noise. J. Phys. A: Math. Gen., 25:843–850, 1992. [11] L. A. Aguirre and S. A. Billings. Identification of models for chaotic systems from noisy data: implication for performance and nonlinear filtering. Physica D, 85(1,2):239–258, 1995. [12] James B. Katdke, Jeffrey Brush, and Joachim Holzfuss. Global dynamical equations and Lyapunov exponents from chaotic time series. Int. J. Bif. Chaos, 3(3):607–616, 1993. [13] S. A. Billings, M. J. Korenberg, and S. Chen. Identification of nonlinear output affine systems using an orthogonal least squares algorithm. Int. J. Systems Sci., 19(8):1559–1568, 1988. [14] S. A. Billings, S. Chen, and M. J. Korenberg. Identification of MIMO nonlinear systems using a forwardregression orthogonal estimator. Int. J. Control, 49(6):2157–2189, 1989. [15] M. Kortmann and H. Unbenhauen. Two algorithms for model structure determination of nonlinear dynamic systems with applications to industrial processes. Technical report, IFAC Symposium on Identification and System Parameter Estimation, Beijing, 1988. [16] L. A. Aguirre and S. A. Billings. Improved structure selection for nonlinear models based on term clustering. Int. J. Control, (62):569–587, 1995. [17] L. A. Aguirre and E. M. A. M. Mendes. Global nonlinear polynomial models: Structure, Term cluster and Fixed points. Int. J. Bif. Chaos, 6(2):279–294, 1996. [18] S. A. Billings and S. Chen. Extended model set, global data and threshold model identification of severely nonlinear systems. Int. J. Control, 50(5):1897–1923, 1989. [19] Q. M. Zhu and S. A. Billings. Recursive parameter estimation for nonlinear rational models. J. syst. Eng., 1: 63–76, 1991. [20] M. J. Korenberg, S. A. Billings, Y. P. Liu, and P. J. McIlroy. Orthogonal parameter estimation algorithm for nonlinear stochastic systems. Int. J. Control, 48(1):193–210, 1988. [21] M. V. Corrêa, E. M. A. M. Mendes, and L. A. Aguirre. Modeling chaotic dynamics with discrete nonlinear rational models. International Journal of Bifurcation and Chaos, 1999. [22] M. Korenberg. Orthogonal identification of nonlinear difference equation models. In Proc. of the 28th Midwest Symposium on Circuits and Systems, pages 90–95, Louisville, K.Y., 1985.

Identification of Nonlinear Dynamic Systems by E. Mendes, E. Nepomuceno and O. Neto

10

[23] Eduardo M. A. M. Mendes. Identification of Nonlinear Discrete Systems with Intelligent Structure Detection. PhD thesis, Department of Automatic Control and Systems Enginnering, University of Sheffield, Mappin Street, S1 4DU, Sheffield, UK, 1995. [24] Krister Forsman. Constructive Commutative Algebra in Nonlinear Control Theory. PhD thesis, Department of Electrical Engineering, Linköping University, S-581 83 Linköping, Sweden, 1991. [25] J. M. Oliveira and E. M. A. M. Mendes. Implicacoes da presença de pontos fixos complexos na identificacao de sistemas discretos nao-lineares. In XII Congresso Brasileiro de Automatica, Uberlandia, Brazil, 1998. [26] O. E. Rossler. An equation for continuous chaos. Physics Letters A, 57(5):397–398, 1976. [27] O. E. Rossler. An equation for hyperchaos. Physics Letters A, 71(2,3):155–157, 1979.

IDENTIFICATION OF NONLINEAR DYNAMIC SYSTEMS WITH REDUCED DEGREE OF NONLINEARITY Eduardo M. A. M. Mendes∗ , Erivelton G. Nepomuceno‡ and Oriane M. Neto∗ ∗

Departamento de Eletricidade Fundação de Ensino Superior de São João Del Rei Praça Frei Orlando 170 - Centro - São João Del Rei, MG - 36300.000 - Brazil Tel.: +55(32)3792552, Fax: +55(32)3792306, E-mails: [email protected], [email protected] ‡ Centro de Pesquisa e Desnvolvimento em Engenharia Elétrica Av. Antônio Carlos 6627 Belo Horizonte-MG CEP 31270-010, Brazil Tel:+55(31)499-5465 Fax:+55(31)499-5480 Email: [email protected]

Abstract In this paper identification of discrete monovariable rational models with reduced degree of nonlinearity from data generated by iterating nonlinear systems is addressed. In order to find the input-output equations and fixed points of such systems, the so-called Gröbner Bases and the concepts of Term Clusters and Clusters Coefficients are reviewed. Keyword:

Chaos, Identification, Nonlinear Discrete Systems, Fixed points

1. Introduction In the evolution of the study of chaotic systems, several distinct but sometimes co-existent phases can be distinguished. In the first phase, chaos was recognized as a deterministic dynamical regime which could be responsible for fluctuations that had been regarded as noise and therefore modelled as a stochastic process [1]. In a subsequent phase, it was necessary both to develop criteria to detect chaotic dynamics and to establish dynamical invariants to quantify chaos [2, 3, 4]. Having succeeded in diagnosing chaos, the next step was to build models which would learn the dynamics from data on the strange attractor [5, 6, 7, 8, 9, 10, 11, 12]. One of the great challenges on the identification of nonlinear dynamic systems is the choice of representation and selection of model structures. As far as model structure is concerned, several techniques have been suggested in the literature which are based mainly on statistical approaches and orthogonal matrix decomposition methods [13, 14, 15]. The models identified using such approaches are frequently used for predictions. However, when the main interest is the use of models for designing controllers for nonlinear systems, it could be necessary to obtain identified models without sobreparametrization. An attempt towards this direction is the use of concepts of Term Clusters and Cluster Coefficients introduced by Aguirre and Billings[16] and further developed by Aguirre and Mendes [17]. The Term clusters are crucial when the objective is to determine the fixed points of the monovariable systems. Besides, the judicious choice of the term clusters can avoid the deleterious effects of the overparametrization. In this work the aim is to identify dynamically valid models from data generated by a multivariable nonlinear system in a chaotic motion. In this case, the calculation of the fixed points are not straightforward: Gröbner Bases may be used when an analytical solution is required. The idea behind Gröbner Bases is the transformation of a given set of polynomials F into a standard form G. This form has the property of isolating one of the variables which allows the calculation of the fixed points. The information generated by the calculation of the fixed points is used to determine whether the identified model is overparametrized. The example used in this work is the so-called Rössler Hyperchaos equations. It will be shown that it is possible to identify dynamically valid models with a reduced degree of the nonlinearity when compared to the original set of equations. It is believed that the results presented here will be helpful when the objetive is to obtain models that can describe the original dynamics and are parsimonious at the same time.

Identification of Nonlinear Dynamic Systems by E. Mendes, E. Nepomuceno and O. Neto

2

This article is divided as follows: in Section 2 basic background necessary to the understanding of this work; in Section 3 the problem of the identification of nonlinear models with reduction of the nonlinearity degree is addressed whereas and the conclusions are presented in Section 4. 2. Background Material

2.1. Rational Representation of Nonlinear Systems Consider the Rational NARMAX model proposed in [18]

y(k) =

a(y(k−1), . . . , y(k−ny ), u(k−1), . . . , u(k−nu ), ... b(y(k−1), . . . , y(k−ny ), u(k−1), . . . , u(k−nu ), a(e(k), . . . , e(k−ne )) , ... b(e(k), . . . , e(k−ne ))

(1)

where u(k) and y(k) are respectively the input and output signals sampled at intervals of Ts . e(k) accounts for uncertainties, possible noise, unmodelled dynamics, etc. ny , nu and ne are the maximum lags of the output, input and noise, respectively. Moreover, such lags need not be the same in a(. . .) and b(. . .) which are nonlinear polynomial functions. In order to estimate the parameters in (1), it is convenient to define the numerator and denominator polynomials respectively as [18] Nn

a(k − 1) =

∑ pn j θn j = ψTn (k − 1)Θn

,

(2)

∑ pd j θd j = ψTd (k − 1)Θd

,

(3)

j=1 Nd

b(k − 1) =

j=1

where θn j , θd j are the parameters of the regressors up to time k − 1. Nn + Nd is the total number of parameters to be estimated. Unfortunately, the use of (1) to perform parameter estimation is not straightforward because such a function is nonlinear in the unknowns. An alternative solution to this problem is to multiply both sides of (1) by b(k − 1) and rearranging terms in order to yield [19] Nd

Y (k) = a(k − 1) − y(k) ∑ pd j θd j + b(k − 1)e(k) j=2

Nn

=

∑

j=1

Nd

pn j θn j − y(k) ∑ pd j θd j + ζ(k) j=2

= ψTn (k − 1)Θn − ψTd1 (k − 1)Θd + ζ(k) ,

(4)

where ψTd (k − 1) = [pd1 ψTd1 (k − 1)], θd1 = 1 and Y (k) = y(k)pd1 =

a(k − 1) pd1 + pd1 e(k) , b(k − 1) !

ζ(k) = b(k − 1)e(k) =

(5)

Nd

∑ pd j θd j

e(k) .

(6)

j=1

Because e(k) (white noise) is independent of b(k − 1) and has zero mean, E[ζ(k)] = E[b(k − 1)]E[e(k)] = 0. (5) reveals that all the terms of the form y(k)ψTd (k − 1), because of y(k), implicitly include the noise e(k) which is correlated with ζ(k). This, of course, results in parameter bias even if the noise e(k) is white. The aforementioned correlation occurs as a consequence of multiplying (1) by b(k − 1) and should be interpreted as the price paid for turning a function which is nonlinear in the parameters to one which is linear in the parameters.

3

Identification of Nonlinear Dynamic Systems by E. Mendes, E. Nepomuceno and O. Neto 2.2. Forward Selection Using Orthogonalisation

To avoid an excessive number of parameters in a rational model, an efficient subset selection procedure based on the orthogonal least squares (OLS) method [20, 13] can be performed. Given the full set of candidate process terms (regressors), the OLS algorithm selects the significant terms one by one until a criterion which indicates the adequacy of the model structure is satisfied. Following the procedure devised in [21], (1) can be rewritten in the well-known form Θ = [ΨT Ψ]−1 ΨT Y ,

(7)

Nn +Nd −1 = ∑i=1 pi (k).

where Θ = [Θn Θd1 T ] and Ψ pi (k) are column-vectors which represent the process terms. T ˆ satisfies the condition The regression vector [p1 . . . pNn +Nd −1 ] forms a set of basis vectors, and the OLS solution Θ ˆ that ΨΘ will be the projection of Y onto the space spanned by these basis vectors. The OLS method [22] involves the transformation of the set of pi into a set of orthogonal basis vectors, and thus makes it possible to calculate the individual contribution to the desired output from each basis vector. The regression matrix Ψ can be decomposed into T

T

Ψ = QR ,

(8)

where R, shown below, is an (Nn + Nd − 1) × (Nn + Nd − 1) triangular matrix with 1’s on the diagonal and 0’s below, as 1 r12 r13 . . . r1(Nn +Nd −1) 0 1 r23 . . . r2(Nn +Nd −1) .. R= (9) 0 0 1 . . . . . . . . . .. .. .. .. .. 0 ... ... ... 1 and Q is an N × (Nn + Nd − 1) matrix with orthogonal columns qi such that QT Q = D, where D is a diagonal matrix. Since the space spanned by the set of orthogonal basis vectors qi is the same space spanned by the original set of vectors (pi ), the regression equation (4) can be rewritten as Y (k) = QT (k − 1)gˆ + ξ(k)

(10)

The OLS solution, g, ˆ can be calculated using gˆ = D−1 QT Y or gˆi =

qTi Y , 1 ≤ i ≤ (Nn +Nd −1). qTi qi

(11)

ˆ can be retrieved by solving the triangular system RΘ ˆ = g. The original set of parameters Θ ˆ A great advantage of the orthogonal estimator is the possibility of selecting the relevant vectors (terms) as a byproduct. To demonstrate this, consider again the orthogonal regression equation (10). In doing so, it is assumed that the orthogonal property qTi q j = 0 for i 6= j holds. Therefore, if (10) is multiplied by itself and the time average is taken, the following equation can be derived 1 T 1 (Nn +Nd −1) 2 T 1 Y Y= ∑ gi qi qi + N ξT ξ N N i=1

(12) (N +N −1)

The output mean square value (MSV) Y T Y /N consists of two terms. The first term, ∑i=1n d g2i qTi qi /N, is the part of the output MSV explained by the regressors whereas the second term, ξT ξ/N, accounts for the unexplained output MSV. Owing to the orthogonal estimator, the increment towards the overall output MSV of each regressor (term or vector) can be computed independently as g2i qTi qi . Expressing this quantity as a fraction of the overall output MSV yields the Error Reduction Error (ERR) [ERR]i =

g2i qTi qi , 1 ≤ i ≤ (Nn +Nd −1) , Y TY

(13)

4

Identification of Nonlinear Dynamic Systems by E. Mendes, E. Nepomuceno and O. Neto

that can be used as a simple and effective means of selecting the most relevant regressors in a forward-regression manner. Therefore ERR imposes a hierarchy of terms according to their contribution towards the overall output MSV. 2.3. Term clustering The deterministic part of a rational NARMAX model can be expanded as the summation of terms with degrees of nonlinearity in the range 1 ≤ m ≤ `. Each m th-order term can contain a p th-order factor in y(k − ni ) and a (m − p) thorder factor in u(k − ni ) and is multiplied by a coefficient α p,m−p (n1 , . . . , nm ) or β p,m−p (n1 , . . . , nm ) as follows nα ,nα

α

p m 1 ` m y u y(k) = × ∑ ∑ ∑ α p,m−p (nα1 , . . . , nαm ) ∏y(k−nαi ) ∏ u(k−nαi ) D m=0 p=0nα ,nα i=1 i=p+1 1

(14)

m

where `β

D= ∑

β

β

m ny ,nu

p

β

β

m

∑ ∑ β p,m−p (n1 , . . . , nβm )∏ y(k−ni ) ∏

m=0 p=0nβ ,nβ 1

m

β

u(k−ni )

(15)

i=p+1

i=1

and α nα y ,nu

∑

nα y

≡

α nα 1 ,nm β

∑

nα u

···

nα 1 =1 β

β

ny ,nu

∑

β β n1 ,nm

∑

,

β

ny

≡

,

nα m =1

∑

β n1 =1

nu

···

∑

. ,

(16)

β nm =1

and the upper limit is ny if the summation refers to factors in y(k − ni ) or nu for factors in u(k − ni ).

Definition 2.1 [23]. nα ,nα

y u ∑nα ,nαm α p,m−p (nα1 , . . . , nαm ) in (14) are the coefficients of the term clusters of numerator Ωαy p um−p , which contain 1 terms of the form y(k − i) p u(k − j)m−p for m = 0, . . . , `α and p = 0, . . . , m. Such coefficients are called cluster coefficients of numerator and are represented as Σαy p um−p . 2

Definition 2.2 [23]. β

β

ny ,nu

∑

β β n1 ,nm

β

β

β

β p,m−p (n1 , . . . , nm ) in (15) are the coefficients of the term clusters of denominator Ωy p um−p , which con-

tain terms of the form y(k − i) p u(k − j)m−p for m = 0, . . . , `β and p = 0, . . . , m. Such coefficients are called cluster β coefficients of denominator and are represented as Σy p um−p . 2

For details about the definition of term clusters and cluster coefficients the reader is referred to [16, 23]. For the purposes of this section it suffices to realize that a term cluster is a set of terms of the same type and the respective cluster coefficient is obtained by the summation of the coefficients of all the terms, of the respective cluster, which are contained in the model. In practice it will be helpful to notice that terms of the same cluster explain the same type of nonlinearity.

Identification of Nonlinear Dynamic Systems by E. Mendes, E. Nepomuceno and O. Neto

5

2.4. Gröbner Bases Gröbner Bases are used for solving problems related to a system of polynomial ideals. The idea is to find the contractions of these polynomials that, hopefully, isolate one of the variable of the system in one of the system equations. In this work, Gröbner Bases will be used as a tool in the solution of the following problems : • conversion from state space form to input-output representation; • calculation of fixed points of multivariable systems [24]. The literature on Gröbner Bases is very extensive and complex. In this work, the main concern is to present the examples that help the reader to understand the importance and versatility of this tool. More details can be found in [25]. For the sake of clarity, an example on how to use Gröbner Bases as a tool for calculating the fixed points of multivariable systems will be given next. In the case of the conversion from state space to I/O representation, the reader is referred to the excellent work by Forsman [24]. Fixed Points of Multivariable Systems One of the applications of the Gröbner Bases is the calculation of the fixed points of multivariable systems. For instance, considerer the equations originally proposed by Rössler [26]: x˙1 x˙2 x˙3

= −x2 − x3 = x1 + σx2 = α + x3 x1 − x3 ρ

(17)

Computing the Gröbner Bases using the routine gbasis of MAPLE V, the following set of equations can be found: x1 − αx3 x2 + x3 α − x3 ρ + σx32

= 0 = 0 = 0

(18)

From (18), it can be noticed that the last polynomial depends on the variable x3. only. Therefore, the fixed points for the variable x3 can be readily calculated. The remaining fixed points, that is, the fixed points for variables x1 and x2 , can be calculated by substituting the fixed points for x3 in the previous polynomials and solving the resulting equations. It can also be noticed that the last polynomial has the same form as the polynomial used for calculating the fixed points for monovariable systems. Unfortunately, this is not always the case. 3. Results In this section, identification of nonlinear systems with reduced degree of the nonlinearity is presented and discussed. For the sake of clarity, the system used as an example is the well-know Rössler Hyperchaos oscillator[27]. Rössler Hyperchaos Oscillator Consider the equations for the Rössler Hyperchaos oscillator [27]: x˙ y˙ z˙ w˙

= = = =

−y − z x + 0.25y + w 3 + xz −0.5z + 0.05w

(19)

6

Identification of Nonlinear Dynamic Systems by E. Mendes, E. Nepomuceno and O. Neto A Poincaré Section for this system is generated by considering the following set of three difference equations:

x(k + 1) = 3.8x(k)(1 − x(k)) − 0.05(y)k) + 0.35)(1 − 2z(k)) y(k + 1) = 0.1((y(k) + 0.35)(1 − 2z(k)) − 1)(1 − 1.9x(k)) z(k + 1) = 3.78z(k)(1 − z(k)) + 0.2y(k)

(20)

In order to obtain the Poincaré section (please note that this section can be considered as a discrete dynamical system), (20) was simulated with initial conditions x=0.085, y=-0.121 and z=0.075. The result can be seen in Figure 1.

Figure 1: Poincaré section of the Rössler Hyperchaos system For the calculation of the fixed points of the Poincaré section, the ideas shown in the previous section were used. Table 1 shows the results.

1 2 3 4 5 6 7 8 9

x 2.06046 0.9229-1.5026i 0.9229+1.5026i -0.7951-0.8361i -0.7951+0.8361i 0.740225 -0.00193 0.00494 0.73007

y 60.7097 48.025+35.43i 48.025-35.43i 13.34-43.548i 13.34+43.548i 0.048379 -0.1112 -0.07178 0.02417

Z 2.1973 0.9352+1.6516i 0.9352-1.6516i -0.9156+0.898i -0.9156-0.898i 0.7389 0.7274 0.0052 -0.00173

Table 1: Fixed points calculated algebraically suing Gröbner basis Note that the system exhibits nine fixed point that cannot be detected simply by inspecting (20). The presence of four complex fixed points for each variable should be also noticed. This can cause serious difficulties when the interest is the identification of global models [25]. Considering that the data range for the system under investigation is between 0.1 and 0.98, it can be seen that only two fixed points are inside of these limits and the other seven (three real and four complex values) are outside. This situation shows that there is a possibility of identifying dynamically valid models with reduced degree of the nonlinearity.

Identification of Nonlinear Dynamic Systems by E. Mendes, E. Nepomuceno and O. Neto

7

For this system in particular, only the variable x and z have an I/O relationship. From the results related to the Gröbner basis calculations, it was determined that the model structure to be used in the identification procedure is rational as pointed in [25]. In addition to that, the variable z leads a much simpler (less terms) rational model than the variable x. However, this equation will not be listed in this work due to the size, 116 terms in the numerator and 15 terms in the denominator. Using the above information and the fact of that there are fixed points located outside of the range of the data, several models with degrees of nonlinearity between l=6 and l=9, maximum lag ny=3 and maximum number of terms equals to 131 were identified for the coordinate z. The best identified model has degree of nonlinearity l=7 and 131 terms of process. The fixed points for this model are shown in Table 2.

1 2 3 4 5 6 7

z 0.09 0.7389 0.7274 0.823+0.129i 0.823-0.129i 0.058+0.207i 0.058-0.207i

Table 2: Fixed Points for the identified model It is worth noticing that the fixed points highlighted in Table 2 are in agreement with the original listed elsewhere in the text. These two fixed points are located inside of data range, which confirms previous results regarding the location of the fixed points: a good estimation of the fixed points is closely related to the fact they are located inside of the data range. Since the other fixed points, that is, four complex and one real (0.09) values, are located outside of the data range, there is no much information on the time seriesnfor identification purposes. This causes the estimation algorithm to place them inside of the data range, that is, between 0.1 and 0.98. The lack of relevant information related to these points impairs a precise identification. On the other hand, the fixed points 0.7389 and 0,7273 are accurately identified since the data contain enough information to do so. In order to provide a better understanding, Figure 2 shows tow-dimensional attractors reconstructed from the original time series and from data collected from iterating the identified model. From these attractors, it can be noticed that the identified model reproduces the dynamics of the Poincaré section of Rössler Hyperchaos system fairly. 4. Conclusions In this work it has been shown that it is possible to obtain models with reduced degree of nonlinearity that can reproduce the dynamic properties of Rössler Hyperchaos system. The influence of the location of the fixed points in relation to the data range is addressed. It has been shown that when the fixed points are located outside of the data range, their influence on the data is such that they can be eliminated and thus leading the parsimonious models. It is believed that the results presented in this paper are general and therefore can be applied to a wider range of nonlinear systems. Acknowledgments E. Mendes acknowledges the support of CNPq under the grant 301313/96-2 and 201557/91-6. E. Nepomuceno ackonowledges the financial support from CNPq and O. Neto acknowledges the financial support from CAPES. References [1] Edward N. Lorenz. Deterministic nonperiodic flow. Journal of The Atmospheric Sciences, 20:131–141, 1963. [2] J. Guckenheimer. Noise in chaotic systems. Nature, 298:358–361, 1982.

Identification of Nonlinear Dynamic Systems by E. Mendes, E. Nepomuceno and O. Neto

(a)

(b)

Figure 2: a) Attractor for the original system; b) Attractor for the identified model

8

Identification of Nonlinear Dynamic Systems by E. Mendes, E. Nepomuceno and O. Neto

9

[3] J. P. Eckmann and D. Ruelle. Ergodic theory of chaos and strange attractors. Rev. Mod. Phys., 57:617–656, 1985. [4] T. A. Denton and G. A. Diamond. Can the analytic techniques of nonlinear dynamics distinguish periodic, random and chaotic signals? Compt. Biol. Med., 21(4):243–264, 1991. [5] J. D. Farmer and J. J. Sidorowich. Predicting chaotic time series. Phys. Rev. Lett., 59(8):845–848, 1987. [6] J. P Crutchfield and B. S. McNamara. Equations of motion from a data series. Complex Systems, 1:417–452, 1987. [7] D. S. Broomhead and D. Lowe. Multivariable functional interpolation and adaptive networks. Complex Systems, 2:321–355, 1988. [8] M. Casdagli. Nonlinear prediction of chaotic time series. Physica D, 35:335–356, 1989. [9] Tiam Li Sze. Radial Basis Function Network for Nonlinear System Identification. PhD thesis, Department of Automatic Control and Systems Enginnering, University of Sheffield, Mappin Street, S1 4DU, Sheffield, UK, 1995. [10] J. B.Elsner. Predicting time series using a neural network as a method of distinguishing chaos from noise. J. Phys. A: Math. Gen., 25:843–850, 1992. [11] L. A. Aguirre and S. A. Billings. Identification of models for chaotic systems from noisy data: implication for performance and nonlinear filtering. Physica D, 85(1,2):239–258, 1995. [12] James B. Katdke, Jeffrey Brush, and Joachim Holzfuss. Global dynamical equations and Lyapunov exponents from chaotic time series. Int. J. Bif. Chaos, 3(3):607–616, 1993. [13] S. A. Billings, M. J. Korenberg, and S. Chen. Identification of nonlinear output affine systems using an orthogonal least squares algorithm. Int. J. Systems Sci., 19(8):1559–1568, 1988. [14] S. A. Billings, S. Chen, and M. J. Korenberg. Identification of MIMO nonlinear systems using a forwardregression orthogonal estimator. Int. J. Control, 49(6):2157–2189, 1989. [15] M. Kortmann and H. Unbenhauen. Two algorithms for model structure determination of nonlinear dynamic systems with applications to industrial processes. Technical report, IFAC Symposium on Identification and System Parameter Estimation, Beijing, 1988. [16] L. A. Aguirre and S. A. Billings. Improved structure selection for nonlinear models based on term clustering. Int. J. Control, (62):569–587, 1995. [17] L. A. Aguirre and E. M. A. M. Mendes. Global nonlinear polynomial models: Structure, Term cluster and Fixed points. Int. J. Bif. Chaos, 6(2):279–294, 1996. [18] S. A. Billings and S. Chen. Extended model set, global data and threshold model identification of severely nonlinear systems. Int. J. Control, 50(5):1897–1923, 1989. [19] Q. M. Zhu and S. A. Billings. Recursive parameter estimation for nonlinear rational models. J. syst. Eng., 1: 63–76, 1991. [20] M. J. Korenberg, S. A. Billings, Y. P. Liu, and P. J. McIlroy. Orthogonal parameter estimation algorithm for nonlinear stochastic systems. Int. J. Control, 48(1):193–210, 1988. [21] M. V. Corrêa, E. M. A. M. Mendes, and L. A. Aguirre. Modeling chaotic dynamics with discrete nonlinear rational models. International Journal of Bifurcation and Chaos, 1999. [22] M. Korenberg. Orthogonal identification of nonlinear difference equation models. In Proc. of the 28th Midwest Symposium on Circuits and Systems, pages 90–95, Louisville, K.Y., 1985.

Identification of Nonlinear Dynamic Systems by E. Mendes, E. Nepomuceno and O. Neto

10

[23] Eduardo M. A. M. Mendes. Identification of Nonlinear Discrete Systems with Intelligent Structure Detection. PhD thesis, Department of Automatic Control and Systems Enginnering, University of Sheffield, Mappin Street, S1 4DU, Sheffield, UK, 1995. [24] Krister Forsman. Constructive Commutative Algebra in Nonlinear Control Theory. PhD thesis, Department of Electrical Engineering, Linköping University, S-581 83 Linköping, Sweden, 1991. [25] J. M. Oliveira and E. M. A. M. Mendes. Implicacoes da presença de pontos fixos complexos na identificacao de sistemas discretos nao-lineares. In XII Congresso Brasileiro de Automatica, Uberlandia, Brazil, 1998. [26] O. E. Rossler. An equation for continuous chaos. Physics Letters A, 57(5):397–398, 1976. [27] O. E. Rossler. An equation for hyperchaos. Physics Letters A, 71(2,3):155–157, 1979.