Neural Networks, IEEE Transactions on - IEEE Xplore

IEEE TRANSACTIONS ON NEURAL NETWORKS, VOL. 14, NO. 1, JANUARY 2003

243

Letters__________________________________________________________________________________________ Computing With Phase Locked Loops: Choosing Gains and Delays José Roberto Castilho Piqueira, Fernando Moya Orsatti, and Luiz Henrique Alves Monteiro

Abstract—We simulate a four-node fully connected phase-locked loop (PLL) network with an architecture similar to the neural network proposed by Hoppensteadt and Izhikevich, using second-order PLLs. The idea is to complement their work analyzing some engineering questions like: • how the individual gain of the nodes affects the synchronous state of whole network; • how the individual gain of the nodes affects the acquisition time of the whole network; • how close the free-running frequencies of the nodes need to be in order to the network be able to acquire the synchronous state; • how the delays between nodes affect the synchronous state frequency. The computational results show that the Hoppensteadt–Izhikevich network is robust to the variation of these parameters and their effects are described through graphics showing the dependence of the synchronous state frequency and acquisition time with gains, free-running frequencies, and delays. Index Terms—Acquisition time, free-running frequencies, phase-locked loops (PLLs), synchronism, synchronous state.

I. INTRODUCTION Computation using oscillators is starting to become a reality inspired in the oscillatory nature of the biological neurons and their synaptic connections [1], [2]. The idea is based on the fact that life is strongly connected to oscillations at the several levels of organization: population phenomena, physiological synchronization, cellular rhythms, and molecular dynamics are ubiquitous in nature [3]–[7]. Thinking about brain information processing, synchronized oscillations are identified through electroencephalography associated to systems as hippocampus, thalamus, cortex, and frontal lobe playing a fundamental role to the sensory processes [8], [9]. Phase-locking oscillations seem to be involved in the emergence of memory and cognitive capacity, generating complex adaptive behaviors for brain controlled organisms [10]–[14]. Using phase-locked loop (PLL) models is an interesting way of interpreting the conversion of sensory and motor signals to firing rates [15] providing realistic models for pattern recognition in neural networks based in phase-locked oscillations [16], [17]. In [16], Hoppensteadt and Izhikevich described a neural network built with commercial PLLs LMC568 or LM565 with interesting experimental results related to pattern recognition. In order to implement

Fig. 1. Phase-locked loop.

the network they considered the theoretical fact expressed in their Theorem 1, stating that a neural network whose nodes are PLL with -close central frequencies and whose synaptic connections are symmetric always captures the synchronous state. In the synchronous state, all the spatial frequency errors vanish and the spatial phase errors remain constant [18] with the phase error pattern providing the recognition [16]. But there are some questions that, from the engineering point of view, need to be discussed as the acquisition time for the steady state and the influence of the delays between the nodes. In this work, using the simulation of a fully connected four node network, we study the influence of the gain of the nodes and the delays between nodes in the acquisition time and in the steady-state frequency. In addition, the changes in the acquisition time and in the steady-state frequency due to the dispersion among the free-running frequencies of the nodes are analyzed. We start describing the dynamics of the nodes and how they form a four-node fully connected architecture with the synaptic connections. Then, the computational results are presented, according to the gain of the single nodes. II. SINGLE-NODE DYNAMICS: SECOND-ORDER PLL PLLs are the basic elements in the electronic extraction of clock signals [19]–[22] and in our models they will be the basic nodes for the full-connected network. Second-order PLLs were chosen for the models, in order to avoid the onset of periodic and chaotic attractors for the phase and frequency errors in an autonomous isolated node of the network that can appear by considering PLL with higher order [23]. PLLs are composed by three elements: a phase detector (PD), a low-pass filter (F) and a voltage controlled oscillator (VCO), connected as shown in Fig. 1. Considering that the input signal vi (t) and the VCO signal v0 (t) are given by vi (t) =Ai sin i (t) v0 (t) =A0 cos 0 (t)

Manuscript received August 4, 2002; revised September 10, 2002. The work of J. R. C. Piqueira and L. H. A. Monteiro was supported by CNPq. J. R. C. Piqueira and F. M. Orsatti are with the Departamento de Engenharia de Telecomunições e Controle, Escola Politécnica, Universidade de São Paulo, CEP05508–900 Sao Paulo SP, Brazil (e-mail: [email protected]). L. H. A. Monteiro is with the Departamento de Engenharia de Telecomunicações e Controle, Escola Politécnica, Universidade de São Paulo and also with Pós Graduação em Engenharia Elétrica, Universidade Presbiteriana Mackenzie, CEP05508–900 Sao Paulo SP, Brazil. Digital Object Identifier 10.1109/TNN.2002.806633

one can prove [20]–[22] that the output of the PD, neglecting the double-frequency terms, can be expressed as being proportional to the sine of the phase-error. That is vd (t) =

Kd sin [i (t) 2

0 0 (t)]

with the constant Kd depending on Ai and A0 and on the construction of the phase detector.

1045-9227/03$17.00 © 2003 IEEE

244


Fig. 2. The four-node network.

The filter is considered to be a linear first-order lag with transfer function [24] F (s) =

!0

25s + !0

Fig. 3. Acquisition time

2 gain for a network without delays.

and the phase of the VCO varies with the output (vc (t)) of the filter according to _0

= K0 v (t) Combining the expression of v (t), the filter transfer function F (s) c

d

and the control of the phase given above, we have the dynamics of the phase of the VCO described by (1) 250 + !0_0 = !0 K0 Kd sin (i 0 0 )

2

Using that the central angular frequency !0 is related to the central frequency f0 by 25, choosing a nondimensional variable T f0 t, using primes to indicate the derivatives related to T and, from now on, using bars over the parameters indicating a division by the central angular frequency !0 , (2) becomes

= K sin [ (T ) 0 0 (T )] (2) =2!0 ) is called the normalized gain of the PLL

000 + 00

i

= (K0 Kd In (2), K and, in what follows, (2) is used as the model for each node of the network providing results that are independent of the frequency band. Therefore, the simulations conducted are valid either for the Megahertz range of the PLL or for the 1–50-Hz range of the biological phenomena considering that a unit of time corresponds to a period of a f0 periodic signal. III. FULLY CONNECTED FOUR-NODE NETWORK In order to evaluate the effects of the gains and delays in a fully connected PLL network, we implemented, using the MATLAB-Simulink [25], [26], a four-node network as shown in Fig. 2. In the simulations we considered si;j as the weight of the synaptic connection between the nodes i and j, the normalized gain of each node and the signal delay between nodes i and j given by i;j . K (j ) Each node was a PLL described by (1) and the input signal vi of the node j was given by (j )

vi

4

(t) =

(k) sk;j v0

(t 0 ) k;j

k=1;k6=j

with 4

sk;j

=1

k=1;k6=j

The condition for a symmetric network was obtained setting

=s1 2 = s2 1 = s3 4 = s4 3 A2 =s2 3 = s3 2 = s1 4 = s4 1 A3 =s1 3 = s3 1 = s2 4 = s4 2 A1

;

;

;

;

;

;

;

;

;

;

;

;

Fig. 4. Synchronous-state frequency


The main objectives of our simulations were to verify the influence of the gains of the nodes, the delays and the free-running frequencies in the acquisition time and in the synchronous-state frequency. The network is considered to be in the synchronous state when all the spatial frequency errors, given by _0i 0 _0j , vanish. Then the normalized synchronous state frequency is obtained examining the phase output of the nodes and the acquisition time is given by the number T of periods of the central frequency signal taken for the phase deviations between nodes to become constant. As a matter of fact there are small amplitude oscillations around the synchronous state, even in the steady-state situation. Therefore we considered the acquisition time as the time taken for the system to reach the steady state and we calculate the normalized synchronous-state frequency as the mean value of the frequency in the steady state. IV. COMPUTATIONAL RESULTS In order to evaluate the influence of the individual PLL gains we set the node normalized free-running frequencies to 0.85, 0.95, 1.05, and 1.15, respectively, with A1 = A2 = :25 and A3 = :5 and the initial phase of the nodes were set to 5, 5=2, 0, 05=2. Fig. 3 shows the acquisition time and Fig. 4 shows the normalized of the synchronous state frequency obtained as the individual gains K nodes, considered to be equal, were adjusted. The simulated network was delay-free and the normalized synchronous state frequencies that were supposed to be one [27]–[29] appeared a little different. The acquisition time showed to be independent from the value of the gains and it is approximately equal to 13 periods of the central frequency signal.


245

Fig. 5. Amplitude of the oscillations of the phase differences in the synchronous state gain for a network without delays.

Fig. 8. Amplitude of the oscillations of the phase differences in the synchronous state gain for a network without delays.

2

Fig. 6. Acquisition time

Fig. 7.


Synchronous-state frequency


In order to see how the oscillations around the synchronous state are, we show in Fig. 5 the amplitude of the oscillations of the phase differences between nodes 1 and 4. These oscillations could be thought as a quality measure for the synchronous state and they depend on the gain. Changing the synaptic connections (A1 = 1=3, A2 = 1=3, and A3 = 1=3) the results, from a qualitative point of view, were the same as shown in Figs. 6–8. An auxiliary variable a was used to test how close the free-running frequencies of the individual nodes need to be in order for the acquisi-

2

Fig. 9. Acquisition time delays.

2 frequencies dispersion for a network without

Fig. 10. Synchronous-state frequency without delays.

2 frequencies dispersion for a network

tion of the synchronous state to be possible and simulations were performed considering that

!0(1) =1 0 1:5a; !0(2) = 1 0 0:5a !0(3) =1 + 0:5a; !0(4) = 1 + 1:5a: The simulated network was delay-free with A1 = A2 = 0:25, A3 = = 2:5. 0:5, and K Fig. 9 shows the acquisition time and Fig. 10 shows the normalized synchronous state frequency as a function of the auxiliary variable. The results show a small variation of the acquisition time according to the

246


Fig. 11. Amplitude of the oscillations of the phase differences in the synchronous state frequencies dispersion for a network without delays.

2

Fig. 14. Synchronous-state frequency with delays.

2 auxiliary variable b for a network

In the network with delays the difference between the free-running frequencies had to be smaller than in the case without delays, otherwise the acquisition is not possible. The normalized synchronous-state frequency is lower than the mean value of the node free-running frequencies [30] and the acquisition time showed to be a little higher than in the situation without delays. The study of the influence of the delays could be done considering a default value of 0 = 0:1=f0 and taking 1;2 =2;1 = 3;4 = 4;3 = 0 (1

0 b)

2;3 =3;2 = 1;4 = 4;1 = 0 1;3 =3;1 = 2;4 = 4;2 = 0 (1 + b):

Fig. 12.

Acquisition time

2 frequencies dispersion for a network with delays.

The node free-running frequencies were set to 0.85, 0.95, 1.05, and 1.15 again and the PLL normalized gains were set to 2.5. Fig. 14 shows how the synchronous state frequency depends on the delays showing that the acquisition is possible but in a lower frequency than in the delay-free case. V. CONCLUSION

Fig. 13. Synchronous-state frequency with delays.

2 frequencies dispersion for a network

value of a and that the synchronous state frequency remains almost constant with a. Fig. 11 shows the amplitude of the oscillations of the phase differences between nodes 1 and 4 at the steady state. One can see that the amplitude of the oscillations is increased with the dispersion auxiliary variable a. The same problem was analyzed considering that the delays between the nodes are equal and set to a value corresponding to 10% of the period of the central frequency signal. The results for the acquisition time and normalized synchronous state frequency are shown in Figs. 12 and 13, respectively.

The neural network proposed by Hoppensteadt and Izhikevich using second-order PLLs is an interesting architecture for computation being robust to several kinds of adjustments and variations. For the individual gain of the nodes, it is important to choose a value for which the synchronous state is captured. This gain value can not be too high because the amplitude of the oscillation of the phase differences in the steady state will become high due to, probably, the second harmonic terms in the output of the phase detectors of the PLL nodes. The free-running frequencies of the individual nodes need not to be so close and the synchronous state can be captured even when there is a high dispersion of their values. The only point to take care is the possible high amplitude of the oscillation of the phase deviations due to the frequencies dispersion. The delays between the nodes do not avoid the acquisition of the synchronous state that, in this case, is obtained in a lower frequency than the usual. REFERENCES [1] F. G. Hoppensteadt and E. M. Izhikevich, “Synaptic organizations and dynamical properties of weakly connected neural oscillators I. Analysis of a canonical model,” Biol. Cybern., vol. 75, pp. 117–127, 1996. [2] , “Synaptic organizations and dynamical properties of weakly connected neural oscillators II. Learning phase information,” Biol. Cybern., vol. 75, pp. 129–135, 1996. [3] L. A. Segel, Modeling Dynamic Phenomena in Molecular and Cellular Biology. New York: Cambridge Univ. Press, 1984.


247

[4] L. Glass and M. C. Mackey, From Clocks to Chaos: The Rhythms of Life. Princeton, NJ: Princeton Univ. Press, 1988. [5] F. C. Hoppensteadt and C. S. Peskin, Mathematics in Medicine and Life Sciences Springer-Verlag, New York, 1992. [6] J. T. Bonner, Life Cycles. Princeton, NJ: Princeton Univ Press, 1993. [7] P. Sassone-Corsi, “Molecular clocks: Mastering time by gene regulation,” Nature, vol. 392, pp. 871–874, 1998. [8] R. D. Traub, M. A. Whittington, I. M. Stanford, and J. G. R. Jefferys, “A mechanism for generation of long-range synchronous fast oscillations in the cortex,” Nature, vol. 383, pp. 621–624, 1996. [9] E. Basar, Brain Functions and Oscillations I: Principles and Approaches. Berlin, Germany: Springer-Verlag, 1998. [10] M. Abeles, Y. Prut, H. Bergman, E. Vaadia, and A. Aerstsen, “Integration, synchronicity and periodicity,” in Brain Theory, A. Aertsen, Ed. Amsterdam, The Netherlands: Elsevier, 1993. [11] M. A. Arbib, Handbook of Brain Theory and Neural Networks. Cambridge, MA: The MIT Press, 1995. [12] K. MacLeod, A. Bäcker, and G. Laurent, “Who reads temporal information contained across synchronized and oscillatory spike trains?,” Nature, vol. 395, pp. 693–698, 1998. [13] E. Basar, Brain Functions and Oscillations II: Integrative Brain Function. Berlin: Springer-Verlag, 1999. [14] E. R. Kandel, J. H. Schwartz, and T. M. Jessel, Principles of Neural Science, 4th ed. New York: Mc Graw Hill, 2000. [15] E. Ahissar, “Temporal-code to rate-code conversion by neuronal phaselocked loops,” Neural Computation, vol. 10, pp. 597–650, 1998. [16] F. C. Hoppensteadt and E. M. Izhikevich, “Pattern recognition via synchronization in phase-locked loop neural networks,” IEEE Trans. Neural Networks, vol. 11, pp. 734–738, May 2000. , “Oscillatory neurocomputers with dynamic connectivity,” Phys. [17] Rev. Lett., vol. 82, no. 14, pp. 2983–2986, 1999. [18] K. Dessouky and W. C. Lindsey, “Phase and frequency transfer between mutually synchronized oscillators,” IEEE Transactions on Communications, vol. COM-32, pp. 110–117, Feb. 1984.

[19] A. J. Viterbi, Principles of Coherent Communication. New York: McGraw-Hill, 1966. [20] W. C. Lindsey, Syncronization Systems in Communication and Control. New Jersey: Prentice-Hall, 1972. [21] F. M. Gardner, Phaselock Technics. New York: Wiley, 1971. [22] R. E. Best, Phase-Locked Loops. New York: McGraw-Hill, 1984. [23] J. R. C. Piqueira, “Aplicação da teoria qualitativa de equações diferenciais a problemas de sincronismo de fase,”, Escola Politécnica-USP, São Paulo, Brazil, 1987. [24] K. Ogata, Modern Control Engineering. Upper Saddle River, NJ: Prentice-Hall, 1997. [25] D. Hanselman and B. Littlefield, Mastering MATLAB. Upper Saddle River, NJ: Prentice-Hall, 1996. [26] Simulink: Dynamic System Simulation for MATLAB. Natick, MA: The Math Works, 1997. [27] M. de Souza Vieira, A. J. Lichtenberg, and M. A. Lieberman, “Nonlinear dynamics of self-synchronizing systems,” Int. J. Bifurcation Chaos, vol. 1, no. 3, pp. 691–699, 1991. , “Self-synchronization of many coupled oscillators,” Int. J. Bifur[28] cation Chaos, vol. 4, no. 6, pp. 1563–1577, 1994. [29] G. Goldsztein and S. H. Strogatz, “Stability of synchronization in networks of digital phase-locked loops,” Int. J. Bifurcation Chaos, vol. 5, no. 4, pp. 983–990, 1995. [30] J. R. C. Piqueira and L. H. Monteiro, “Full-connected three node phaselocked loop network,” in Nonlinear Dynamics, Control, Chaos and Applications to Engineering Sciences, J. M. Balthazar, P. B. Gonçalves, R. M. F. L. R. F. Brasil, I. L. Caldas, and F. B. Rizatto, Eds. Campinas, Brazil: American Academy of Mechanics and Associação Brasileira de Ciências Mecânicas, 2000, vol. 5, pp. 6–18.