Spiking Neuron Models: A Review

Spiking Neuron Models: A Review Ahmed A. Abusnaina, Rosni Abdullah

Spiking Neuron Models: A Review 1

Ahmed A. Abusnaina, 2Rosni Abdullah School of Computer Sciences, Universiti Sains Malaysia, [email protected] 2 School of Computer Sciences, Universiti Sains Malaysia, [email protected]

*1

Abstract This paper illustrates various spiking neuron models, which could be used in simulation or implementation of different spiking neural network applications; such as brain simulation and engineering problems. Biologically plausible and computational efficiency are the most important factors that could be taken in consideration in order to choose one from the available spiking neuron models. The chosen model should be the best to fit and meet the application specifications. This paper discusses these spiking neuron models mathematically and how it could be used in simulating hippocampus. As well as, this paper considers valuable comparisons between these models which they are provided according to the biological plausibility, computational efficiency, number of variables and complexity.

Keywords: Spiking neuron model, Spiking neural network, Action potential, Biological neuron behaviors.

1. Introduction Biological neurons use short and sudden increases in voltage (signal) to send information to neighboring neurons. These signals are more commonly known as action potentials, membrane potential, or spikes as shown in Figure1. Recent neurological research has shown that neurons encode information in the timing of single spikes, and not only just in their average firing frequency. Networks of spiking neurons are more powerful than their non-spiking predecessors as they can encode temporal information in their signals [1]. Neural networks classified according to computational units (neurons) as three generations, [2- 4]: First generation: Networks based on McCulloch and Pitts’ neurons as computational units, i.e. with only digital outputs. Second generation: Networks based on computational units that apply an activation function with a continuous set of possible output values, such as sigmoid or polynomial or exponential functions (e.g. feed-forward and recurrent networks). The real-valued outputs of such networks can be interpreted as firing rates of natural neurons. Third generation: Networks which employ spiking neurons as computational units, i.e. spiking neural networks. These networks taking into account the precise firing times of neurons for information coding. Spiking neural Network (SNN) models capture neuronal behavior more accurately than a traditional neural network. SNN include the concept of time into their operating model [3]. The idea is that neurons in the SNN do not fire at each propagation cycle (as it happens with traditional neural networks), but rather they fire only when a membrane potential reach its threshold. Spiking neural networks (SNNs) exhibit interesting properties that make them particularly suitable for applications that require fast and efficient computation and where the timing of input/output signals carries important information. [5]. Spiking neural networks models provide powerful tools and solutions for wide range of problems, such as elementary processes in the brain, including neural information processing, plasticity, learning and formation of associative memories, simulation of brain parts such as hippocampus. As well as problems in applied engineering, e.g. fast signal-processing, event detection, classification, speech recognition, spatial navigation or motor control [6-8], however, classical Neural Networks (NN) is still used to solve some of such problems, e.g. Decision Support System [28] and Robot Path Planning [29].

International Journal of Digital Content Technology and its Applications(JDCTA) Volume8, Number3, June 2014

14


Figure 1. A) Spiking Neural Networks and their connections VS. B) Biological neurons. part B) taken from [26]. The rest of this paper is organized as follows, in section two there is a description of each spiking neuron model, section three provides a comparison between spiking neuron models and general discussion, as well as it concludes the paper.

2. Spiking Neuron Models Spiking Neurons can be simulated using various models. These models differ in neuronal behaviors that can exhibit; Figure 2 shows some of the spiking neuron behaviors. Simulation of spiking neural networks can be in large-scale; where the number of neurons more than 10 5 and the number of synaptic connections more than 10 7 [9]. As well as the simulation could be done with small number of neurons (about 100 neuron, i.e. [10][11]), but in this case it does not reflect the realism of biological environment.

Figure 2. Spiking neuron behaviors. This figure is reproduced with permission from www.izhikevich.com. (Electronic version of the figure and reproduction permissions are freely available at www.izhikevich.com).

15


There are two factors that can characterize each spiking neuron model: 1- Biologically plausible: this means that the spiking neuron will be able to produce a set of firing patterns or behaviors exhibited by real biological neurons; such patterns or behaviors shown in Fig.2. 2- Computational efficiency: this factor is determined by the number of variables used in order to represent the neuron model (activation function) and the number of floating point operations needed to accomplish one mille second (ms) of model simulation. These operations affect the time needed to accomplish the simulation, as well as it affect computational resources such as processing units and memory usage. The model that has less number of variables and floating point operations is more efficient. So, the complexity of neuron model can be classified into five categories Very Low, Low, Medium, High and Very High, as shown in Table1.

2.1. Izhikevich Model It is biologically plausible. As well as it is computationally efficient, as it takes 12 floating points operations. Izhikevich Model depends on four parameters, which is capable to reproduce spiking and bursting behavior of known types of cortical neurons. Hippocampus neurons can be described by this model, and it is more suitable for large-scale simulations [12]. The model represented by two differential equations (1) and (2), where equation (3) is used to adjust membrane voltage v and the recovery variable u, as following:    

= 0.04v2 + 5v + 140 - u + I

(1)

= a (bv - u)

(2)

With the auxiliary after-spike resetting, expressed in equation (3) if v ≥ 30 mV;

= then   =+

(3)

Where: v represents the membrane potential of the neuron and u represents a membrane recovery variable, which accounts for the activation of K+ ionic currents and inactivation of Na+ ionic currents, and it provides negative feedback to v. I : Synaptic currents or injected dc-currents. a, b, c, and d are dimensionless parameters. Izhikevich Model exhibits all neuron behaviors [13]. In addition it is widely used in benchmarking and simulation of spiking neural networks.

2.2. Integrate-and-Fire Model (I&F) I&F one of the most used models by neuroscientist to prove their analytical results [1] [13-14]. I&F model plays an important role in understanding the relationship between the variability of inputs to neurons and the variability of their outputs. In addition I&F model is useful in understanding the basic properties of large networks of neurons. The mechanism of this model is: the neuron membrane charged by currents flowing into it, upon the membrane potential exceeding a certain threshold θ, firing the action potential (spike) and discharging. The I&F model can be represented by an RC circuit (Resistor, Capacitor and current source in parallel). When the voltage is below the threshold θ, its value is determined by equations (4) and (5):Cm

 

=–

 

if V ≥ Vthreshold

+I

then V = Em

(4) (5)

Where: - V: membrane potential, Cm: membrane capacitance, Em: membrane equilibrium potential, Rm: membrane resistance, I: the total currents flowing into the neuron cell.

16


Because of its simplicity e.g. depends on one variable, I&F fail to exhibit all neuron behaviors e.g. phasic spiking, bursting, and rebound responses. So, it can be concluded that I&F is not good enough in simulations [16][22]. Many improvements done to enhance I&F model, such as: I&F with adaptation, Integrate-and-Fire-or-Burst, Resonate-and-Fire, Quadratic I&F and IF-FHN, each model concentrate on a specific neuron behavior [19-22].

2.3. Hodgkin-Huxley Model (HH) HH model describes how the membrane potential charges over time in a small region of squid giant axon. Action potential (spike): - is a sharp increase in the membrane potential followed by a somewhat less sharp decrease toward the resting potential. This can be described using HH model [13-14] by equation (6) : Cm

 

= - gL (V - EL) – gNa m3 h (V - ENa) - gK n4 (V - EK) + I

(6)

Where: - V: membrane potential, Cm: membrane capacitance, g: Conductance, E: Equilibrium potential, I: local circuit current, m, h, n: constants. Hodgkin-Huxley neuron is able to model biophysically and biologically meaningful properties of neuron.

2.4. BVP Model It is a simple representation of Hodgkin-Huxely Model [15]. But even though it cannot exhibit all neuron behaviors; e.g. it cannot exhibit chaotic dynamics or bursting [13]. The BVP model is described by equations (7), (8) and (9):   

= c ( y + v - v3/ 3 + z ) =

– ∗    

1 -2b/3 < a < 1,

(8)



0 < b < 1,

(7)

b < c2

(9)

Where: a,b,c are constants, v: membrane voltage, z: stimulus current.

2.5. Wilson Model Wilson model can exhibit all neurocomputational properties, which can be described by 4 polynomial differential equations [13][16], as following Ij = gj (V - Ej) c

 

(10)

= – m∞ (V) (V - ENA)+ gK R (V – EK) + I  

=

 

(-R + R∞ (V ) )

(11) (12)

m∞ (V) = 17.8 + 47.6 + 33.8V2 ,

(13)

2

R∞ (V) = 1.24 + 3.7V + 3.2 V . The first equation describes changes in the membrane potential V due to membrane capacitance (C), ionic currents, and the stimulating current I (in nA). ENa and EK are the equilibrium potentials for Na+ and K+, respectively, while m∞ is the Na+ activation function. The second equation describes K+

17


channel activation as a function of the time constant  and the equilibrium state R∞ (V). The variable in this equation has been designated R to signify that it represents a recovery variable.

2.6. Morris–Lecar Model This model became quite popular in computational neuroscience community. It consists of a membrane potential equation with instantaneous activation of Ca current and an additional equation describing slower activation of K current, the model can exhibit various types of spiking [13][17]. = C V\ +gL(VL) + gca M ( V – Vca ) + gKN(V- VK)

I

(14)

\

(15)

\

N = λN (V) [N∞ (V) – N]

(16)

M∞ (V) = 1/2 {1 + tanh [(V- V1)/V2]}

(17)

M = λM (V) [M∞ (V) – M]

\

λM (V) = λM cosh ([V- V1] /2V2)

(18)

N∞ (V) = 1/2 { l + tanh [(V - V3)/V4]}

(19)

\

λN (V) = λN cosh ([V- V3]/2V4)

(20)

For a list of definitions for these variables and parameters refer to [17].

2.7. Hindmarsh–Rose Model This model is biologically plausible; it can reproduce biological neuron behaviors. But the problem is how to find the F, G, and H functions that make it plausible.      

= y – F(v) – z +I + u

(21)

= G(v) – y

(22)

=

 

* H(v) – z

(23)

Where v(t) represents the membrane potential, usually considered as the natural output of the cell, y(t) and z(t) are the recovery and the adaptation variables, respectively. The external current stimulus is given by I constant and input u. Furthermore, is the time constant and the functions F(v), G(v), and H(v) are chosen to display the generation of bursts of spikes and are usually third-, second-, and ﬁrst-order polynomials, respectively [18][27].

2.8. Spike-Response Model (SRM) The SRM inspired biologically, flexible model of spiking neuron. It is based on kernel functions that describe the effect of spike reception and emission on the membrane potential of the neuron [23]. The Spike Response Model is more general than other classical neuron models such as Integrateand-Fire (IF) model, because of that the kernel function κ is not restricted and it is time-dependent. As well as the model is described in a more realistic manner [24]. Spike Response Model is described by equations (24) and (25): ∞

() = η(t − tˆ) + ∫∞ κ(t − tˆ , s) I(t − s) 

(24)

In order to account for spike generation there is a threshold condition (θ), therefore:

18


if v(t) ≥ θ and v\ (t) > 0, then tˆ = t

(25)

Where: v(t) represents the neuron variable state which it is the membrane potential, ˆt is the time where the neuron fired its last spike , I(t) is the external current, The kernel η includes the form of the spike itself as well as the after-hyperpolarization potential (AHP), if needed. A complete definitions and descriptions for the kernels η and κ functions can be founded in [25].

3. Discussion and Conclusions The eight various mentioned neuron models were described and demonstrated. This section provides a comparison between them in terms of computational efficiency; number of floating point operations needed to accomplish one mille second (ms) of model simulation, number of variables and the overall complexity as shown in Table 1. In addition, the eight neuron models are compared in terms of their biological plausibility; the neuronal properties or behaviors that can be exhibited by each model as indicated in Table 2. Integrate-and-Fire model is the lowest model in consumption of computational power; which it could be used in simple simulation that accuracy is not an important manner. While Hodgkin-Huxely model exhibit all neural behaviors which could be used in applications where every single detail is needed, but this model require very huge computational power. Izhikevich model exhibit most of neural behaviors and does not require huge computational power, which it is the best model that could be used in any simulation or implementation of spiking neural networks, e.g. hippocampus simulation, classification, or solving engineering problems. Table 1. Spiking Neuron Models comparisons in term of: number of floating-point operations, variables and complexity. Number of floatingNumber of Model Complexity point operations Variables Integrate-and-Fire

5

1

Very Low

Izhikevich

13

2

Very Low

Spike- Response Model

50

1

Low

BVP

72

2

Low

Hindmarsh–Rose

120

3

Medium

Wilson

180

2

Medium

Morris–Lecar

600

3

High

Hodgkin-Huxely

1200

1

Very High

19


Table 2. Spiking Neuron Models comparison in terms of the neuronal behavior. Adapted from [13] with modification.* Neuronal Behavior Hindmarsh– Morris– I&F Izhikevich BVP Wilson HH Rose Lecar Biophysically Meaningful Tonic Spiking Phasic Spiking Tonic Bursting Phasic Bursting Mixed Mode Spike Frequency Adaptation Class One Excitable Class Two Excitable Spike Latency Sub-threshold Oscillations Resonator Integrator Rebound Spike Rebound Burst Threshold Variability Bi-stability Accommodation Inhabitation-induced Spiking Inhabitation-induced Bursting Chaos

×

×

×

×

×

√

√

√ × × ×

√ √ √ √

√ √ ×

√ √ √

√ √ √

√ √ ×

√ √ √

-

√

×

-

-

-

×

×

-

×

√

×

√

√

×

√

√ × ×

√ √ √

√ × √

√ √ √

√ √ √

√ √ √

√ √ √

×

√

√

√

√

√

√

× √ × × × × ×

√ √ √ √ √ √ √

√ × √ × √ √ √

√ √ √ √ √ √ √

√ √ √ √ √

√ √ √

√

√ √ √

√ √ √ √ √ √ √

×

√

√

√

-

√

√

×

√

×

-

×

-

×

√

×

√

-

×

√

-

-

* √ : The spiking neuron model can exhibit the property. × : The spiking neuron model cannot exhibit the property. - : No available information if the spiking neuron model can exhibit the property or not.

4. Acknowledgment This research is supported by UNIVERSITI SAINS MALAYSIA and has been funded by the Research University Cluster (RUC) grant titled by “Reconstruction of the Neural Microcircuitry or Reward-Controlled Learning in the Rat Hippocampus” (1001/PSKBP/8630022).

5. References [1] Jilles Vreeken, “Spiking neural networks, an introduction”. Technical Report UU-CS-2003-008 Institute for Information and Computing Sciences, Utrecht University. pp.1-5, 2002. [2] Wolfgang Maass, “Networks of Spiking Neurons: The Third Generation of Neural Network Models”. Neural Networks, vol. 10, no. 9, pp. 1659-1671, 1997. [3] Samanwoy Ghosh-Dastidar, Hojjat Adeli, “Third Generation Neural Networks: Spiking Neural Networks”, Advances in Computational Intell., AISC 61, Springer-Verlag Berlin Heidelberg, pp. 167–178, 2009. [4] Samanwoy Ghosh-Dastidar, Hojjat Adeli, “SPIKING NEURAL NETWORKS”, International Journal of Neural Systems, vol. 19, no. 4, pp. 295–308, 2009. [5] Miss.Chinki Chandhok, Mrs.Soni Chaturvedi, “Adaptation of Spiking Neural Networks for Image Clustering”, International Journal of Video & Image Processing and Network Security IJVIPNSIJENS, vol.12, no. 03, pp.8-13, 2012. [6] Sander M. Bohte, Joost N. Kok, “Applications of Spiking Neural Networks”, Journal of Information Processing Letters, vol. 95, no. 6, pp. 519-520, 2005.

20


[7] Filip Ponulak, Andrzej Kasiński, “Introduction to spiking neural networks: Information processing, learning and applications”, Acta Neurobiologiae Experimentalis, vol.71, no. 4, pp. 409– 433, 2011. [8] Ammar Belatreche, Liam P. Maguire, Martin McGinnity, “Advances in design and application of spiking neural networks”. Soft Computing, vol. 11, no.3, pp. 239–248. 2007. [9] Jayram Moorkanikara Nageswaran, Nikil Dutt, Jeffrey L. Krichmar, Alex Nicolau, Alexander V. Veidenbaum “A configurable simulation environment for the efficient simulation of large-scale spiking neural networks on graphics processors”, Journal of Neural Networks, vol.22, no.5-6, pp.791-800, 2009 [10] Zhao Wang-Xiong; Qiao Qing-Li; Wang Dan, “Hippocampus function modeling with neural network”, In Proceeding(s) of IEEE International Symposium on IT in Medicine & Education. China, pp. 1128 – 1131, 2009. [11] Zhenguo Xiao, Xin Tian, “Neuronal Ensemble Coding of Spike Trains in the Hippocampus CA3 via Small-world Network”, journal of computers, vol. 5, no. 3, pp. 448 – 455, 2010. [12] Eugene M. Izhikevich, “Simple model of spiking neurons”, IEEE Trans. Neural Networks, vol. 14, pp. 1569–1572. 2003. [13] Eugene M. Izhikevich, “Which Model to Use for Cortical Spiking Neurons?,” IEEE Trans. Neural Networks, vol. 15, pp.1063–1070. 2004. [14] David Sterratt, Bruce Graham, Andrew Gillies, “Principles of Computational Modelling in Neuroscience”, Cambridge University Press, 2011. [15] Richard FitzHugh, “Impulses and physiological states in theoretical models of nerve membrane”, Biophysical Journal. vol. 1, pp. 445–466, 1961. [16] Hugh R. Wilson, “Simplified Dynamics of Human and Mammalian Neocortical Neurons”, Journal of theor. Biol.,vol. 200, no. 4, pp. 375–388. 1999. [17] CATHERINE MORRIS, HAROLD LECAR, “VOLTAGE OSCILLATIONS IN THE BARNACLE GIANT MUSCLE FIBER”, BIOPHYS. J., vol. 35, pp. 196–213, 1981. [18] Paolo Checco, Marco Righero, Mario Biey, Ljupco Kocarev, “Synchronization in Networks of Hindmarsh–Rose Neurons”, IEEE transactions on circuits and systems—ii: express briefs, vol. 55, no. 12. pp. 1274- 1278, 2008. [19] Gregory D. Smith, Charles L. Cox, S. Murray Sherman, John Rinzel, “Fourier analysis of sinusoidally driven thalamocortical relay neurons and a minimal integrate-and-fire-or-burst model”, J. Neurophysiology, vol. 83, pp. 588–610, 2000. [20] Eugene M. Izhikevich, “Resonate-and-fire neurons”, Neural Network, vol. 14, pp. 883–894, 2001. [21] P. E. LATHAM, B. J. RICHMOND, P. G. NELSON, S. NIRENBERG, “Intrinsic dynamics in neuronal networks. I. Theory,” J. Neurophysiol., vol. 83, pp. 808–827, 2000. [22] Jianfeng Feng, “Is the integrate-and-fire model good enough? – a review”, Neural Networks vol. 14, no. 6-7, pp. 955-975, 2001. [23] Murilo Saraiva de Queiroz, Roberto Coelho de Berrêdo, Antônio de Pádua Braga, “Reinforcement learning of a simple control task using the spike response model”, Neurocomputing, vol. 70, no. 13, pp. 14-20, 2006. [24] Renaud Jolivet,Timothy J. Lewis, Wulfram Gerstner, “The spike response model: A framework to predict neuronal spike trains”, In Processing Artificial Neural Networks and Neural Information — ICANN/ICONIP Lecture Notes in Computer Science, vol. 2714, pp. 846-853, 2003. [25] Gerstner and Kistler, “Spiking Neuron Models. Single Neurons, Populations, Plasticity”, Cambridge University Press, 2002. [26] Eugene M. Izhikevich, “Dynamical Systems in Neuroscience The Geometry of Excitability and Bursting”. The MIT Press, London, 2005. [27] Marco Storace, Daniele Linaro, Enno de Lange, “The Hindmarsh–Rose neuron model: Bifurcation analysis and piecewise-linear approximations”, CHAOS, vol. 18, no. 3, 2008. [28] Xue-Feng Jiang “An Effective Decision Support System based on Neural Networks and Its Application”, International Journal of Advancements in Computing Technology(IJACT), vol. 5, No. 1, pp.37-45, 2013. [29] Xianmin Wei “Efficient Robot Path Planning Based on Simulated Annealing Artificial Neural Networks”, International Journal of Advancements in Computing Technology (IJACT) vol. 5, No. 4, pp.590-597, 2013.

21