ASIC Oriented Comparative Analysis Of Biologically

Inspired Neuron Models

Ahmed J. Abd El-Maksouda, Youssef O. Elmasrya, Khaled N. Salamac, Hassan Mostafaa,b a Electronics and Communications Engineering Department, Cairo University, Giza 12613, Egypt

bNanotechnology and Nanoelectronics Program, Zewail City for Science and Technology c King Abdullah University of Science and Technology (KAUST), Thuwal, Saudi Arabia

Email: khaled.salama@kaust.edu.sa and hmostafa@uwaterloo.ca

Abstract—This paper introduces the hardware and the ASIC

implementations of the four most popular biologically inspired

neuron models. The models are quartic, Izhikevich, Hindmarsh

Rose and Fitzhugh-Nagumo. Moreover, some approximate

computing techniques are applied on these models to reduce the

area and power consumption. In addition, ASIC implementations

of these models and their approximate versions are carried out.

Also, spiking behavior error between these models and the

Hodgkin Huxley model, the reference accurate model, is

presented. Finally, a fair comparative analysis is discussed to help

the Spiking Neural Networks designers to select the best neuron

model hardware implementation from the power, area and

accuracy perspectives.

Keywords— Neuromorphic computing, spiking neural networks,

biologically inspired models

I. INTRODUCTION

In the last two decades, Neuromorphic computing has become a challenging field and initiates a wide range of useful applications and solutions for many problems. The main potential for opening this field is to create new architectures that can learn and adapt. These architectures are designed to accelerate the running of machine learning and deep learning algorithms.

Neuromorphic computing is employed in Neuroscience to use it in medical treatment and diseases discovery. In addition, von Neumann architectures reached its peak [1]. Although they have a great computational power, they have some issues such as von Neumann bottleneck and inability to run machine learning algorithms efficiently. This leads to thinking about new architectures and innovating new powerful algorithms to replace von Neumann architecture.

As nature has always been the main source of inspiration for humans; the name and working methodology of Neuromorphic computing are inspired from the human brain. Human brain have billions of neurons and massive computational power with low power dissipation and operating frequency. A lot of research and development in brain inspired computing have been conducted recently.

One of the main research topics is neuron modeling. There are lots of models for the neurons that are built[2] to mimic the behavior of the brain neuron cell [3], and these models vary in their complexity and accuracy. One category of these neuron models is the Spiking neural networks (SNN) that have become a subject for a variety of research due to its extensive use in cognitive computing[4] and artificial intelligence. SNN models are categorized according to their complexity and biological inspiration [5] such as the biologically inspired models that are

less complex than the biologically plausible models which are taken as the reference neuron model. However, biologically inspired models are trying to model the function of the living cells without emulating physical activity of the biological systems, and these models are the focus of this paper.

Biologically inspired models are relatively complex to be

implemented in hardware and need relatively large area and

power consumption. Approximate computing [6][7] is applied

to minimize the hardware, power consumption and area needed

to implement these models. Several techniques of approximate

computing are developed to simplify either the mathematical

(models and operations) or to simplify the hardware

implementation such as the approximate multipliers [8].

These faulty multipliers produce approximate outputs which

have an impact on the final result accuracy according to the

number of reduced bits. Another technique of approximate

multipliers in [9] is to use shifting instead of multiplication or

division.

The idea behind adopting approximations in neural

networks is that they are basically used to mimic the behavior

of the human brain which produces errors depending on the

training of the network and the dataset used to apply any

specific deep learning algorithm. Accordingly, approximate

neuromorphic computing is the best candidate to trade-off the

power/area and accuracy.

The SNN have been implemented in many research works

in the literature. The objective of this work is to perform the

hardware implementations of the most popular biologically

inspired neuron models [10][11]. In addition, this work presents

some approximate computing techniques such as CORDIC

based multiplication [12][13] and piecewise linear

approximation of quadratic model [14][15].

In this paper, hardware implementations are carried out for

the most popular biologically inspired neuron models. As SNN

neurons is relatively complex, approximate computing is

applied on these models to reduce area and power consumption.

Also, full width at half maximum (FWHM) error is determined

with respect to the Hodgkin Huxley reference model to show

the power-accuracy trade-off of these different neurons models.

In addition, ASIC implementations are performed on these

models to determine accurate area and power consumption.

Moreover, a comparative analysis is presented to help the SNN

designers to select the suitable neural model implementation

considering power, area and accuracy.

978-1-5386-7392-8/18/$31.00 ©2018 IEEE 504

The paper is organized as follows Section II discusses the

hardware implementations of the four most popular biologically

inspired neuron models. Section III discusses the hardware

Implementations of the approximate neuron models. In Section

IV, ASIC implementations of the models after applying the

approximate computing techniques are presented. In Section V,

a comparative analysis among these models is conducted. This

work is concluded in Section VI.

II. HARDWARE IMPLEMENTATIONS

In this section, the definitions of the most popular biologically inspired models are presented. These models are Quartic, Izhikevich, Hindmarsh-Rose and Fitzhugh-Nagumo. These models have different characteristics in terms of mathematical complexity and accuracy. In the hardware implementation, the number of registers is equal to the number of model's differential equations to save its state. In addition, combinational blocks are added to calculate each state variable.

1) Quartic model

The behavior of the Quartic model [16] is governed by two

coupled differential equations and a reset condition. The model

controlling variables are the synaptic input current I and

parameters a and b. The differential equations controlling the

model’s dynamics are as follows:

{�̇� = 𝑣4 + 2𝑎𝑣 − 𝑤 + 𝐼 �̇� = 𝑎(𝑏𝑣 − 𝑤)

(1)

The spikes are emitted when the membrane voltage cross a

constant threshold, α, defined by the model. When the

membrane potential exceeds α, the membrane voltage resets to

a reference value 𝑣𝑟 and the variable w is updated.

If 𝑣(𝑡−) > 𝛼 then {𝑣(𝑡) = 𝑣𝑟

𝑤(𝑡) = 𝑤(𝑡−) + 𝑑 (2)

Also, there are 𝑣𝑟 and d which are the parameters controlling the reset condition. The spikes produced by the Quartic model are shown in Fig. 1. The simulated output is for the Tonic Spiking and Tonic Bursting.

2) Izhikevich model

For the Izhikevich model [17], it is a reduced version of the

accurate biologically plausible model Hodgkin-Huxley. It

consists of two-dimensional system of ordinary differential

equations:

{�̇� = 0.04𝑣2 + 5𝑣 + 140 − 𝑢 + 𝐼 �̇� = 𝑎(𝑏𝑣 − 𝑢) (3)

(a)

(b)

Fig. 1. a) quartic tonic spikng mode b) quartic bursting mode

(a)

(b)

Fig. 2. a) Izhikevich tonic spikng mode b) Izhikevich bursting mode

With spiking condition:

If 𝑣(𝑡−) ≥ 30 𝑚𝑉 , then {𝑣(𝑡) = 𝑐

𝑢(𝑡) = 𝑢(𝑡−) + 𝑑 (4)

The rest condition is invoked when the membrane potential

exceeds a constant threshold. In (3), v is the membrane potential

and u is the membrane recovery variable. v and u are

dimensionless variables and a, b, c, d are dimensionless

parameters. The spikes produced by the Izhikevich model are

shown in Fig.2. The simulated output is for the Tonic Spiking

and Tonic Bursting modes as shown in Fig.2 (a) and (b).

3) Hindmarsh-Rose model

The Hindmarsh-Rose model [18] is characterized by three

coupled differential equations with variables of order two and

three and the equations are as follows:

{

𝑥 ̇ = 𝑦 − 𝑎𝑥3 + 𝑏𝑥2 − 𝑧 + 𝐼

�̇� = 𝑐 − 𝑑𝑥2 − 𝑦

�̇� = 𝑟(𝑠(𝑥 − 𝑥1) − 𝑧)

(5)

where x is the membrane potential, y is the recovery current, z

is the adaptation current, and I is the input of the neuron with

which the model can go from no spiking to infinite spiking.

Finally, 𝑥1 is the first equilibrium point for the model with no

adaptation. The spikes produced by the Hindmarsh-Rose model

are shown in Fig.3 (a) and (b). The simulated output is for the

Tonic Spiking and Tonic Bursting modes.

4) Fitzhugh-Nagumo model

The Fitzhugh-Nagumo model [19] has a two coupled

differential equations. It has only the regular spiking mode and

the equations that control its dynamics are as follows:

{

�̇� = 𝑓(𝑣 + 𝑣𝑒𝑞) − 𝑓(𝑣𝑒𝑞) − 𝑤

�̇� = Ɛ(𝑣 − ɣ𝑤)

𝑓(𝑣) = 𝑣(𝑣 − 𝛼) (1 − 𝑣)

(6)

Where v represents the potential, w represents the sodium

gating. α, γ and ε are variables controlling the dynamics of the

model. The regular spiking mode output of the Fitzhugh-

Nagumo model is presented in Fig.4 for regular spiking mode.

It is based on 32-bit signed number (i.e., 24-bit fraction part and

8-bit integer part).

(a)

(b)

Fig. 3. a) Hindmarsh-Rose tonic spikng mode b) Hindmarsh-Rose bursting

mode

Fig. 4. Fitzhugh-Nagumo regular spiking mode

505

III. APPROXIMATE NEURON MODELS

Approximate computing is used to reduce the power consumption and area of neuron models. The approximate computing techniques used in this work are reducing the word length to reduce the hardware area and replacing multiplication and division by 2's by shifting left or right.

For the Quartic model, the number of bits is reduced to 24-

bit (i.e., 12-bit fraction part and 12-bit integer part) to optimize

the power. The output is shown in Fig.5 (a) and (b). For the

Izhikevich model, the number of bits is reduced to 28-bit (i.e.,

14-bit fraction part and 14-bit integer part). The output is shown

in Fig.5 (c) and (d). For Hindmarsh-Rose model, the number of

bits is reduced to 24-bit (i.e., 16-bit fraction part and 8-bit

integer part). The output is shown in Fig.5 (e) and (f). For the

Fitzhugh-Nagumo model, the number of bits is reduced to 12-

bit (i.e., 8-bit fraction part and 4-bit integer part). The output is

shown in Fig.5 (g).

IV. ASIC IMPLEMENTATION

ASIC implementations of the proposed neuron models have

been carried out to compare the models in terms of area, power

consumption and maximum operating frequency. These layouts

is made by using industrial hardware-calibrated TSMC 130nm

CMOS technology. All models are simulated under frequency

of 20 MHz. The layouts of the models are shown in Table II to

scale.

(a)

(b)

(c)

(d)

(e)

(f)

(G)

Fig. 5. a) quartic tonic spikng mode b) quartic bursting mode c) Izhikevich tonic spikng mode d) Izhikevich bursting mode e) Hindmarsh-Rose tonic

spikng mode f) Hindmarsh-Rose bursting mode g) Fitzhugh-Nagumo regular

spikng mode

V. COMPARATIVE ANALYSIS AND DESIGN RECOMMENDATIONS

In Table II, a comparison among the presented models and

their approximated ones is presented in terms of number of

differential equations (representing the complexity), number of

modes, power consumption, maximum frequency and layout

area. It is clear that the quartic model has the largest area and

power consumption, however, it is the most accurate model

with respect to Hodgkin Huxley plausible reference model.

After applying some approximate computing techniques, the

area and power consumption is cut down by 25% for quartic

model.

Izhikevich model is the best model in terms of number of

modes as it has 16 modes. Also, with respect to the number of

the differential equations, quartic model has the highest order

and this increases the area and power consumption. On the other

hand, Fitzhugh-Nagumo is the best choice in terms of lowest

power consumption, area and simple implementation, at the

expense of lower accuracy.

Another comparative perspective for these neuron models is

full width at half maximum error (FWHM). This error defines

the accuracy of the proposed models with respect to the

plausible model as a reference. In this error definition, the width

of the spikes in different models at the half way to the maximum

is measured to compare it to that of the Hodgkin Huxley model.

This error can be formulated as follows:

𝐹𝑊𝐻𝑀 = |𝐹𝑊𝐻𝑀ℎ𝑢𝑥𝑙𝑒𝑦−𝐹𝑊𝐻𝑀𝑚𝑜𝑑𝑒𝑙

𝐹𝑊𝐻𝑀ℎ𝑢𝑥𝑙𝑒𝑦| × 100 (7)

Table I shows the error between the four most popular models

and Hodgkin Huxley model as a reference.

TABLE I. FWHM ERROR BETWEEN PRESENTED FOUR NEURON MODELS AND

HODGKIN HUXLEY MODEL

Qaurtic Izhikevich Hindmarsh-Rose Fitzhugh-

Nagumo

FWHM (%) 1.41% 71.42% 3.51% 321.23%

From Table I and Table II, several design recommendations are

extracted and listed as follows to help the SNN designers to

select the most suitable neuron model considering power/area

and accuracy:

• The most accurate model is quartic model compared

with reference model, however it has the largest area and

consumption. Its power consumption is 2.1X larger than

the Fitzhugh-Nagumo neuron model power

consumption.

• Fitzhugh-Nagumo is the simplest model in terms of

hardware specifications, however it has the least

accuracy.

• Izhikevich has the largest number of modes to model the

brain behavior, however it has large FWHM error

compared to the quartic model.

• Hindmarsh-Rose has high accuracy compared to its

hardware specifications, however it has only two modes.

506

VI. CONCLUSION

In this paper, ASIC oriented comparative analysis of

biologically inspired neuron models is carried out. The

hardware implementations of all these models are presented. In

addition, approximate computing is applied to the models which

resulted in reducing the area by range from17% to 84% and

power consumption with range from 27% to 87% for four

different models. Moreover, the results of the ASIC

implementations are presented by comparing these models in

terms of their hardware specifications. Finally, FWHM error is

adopted to quantify the accuracy of these models compared to

the Huxley model.

ACKNOWLEDGMENT

This research was partially funded by ONE Lab at Cairo

University, Zewail City of Science and Technology, and

KAUST.

REFERENCES

[1] S. B. Furber, “Brain-inspired computing,” IET Comput. Digit. Tech., vol. 10, no. 6, pp. 299–305, 2016.

[2] R. Naous, M. Al-Shedivat, E. Neftci, G. Cauwenberghs and K. N. Salama, "Stochastic synaptic plasticity with memristor crossbar arrays," IEEE International Symposium on Circuits and Systems (ISCAS), pp. 2078-2081, 2016.

[3] A. Schmid, “Neuromorphic microelectronics from devices to hardware systems and applications,” Nonlinear Theory Its Appl. IEICE, vol. 7, no. 4, pp. 468–498, 2016.

[4] O. Krestinskaya, K. N. Salama, and A. P. James, "Analog Backpropagation Learning Circuits for Memristive Crossbar Neural Networks," IEEE International Symposium on Circuits and Systems (ISCAS), pp. 1-5, 2018.

[5] C. D. Schuman et al., “A Survey of Neuromorphic Computing and Neural Networks in Hardware,” arXiv:1705.06963, pp. 1–88, 2017.

[6] S. Mittal, “A Survey of Techniques for Approximate Computing,” ACM Comput. Surv., vol. 48, no. 4, pp. 1–33, 2016.

[7] R. Naous and K. N. Salama, “Approximate Computing with Stochastic

Memristors,” 15th International Workshop on Cellular Nanoscale Networks and their Applications (CNNA), pp. 1-2, 2016.

[8] V. Mrazek, S. S. Sarwar, L. Sekanina, Z. Vasicek, and K. Roy, “Design of power-efficient approximate multipliers for approximate artificial neural networks,” Proc. 35th Int. Conf. Comput. Des. (ICCAD), pp. 1–7, 2016.

[9] P. Panda et al., “Cross-layer approximations for neuromorphic computing: from devices to circuits and systems,” Proc. 53rd Annu. Des. Autom. Conf. (DAC), pp. 1–6, 2016.

[10] V. Rangan, A. Ghosh, V. Aparin, and G. Cauwenberghs, “A subthreshold a VLSI implementation of the Izhikevich simple neuron model,” IEEE Eng. Med. Biol. Soc. (EMBC), pp. 4164–4167, 2010.

[11] A. van Schaik, C. Jin, A. McEwan and T. J. Hamilton, "A log-domain implementation of the Izhikevich neuron model," IEEE International Symposium on Circuits and Systems (ISCAS), pp. 4253-4256, 2010.

[12] M. Heidarpour, A. Ahmadi, and R. Rashidzadeh, “A CORDIC Based Digital Hardware for Adaptive Exponential Integrate and Fire Neuron,” IEEE Trans. Circuits Syst. I Regul. Pap., vol. 63, no. 11, pp. 1986–1996, 2016.

[13] A. Elnabawy, H. Abdelmohsen, M. Moustafa, M. Elbediwy, A. Helmy, and H. Mostafa, “A Low Power CORDIC-Based Hardware Implementation of Izhikevich Neuron Model,” IEEE International NEW Circuits and Systems Conference (NEWCAS), pp. 1–4, 2018.

[14] H. Soleimani, a. Ahmadi, and M. Bavandpour, “Biologically Inspired Spiking Neurons: Piecewise Linear Models and Digital Implementation,” IEEE Trans. Circuits Syst. I Regul. Pap., vol. 59, no. 12, pp. 2991–3004, 2012.

[15] E. M. Izhikevich, "Which model to use for cortical spiking neurons?," IEEE Transactions on Neural Networks, vol. 15, no. 5, pp. 1063-1070, Sept. 2004.

[16] F. Grassia, T. Levi, T. Kohno, and S. Saïghi, “Silicon neuron: digital hardware implementation of the quartic model,” Artif. Life Robot., vol. 19, no. 3, pp. 215–219, 2014.

[17] E. M. Izhikevich, "Simple model of spiking neurons," IEEE Transactions on Neural Networks, vol. 14, no. 6, pp. 1569-1572, Nov. 2003.

[18] J. Lu, J. Yang, Y. Bin Kim, J. Ayers, and K. K. Kim, “Implementation of excitatory CMOS neuron oscillator for robot motion control unit,” J. Semicond. Technol. Sci., vol. 14, no. 4, pp. 383–390, 2014.

[19] B. Linares-Barranco, E. Sanchez-Sinencio, A. Rodriguez-Vazquez and J. L. Huertas, "A CMOS implementation of FitzHugh-Nagumo neuron model," IEEE Journal of Solid-State Circuits, vol. 26, no. 7, pp. 956-965, Jul 1991.

TABLE II. COMPARISON BETWEEN FOUR BRAIN INSPIRED NEURON MODELS AND THEIR APPROX. VERSIONS

QUARTIC IZHIKEVICH HINDMARSH-

ROSE

FITZHUGH-

NAGUMO

QUARTIC

Approx.

IZHIKEVICH

Approx.

HINDMARSH-

ROSE

Approx.

FITZHUGH-

NAGUMO

Approx.

Num. of ODE'S 2 2 3 2 2 2 3 2

order 4 2 3 3 4 2 3 3

Num. of modes 4 16 2 1 4 16 2 1

Max Frequency 41.66 Mhz 50 Mhz 45.45 Mhz 45.45 Mhz 45.45 Mhz 50 Mhz 50 Mhz 100 Mhz

Area µ𝑚2 422500 200000 122500 160000 302500 160000 90000 25600

Power

consumption

(mW)

0.216175

0.114245

0.0701951

0.102609

0.154097

0.088202

0.0462406

0.0140529

Dimensions µm 600*600 400*500 350*350 400*400 550*550 400*400 300*300 160*160

Layout

507