January 10, 2007 21:0 00084

International Journal of Neural Systems, Vol. 16, No. 6 (2006) 445–456c© World Scientific Publishing Company

ANALOG VLSI IMPLEMENTATION OFRESONATE-AND-FIRE NEURON

KAZUKI NAKADAGraduate School of Life Science and Systems Engineering,

Kyushu Institute of Technology,2-4 Hibikino, Kitakyushu, Fukuoka 808-0196, Japan

nakada@brain.kyutech.ac.jp

TETSUYA ASAIGraduate School of Information Science and Technology,

Hokkaido University Kita 14, Nishi 9,Kita-ku, Sapporo, Hokkaido 060-0814, Japan

HATSUO HAYASHIGraduate School of Life Science and Systems Engineering,

Kyushu Institute of Technology,2-4 Hibikino, Kitakyushu, Fukuoka 808-0196, Japan

Received 20 April 2006Revised 24 July 2006

Accepted 15 August 2006

We propose an analog integrated circuit that implements a resonate-and-fire neuron (RFN) model basedon the Lotka-Volterra (LV) system. The RFN model is a spiking neuron model that has second-ordermembrane dynamics, and thus exhibits fast damped subthreshold oscillation, resulting in the coincidencedetection, frequency preference, and post-inhibitory rebound. The RFN circuit has been derived fromthe LV system to mimic such dynamical behavior of the RFN model. Through circuit simulations, wedemonstrate that the RFN circuit can act as a coincidence detector and a band-pass filter at circuit leveleven in the presence of additive white noise and background random activity. These results show thatour circuit is expected to be useful for very large-scale integration (VLSI) implementation of functionalspiking neural networks.

Keywords: Analog integrated circuit; very large-scale integration (VLSI); spiking neural networks;resonate-and-fire Neuron (RFN) model; coincidence detection; frequency preference.

1. Introduction

Dynamical properties of spiking neural networks andtheir functional roles in information processing havebeen considered to be highly attractive from a pointof view of hardware implementation of neuromorphicsystems.1 In fact, findings in neuroscience3–14 haveprovided valuable insights into implementation offunctional networks of silicon spiking neurons. Basedon these findings, such networks of silicon spikingneurons can increase noise robustness and tolerance

in sensory coding29,32,33 and reduce the influence ofdevice deviation on their operation29,31–34,40,41 inaddition to the abilities of information processingincluding image processing,18–23,28 auditory featureextraction,24,25 onset detection,29 and learning andmemory.30,31, and of sensori-motor integration.16

Recent research efforts for functional networksof silicon spiking neurons concentrate on real-timeevent-based computation,25–30,46–48 whereas majorprevious works have been related with rate-based

445

January 10, 2007 21:0 00084

446 K. Nakada, T. Asai & H. Hayashi

computation.17–24 Furthermore, this trend would beaccelerated toward multi-chip system integration byrapid progress in the address-event representation(AER) protocol,15 which is an asynchronous inter-chip communication protocol for reducing massivelyconnections in very large-scale integration (VLSI)systems.

For designing of functional networks of siliconspiking neurons, a major requirement is the abilityto extract meaningful information embedded in spikesequences. Temporal filtering properties are neededto extract the frequency of spike sequences, whichcarry information in the event-based computation aswell as the rate-based computation. Coincidence orsynchrony detection are considered to play essentialroles in information processing in the event-basedcomputation since fine temporal structure of spikesequences may be another carrier of information.

Such functions of the information selectivity insilicon spiking neural networks are limited by theperformance of components of the networks: neuronand synaptic circuits. For instance, the Axon-Hillockcircuit,1 widely known as an electronic analogue ofthe integrate-and-fire neuron (IFN) model, can onlyact as a low-pass filter for a sequence of pulses. Forincreasing the information selectivity, spiking neu-ron circuits,31,42–49 such as a low-power IFN cir-cuit with frequency adaptation31 and asynchronouschaotic spiking neuron circuits,47,48 have been alter-natively developed. These neuron circuits providefunctions of synchrony detection and temporal filter-ing to the network circuits. Synaptic circuits can alsoincrease the information selectivity of silicon spik-ing neural networks.29,31,32,35–41 These synaptic cir-cuits act as a nonlinear temporal filter by themselves.However, most of these circuits can work more effec-tively at network level. For instance, it is required tocombine multiple synaptic circuits for constructing aspatio-temporal filter or a band-pass filter.

In this paper, we propose an analog integratedcircuit that implements a resonate-and-fire neuron(RFN) model proposed by Izhikevich.50 The RFNmodel is a spiking neuron model that has second-order membrane dynamics, and thus exhibits fastsubthreshold oscillation, resulting in the coincidencedetection, frequency preference, and post-inhibitoryrebound. Such dynamic behavior was implementedas an analog integrated circuit using complementarymetal-oxide-semiconductor (CMOS) technology. We

will demonstrate that the RFN circuit can act as acoincidence detector and a band-pass filter at circuitlevel through circuit simulations using the simulationprogram with integrated circuit emphasis (SPICE).We will further investigate (i) the effects of whitenoise and (ii) background random activity on theperformance of the signal detection; the coincidencedetection and the frequency preference. Finally, wewill consider (iii) the influence of device deviation onthe circuit operation.

This paper is organized as follows. In Sec. 2, thedynamical behavior and the related functions of theRFN model are described. In Sec. 3, analog inte-grated circuit implementation of the RFN model isillustrated in detail. In Sec. 4, we will show the sim-ulation results for the signal detection of the RFNcircuit. The summary of the present research is givenin Sec. 5.

2. Resonate-and-Fire Neuron (RFN)Model

We here describe a resonate-and-fire neuron (RFN)model proposed by Izhikevich.50 The RFN model isa spiking neuron model with second-order membranedynamics and a firing threshold. The dynamics of theRFN model are as follows:

x = bx − wy + I (1)

y = wx + by (2)

or given by an equivalent complex form:

z = (b + iw)z + I (3)

where z = x + iy is a complex state variable. Thereal and imaginary parts, x and y, are the current-and voltage-like state variables, respectively. Theconstants, b and w are parameters, and I is an exter-nal input. If the variable y exceeds a certain thresh-old ath, the variable z is reset to an arbitrary value zo,which describes activity-dependent after-spike reset.This model exhibits a damped subthreshold oscilla-tion of membrane potential due to the second-ordermembrane dynamics. As a result, the RFN model issensitive to the timing of stimuli.50

Figure 1 shows typical behavior of the RFNmodel in response to input pulses. In this case, weused the following parameters: b = −0.1, w = 1,ath = i, zo = −0.5 + i, and the input pulses weregiven by: I = imax · (t/τ)exp(1− t/τ), where thetime constant τ = 50ms, the maximal amplitude

January 10, 2007 21:0 00084

Analog VLSI Implementation of Resonate-and-Fire Neuron 447

A B C D E

zozo athathathathath

input pulses

Fig. 1. Behavior of the resonate-and-fire neuron model in response to (A) a single pulse, (B) a coincident doublet, (C) anon-resonant doublet, (D) a resonant doublet, and (E) a non-resonant doublet. The output pulses are manually writtento clarify the firing time in (B) and (D). Phase plane portraits show the sensitivity to the timing of the inputs, where thearrows represent the direction of the corresponding inputs.

imax = 1.2, and t was an elapsed time since a pulsewas given. When a weak pulse that cannot evokean action potential alone arrives at an RFN model,a damped subthreshold oscillation occurs, as shownin Fig. 1A. When a pair of pulses arrives at theRFN model at a short interval (a coincident doublet,e.g. 2.5ms, Fig. 1B), or at an interval nearly equalto the intrinsic period of the subthreshold oscilla-tion (a resonant doublet, 12.5ms, Fig. 1D), the RFNmodel fires a spike. However, the RFN model doesnot fire a spike in response to a doublet with an inter-val in other ranges, e.g. 7.5ms (Fig. 1C) or 15.0ms(Fig. 1E).

These results indicate that the RFN model candetect coincidence pulses (coincidence detection),and resonate with a sequence of pulses at a resonancefrequency (frequency preference). In addition it hasbeen reported that the RFN model can fire a spikein response to an inhibitory pulse (post-inhibitoryrebound).50

3. Circuit Implementation

We here propose an analog integrated circuit thatimplements the RFN model.49 The circuit consistsof a membrane circuit, a threshold-and-fire circuit,and excitatory and inhibitory synaptic circuits,2,43

as shown in Fig. 2.

3.1. Membrane circuit

The membrane circuit was designed based on theLotka-Volterra (LV) system having periodic solutionsfor mimicking the subthreshold membrane dynamicsof the RFN model.49

3.1.1. Lotka-Volterra (LV) system

The LV system53 is modeled after the interactionsbetween multiple species in an ecological system,which is represented by the following equations:

zi = zi

(ri +

N∑j

aijzj

), (i, j = 1, 2, . . . , N) (4)

where N denotes the number of kinds of species Czi

and ri the population and the linear growth rate ofthe i-th species, respectively, and aij the interactionbetween the species. By introducing the nonlineartransformation:

zi = bi expxi, bi = const. (5)

into the LV system, the following equation:

xi = ri +N∑j

aijbj exp xj , (6)

are obtained. The resulting equations are suitable foranalog integrated circuit implementation because of

January 10, 2007 21:0 00084

448 K. Nakada, T. Asai & H. Hayashi

Fig. 2. Schematic of the resonate-and-fire neuron circuit. The circuit consists of a membrane circuit, a threshold-and-firecircuit, and excitatory and inhibitory synaptic circuits. The membrane circuit has second-order dynamics containing twostate variables Ui (a current-like variable) and Vi (a voltage-like variable), and shows oscillatory behavior mediated andmodulated by a summation of synaptic inputs Iin =

Pj IEPSCi,j

− Pj IIPSCi,j

.

(i) reducing the multiplication terms of the systemvariables of the original system, and (ii) exponentialnonlinearity, which is an essential characteristic ofsemiconductor devices.46

3.1.2. Subthreshold current of MOS FETs

The current-voltage relationships of metal-oxide-semiconductor field-effect-transistors (MOS FETs)operating in the subthreshold region are describedas follows:1

Ids,n = SIno exp(

κnVg − Vs

VT

)

×(

1 − exp(−Vd − Vs

VT

)+

Vd − Vs

VE,n

)(7)

Ids,p = SIpo exp(−κpVg + Vs

VT

)

×(

1 − exp(

Vd − Vs

VT

)+

Vs − Vd

VE,p

)(8)

where Ids,n and Ids,p represent the currents of annMOS and a pMOS FET, respectively, Vg, Vd andVs the gate, drain, and source voltage, κn and κp thecapacitive coupling ratio from the gate to channel,VT ≡ kT/q the thermal voltage (k: Boltzmann’s con-stant, T : temperature, and q: electron charge), VE,n

and VE,p the Early voltages, S = W/L the aspectratio of MOS FETs (W : the channel width and L:

the channel length), and Ino and Ipo pre-exponentialcurrents.2

When subthreshold MOS FETs are saturated(Vd − Vs ≥ 4VT ≈ 100mV at room temperature)and the Early voltages VE,n and VE,p are sufficientlylarger than |Vds| = |Vd − Vs|, (7) and (8) can beapproximately rewritten as follows:

Ids,n = SIno exp(

κnVg − Vs

VT

), (9)

Ids,p = SIpo exp(−κpVg + Vs

VT

). (10)

Thus, the subthreshold currents of MOS FETs areexponential functions of input voltages.

3.1.3. Lotka-Volterra oscillator circuit

The LV system consisting of one predator and oneprey is given in the following form:

z1 = z1(r − z2), r > 0 (11)

z2 = z2(z1 − 1) (12)

where z1 represents the prey population and z2 thepredator population, respectively, and r is a positiveconstant. This system has a conserved quantity:

c = z1 + z2 − lnz1 − rlnz2 (13)

where c is a positive constant determined by initialconditions. Thus the LV system is a conservative sys-tem and has periodic solutions depending on the ini-tial conditions.

January 10, 2007 21:0 00084

Analog VLSI Implementation of Resonate-and-Fire Neuron 449

Fig. 3. Schematic of the Lotka-Volterra (LV) oscillator.

By applying the nonlinear transformation (5),(11) and (12) are transformed into:

x1 = r − b2 exp x2 (14)

x2 = b1 exp x1 − 1 (15)

where b1 and b2 represent scaling constants. Equa-tions (14) and (15) can be implemented into siliconchips using the exponential current-voltage relation-ships described by (9) and (10).

Figure 3 shows the schematic of the LV oscillatorcircuit.46 Applying Kirchhoff’s current law at eachnode with capacitance in Fig. 3, we can obtain thefollowing circuit dynamics:

C1dUi

dt= IUi − SInoexp(κnVi/VT ) (16)

C2dVi

dt= SInoexp(κnUi/VT ) − IVi (17)

where Ui and Vi represent the node voltages, C1 andC2 the capacitance, and IUi and IVi the bias cur-rent for a current mirror. When MOS FETs con-tained in the LV oscillator circuit operate undersaturation, the circuit becomes a conservative sys-tem, and thus shows oscillations depending on initialconditions.

3.1.4. Dynamics of membrane circuit

We designed the membrane circuit by introducingdissipation terms and diode-connected transistors

into the LV oscillator circuit, as shown in Fig. 2.The dynamics of the membrane circuit is describedas follows:

C1dUi

dt= −g(Ui − Vrst) + Iin + IUi

−SInoexp(

κ2n

κn + 1Vi

VT

)(18)

C2dVi

dt= SInoexp

(κ2

n

κn + 1Ui

VT

)− IVi (19)

where Ui and Vi represent the state variables, C1

and C2 the capacitance, g the conductance for thetransistor M11, Vrst the reset voltage, and Iin thesummation of the synaptic currents:

Iin =∑

j

IEPSCi,j −∑

j

IIPSCi,j (20)

where IEPSCi,jand IIPSCi,j

represent the i-th post-synaptic currents through the jth excitatory andinhibitory synaptic circuits, respectively. Currents,IUi and IVi are described as follows:

IUi = αIUi

(1 +

VDD − Ui

VE,p

)(21)

IVi = βIVi

(1 +

Vi

VE,n

)(22)

where IUi and IVi represent the bias currents for thecurrent mirror, VDD the power-supply voltage, andα and β the dimensionless constants:

α =(

1 +VDD − Vg1

VE,p

)−1

(23)

β =(

1 +Vg2

VE,n

)−1

(24)

where Vg1 and Vg2 represent the gate voltages of M7–M8 and M9-M10, which are determined by the biascurrents IUi and IVi . As a result of introducing (21)and (22), the membrane circuit becomes a dissipativesystem.

The equilibrium point of the membrane circuit,(Uo, Vo), can easily be calculated, and the stability ofthe point can be analyzed by the eigenvalues of the

January 10, 2007 21:0 00084

450 K. Nakada, T. Asai & H. Hayashi

Jacobian matrix of the membrane circuit,

J =

−αIUi

VE,p− κ2

n

κn + 1IVo

VT

κ2n

κn + 1IUo

VT−βIVi

VE,n

(25)

=

− IUi

VE,p + VDD − Vg1− κ2

n

κn + 1IVo

VT

κ2n

κn + 1IUo

VT− IVi

VE,n + Vg2

(26)

where IUo and IVo represent the equilibrium currentat the equilibrium point. It is assumed that the leakconductance g is zero. We used diode-connected tran-sistors M1 and M3 to obtain small coefficients forIUo and IVo , and short transistors that have smallEarly voltages for M7–M10 to obtain large coeffi-cients for IUi and IVi . Consequently, the equilibriumpoint becomes a focus, and the membrane circuitexhibits damped oscillation in response to an input.In this case, the circuit dynamics is qualitativelyequivalent to the membrane dynamics of the RFNmodel near the equilibrium point.

3.2. Threshold-and-fire circuit

The threshold-and-fire circuit was constructed usinga comparator circuit and an inverter. When inputvoltage Vi increases or decreases monotonously, thecircuit generates output voltage Vpulse, which can beapproximately described as follows:

Vpulse =

{VDD Vi ≥ Vth + δVth, (27)

GND otherwise. (28)

where VDD represents a power-supply voltage, andGND a ground voltage. Effective threshold voltage isdivided into a bias voltage for the comparator, Vth,and a time-dependent hysteresis voltage δVth, whichis approximately described as follows:

δVth =∫ T

0

dVi

dtdt (29)

where T is an elapsed time since the voltage Vi haspassed over the bias voltage Vth, which is describedas follows:

T =

Cinv

Ibias· (VDD − Vinv)

dVi

dt> 0, (30)

Cinv

Ibias· Vinv

dVi

dt< 0. (31)

where Cinv and Vinv represent an input capaci-tance and a reversal voltage of the inverter, respec-tively, and Ibias is a bias current for the comparator.In short, the threshold-and-fire circuit has hysteresisdue to charging or discharging the input capacitanceof the inverter.

3.3. Resonate-and-fire neuron circuit

We here describe behavior of the RFN circuit. Pulsecurrents through excitatory or inhibitory synapticcircuits change the voltage Ui instantaneously andthe trajectory of voltage oscillation of the membranecircuit is disturbed. If Vi exceeds a threshold voltageVth, the threshold-and-fire circuit generates a spike(a pulse voltage Vpulse) after a slight delay, which isdescribed by (30). Consequently, the transistor M11

turns on state, and then Ui is reset to the bias voltageVrst instantaneously. As a result of resetting Ui, Vi

decreases monotonously. Due to the time-dependenthysteresis described by (31), weak noise and jittersaround Vth may not disturb threshold detection ofthe RFN circuit. After that, Vi decreases, and Vpulse

is returned to 0. Therefore, the width of Vpulse isbrief C and the RFN circuit seems to fire a spike. Itcan be said that such behavior of the RFN circuit isqualitatively the same as that of the RFN model.

4. Results

We verified the performance of signal detection ofthe RFN circuit with SPICE simulations. We usedthe circuit simulator, T-Spice Pro, and the modelparameters of the BSIM 3v3 LEVEL 49 model forthe AMI 0.35µm CMOS process.

Through the following simulations, the supplyvoltages were set at VDD = 1.5V, Vth = 850mV,and Vrst = 750mV, the bias currents were set atIUi = IVi = 10nA, and Ibias = 250nA, and thecapacitance were set at C1 = C2 = 1.2 pF. We usedpulse currents (width: 0.3 µsec) as synaptic inputs,and the capacitance for excitatory and inhibitorysynaptic circuits were set at Cp = 0.02pF.

4.1. Response to excitatory inputs

Figure 4 shows typical behavior of the RFN circuitin response to excitatory synaptic inputs. When apulse current (EPSC, amplitude: 300nA) that could

January 10, 2007 21:0 00084

Analog VLSI Implementation of Resonate-and-Fire Neuron 451

(A)

(B)

(C)

(D)

(E)

Fig. 4. Responses to excitatory synaptic inputs, wheresolid and dashed lines represent Vi and Ui, respectively.

not make the voltage Vi exceeded the threshold volt-age Vth was fed into the circuit, a damped oscilla-tion of the voltages Ui and Vi occurred, in which theperiod of the oscillation was about 50µs (Fig. 4A).When a pair of pulses (EPSCs) with an interpulse-interval (IPI) in the range of 0 to 8.5µs was given

(A)

(B)

Fig. 5. Responses to inhibitory synaptic inputs, wheresolid and dashed lines represent Vi and Ui, respectively.

into the RFN circuit, Vi exceeded Vth and the RFNcircuit generated a pulse voltage Vpulse, and then Ui

was reset to the bias voltage Vrst instantaneously.Thus, the RFN circuit seemed to fire a spike sim-ply. Figure 4B shows a response to a pair of pulseswith an IPI of 5µs. When we gave two pulses withan IPI in the range of 45 to 55µs around the reso-nance interval of the RFN circuit, the RFN circuitalso generated Vpulse. Figure 4C shows a response toa pair of pulses with the IPI of 50µs. However, theRFN circuit did not fire a spike when the IPI was inother ranges, e.g. 30µs (Fig. 4D) and 70µs (Fig. 4E).

These results indicate that the RFN circuit canact as a coincidence detector and a bandpass filterfor sequences of input pulses.

4.2. Response to inhibitory inputs

Figure 5 shows typical behavior of the RFN circuitin response to inhibitory synaptic inputs. When aninhibitory pulse current (IPSC, amplitude: −500nA)was given into the circuit, a damped subthresholdoscillation of Ui and Vi occurred as well as when anexcitatory pulse was given (Fig. 5A). The amplitudeof the oscillation increased with the amplitude of theinhibitory synaptic input. When a strong inhibitorypulse (amplitude: −900nA) was given, Vi exceededVth as a result of a rebound effect of the inhibitoryinput, as shown in Fig. 5B. Thus, the RFN cir-cuit generated Vpulse and Ui was reset to Vrst aftera slight delay. These results are indicative of the

January 10, 2007 21:0 00084

452 K. Nakada, T. Asai & H. Hayashi

property of the post-inhibitory rebound of the RFNcircuit.

4.3. Effects of additive white noise

We here consider the effects of white noise on thecircuit operation. It is suggested that a weak noisyinput induces a sustained subthreshold oscillationin the RFN model.51 When we give a weak noisyinput εI(t) into the RFN model as described asfollows:

z = (−ε + iw)z + εI(t) (32)

the noisy input induces a sustained oscillation, whichis described as follows:

z = I∗(w) exp(iwt) (33)

where I∗(w) represents the Fourier coefficient of theI(t) corresponding to the frequency w, and |I∗(w)|the average amplitude of the oscillation, as proven byIzhikevich.51 As in the case of the RFN model, weaknoise induces a sustained subthreshold oscillation inthe RFN circuit. If white noise in the RFN circuit,which has little effect on the equilibrium point and itsstability, the average amplitude of the oscillation canbe estimated in a similar way to (32) and (33). Theaverage frequency of the oscillation is nearly equalto the natural frequency of the RFN circuit sincethe power spectrum of the white noise is frequencyindependent.

Although the white noise induces the fluctuationof the voltages Ui and Vi, such fluctuation may notaffect the operation of the threshold-and-fire circuitdue to the time-dependent hysteresis described inSec. 3.2. Thus, the RFN circuit can operate correctlyeven in the presence of the white noise.

4.4. Effects of background randomactivity

We then consider the effects of background randomactivity on the performance of the signal detection inthe RFN circuit. We employed a sequence of randompulses as background activity. The IPI distributionis Gaussian, the ratio of the standard deviation tothe mean of IPI is 0.1, and the amplitude of pulsesare 60 nA for excitation and −60nA for inhibition,

RFN

trigger input

background random activity

Fig. 6. RFN circuit under the influence of backgroundrandom activity. Open and closed circles are excitatoryand inhibitory synaptic circuits, respectively.

respectively (Fig. 6). It should be noted that suchrandom activity can be regarded as strong noise.

4.4.1. Resonance interval modulation

We investigated resonance interval modulation of theRFN circuit under the influence of the backgroundrandom activity. The resonance interval of the RFNcircuit decreased with increase in the mean IPI ofthe inhibitory random activity, and increased withincrease in the mean IPI of the excitatory randomactivity, as shown in Fig. 7. This is because theexcitatory random activity increases the equilibriumvoltages (Uo, Vo), and then the equilibrium currents(IUo , IVo), that is the RFN circuit is depolarized. Incontrast to the excitation, the inhibitory backgroundactivity decreases the equilibrium voltages, and thenthe equilibrium currents, that is the RFN circuit ishyperpolarized. As shown in the Jacobian matrix(26), the frequency of the subthreshold oscillationdepends on the equilibrium currents, and thus theresonance interval changes.

30

40

50

60

70

80

0 2 4 6 8 10

Excitatory modulation

Inhibitory modulation

Mean Interpulse-Interval (µsec)

Reso

nance

Inte

rval (

µsec)

Fig. 7. Resonance Interval Modulation.

January 10, 2007 21:0 00084

Analog VLSI Implementation of Resonate-and-Fire Neuron 453

RFN

background random activity

synchronous doublets

Fig. 8. RFN circuit with five excitatory synaptic cir-cuits, in which each carries synchronous doublets embed-ded in background random activity.

4.4.2. Robustness of frequency preference

We here describe how background random activityaffects the performance of the frequency preferenceof the RFN circuit. We consider an RFN circuit withfive excitatory synaptic circuits, each of which car-ries random pulses (the mean of IPI: 25µs) as back-ground activity, and synchronous doublets to theRFN circuit, as shown in Fig. 8.

The RFN circuit can detect synchronous dou-blets with the resonance interval embedded in back-ground random pulses. If the mean of IPI of thebackground pulses is smaller than the resonanceinterval of the RFN circuit, the background pulsesare filtered due to the band-pass characteristics ofthe RFN circuit. Furthermore, if the mean intervalof the synchronous doublets is nearly equal to theresonance frequency modulated by the backgroundrandom activity, the synchronous doublets can bedetected. Figure 9 shows that the RFN circuit can beresonance with synchronous doublets given at about300µs and 350µs even in the presence of the weaksubthreshold oscillation mediated by the the back-ground random activity.

4.5. Influence of device mismatch

We further consider the influence of device mismatchon the circuit properties of the RFN circuit. Thedevice mismatch may mainly affects the propertiesof the membrane circuit. We assumed the varia-tion of threshold voltage of transistors, ∆VTH, asdevice mismatch, and evaluated how such variation

6800 100 200 300 400 500 600 700 800

880

time (µsec)

U

(A)

880

time (µsec)

6800 100 200 300 400 500 600 700 800

Timing of inputs (IEPSC)

Timing of output (Vpulse)

(B)

Fig. 9. Coincidence detection of synchronous doubletsembedded in background random activity.

spreads the equilibrium voltages and resonance inter-val of the membrane circuit using Monte-Carlo simu-lations. Focusing on symmetry in the membrane cir-cuit (Fig. 2), we only considered the local variation ofthe transistors comprising the membrane circuit, andset the standard deviation of the threshold voltageVTH between a pair of transistors (e.g. M1 and M3,and M5 and M6), σ(VTH) = 1 mV. Figure 10 showsthe statical distribution of the equilibrium voltagesUo and Vo. One of the influence of the variation of theequilibrium voltages is on the variation of distancebetween the equilibrium voltage Vo and the thresh-old voltage of the threshold-and-fire circuit, Vth. Thisinfluence can be reduced by tuning the ratio of thevariation and the distance. Another influence is onthe variation of the resonance interval, as shown inFig. 11. The ratio between the mean and standarddeviation of the resonance interval was calculated at2.3%, and thus such influence may be not critical forthe frequency preference of the RFN circuit.

The influence of the device mismatch consideredhere can also be reduced by enlarging the size of thetransistors comprising the RFN circuit. In addition,MOS transistor scaling supports this approach sincethe device mismatch of MOS transistors is generallyproportional to the oxide thickness, which decreaseswith the scaling. The influence of device mismatchmay decrease with decreasing the oxide thickness dueto the MOS transistor scaling if the size of the RFNcircuit is constant. We can also estimate the local

January 10, 2007 21:0 00084

454 K. Nakada, T. Asai & H. Hayashi

Fig. 10. Statical distribution of equilibrium voltages.

variation of device parameters as the oxide thicknessand the size of transistors are given. For instance,the local variation of the threshold voltage can beestimated as follows:

σ(VTH) ≈ ATHtox√LW

(34)

where tox represents the oxide thickness, L andW the channel width and length, respectively, andATH = 1V a proportional constant at a standardprocess. We assumed the oxide thickness tox = 8.0nm in the Monte-Carlo simulations. Thus, if we wantto obtain σ(VTH) = 1mV, the area of a transistorshould be more than LW = 64 µm2 .

4.6. Estimated power consumption

The power consumption of the RFN circuit at asteady state can be calculated as follows:

P = VDD · (IUo + 2IVo + IUi + IVi + 3Ibias) (35)

Fig. 11. Statical distribution of resonance interval.

where IUo and IVo represent the equilibrium cur-rents of the membrane circuit, IUi and IVi the biascurrents for the membrane circuit, and Ibias thebias current for the threshold-and-fire circuit. Byassuming the same parameters as used in the previ-ous simulations, we measured the currents as follows:IUo = 11.8nA, IVo = 12.8 nA, IUi = IVi = 10nA,and Ibias = 500nA. Thus, the power consumptionwas estimated at 2.34µW.

5. Summary and Concluding Remarks

In this paper, we have proposed analog integratedcircuit implementation of a resonate-and-fire neuron(RFN) model. The RFN circuit has been derivedfrom the Lotka-Volterra (LV) oscillator circuit tomimic the dynamical behavior of the RFN model:the damped subthreshold oscillation, resulting intothe coincidence detection, frequency preference, andpost-inhibitory rebound. Due to such properties, theRFN circuit can act as a coincidence detector, andat the same time as a band-pass filter for a spikesequence even in the presence of the additive whitenoise and the background random activity. We havealso considered the influence of the device mismatchon the circuit properties of the RFN circuit. Suchinfluence can be reduced by enlarging the size ofthe RFN circuit, and the scaling of MOS transistorsis suitable for this approach. The typical operatingtime-scales we used here are in the order of tens ofmicroseconds. Therefore, our circuit is insufficient fordealing with natural signals in speech and visual pro-cessing, in which time scales are in the order of mil-liseconds. Our circuit would be applied to high-order

January 10, 2007 21:0 00084

Analog VLSI Implementation of Resonate-and-Fire Neuron 455

neuromorphic processing using the event-based com-putation, concerned with associative memory30 andsequential coding.34

In addition, Izhikevich has recently suggested inhis literature that biological neurons are classifiedinto two categories; integrator and resonators.51 Itis interesting to reflect that cortical neurons actas integrators or resonators depending on whetherthey are excitatory or inhibitory, respectively. Thisfact implies that integrators and resonators in bio-logical neural network have a possibility to shareroles in information processing with each other. Ina similar way, the RFN circuit plays a complemen-tary role to IFN circuits. Thus, the RFN circuit isa candidate as essential component for large-scaleintegration of functional networks of silicon spikingneurons.

References

1. C. A. Mead, Analog VLSI and Neural Systems(Addison-Wesley, Reading, 1989).

2. S.-C. Liu, J. Kramer, G. Indiveri, T. Delbruck andR. Douglas, Analog VLSI: Circuits and Principles(MIT Press, 2002).

3. W. Maass and C. M. Bishop (ed.), Pulsed NeuralNetworks (MIT Press, 1998).

4. T. Fukai, Competition in the temporal domainamong neural activities phase-locked to subthresh-old oscillations, Biol. Cybern. 75 (1996) 453–461.

5. M. Hanagata, Y. Horio and K. Aihara, Asyn-chronous pulse neural network model for VLSIimplementation, IEICE Trans. Fund. E81-A(9)(1998) 1853–1859.

6. W. Senn, I. Segev and M. Tsodyks, Reading neuronalsynchrony with depressing synapses, Neural Compu-tation 10(4) (1998) 815–819.

7. D. J. Mar, C. C. Chow, W. Gerstner, R. W. Adamsand J. J. Collins, Noise shaping in populations ofcoupled model neurons, in Proc. Natl. Acad. Sci.96(18) (1999) 10450–10455.

8. F. Worgotter, A. Cozzi and V. Gerdes, A paral-lel noise-robust algorithm to recover depth informa-tion from radial flow fields, Neural Computation 11(1999) 381–416.

9. S. Fusi and M. Mattia, Collective behavior of net-works with linear (VLSI) Integrate and Fire Neu-rons, Neural Computation 11 (1999) 633–652.

10. S. Fusi, M. Annunziato, D. Badoni, A. Salamon andD. J. Amit, Spike-driven synaptic plasticity: Theory,simulation, VLSI implementation, Neural Computa-tion 12 (2000) 2227–2258.

11. T. Fukai and S. Kanemura, Noise-tolerant stimulusdiscrimination by synchronization with depressingsynapses, Biol. Cybern. 85 (2001) 107–116.

12. G. Bugmann, Synaptic depression increases the selec-tivity of a neuron to its preferred pattern and bina-rizes the neural code, Biosystems 67 (2002) 17–26.

13. R. Reeve and B. Webb, New neural networks forrobot phonotaxis, Philos. Trans. A Math. Phys. Eng.Sci. 361 (2003) 2245–2266.

14. J. J. Hopfield and C. D. Brody, Learning rules andnetwork repair in spike-timing-based computationalnetworks, in Proc. Nacl. Acad. Sci. 101(1) (2004)337–342.

15. M. Mahowald, An Analog VLSI System for Stereo-scopic Vision (Boston, MA: Kluwer Academic Pub-lishers, 1994).

16. F. Tenore, R. Etienne-Cummings and M. A. Lewis,A programmable array of silicon neurons for the con-trol of legged locomotion, in Proc. IEEE Int. Symp.Circ. Syst. (2003).

17. D. H. Goldberg, G. Cauwenberghs and A. G.Andreou, Probabilistic synaptic weighting in areconfigurable network of VLSI integrate-and-fireneurons, Neural Networks 14(6–7) (2001) 781–793.

18. S.-C. Liu, J. Kramer, G. Indiveri, T. Delbruck,T. Burg and R. J. Douglas, Orientation-selectiveaVLSI spiking neurons, Neural Networks 14(6–7)(2001) 629–643.

19. E. Culurciello, R. Etienne-Cummings and K. Boa-hen, A biomorphic digital image sensor, IEEE Jour-nal of Solid-State Circuits 8(2) (2003) 281–294.

20. D. C. Ng, K. Isakari, A. Uehara, K. Kagawa,T. Tokuda, J. Ohta and M. Nunoshita, A studyof bending effect on pulsed frequency modulationbased photosensor for retinal prosthesis, Jpn. J.Appl. Phys. 42(12) (2003) 7621–7624.

21. T. Y. W. Choi, B. E. Shi and K. A. Boahen, AnON-OFF orientation selective address event repre-sentation image transceiver chip, IEEE Trans. Circ.Sys.-I 51(2) (2004) 342–353.

22. R. J. Vogelstein, U. Mallik, G. Cauwenberghs,E. Culurciello and R. Etienne-Cummings, Saliency-driven image acuity modulation on a reconfigurablesilicon array of spiking neurons, Adv. Neural Info.Proc. Syst. 16 (2005).

23. A. Linares-Barranco, G. Jimenez-Moreno, B.Linares-Barranco and A. Civit-Ballcels, On algorith-mic rate-coded AER generation, IEEE Trans. onNeural Networks in printing.

24. N. Kumer, W. Himmelbauer, G. Cauwenbeghs andA. Andreou, An analog VLSI chip with asyn-chronous interface for auditory feature extraction,IEEE Trans. Circ. Sys.-II 45(5) (1998) 600–606.

25. M. Cheely and T. Horiuchi, Analog VLSI models ofrange-tuned neurons in the bat echolocation system,EURASIP J. Appl. Signal Proc. 7 (2003) 649–658.

26. R. Reeve, B. Webb, A. Horchler, G. Indiveri andR. Quinn, New technologies for testing a model ofcricket phonotaxis on an outdoor robot platform,Robotics and Autonomous Systems 51(1) (2005)41–54.

January 10, 2007 21:0 00084

456 K. Nakada, T. Asai & H. Hayashi

27. A. Van Schaik and S.-C. Liu, AER EAR: A matchedsilicon cochlea pair with address event representa-tion interface, in Proc. IEEE Int. Symp. on Circ.and Syst. Kobe, Japan, May V (2005) 4213–4216.

28. S.-C. Liu and R. J. Douglas, Temporal coding in anetwork of silicon integrate-and-fire neurons, IEEETrans. Neural Networks 15(5) (2004) 1305–1314.

29. Y. Kanazawa, T. Asai, M. Ikebe and Y. Amemiya,A novel CMOS circuit for depressing synapse andits application to contrast-invariant pattern classifi-cation and synchrony detection, Int. J. Robotics andAutomation 19(4) (2004) 206–212.

30. H. Tanaka, T. Morie and K. Aihara, Associa-tive memory operation in a Hopfield-type spikingneural network with modulation of resting mem-brane potential, presented at Int. Symp. on Non-linear Theory and its Applications, Bruges, Belgium(2005).

31. G. Indiveri, E. Chicca and R. J. Douglas, A VLSIarray of low-power spiking neurons and bistablesynapse with spike-timing dependent plasticity,IEEE Trans. Neural Networks, to be appeared.

32. D. Muir, G. Indiveri and R. Douglas, Form specifiesfunction: Robust spike-based computation in ana-log VLSI without precise synaptic weights, in Proc.the IEEE Int. Sympo. Circ. and Syst., Kobe, Japan(May 2005).

33. A. Utagawa, T. Asai, T. Hirose and Y. Amemiya, Aneuromorphic LSI performing noise-shaping pulse-density modulation with ultralow-power subthresh-old neuron circuits, 10th International Conferenceon Cognitive and Neural Systems, Boston, USA(May 2006).

34. J. V. Arthur and K. Boahen, Learning in silicon:Timing is everything, Adv. Neural Info. Proc. Syst.17 (2006).

35. C. Rasche and R. H. R. Hahnloser, Silicon synapticdepression, Biol. Cybern. 84 (2001) 57–62.

36. S.-C. Liu, Analog VLSI circuits for short-termdynamic synapses, EURASIP J. Appl. Signal Proc.7 (2003) 620–628.

37. R. Z. Shi and T. Horiuchi, A summating,exponentially-decaying CMOS synapse for spikingneural systems, Adv. Neural Info. Proc. Syst. 16(2004).

38. R. J. Vogelstein, F. Tenore, R. Philip, M. S. Adler-stein, D. H. Goldberg and G. Cauwenberghs, Spiketiming-dependent plasticity in the address domain,Adv. Neural Info. Proc. Syst. 15 (2003).

39. A. Bofill-i-Petit and A. F. Murray, Synchrony detec-tion and amplification by silicon neurons with STDPsynapses, IEEE Trans. Neural Networks 15(5)(2004) 1296–1304.

40. K. Cameron, V. Boonsobhak, A. Murray andD. Ranshaw, Spike timing dependent plasticity(STDP) can ameliorate process variations in neu-romorphic systems, IEEE Trans. Neural Networks16(6) (2005) 1626–1637.

41. G. M. Tovar, T. Hirose, T. Asai and Y. Amemiya,Precisely-timed synchronization among spiking neu-ral circuits on analog VLSIs, in Proc. the 2006RISP International Workshop on Nonlinear Circuitsamd Signal Processing Honolulu, USA (March 2006),pp. 62–65.

42. S. R. Schultz and M. A. Jabri, Analogue VLSIintegrate-and-fire neuron with frequency adaptation,Electronic Letters 31(16) (1995) 1357–1358.

43. K. A. Boahen, Retinomorphic Vision Systems:Reverse Engineering the Vertebrate Retina, Ph.D.Thesis, California Institute of Technology, PasadenaCA (1997).

44. A. Van Schaik, Building blocks for electronic spik-ing neural networks, Neural Networks 14(6–7) (2001)617–628.

45. S.-C. Liu and B. A. Minch, Homeostasis in a siliconintegrate-and-fire neuron, Adv. Neural Info. Proc.Syst. 13 (2001).

46. T. Asai, Y. Kanazawa and Y. Amemiya, A sub-threshold MOS neuron circuit based on the Volterrasystem, IEEE Trans. Neural Networks 14(5) (2003)1308–1312.

47. H. Nakano and T. Saito, Grouping synchronizationin a pulse-coupled network of chaotic spiking oscil-lators, IEEE Trans. Neural Networks 15(5) (2004)1018–1026.

48. Y. Horio, T. Taniguchi and K. Aihara, An asyn-chronous spiking chaotic neuron integrated circuit,Neurocomputing 64 (2005) 447–472.

49. K. Nakada, T. Asai and H. Hayashi, A siliconresonate-and-fire neuron based on the Volterra sys-tem, in Proc. the 2005 International Symposiumon Nonlinear Theory and Its Applications, 82–85,Bruges, Belgium (2005).

50. E. M. Izhikevich, Resonate-and-fire neurons, NeuralNetworks 14 (2001) 883–894.

51. E. M. Izhikevich, Dynamical Systems in Neuro-science: The Geometry of Excitability and Bursting(The MIT press, 2006).

52. M. J. M. Pelgrom, A. C. J. Duinmaijer and A. P.G. Welbers, Matching properties of MOS transistors,IEEE Journal of Soilid-State Circuits SC-24 (1989)1433–1440.

53. S. N. Goel, C. S. Maitra and W. E. Montroll,On the Volterra and other nonlinear models ofinteracting populations, Rev. Mod. Phys. 43 (1971)231–276.