C

M

Ia

b

a

ARRA

KMSC

1

riWroriBttirpeee

e

0h

Journal of Neuroscience Methods 208 (2012) 18–33

Contents lists available at SciVerse ScienceDirect

Journal of Neuroscience Methods

journa l homepage: www.e lsev ier .com/ locate / jneumeth

omputational Neuroscience

odeling and analyzing higher-order correlations in non-Poissonian spike trains

mke C.G. Reimera, Benjamin Staudea, Werner Ehmb, Stefan Rottera,∗

Bernstein Center Freiburg and Faculty of Biology, Albert-Ludwig University, Freiburg, GermanyInstitute for Frontier Areas of Psychology and Mental Health, Freiburg, Germany

r t i c l e i n f o

rticle history:eceived 5 December 2011eceived in revised form 17 April 2012ccepted 18 April 2012

eywords:ultiple single-unit spike trains

urrogate dataalibration of correlation measures

a b s t r a c t

Measuring pairwise and higher-order spike correlations is crucial for studying their potential impacton neuronal information processing. In order to avoid misinterpretation of results, the tools used fordata analysis need to be carefully calibrated with respect to their sensitivity and robustness. This, inturn, requires surrogate data with statistical properties common to experimental spike trains. Here, wepresent a novel method to generate correlated non-Poissonian spike trains and study the impact of single-neuron spike statistics on the inference of higher-order correlations. Our method to mimic cooperativeneuronal spike activity allows the realization of a large variety of renewal processes with controlledhigher-order correlation structure. Based on surrogate data obtained by this procedure we investigate

the robustness of the recently proposed method empirical de-Poissonization (Ehm et al., 2007). It assumesPoissonian spiking, which is common also for many other estimation techniques. We observe that somedegree of deviation from this assumption can generally be tolerated, that the results are more reliablefor small analysis bins, and that the degree of misestimation depends on the detailed spike statistics.As a consequence of these findings we finally propose a strategy to assess the reliability of results forexperimental data.

. Introduction

Despite more than 50 years of experimental and theoreticalesearch, the role of correlated spike activity for neural cod-ng and cortical information processing remains highly debated.

hile earlier studies focused on pairwise spike correlations, moreecently higher-order correlations have also attracted the interestf researchers. Various simulation studies have revealed that neu-ons are indeed very sensitive to the higher-order structure in theirnput (Diesmann et al., 1999; Bohte et al., 2000; Kuhn et al., 2003;enucci et al., 2007). Advancements in multi-electrode recordingechniques nowadays allow to investigate the spiking activity ofhree or more neurons simultaneously, and first findings about thencidence of higher-order correlations in neuronal data have beeneported, along with speculations about their role for informationrocessing (e.g. Shlens et al., 2006; Schneidman et al., 2006; Tangt al., 2008; Montani et al., 2009; Ohiorhenuan et al., 2010; Ganmor

t al., 2011; Yu et al., 2011; Shimazaki et al., 2012, see also Macket al., 2011 for possible explanations).

Abbreviations: CPP, compound Poisson process; CV, coefficient of variation; EDP,mpirical de-Poissonization; PPD, Poisson process with dead time.∗ Corresponding author. Tel.: +49 761 203 9316.

E-mail address: stefan.rotter@biologie.uni-freiburg.de (S. Rotter).

165-0270/$ – see front matter © 2012 Elsevier B.V. All rights reserved.ttp://dx.doi.org/10.1016/j.jneumeth.2012.04.015

© 2012 Elsevier B.V. All rights reserved.

The measurement of neuronal correlations, and hence theirinterpretation, can be affected by various factors (Cohen and Kohn,2011), in particular the brain state (Kohn et al., 2009), spike sortingerrors (Bar-Gad et al., 2001; Pazienti and Grün, 2006) and sys-tematic deviations from the (implicit) assumptions made by theemployed data analysis method (Tetzlaff et al., 2008; Grün, 2009).By far the most common assumption is that measurements in sub-sequent time bins are independent of each other. For single-neuronspike trains, this effectively implies Bernoulli or Poisson statistics.However, recent studies emphasize that neuronal spike trains oftendeviate from this assumption. In particular, neurons can be moreirregular or more regular than a Poisson process, and the degree ofspiking irregularity varies with the brain area (Shinomoto et al.,2009; Maimon and Assad, 2009). For pairwise correlations, theinfluence of single-neuron spike statistics has been well investi-gated, and methods to account for model violations are available(see e.g. Tetzlaff et al., 2008; Grün, 2009; Louis et al., 2010a,b). Inorder to perform similar studies for estimators of higher-order cor-relations, proper surrogate data are inevitable. Currently, however,no method is available to generate non-Poissonian point processeswith defined higher-order correlations.

In the first part of this paper (Section 2), we will therefore intro-

duce a new method to simulate non-Poissonian spike trains withcontrolled higher-order correlations. It combines two proceduresof generating point processes: First, Poisson processes with higher-

urosci

o(g2a1Pttt

nteTiaEapmcac2i

2h

Pn

2

2

td(

ddtt

�

w∫ioLt

�

2

pcrp

of the hazard function of the target non-Poissonian process.

I.C.G. Reimer et al. / Journal of Ne

rder correlations can be obtained by injecting coincident spikesi.e. higher-order correlations) into a background of independentlyenerated spikes (Kuhn et al., 2003; Ehm et al., 2007; Staude et al.,010b,c). Second, a renewal process can be generated by appropri-te deletion of events (“thinning”) of a Poisson process (Devroye,986). Applying the thinning procedure to each of the correlatedoisson processes then yields a population of non-Poissonian spikerains with higher-order correlations. We study the influence ofhinning and devise a strategy to prescribe the higher-order struc-ure of the resulting non-Poissonian population.

In the second part (Section 3), we investigate the influence ofon-Poissonian spiking statistics on estimates of higher-order fea-ures. As a result, we propose an a posteriori strategy to judge therror of higher-order correlation estimates from experimental data.he method employed here to estimate higher-order correlationss empirical de-Poissonization (EDP; Ehm et al., 2007) which willlso be briefly introduced. Based on the population spike count,DP estimates a population average of the correlation structure,nd a lower confidence limit for the maximal order of correlationresent in the data. In contrast to other higher-order analysis-ethods (e.g. Martignon et al., 1995, 2000; Amari et al., 2003) its

ompact parametrization of higher-order correlations renders EDPpplicable to large neuronal populations and sample sizes that areompatible with current experimental settings (see also Ehm et al.,008). However, the underlying model assumes that neurons fire

n a Poisson manner.

. Generation of non-Poissonian processes withigher-order correlations

In order to introduce our new method to generate non-oissonian processes with higher-order correlations some termi-ology and methodology of renewal theory is required.

.1. Renewal point process

.1.1. DefinitionA renewal process is a point process where the times between

wo point-like events (i.e. the inter-spike intervals) are indepen-ent and identically distributed with probability density functionp.d.f.) f(x) (see e.g. Cox, 1962; Perkel et al., 1967).

Equivalently to this definition, a renewal process is completelyetermined by its hazard function. This function describes the con-itional rate for the occurrence of an event (i.e. spike) with respecto the time x passed since the last event. Formally, the hazard func-ion is defined as

(x) := lim�x→0

P(x < X ≤ x +�x | x < X)�x

= f (x)1− F(x)

, (1)

here X is the inter-event interval and F(x) := P(X ≤ x) =x

0f (y) dy is the cumulative distribution function.Useful quantities to describe a renewal process include the

ntensity � (i.e. the spike rate) and the coefficient of variation CVf the inter-spike interval distribution (i.e. the spiking irregularity).et E[X] denote the expectation of X and Var[X] the variance of X,hen � and CV are given by

= 1E[X]

and CV =√

Var[X]E[X]

. (2)

.1.2. ExamplesThe most prominent example of a renewal process is the Poisson

rocess with exponentially distributed inter-spike intervals and aonstant hazard function. In neuroscience common non-Poissonianenewal processes are Poisson processes with dead time, gammarocesses and lognormal processes. In particular, they are fre-

ence Methods 208 (2012) 18–33 19

quently used to fit experimentally observed inter-spike intervalsand to mimic non-Poissonian spike activity in theoretical studiesof neural processing (see e.g. Kuffler et al., 1957; Stein, 1965; Beyeret al., 1975; Burns and Webb, 1976; Tuckwell, 1988; Levine, 1991;McKeegan, 2002; Nawrot et al., 2008; Maimon and Assad, 2009;Minich et al., 2009; Grün, 2009; Ly and Tranchina, 2009; Deger et al.,2010, 2011; Rosenbaum and Josic, 2011). We briefly describe someof its main features and refer the reader to Appendix A for moredetails.

2.1.2.1. Poisson process with dead time. Poisson processes withdead time (PPD) are quite similar to Poisson processes but addi-tionally they allow to account for the refractory period of a neuron.That is, the rate for the occurrence of a new spike after a spikeis zero up to the dead time d. This is depicted for the inter-spikeinterval distribution and for the corresponding hazard function inFig. 1A and 1D (compare orange dashed line, Poisson process, withthe solids lines, PPD). The spiking irregularity which can be mim-icked by a PPD is restricted to a coefficient of variation CV smallerthan one (cf. Appendix A).

2.1.2.2. Gamma process. In contrast, any coefficient of variationwithin the interval [0,∞[ can be realized by gamma processeswhich have gamma-distributed inter-spike intervals. As for PPDsthe Poisson process constitutes a special case for CV = 1. Fig. 1B andE shows that for CV > 1 small inter-spike intervals are very frequent,in contrast to neuronal spike activity. For CV < 1 small inter-spikeintervals can occur but they are rare.

2.1.2.3. Lognormal process. As opposed to gamma processes theoccurrence of small inter-spike intervals is very unlikely for pro-cesses with lognormal-distributed inter-spike intervals even forCV > 1. Accordingly, the hazard function of lognormal processesexhibits a different behavior (compare panels E and F in Fig. 1). Fur-thermore, a lognormal process with CV = 1 is not a Poisson process,as can be seen from the non-flat hazard function (Fig. 1F).

2.1.3. Generation of a renewal process via thinning of a Poissonprocess

An adaptation of the method proposed by Lewis and Shedler(1979) on generating non-stationary Poisson processes enablesone to generate renewal processes via a thinning procedure (seeDevroye, 1986, chapter VI.2.4). An intuition for its main idea caneasily be obtained for Poisson processes with dead time. As hasbeen mentioned earlier, Poisson processes with dead time are likePoisson processes except for the fact that given an event at time t, noevent can occur up to time t + d, where d is the imposed dead time.Hence, one obtains such a process by first simulating a “source”Poisson process and then deleting all events which occur too earlywith respect to the previous event (see Fig. 2A). That is, one keepsan event only if the hazard function of the corresponding inter-event interval has a non-zero value. In doing so, the rate is reducedand therefore the initial rate of the source process has to be chosensufficiently high.

The procedure generalizes to processes with more complicatedhazard functions �(x) by deleting events with a probability propor-tional to the hazard function evaluated at the time x elapsed sincethe most recent event (see Fig. 2B). In order to obtain a certain ratein the “target” process after thinning, the rate of the source Poissonprocess has to be chosen at least as high as max

x�(x), the maximum

Let the list of spike times S = {t1, t2, . . . tn} and S′ = {t′1, t′2, . . .}denote the spike trains of the source and the target process, respec-tively. Then the thinning algorithm is described by the followingsteps:

20 I.C.G. Reimer et al. / Journal of Neuroscience Methods 208 (2012) 18–33

Fig. 1. Probability density functions f(x) of the inter-spike intervals (A–C) and corresponding hazard functions �(x) (D–F) for various renewal processes with differentc gend)p r Poiss

0

1

2

Iiees

tmadpvtc

2

gbtwcnS

B

U

Haz

ard

Rm

AR

m

d

Haz

ard

Time

Time

Fig. 2. Thinning of a Poisson process (grey and red ticks at bottom of A, B) accordingto the hazard function �(x) (top) of the target renewal process (red ticks at bottom).The dashed line (top) indicates an upper bound Rm of the hazard function whichis chosen as the rate of the source Poisson process. (A) Target process is a Poissonprocess with dead time. Events which occur within the dead time d are deleted. (B)

oefficient of variations with CV∈ [0.45, 3] (solid lines, CV encoded by color, see lerocesses (B and E) and lognormal processes (C and F) in comparison with those fo

. a) choose inter-event interval distribution of the target processwith � and CV according to Eq. (2)

b) determine �(x) via Eq. (1) and maxx

�(x)

. a) set Rm such that Rm ≥maxx

�(x)

b) generate a Poisson process S with rate Rm

. set t′latest ← t1 % t′latest is latest renewal event timeset S′ ← {t′latest}for i = 2 : n do

draw U with U∼U(0, Rm)if U ≤ �(ti − t′latest) then

t′latest ← ti

S′ ← {S′, t′latest}end if

end for

Note that the first event of the target process is a Poisson event.n order to obtain a renewal process in “equilibrium” where allnter-spike intervals follow the desired distribution, one has tomploy a “warm-up time” (Cox, 1962; Nawrot et al., 2007). Oth-rwise, strong onset transients may contaminate the spike counttatistics (Muller et al., 2007; Deger et al., 2010).

Step 1. a) of the algorithm implies that the hazard function ofhe target process is bounded. Given this condition the thinning

ethod allows to generate a renewal process with arbitrary haz-rd function. This could be an empirical one obtained from neuronalata, or a parametrized one. Examples for the latter are Poissonrocesses with dead time, gamma processes with a coefficient ofariation smaller than one and lognormal processes (see Fig. 1, bot-om, and Appendix A). The gamma process with CV > 1, however,annot be simulated (see also discussion).

.2. Correlated renewal point processes

The method described in the previous section enables one toenerate single spike trains with any inter-spike interval distri-ution given its hazard function is bounded. We will now usehis technique to simulate populations of renewal point processes

ith higher-order interactions. To that end, we define higher-order

orrelations and review how Poisson processes with these multi-euron events can be obtained (cf. Ehm et al., 2007; Brette, 2009;taude et al., 2010a,c).

. Shown are the functions for Poisson processes with dead time (A and D), gammaon processes (dashed orange line), each with � = 6 Hz.

2.2.1. Higher-order correlationsFig. 3C shows the spike trains {si(t)}i of N neurons. The col-

ored ticks indicate times with simultaneous spikes in at least twoneurons, where color encodes the number of neurons that spikesynchronously. As can be seen in Fig. 3A via corresponding colors,we represent a pattern of exactly n coincident spikes by a processyn(t) with intensity �n (see B). In other words, the events of yn(t)indicate all instants when such a multi-neuron event of order n (i.e.only a single spike in case of y1(t)) occurs.

Target process is a renewal process with bounded hazard function. One successivelytests whether the interval x between a Poisson event and the latest renewal event isin accordance with the hazard function. That is, one draws a random number U withU∼U(0, Rm) (indicated by crosses, top). If U≤�(x) the event is kept and assigned tothe target renewal process (marked red).

I.C.G. Reimer et al. / Journal of Neuroscience Methods 208 (2012) 18–33 21

Cs

1(t)

s2(t)

sN(t)

.

.

.

A y1(t)

y2(t)

yN(t)..

B ν1

ν2

νN

.

.

ξ

DZ

l

Time

{

h

E

k

P(Z

l=k)

EDP

Fig. 3. Modeling and measuring higher-order correlations. (A) Raster plot of the component processes yn(t) which constitute the compound Poisson process z(t) =∑N

n=1nyn(t). Each process yn(t) represents the synchronized activity of some order n. (B) Correlation structure (�1, �2, . . ., �N) where the bar length indicates �n , the

rate of the component process yn(t) shown in (A). Arrow indicates maximal order of correlation �. (C) Raster plot of a population of N correlated spike trains si(t), 1≤ i≤N.The bars are color-coded according to the order n of synchronization and correspond to the component process yn(t) in (A). (D) Population histogram. The number of spikesi intere e pop( cess (

b

z

Tsn“�erMf

2c2{pstyyrcit

maf

n (B) are counted in time bins of length h, and Zl denotes this number for the timempirical de-Poissonization (EDP, see Section 3.1) is to relate the distribution of thindicated by arrow from E to B). In doing so, it assumes the compound Poisson pro

The summed spike activity∑N

n=1si(t) of the population can thene represented as

(t) =N∑

n=1

nyn(t). (3)

hus, if �n > 0 for some n≥2, there are groups of n neurons showingynchronized activity, which implies that the corresponding singleeuron spike trains are correlated with each other. We therefore saythe neuronal spike activity exhibits correlations of order at least” if the rate �n is non-zero for at least one n≥ � (see also Staudet al., 2010c, Theorem 1). Accordingly, the vector of componentates (�1, �2, . . ., �N) will be referred to as the “correlation structure”.oreover, we call its representation as a probability mass function

A(k) = �k/�+ with �+ =∑N

n=1�n the “amplitude distribution”. 1

.2.2. Generation of Poisson processes with higher-orderorrelations.2.2.1. Compound Poisson process. If the component processesyn(t)}n are independent stationary Poisson processes, the sumrocess z(t) is a compound Poisson process (CPP). This model canerve as the basis for the generation of various populations of spikerains. That is, given N independent component Poisson processesn(t) with intensity �n (n = 1, . . ., N), copying the event times ofn(t) to n out of N neurons will result in a population with cor-elation structure (�1, �2, . . ., �N). Here, many rules of assignment

an be considered (see Staude et al., 2010b, for more examples): fornstance, if �n = 0 for n≥2 and the events are assigned alternativelyo one out of two neurons, this will result in two gamma processes

1 This name is motivated by considering the compound Poisson process as aarked Poisson process. That is, z(t) =

∑jı(t− tj) · aj where tj are Poisson events,

nd aj the corresponding amplitude (number of synchronously firing neurons). Thus,A(k) = P(aj = k).

val [(l−1)h, lh]. (E) Distribution of the population spike count Zl in (C). The aim ofulation spike counts derived from experimental data to the correlation structure

A) as a model for the summed spike activity of the considered population (C).

with shape parameter ˛ = 2 (cf. e.g. Baker and Gerstein, 2000) andno coincident spikes.

2.2.2.2. Homogeneous Poisson population. If, however, event timesare ascribed to each neuron independently one will end up withPoisson processes (Ehm et al., 2007; Brette, 2009; Staude et al.,2010c). The simplest procedure is to assign the event times uni-formly to the N neurons which gives a homogenous population. Inmore detail, let A be a random variable representing the correlationstructure, i.e. having the probability mass function fA(k) as definedin Section 2.2.1. Then, the single cell processes si(t) have the rate

� = E[A]N

�+. (4)

Moreover, with �c denoting the rate of coincident firing of any twocells,

cp := �c

�= E[A2]− E[A]

E[A](N − 1)(5)

is the pairwise coincidence probability, which in the case of Pois-son processes coincides with Pearson’s count correlation coefficient(Kuhn et al., 2003; Tetzlaff et al., 2008). Note that Eqs. (4) and (5)depend only on the first two moments of the amplitude distribu-tion fA(k), and more details of it are not incorporated. Therefore, itis possible to construct Poisson populations with identical singleand pairwise statistics that differ with regard to their higher-ordercorrelation structure (Kuhn et al., 2003; Staude et al., 2010c).

2.2.2.3. Heterogeneous Poisson population. Correlated Poisson pro-cesses with different rates and pairwise correlations as well asnon-stationarities can be obtained by more complex rules accord-ing to which the event times of the component processes are

independently assigned to the different neurons. Fig. 4 (right panel)shows an example of a heterogenous population where the rasterplot (bottom) visualizes four different subpopulations. These spiketrains have been obtained by assigning the event times of y1(t) to

22 I.C.G. Reimer et al. / Journal of Neuroscience Methods 208 (2012) 18–33

10

0.5

1A

fA(k

)

3 5 7 9 11 13 150

0.01

0.02

0.03d=0.05c

p=0.059

cp=0.051

cp=0.05

k

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8

10

20

30

40

50C

Time [s]

Neu

ron

ID

10

0.5

1B

3 5 7 9 11 13 150

0.02

0.04d=0.02c

p=0.029

cp=0.015

cp=0.015

k

prepostbinomial

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8

20

40

60

80

100D

Time [s]

Fig. 4. Generation of correlated non-Poissonian processes via thinning of Poisson processes. (A) and (B) shows the amplitude distribution fA(k) of the Poisson processesbefore thinning (marked grey, named ‘pre’), the distribution of the resulting processes after thinning (marked red, named ‘post’), and the distribution given by the binomialdescription (marked green, named ‘binomial’). The pairwise coincidence probability cp estimated via the right-hand side of Eq. (5) is colored accordingly. The deviation offApost from fAbinomial

is quantified by d(�binomial, �post) (see Eq. (22)). In (C) and (D) raster plots show the Poisson spikes (grey and red dots) along with those which survived thet nd B)D or dett

tmiicpawa

2c

fcep

2nspIa

ofiw

sFaeao

hinning procedure (marked red) and constitute the non-Poissonian processes. (A a) 100 lognormal processes consisting of four disjoint homogeneous assemblies, f

hinning procedure.

he neurons with different probabilities according to their groupembership. Moreover, only those higher-order events belong-

ng to the same order of synchronized activity have been copiednto one group. Note that the calculation of the mean pairwiseoincidence probability for two spike trains of a non-homogenousopulation needs to take into account the specific assignment prob-bilities (cf. Brette, 2009). Hence, for non-homogenous populationse will rather refer to cp given by the right-hand side of Eq. (5) asmeasure of correlation strength (see Staude et al., 2010c).

.2.3. Generation of non-Poissonian processes with higher-orderorrelations

We use the previous techniques to introduce a new methodor the generation of non-Poissonian processes with higher-orderorrelations. First, we describe the main procedure and then welaborate on how the higher-order correlation structure can beredefined.

.2.3.1. General procedure. Fig. 4C and D shows raster plots ofon-Poissonian processes (red dots). These processes have beenimulated by first generating a population of correlated Poissonrocesses (all spikes in the raster plot) as described in Section 2.2.2.

n doing so, the rate of each source Poisson process has been chosens Rm =max

x��,CV(x), where ��,CV(x) denotes the hazard function

f the target spike activity of one neuron with spike rate � and coef-cient of variation CV. Then the thinning procedure of Section 2.1.3as applied to each source process.

Evidently, the thinning affects not only the single neuron spiketatistics but also the correlation structure in the population (seeig. 4). As the spikes in the source spike trains are deleted randomly

nd independently, thinning reduces the order of higher-ordervents in a random fashion (for details, see discussion below). Asconsequence, the random variable describing the post-thinningrder of the multi-neuron event that had order n in the source pop-

50 Poisson processes with dead time with � = 10 Hz, CV = 0.85 and cp = 0.05. (C andails see Appendix D. Simulation time is 1000 s. See Section 2.2.3 for details on the

ulation is binomially distributed with parameters n and p, where pis the survival probability. For an arbitrary amplitude distribution,the above argument implies that the amplitude distribution afterthinning, fA, is a superposition of binomial distributions

fA(k) ∝∑

n

fApre (n) ·B(k|n, p), (6)

where fApre (n) is the amplitude distribution before thinning. Notethat the binomial distribution assigns a non-zero probability to allk≥0, but within our model events of order zero (k = 0) are notaccounted for. Hence, to obtain a well-defined probability massfunction fA(k) the right-hand side in Eq. (6) has to be renormalized.

2.2.3.2. Predefining the correlation structure. Not only firing ratesand the coefficient of variation CV, but also the correlation structurecan be controlled, as we will now explain with a simple example.Consider that we start with a two-point amplitude distribution fApre

that only allows for independent spikes and synchronized spikes ofsome order �syn > 1 only, as illustrated in Fig. 4A (grey bars) and C(grey and red dots). Then

fApre (k) = � · ı1,k + (1− �) · ı�syn,k, (7)

where ıi,j denotes the Kronecker delta (ıi,j = 1 if i = j, otherwiseıi,j = 0). The probability that a spike survives thinning is �/Rm. Thus,following the above arguments (cf. Eq. (6)), the amplitude distribu-tion after thinning is a superposition of binomial distributions withparameters (1, �/Rm) and (�syn, �/Rm), which after normalization is

fAbinomial(k) =

� · �Rm· ı1,k + (1− �) ·B(k|�syn, �

Rm)

� · �Rm+ (1− �) · [1− (1− �

Rm)�syn ]

. (8)

uroscience Methods 208 (2012) 18–33 23

Dn

�

Add

rsoIhsvf

�

gnsitnkaantcfta

2dfBhwao

t(

0

ir

iat

sp

s�

tc

0 100 2000

0.5

1

1.5

2A

Coi

nc. e

vent

s [1

/s]

−60 0 60

0

0.1

0.2

0.3

0 100 2000

0.5

1

1.5

2B

−60 0 60

0

0.1

0.2

0.3

0 100 2000

0.5

1

1.5

2C

Lag [ms]

Coi

nc. e

vent

s [1

/s]

−60 0 60

0

0.1

0.2

0.3

0 100 2000

0.5

1

1.5

2D

Lag [ms]

−60 0 60

0

0.1

0.2

0.3

Fig. 5. Auto- and cross-correlation histograms of Poisson processes with dead time(PPDs). A population of N = 200 PPDs with correlation structure as described by Eq.(8) has been generated by our thinning method for 1000 s. Parameters have beenchosen as � = 10 Hz, � = 20, Rm = 50 and cp = 0.1 (i.e., spikes of two neurons coincide onaverage once per second). The auto-correlation histograms are depicted in grey andthe cross-correlation histograms in red. The histograms have been determined witha bin width of 1 ms and then averaged over 100 samples for PPDs with CV = 0.9 (A, C)and CV = 0.45 (B, D) which corresponds to a dead time of 10 ms and 55 ms, respec-tively. Insets depict the same pictures with different limits on the axis. Note that

I.C.G. Reimer et al. / Journal of Ne

isregarding the independent spikes, the mean order of synchro-ized activity now is

= �syn · �

Rm. (9)

s can be seen in Fig. 4 (left top panel) for our above example theistribution fAbinomial

(k) (green bars) fits the empirical amplitudeistribution after thinning fApost (k) (red bars) quite well.

This binomial description enables one not only to predefine theate � and the coefficient of variation CV but also the correlationtructure � = (�1, �2, . . ., �N) of the target processes. The locationf the mean order of correlation � can be adjusted via Eq. (9).n order to get a certain pairwise coincidence probability cp, oneas to determine � and thereby the amplitude distribution of theource processes (see Eq. (7)). This can be achieved by calculating cp

ia the right-hand side of Eq. (5) for the probability mass functionAbinomial

(k) and solving for �. This yields

=� · (� − �

Rm− cp(N − 1))

� · (� − �Rm− cp(N − 1))+ cp(N − 1) · �

Rm

. (10)

It should be noted, however, that the binomial description isood only if the interactions of the source Poisson processes areot too strong. In spike trains which are dominated by high orderpike patterns deletion of spikes tends to affect correlated neuronsn a non-independent manner. In a nutshell, if the spikes belongingo a multi-neuron event survive in two neurons, the same synchro-ized hazard decides about which spikes of the source processes areept or rejected next. So should the corresponding follow-up spikesgain belong to a common multi-neuron event, then a non-flat haz-rd will imply the same bias in both neurons. That is, rejection isot independent across neurons any more. This also shows, whyhis lack of independence depends on both the degree of pairwiseorrelation (overall rate of patterns) and the degree of deviationrom Poissonian firing (non-flat hazard). If not stated otherwise, inhe following sections we refer to the regime where the binomialpproximation holds.

.2.3.3. Auto- and cross-correlation functions. The thinning proce-ure results in non-Poissonian spike trains with an auto-correlationunction that is defined by the chosen hazard function (see Fig. 5A-, grey lines). The cross-correlation function of two target processesas a delta-peak at time lag zero as depicted for Poisson processesith dead time in Fig. 5A-B (red lines). Its height is determined by cp

nd has an upper bound which depends on the correlation structuref the source population and the probability of spike deletion.

For instance, for the population considered in the previous sec-ion one obtains from the constraint that 0≤�≤1 together with Eq.10) and Eq. (9)

≤ cp ≤� − �

Rm

N − 1= �syn − 1

N − 1· �

Rm. (11)

Thus, as �syn ≤N, the maximal pairwise coincidence probabil-ty is given by �/Rm. Hence, choosing Rm as small as possible isecommended to keep the range of possible values for cp large.

More generally, the strongest correlation in the target processess obtained by starting with two source processes with cp = 1. In

homogeneous population the probability that a spike surviveshinning is �/Rm and thus the probability that the same spike time

urvives in both spike trains is(

�/Rm

)2. The pairwise coincidence

robability of the target processes is given by the rate of coincident

pikes (here:(

�/Rm

)2 ·Rm) divided by the rate of a spike train (here:

) which yields �/Rm.

Note that the effect of non-independent spike deletion duringhe thinning procedure (cf. discussion above) is also visible in theross-correlation function of the target processes. Fig. 5A shows

all central peaks are truncated. (A and B) Precise coincidences. (C and D) Imprecisecoincidences. The spike times of the source processes have been jittered accordingto a uniform distribution with support [−10 ms, 10 ms].

that the average cross-correlation histogram of two PPDs withCV = 0.9 exhibits a trough around lag 0. This feature is similar tothe one in the corresponding autocorrelations. The amplitude ofthe former is, however, very small in comparison. The trough inthe cross-correlation histogram is more prominent for processesdeviating considerably from Poisson (compare Fig. 5A, CV = 0.9 andB, CV = 0.45) and the stronger the processes are correlated (notshown). Additionally, the height of the peak deviates more fromthe predefined correlation strength.

Our algorithm yields precise coincidences, however, correla-tions expanded in time can easily be obtained. By adding a randomdisplacement to each spike time of the source Poisson processesone generates non-instantaneous correlations (cf. e.g. Bäuerle andGrübel, 2005; Brette, 2009), and this jitter is preserved under thin-ning. Fig. 5C, D show the average cross-correlation histograms (redlines) with the same single neuron spike statistics as in A, B butnon-instantaneous coincidences. More precisely, the event timeshave been jittered according to a uniform distribution with support[−10 ms, 10 ms] before the thinning procedure has been applied.This is reflected in the cross-correlation function by a triangulararound lag 0 with a corresponding extent in time.

2.2.3.4. Correlation structure. In paragraph 2.2.3.2 we onlyexplained how the correlation structure can be predefinedwhen the source population exhibits synchronized activity of onlyorder �syn. The ansatz outlined above can easily be extended tomore complicated correlation structures. For instance, this can beachieved by describing the amplitude distribution with coincidencepatterns of various orders �1

syn, �2syn, . . . before thinning as

fApre (k) = � · ı1,k + (1− �) ·Nsyn∑i=1

ωiı�isyn,k. (12)

2 urosci

Hdp

t

f

PttAA

2uohengoi0coudbpt

cPh

writplHia

clfipoWc

3e

erEs

4 I.C.G. Reimer et al. / Journal of Ne

ere, Nsyn denotes the number of non-zero entries in the amplitudeistribution for k > 1 and ωi determines the relative height of eacheak with

∑ωi = 1.

Assuming independent deletion of spikes, the resulting ampli-ude distribution can be described as

Abinomial(k) =

� · �/Rm · ı1,k + (1− �) ·Nsyn∑i=1

ωiB(K∣∣�i

syn, �/Rm)

� · �/Rm + (1− �) ·N∑

k=1

Nsyn∑i=1

ωiB(K∣∣�i

syn, �/Rm)

. (13)

roceeding as above yields an expression for � which then, addi-ionally to the choice of the weights ωi, allows to predefinehe correlation structure of the target non-Poissonian processes.n example with Nsyn = 2 is employed in Section 3.3.2 (see alsoppendix D).

.2.3.5. Heterogeneous populations. Thus far, we focused on pop-lations of homogeneous non-Poissonian processes. However,ne can also employ the thinning procedure to generate non-omogeneous populations. Fig. 4, right column, shows such anxample. The population consists of four cell assemblies whereeuronal spiking is correlated within each group, but not acrossroups (e.g. Berger et al., 2007). More precisely, four populationsf lognormal processes (N = 30, 20, 15 and 35 neurons) have beenndependently generated with fixed firing rate (� = 1.5, 2.5, 12 and.5Hz), coefficient of variation (CV = 1.5, 2.5, 2 and 1.2), pairwiseoincidence probability (cp = 0.03, 0.08, 0.12 and 0.005) and meanrder of synchronized activity (� = 2, 6, 7 and 3), resulting in a pop-lation that is heterogeneous in several respects. The amplitudeistribution of the whole population (red bars, B) is well describedy the superposition of the amplitude distributions of the four sub-opulations (green bars, B). See also Section 3.3 where we studyhis example in more detail.

Another option to obtain a non-homogeneous population is tohoose different hazard functions according to which the sourceoisson processes are thinned. For instance, if one starts with aomogeneous Poisson population with rate Rm =max

i,x��i,CVi

(x)

here ��i,CVi(x) denotes the hazard function of the i-th target

enewal process, then the amplitude distribution is well approx-mated (and hence controllable) by Eq. (8) by substituting � withhe mean rate 〈�i〉 (not shown). However, the pairwise coincidencerobability cp will not be the same across all neurons in the popu-

ation, if the deletion probabilities vary strongly across processes.ence, populations with similar spike rates, but different types of

nter-spike interval distributions can be better controlled by thispproach than populations with different spike rates.

By appropriately choosing the parameters Rm, �syn and � of theorrelated source Poisson processes our method allows to simu-ate various kinds of surrogate data with certain statistics beingxed, while others are varied. For instance, it is possible to generateopulations of spike trains with same rates and similar higher-rder correlation structures, but different degrees of irregularity.e employ this unique feature of our method in the following

hapter (see also Appendix C).

. Impact of single neuron spike statistics on thestimation and inference of higher-order correlations

We study the impact of single neuron spike statistics on the

stimation and inference of higher-order correlations using theecently proposed method of empirical de-Poissonization (EDP,hm et al., 2007). Here, we briefly review how the correlationtructure (�1, �2, . . ., �N) and the maximal order of correlation �

ence Methods 208 (2012) 18–33

are estimated, but refer to Ehm et al. (2007) for a mathematicalderivation and more details.

3.1. Empirical de-Poissonization

Let Zl denote the population spike count, i.e. the number ofspikes recorded from a neuronal population within the time inter-val [(l−1)h, lh] of length h, as depicted in Fig. 3D. The distribution ofthese bin counts (Fig. 3E) evidently depends on the spike rates of thesingle cells, but also on their higher-order correlations, since corre-lations of any order put more weight on the tail of the distributionas compared to the case of independent firing. The distribution thusprovides important information about the strength and the orderof the correlations. In fact, EDP can reconstruct the entire unob-servable correlation structure from that distribution (as indicatedby the arrow in Fig. 3 from E to B).

In order to achieve this, EDP assumes the CPP as a model forthe population spike activity. Practically, it is reasonable to think ofthe single-neuron spike trains as Poisson processes, although thisis not a mathematical necessity. Apart from this property a large setof different populations can be described by this CPP model and, inparticular, by the same correlation structure (cf. Section 2.2.2).

By employing the binned population spike activity one circum-vents an apparent shortcoming of the underlying CPP model. Acomponent process yn(t) represents the exact coincidence of nspikes, whereas correlated firing of neurons does not show sucha high precision. However, the value of the bin count Zl will beapproximately the same if these n spikes do not occur at the exactsame time but with a jitter smaller than the bin size h.

3.1.1. Estimating the correlation structureWithin the framework of the CPP model the rate of synchronized

activity of order n can be regarded as the ‘de-Poissonized’ versionof the ‘Poissonized’ bin counts Zl:

�n = 12

∫

−

h−1 log �h(�)e−in�d�. (14)

Here, �h(�) = E[ei�Zl ] denotes the characteristic function of thebin counts Zl which uniquely determines, and is uniquely deter-mined, by the distribution.

By substituting the unknown �h(�) in Eq. 14 with the empiricalcharacteristic function

�h(�) = L−1L∑

l=1

ei�Zl (15)

one obtains estimates �n of the correlation strengths �n of ordern = 1, 2, . . .. Note that the estimates �n can, in principle, take neg-ative values. If the sample size L and the number of zero bins (i.e.,bins with Zl = 0) are large, �n is a consistent estimate of �n for eachn. In the present manuscript these conditions are satisfied, and werefer the reader to Ehm et al. (2007) for more details and some hintson corrective methods.

3.1.2. A lower bound for the maximal order of correlationIn addition to the direct rate estimates �n, Ehm et al. (2007)

proposed the maximal order of correlation � as a measure of interest(cf. also Staude et al., 2010b,c). The (unknown) true � as defined inSection 2.2.1 can formally be expressed as

� =max{

m | m > 0}

, (16)∑

where m := ∞

n=m�n.Hence, an estimate for � can be obtained by successively esti-

mating the rate tail m and testing whether it deviates significantlyfrom 0. More precisely, one starts with m = 2 and increases m until

urosci

tlt

�

m

t

�

T

p

wd

mt�

3

cAmT

1CiCbgdttnnbcPitcocd

3

i

I.C.G. Reimer et al. / Journal of Ne

he null hypothesis Hm0 : m = 0 can not be rejected at a significance

evel ˛. Let pm denote the p-value associated with such a hypothesisest, then by this procedure the estimated � is given as

ˆ =min {m | pm > ˛} − 1. (17)

The p-value is determined in the following way: Under the CPPodel m can be represented as

m = 12

∫

−

h−1 log �h(�)e−im�

1− ei�d�. (18)

Again plugging in �h(�) for �h(�) in Eq. 18 yields an estimateˆm for sampled data. Under Hm

0 , m is approximately normally dis-ributed with mean zero and variance �2

m = (Lh)−1�m,m where

m,m = 1

(2)2

∫

−

∫

−

1h

(�h(�1 + �2)

�h(�1)�h(�2)− 1

)

× e−m�1

1− e−i�1

e−m�2

1− e−i�2d�1d�2. (19)

his expression, too, can be estimated by plugging in �h(�).An approximate p-value pm for testing Hm

0 : m = 0 against Hm1 :

m > 0 is thus given by

m = 1−�

( m

�m

)(20)

here � denotes the cumulative distribution function of the stan-ard normal distribution.

As shown in Appendix B, � as defined by Eq. (17) is, approxi-ately, a lower confidence limit for � at the level ˛. This means

hat irrespective of the underlying spike rates, the probability thatˆ is larger than the unknown parameter � is at most ˛.

.2. Robustness of empirical de-Poissonization

The analysis of higher-order correlations with EDP requireshoosing a bin size h to determine the population spike count.ctually, this is the only adjustable parameter given a set of experi-ental data. Picking h is hence the only possibility to affect results.

herefore, we study the robustness of EDP in dependence of h.Fig. 6A shows the population spike count for a population of

5 correlated lognormal processes with coefficient of variationV = 0.65 (dark blue) and 15 Poisson processes (light blue) with

dentical correlation structure, averaged over 100 realizations (seefor details on the generation of the surrogate data). For a small

in size of 2 ms (hatched bars) the distributions are almost indistin-uishable. However, for a bigger bin size of 10 ms the distributionsiffer from each other (see e.g. k = 0, 1). Accordingly, applying EDPo these data sets yields similar results only for small bin sizes. Therue correlation structure (red bars in Fig. 6B) is well estimatedot only for the Poisson processes (light blue) but also for the log-ormal processes (dark blue) given a bin size of h = 2 ms (hatchedars). While for a bin size of 10 ms (filled blue bars) the estimatedorrelation structure (�1, �2, . . . , �N) is on average correct for theoisson processes, the processes violating the model assumptionsnduce a biased estimation. Correspondingly, for a small bin sizehe estimated maximal order of correlation for the lognormal pro-esses (dark blue in Fig. 6C) lies on average only slightly below thene estimated for the Poisson processes (light blue). While in bothases � decreases with increasing bin size it appears that the resultsiffer more for larger bins.

.2.1. Estimated correlation structureWe quantify the degree to which non-Poissonian spiking

mpairs the estimation of the correlation structure in relation to the

ence Methods 208 (2012) 18–33 25

situation of Poissonian spiking by using the following estimationerror

error =⟨∑

kk · |�non−PPk

− �truek| −

∑kk · |�PP

k− �true

k|∑

kk ·�truek

⟩(21)

where the angular brackets denote averaging over 100 realizations,�(non−)PP

kis the rate of synchronous spiking of order k estimated by

EDP for (non-)Poisson processes, and �truek

is the actual rate in theconsidered data set. Note that our measure also accounts for theestimation error of EDP on Poissonian data: it measures the meanfraction of the misestimated overall rate (the population spike rateis

∑kk ·�k), or number of spikes, which is due to the deviation of

the population spiking from the CPP model.We determined the estimation error for populations of lognor-

mal processes with different spiking irregularity, but the samecorrelation structure. For almost all values of CV in the range[0.3, 3.5] we find a strong bin size dependence of the results (seeFig. 7A). However, the estimation error does not only depend on thebin size, but also on the CV. For very irregular spiking and spikingwith a coefficient of variation close to 1 the error is considerablelarger than for processes with CV close to 2 or to 0.3.

The dependence of the error on the coefficient of variation isdifferent when the spike activity is not simulated as lognormal pro-cesses but as gamma processes (Fig. 7C) or Poisson processes withdead time (PPDs, 7D). Note that all remaining parameters regardingfiring rate and correlation structure of the analyzed populations areidentical. We conclude that the CV alone is not sufficient to measurethe influence of non-Poissonian spiking on the estimation error.

While the estimation error has the same order of magnitudefor all three types of point processes it is considerably reducedin Fig. 7B. In this example, the same correlation structure is real-ized by more neurons. Increasing the population size threefold and,hence, decreasing the spike rate of each neuron three-fold, leads toa maximal error which is approximately three times smaller.

3.2.2. Lower bound for the maximal order of correlationIn order to investigate the effect of the bin size also on the maxi-

mal order of correlation �, we quantify its error by the average ratiofor non-Poissonian and Poissonian populations, �nonPP/�PP. As canbe seen in Fig. 8A, for our population of 15 correlated lognormalprocesses � tends to be underestimated for CV < 2, and overesti-mated for CV > 2. While for small bin sizes the error is negligible,for larger bin sizes it can be substantial. A similar degree of under-estimation is obtained for gamma processes (Fig. 8C) and PPDs(Fig. 8D). As observed before, the precise dependence of the erroron the coefficient of variation varies with the point process modelemployed.

In line with our findings regarding the estimated correlationstructure, the consequences of non-Poissonian firing for the maxi-mal order of correlation are less pronounced when the populationsize is increased, without changing the correlation structure (com-pare Fig. 8A and 8B). Furthermore, the value of CV marking theturning point between under- and overestimation of � is shifted tothe right for N = 45 as compared to N = 15.

In summary, we find that for small bin sizes both the correlationstructure and the maximal order of correlation is well estimatedalso for non-Poissonian processes. However, the actual estimationerror depends non-trivially on various factors (type of inter-spikeinterval distribution, coefficient of variation, population size and

spike rate) and, hence, can hardly be predicted for correlated pro-cesses not considered here. Consequences and conclusions of ourresults are drawn in the following section and discussed in Section4.2.

26 I.C.G. Reimer et al. / Journal of Neuroscience Methods 208 (2012) 18–33

Fig. 6. Distribution of population spike count (A), corresponding (estimated) correlation structure (B) and inferred maximal order of correlation (C). (A) Population spikecount of 15 lognormal processes with a coefficient of variation CV = 0.65 (non-PP, dark blue) and Poisson processes (PP, light blue) for different bin sizes h (hatched bars:2 ms; filled bars: 10 ms) averaged over 100 realizations of 500 s each. Apart from the type of inter-spike interval distribution the populations have identical statistics, witheach neuron having a spike rate of 6 Hz. (B) The underlying correlation structure is indicated by red bars, which corresponds to a pairwise coincidence probability of cp = 0.05.B n spikb and ni acros

3

tmm

FtCt�ic

lue bars show the estimates of these true rates obtained by EDP from the populatioy EDP is depicted in dependence of the bin size h for Poisson processes (light blue)

s depicted by red horizontal bars on the left. Error bars indicate standard deviation

.3. Validation of empirical de-Poissonization in neuronal data

As illustrated by our robustness studies, a deviation fromhe model assumptions can, under certain conditions, lead to a

arked misestimation of higher-order correlations and hence, to aisinterpretation of experimental data. Therefore, a strategy is

A 0.5 1 1.5 2 2.5 3 3.5

0

0.05

0.1

0.15

0.2

CV

erro

r

B 0.5

4 2.77 2.04 1.56 1.23 1

C

α

0.5 0.6 0.7 0.8 0.9 1

0

0.05

0.1

0.15

0.2

erro

r

CV

116

D 0.3

ig. 7. Error in estimating the correlation structure when model assumptions are not fulhe error measure (Eq. (21)). This bias is depicted for different populations of renewal pr). Results are shown in dependence of the coefficient of variation (CV) for various bin wo 20 ms (light blue line). A: Population of N = 15 correlated lognormal processes with fir= 2 Hz. C: Population of N = 15 correlated gamma processes with � = 6 Hz. Additionally,

ndicated. D: Population of N = 15 correlated Poisson processes with dead time with � =orresponding to CV is marked.

e count (same coding as in A). (C) The mean maximal order of correlation estimatedon-Poissonian processes (dark blue). The corresponding true correlation structures independent trials.

needed to either avoid the misestimation a priori, or, if not possible,to at least quantify its degree a posteriori.

Our findings suggest to choose the bin width of the populationhistogram small in order to get reliable results. However, there isa lower bound on the bin size to render the method insensitive toimprecise coincidences (see Section 3.1). Besides, we saw exam-

1 1.5 2 2.5 3 3.5

CV

0163350668399

d [ms]

0.4 0.5 0.6 0.7 0.8 0.9 1

CV

h

1ms

2ms

4ms

6ms

8ms

10ms

12ms

14ms

16ms

18ms

20ms

filled. The bias due to the violation of the CPP/Poisson assumption is quantified byocesses with identical correlation structures as illustrated in Fig. 6 (for details seeidths h of the population histogram. The bin size ranges from 1 ms (dark red line)ing rate � = 6 Hz each. B: Population of N = 45 correlated lognormal processes withthe order parameter ˛ of the gamma distribution (cf. A.2) corresponding to CV is

6 Hz. Additionally, the dead time d of the inter-spike interval distribution (cf. A.1)

I.C.G. Reimer et al. / Journal of Neuroscience Methods 208 (2012) 18–33 27

A 0.5 1 1.5 2 2.5 3 3.5

0.8

1

1.2

1.4

CVξ

no

nP

P /ξ

PP

B 0.5 1 1.5 2 2.5 3 3.5

CV

4 2.77 2.04 1.56 1.23 1

C

α

0.5 0.6 0.7 0.8 0.9 1

0.8

1

1.2

1.4

ξn

on

PP /

ξP

P

CV

0163350668399116

D

d [ms]

0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

CV

h

1ms

2ms

4ms

6ms

8ms

10ms

12ms

14ms

16ms

18ms

20ms

Fig. 8. Mean ratio of the lower confidence bound on the maximal order of correlation of non-Poissonian processes (�nonPP) and Poisson processes (�PP). Same populations asi variatc

podm

fEso

Ff2aufdTc

n Fig. 7 have been analysed. Results are shown in dependence of the coefficient ofolor coding as in Fig. 7.

les where bin sizes of 5 ms yielded reasonable results but alsones where bin sizes of even 20 ms were acceptable. Hence, it isifficult to decide a priori which bin size is appropriate to avoid aisestimation of higher-order correlations.As a consequence, we need a possibility to assess the per-

ormance of EDP. In our robustness studies the performance of

DP could easily be checked because the underlying correlationtructure was known. This is not the case in applications wherenly the experimental data is given, and we will circumvent this

A I. Analysis of neuronal data

ν=(ν1,...,ν

N)

(unknown)EDP ν

fit fA

binomial νfit

?

II. Imit

IV. Evalua

3 5 7 9 11 13

0

2

4

6

8

10

n

ννν fi t

1

0

50

100

150

200

250

B

νn

[Hz]

ig. 9. Validation of empirical de-Poissonization in neuronal data. (A) Concept. Step I coor the true unknown correlation structure � = (�1, �2, . . ., �N) (pink). Thereafter, one imit.2.3) correlation structure �fit to the estimated one � and the estimation of various statistre generated, and the higher-order correlations estimated (�sur , dark blue) (step III). Thnknown � and the estimated � (dashed arrow, step IV). (B and C) Example, same color cour disjoint cell assemblies (see Fig. 4, right column, and D). The correlation structure esetails on the fitted correlation structure �fit (yellow bars), see D and text. Surrogate datahe average results by EDP with a bin size of 5 ms are depicted by the dark blue bars for lorrelation structure. Error bars denote standard deviation.

ion (CV) for various bin widths h of the population histogram. Order of panels and

problem by the use of surrogate data. We will first propose themain procedure (Fig. 9A) before we illustrate its performance withan example (Fig. 9B and Fig. 9B).

3.3.1. General procedure

Validation of EDP in neuronal data proceeds in four steps.

First, one determines the EDP estimate of the correlationstructure, � = (�1, �2, . . .), from the given data (step I, Fig. 9A, left).The task then is to find a way to generate surrogate data that mimic

ation of neuronal data

use νfit, λ, CV

ISI−distribution

III. Analysis of surrogate data

νsur νsur^EDP

tion of results

3 5 7 9 11 13

0

2

4

6

8

10

n

ν s u r

ν s u r

ν s u r , P P

1

0

50

100

150

200

250

C

νn

[Hz]

nsists of the application of EDP to neuronal data to obtain an estimate � (purple)ates the neuronal data (step II). This implies to fit a realizable (according to sectionics of the experimental spike trains. Surrogate data with correlation structure ≈�fit

e comparison of �sur with �sur gives an approximation of the relation between theoding. Shown is the true correlation structure � (pink bars, B) of a population withtimated by EDP for a bin size of 5 ms from a 200 s sample is depicted in purple. Forwith true correlation structure �sur (red bars, C) have been simulated for 100 times.ognormal processes, and by light blue bars for the Poisson processes with identical

2 urosci

tutcftTca3esolw�mmv

d

Toswdls

3

tc

3EI

3iditstHbsa�fiwtcptfinrth�s

8 I.C.G. Reimer et al. / Journal of Ne

he basic statistical features of the original data, particularly thenknown correlation structure �, as close as possible (step II, cen-er). We consider the scenario where the experimental spike trainsan be approximated by a renewal process with a bounded hazardunction. Corresponding surrogate data can then be simulated byhinning correlated Poisson processes as described in Section 2.2.3.he correlation structure of the latter has to be chosen such that theorrelation structure resulting after thinning, �sur, matches � as wells possible (realized by the fit of fAbinomial

in Fig. 9A, left, see Section.3.2 for more details). Finally, one determines the respective EDPstimate �sur for each surrogate data set generated according to thatcheme (step III, Fig. 9A, right). On repeating this step many times,ne obtains a Monte Carlo estimate of the distribution of �sur (andikewise of the difference �sur − �sur, or other related statistics),

hich then serves as a substitute for the unknown distribution ofˆ one is actually interested in (step IV, dashed arrow in Fig. 9A). A

eans appropriate for assessing the bias of the EDP estimates underodel misspecification would be the distribution of the random

ariable

(�, �) =(∑

k · |�k − �k|)

/(∑

k ·�k

). (22)

he closer it is concentrated near zero, the better the performancef EDP. Of course, this distribution is unknown in applicationsince � is unknown. However, using our surrogate data methode may estimate it by the empirical distribution of the quantities

(�sur, �sur(j)) where �sur(j) denotes the EDP estimate of the corre-ation structure obtained in the j-th simulation of a surrogate dataet.

.3.2. ExampleAn example is shown in Fig. 9 (B and C). The pink bars (left) depict

he correlation structure of an irregularly spiking population whichonsists of four disjoint cell assemblies as in Fig. 4 (right column).

.3.2.1. Step I. The correlation structure � has been estimated byDP from a 200 s sample using a bin size of 5 ms (purple bars, 9B).t slightly deviates from the true correlation structure.

.3.2.2. Step II. As in a real data scenario where this comparisons not possible, we validate the performance of EDP by the proce-ure outlined above. Initially, several parameters of the population

n question have to be estimated: The number of neurons N andheir firing characteristics, in particular their firing rates and inter-pike interval irregularity. Then we generate surrogate data withhese parameters, including the estimated correlation structure �.owever, as has been described in Section 2.2.3 it is not possi-le to generate non-Poissonian processes with arbitrary correlationtructure by our procedure. Therefore, we have to approximate � bycorrelation structure �fit which can be simulated. In our example,

ˆ looks like a distribution with two peaks for �2, . . ., �N. Hence, wetted the correlation structure �fit (yellow bars) defined by Eq. (13)ith Nsyn = 2 to � (see Appendix D for more details). This fit yields

he parameters needed to construct the correlated Poisson pro-esses, which then have to be thinned to obtain the non-Poissonianrocesses with correlation structure �fit. In doing so, we assumedhat we had properly estimated the number of neurons and identi-ed the inter-spike interval distribution as lognormal. However, weeglected the fact, that we had a heterogeneous population withespect to the correlation structure because EDP wouldn’t allowhis inference from real data either. Furthermore, we simulated a

omogenous population of lognormal processes with uniform rate= 〈�i〉 and coefficient of variation CV = 〈CVi〉, 1≤ i≤N, i.e. the mean

pike statistics of all neurons.

ence Methods 208 (2012) 18–33

3.3.2.3. Step III. The resulting surrogate data has a mean correlationstructure �sur (red bars in Fig. 9C). Since Eq. (8) is only an approxima-tion, �fit≈�sur. This surrogate data has been generated and analyzedwith EDP for 100 times.

3.3.2.4. Step IV. For our particular example, a comparison of thetrue correlation structure (�sur, red bars) and the estimated one(�sur , dark blue bars) of the surrogate data suggests that the truecorrelation structure in the neuronal data is slightly misestimated,but that its overall shape is approximated quite well. As outlinedabove, the degree of misestimation can be quantified via the errormeasure d(�, �) (Eq. (22)). Assuming consistency and normality, a95% confidence interval for the deviation of � from � (i.e. d(�, �)) isapproximately given by the mean of d(�sur

k, �sur

k) plus/minus twice

its standard deviation. This corresponds to [0.09, 0.33] in our exam-ple and hence contains the value of 0.16 which we obtained for theneuronal data set.

In order to get an idea how well EDP would have performedif model assumptions were fulfilled, one can additionally gener-ate and analyze Poisson processes with a correlation structure �sur

identical to the one of the non-Poissonian processes. In our partic-ular example, with 〈d(�sur

k, �sur,PP

k)〉 = 0.2± 0.07 the results for the

Poisson processes (light blue bars) are of a quality similar to the onefor lognormal processes. Taken together with the rather large errorbars of the rate estimates, this suggests that the misestimation inthe neuronal data might be rather due to a small sample size, andless due to a deviation from model assumptions.

4. Discussion

We proposed a novel method to simulate non-Poissonian pointprocesses with predefined higher-order correlations. We employedthe new method to investigate the impact of single cell spikingstatistics on the inference of higher-order correlations from empir-ical spike data. Furthermore, we outlined a strategy to assess thereliability of results obtained by empirical de-Poissonization onexperimental data.

We will discuss the limitations and possible extensions of ourapproach to generate correlated spike trains and briefly compare itwith methods that have recently been proposed in other contexts.Furthermore, we will contrast the results of our robustness studywith those of others as far as possible, and explain the relevanceand consequences of our findings for the analysis of biological spiketrains.

4.1. Generation of non-Poissonian processes with higher-ordercorrelations

4.1.1. Limitations and possible extensionsThe method proposed here is based on the deletion of Poisson

events. A bounded hazard function of the target process is required,such a property is shared by various renewal processes commonlyemployed in neuroscience. However, irregular gamma processeswith a coefficient of variation of the ISI distribution larger than onerepresent a counter-example. Yet its overrepresentation of shortinter-spike intervals is inconsistent with neuronal refractory peri-ods and, hence, it is not first choice for mimicking irregular spikingin any case.

There are some constraints with respect to the across-neuronscorrelation structures that can be simulated employing the pro-posed method. First of all, we are restricted to positive correlations.That is, within our framework only surplus coincidences of spikes

can be realized, but not a systematic lack of spikes, e.g. due to adirect inhibitory effect of one neuron on another one. This is due tothe fact that based on their construction the Poisson processes canonly be positively correlated (cf. Bäuerle and Grübel, 2005; Johnson

urosci

attcscaodissinrdsdTAhpt

senetdr

pvsnnc2TgaIdpPesaorxtnnfs

4

rTptvo

I.C.G. Reimer et al. / Journal of Ne

nd Goodman, 2009), and these correlations are only weakened byhinning. While there is no way to overcome this constraint withinhe framework proposed here, the issue of imprecise coincidencesan be addressed in our model by adding a random jitter to eachpike time. In a similar way, elaborated shifting of the source pro-esses against each other in time can yield precise firing patternscross neurons (Abeles et al., 1993; Prut et al., 1998). Moreover,ur method does not allow to the realization of arbitrary amplitudeistributions in combination with non-Poissonian processes which

s possible for Poisson processes. For instance, we are not able toimulate a population of lognormal processes where synchronouspiking only occurs in a fixed number � of neurons (i.e. �k /= 0 onlyf k ∈

{1, �

}). Although the shape of amplitude distributions in real

euronal networks is unknown, it is quite unlikely that this scenarioepresents a biologically relevant example. Instead, an amplitudeistribution with many nonzero entries is to be expected. Anduch a case can be realized by our method. Admissible amplitudeistributions are superpositions of several binomial distributions.his allows a great variety of correlation structures to be modeled.lthough the binomial approximation of the correlation structureolds only for weak interactions, recent reports about very smallairwise correlations (Ecker et al., 2010; Renart et al., 2010) suggesthat our description fits biologically relevant parameter ranges.

We have outlined the generation of correlated spike trains withpike statistics which vary across neurons (see Section 2.2.3). How-ver, a general approach to simulate a heterogeneous population ofon-Poissonian processes with predefined correlation structure isxpected to be rather difficult: As explained in Brette (2009), evenhe construction of correlated Poisson processes with arbitrarilyefined single-neuron rates and pairwise correlations without anyestrictions on the higher-order correlations can be cumbersome.

In this study, we focused on the generation of non-Poissonianrocesses in a stationary regime. However, neurons exhibit time-arying spike rates especially when they are driven by an externaltimulus. Hence, generalizing our method to non-stationary sce-arios is of major importance. While higher-order correlations inon-stationary spike trains have recently been considered for pro-esses constituting a compound Poisson process (Staude et al.,010b), here an extension is required for non-Poissonian processes.wo possible solutions are worth mentioning. The first idea is toenerate correlated non-Poissonian processes in operational time,nd to apply nonlinear time warping in a second step (Cox andsham, 1980; Brown et al., 2002). If the correlation structure isefined by (exact) coincidences in operational time, they will bereserved if they are subject to the same time warp. Hence, non-oissonian processes exhibiting the same non-stationarity can bequipped with a predefined higher-order correlation structure. Theecond ansatz is based on the fact that the thinning procedurepplied to Poisson processes can be adapted straightforwardly tobtain non-stationary non-Poissonian processes. A non-stationaryenewal process is determined by its hazard function �(x, t), wheredenotes the time elapsed since the last event, and t is the clock

ime (cf. Kass and Ventura, 2001). Thus, the generation of correlatedon-stationary non-Poissonian processes is possible again via thin-ing of correlated Poisson processes by use of a 2-parameter hazard

unction. However, it remains to be explored how the correlationtructure is affected by the thinning procedure.

.1.2. Comparison with existing methodsOur approach to simulating correlated spike trains is weakly

elated to the one suggested by Baker and Gerstein (2000).hey generated correlated non-Poissonian processes via non-

robabilistic thinning of correlated Poisson processes. However,heir procedure was restricted to gamma processes with integer-alued order parameters (this implies CV < 1), and they considerednly two parallel processes. Here, we have adapted this idea to more

ence Methods 208 (2012) 18–33 29

than two processes (Staude et al., 2007, cf.][for an extension of theiransatz to higher-order correlations). The method described here,however, employs independent probabilistic thinning, which canbe applied to any renewal process with a bounded hazard function.

Within the last years various methods have been suggested togenerate populations of correlated spike trains (e.g. Bohte et al.,2000; Kuhn et al., 2003; Niebur, 2007; Brette, 2009; Krumin andShoham, 2009; Macke et al., 2009; Onken et al., 2009; Onken andObermayer, 2009; Gutnisky and Josic, 2010; Lyamzin et al., 2010;Krumin et al., 2010). However, in most approaches higher-ordercorrelations merely occur as a by-product. Sometimes, the result-ing probability structure can be mathematically described as e.g. inKrumin and Shoham (2009). Only the copula based framework (e.g.Onken et al., 2009; Onken and Obermayer, 2009) allows the controlof both second-order and higher-order interactions. In contrast toour work, they can also generate negative correlations. However,spike counts, rather than spike times, are targeted and, therefore, itis not clear what the corresponding spike trains would look like. Inturn, it is to be expected that for a given type of spike trains definedin continuous time the counting properties strongly depend on thebin size used to represent them (Tetzlaff et al., 2008). We wouldlike to stress that the different frameworks of correlated spike trainmodeling also adopt different concepts of higher-order correlationsand, hence, the choice of any particular model must be carefullyjustified in view of its envisaged application (Staude et al., 2010a).

4.1.3. ApplicationAlthough neuronal spike activity is known to deviate from Pois-

sonian statistics (see e.g. Kuffler et al., 1957; Beyer et al., 1975;Burns and Webb, 1976; Levine, 1991; Amarasingham et al., 2006;Nawrot et al., 2008; Minich et al., 2009; Maimon and Assad,2009; Shinomoto et al., 2009), Poisson processes are neverthe-less often employed in modeling and for analyzing experimentaldata. Recently, the need to consider non-Poissonian spiking toinvestigate neural computation has been pointed out (Câteau andReyes, 2006; Lindner, 2006; Ly and Tranchina, 2009; Deger et al.,2011). Our proposed method enables the simulation of such spiketrains. While we have focused here on higher-order correlations,our method also contributes to the available possibilities to gener-ate pairs of correlated neurons within the renewal framework (seeabove) and hence allows to study e.g. the transfer of pairwise cor-relations in a richer and biologically more realistic environment (cf.Rosenbaum and Josic, 2011).

Here, we used non-Poissonian processes with higher-order cor-relations to investigate the impact of non-Poissonian spiking on theestimation and inference of higher-order correlations by empiri-cal de-Poissonization. Our point process model can, of course, alsobe employed for the analysis of the sensitivity and robustness ofother methods, which aim to detect coordinated spike activity (ase.g. Grün et al., 2002a,b; Pipa et al., 2008; Staude et al., 2010b,c;Lopes-dos Santos et al., 2011).

4.2. Impact of single neuron spike statistics on the estimation andinference of higher-order correlations

4.2.1. Comparison to other resultsOur robustness study showed that non-Poissonian spike trains

only weakly distort the estimation of higher-order correlationswhen using very small bin sizes. This finding reflects the fact that,for very small bin sizes non-zero bin counts of a single spike trainare sparse, and the corresponding count distribution is very close to

a Poisson distribution. However, non-Poissoninan spike trains mayalso imply serial correlations between consecutive spike counts ifthe bins are not chosen large enough, and this property can be moreimportant for other analysis tools (see Roudi et al., 2009).

3 urosci

imbinSTbf2mcslm

ga(PbnUbtqrpsw

rhnrP

4

scpre2sdae

oua(tsdocatPd

e

0 I.C.G. Reimer et al. / Journal of Ne

To our knowledge, there are currently no other studies whichnvestigate the impact of single-neuron spike statistics on the esti-

ation and inference of higher-order correlations. While the worky Onken et al. (2009) does treat higher-order correlations, their

nvestigation of the importance of marginal spike statistics doesot directly incorporate correlations. Rather, they considered thehannon information of the single neuron spike count distribution.hey found that violations of the Ising model assumption (i.e. ainomial distribution of the single-neuron spike count), which isrequently exploited in correlation analysis (e.g. Schneidman et al.,006; Shlens et al., 2006), result in notable deviations of the esti-ates from the true entropy. In agreement with our findings, they

oncluded that it is important to use the proper single-neuronpike count distribution. Due to the different frameworks under-ying their and our studies, it is difficult to compare the results in

ore detail.The robustness study reported in Grün (2009) and the investi-

ations performed in Tetzlaff et al. (2008) are more in line with ournalysis, though restricted to pairs of spike trains. Unitary eventUE) analysis (Grün et al., 2002b,a), which assumes independentoissonian firing in its null-hypothesis, exhibits slightly differentehavior for very irregular spiking where, in contrast to our results,o false positives occur. This is due to the fact that, contrary to EDP,E operates on binned and clipped spike activity; i.e. spikes elicitedy a neuron within one time bin are counted as one event irrespec-ive of the actual number of spikes. Tetzlaff et al. (2008) revealed aualitatively similar bin size dependence for Pearson’s count cor-elation coefficient as an estimator for the pairwise coincidencerobability just as we found for our higher-order correlation mea-ures: in the case of non-Poissonian spiking the estimates becomeorse for larger bin sizes.

An alternative to EDP for estimating the maximal order of cor-elation is a method called CuBIC (Cumulant Based Inference ofigher-order Correlations; Staude et al., 2010b,c). Here we showo full analysis of CuBIC’s performance, but initial investigationsevealed that CuBIC (Staude et al., 2010c) can be affected by non-oissonian spiking in a qualitatively similar way as EDP.

.2.2. Relevance and consequences of our findingsThe parameter range, in which we investigated the impact of

ingle-neuron spike statistics on the estimation of higher-orderorrelations, has been chosen to be as biologically reasonable asossible. That is, we matched the coefficient of variations to thoseeported by various studies (e.g. Softky and Koch, 1993; Nawrott al., 2007, 2008; Maimon and Assad, 2009; Ponce-Alvarez et al.,010) where we also considered ranges of much more irregularpiking (i.e. CV > 2). Spike rates have been set to rather small valuesue to recent reports on very low firing rates in both spontaneousnd evoked activity (Lee et al., 2006; Hromádka et al., 2008; Haidert al., 2010; Wolfe et al., 2010).

Hence, care must be taken in interpreting measured higher-rder correlations, especially when comparing results obtainednder different stimulus conditions and from different corticalreas, where the neuronal spiking irregularity seems to varyMaimon and Assad, 2009; Shinomoto et al., 2009). In order to keephe estimation bias as small as possible it is recommended that amall bin width is chosen. This is suggested by our results on bin sizeependence, which is in line with the theory that the superpositionf independent renewal processes locally constitutes a Poisson pro-ess (see e.g. chapter 6 in Cox (1962) and references therein, as wells e.g. Lindner (2006)). That is, one can always chose the bin sizeo be so small that the superposed spike count is approximately

oisson distributed, and the larger the bins chosen the more oneeviates from this condition.

However, as we pointed out in Section 3.3, this can neitherntirely solve all remaining problems, nor is there a need for this.

ence Methods 208 (2012) 18–33

Coordinated neuronal activity has only a certain degree of preci-sion which has to be accounted for by a sufficiently large bin size.Besides, the degree of misestimation can be negligible even forrather large bin sizes, as we found for the examples of large sparselyspiking populations. Instead, we therefore suggest the use of ourproposed method to evaluate the reliability of results by EDP forthe analysis of some experimental data.

4.3. Validation of results by empirical de-Poissonization inneuronal data

While it is common to use surrogate data to judge the sig-nificance of correlations and spike patterns (see e.g. Gerstein(2004), Grün (2009), Louis et al. (2010b,a)) we had to adapt theseapproaches as they are based on the assumption that the null-hypothesis includes independent spiking. In contrast, by usingEDP we either directly estimate the synchronized activity of var-ious orders, or we infer a lower bound for the maximal order ofcorrelation via successively testing null-hypotheses, assuming cor-relations only up to a certain maximal order. That is, consideringindependent processes is insufficient, and we outlined how corre-lations can be taken into account as well.

We proposed to characterize the performance of EDP for aparameter set matching the experimental data and concluded thatour results for the neuronal data can be trusted to a similar degree.In doing so, we assume that the results by EDP change smoothlywithin the large parameter space of non-Poissonian firing, affirmedby our robustness studies. Furthermore, as in other approaches, theperformance of our method depends on the reliability of the estima-tion of single-neuron spike statistics. However, as we showed by anexample, the quality of the results can already be well judged basedon population averaged spike rates and coefficient of variations.

We introduced a method to assess the estimated correlationstructure. With the help of additional Poisson processes it can, ofcourse, also be adapted for the evaluation of the inferred maximalorder of correlation (cf. Section 3.2.2).

4.4. Conclusions

We presented a novel method to generate non-Poissonian pro-cesses with defined higher-order correlations. Furthermore, weused data generated by this technique to perform a robustnessstudy with empirical de-Poissonization (EDP, Ehm et al., 2007).The results emphasize the need to carefully calibrate estimators ofhigher-order correlations prior to their application. More precisely,the single neuron spike statistics represent crucial parameterswhich have to be taken into account when interpreting measure-ment for surrogate and experimental data. As a consequence of ourfindings, we ultimately proposed a method to assess the reliabilityof results obtained by EDP.

Acknowledgments

We thank A. Jasper, F. Siegfried and D. Suchanek for helpfulcomments on an earlier version of this manuscript. Supportedby the German Ministry for Education and Research (BMBF grant01GQ0420 to the BCCN Freiburg), the IGPP Freiburg and the GermanResearch Foundation (DFG SFB 780).

Appendix A. Renewal processes and its hazard functions

Our proposed method to generate non-Poissonian processeswith higher-order correlations requires that the hazard functionof the renewal process is bounded (see Section 2.1.3). We describe

urosci

ho

A

fa

f

a

�

wov

�

A

d

f

wbv

˛

Tcms

A

r

f

w

�

Itan

Ao

pt

eHtc

P

I.C.G. Reimer et al. / Journal of Ne

ere the necessary details of the processes which we considered inur manuscript.

.1. Poisson process with dead time

The inter-spike interval distribution f�p,d(x) and the hazardunction ��p,d(x) of a Poisson process with dead time read formallys

�p,d(x) = �pe−�p(x−d)1[d,∞[(x) (A.1)

nd

�p,d(x) = �p1[d,∞[(x), (A.2)

here 1A(x) equals 1 if x∈A and 0 otherwise. Hence, the maximumf the hazard function is �p. The spike rate � and the coefficient ofariation CV relate to the parameters �p and d via:

p = �

CVand d = 1− CV

�. (A.3)

.2. Gamma processes

The inter-spike intervals follow a gamma distribution which isefined as

˛,ˇ(x) = ˇ(ˇx)˛−1e−ˇx

�(˛), (A.4)

here � denotes the gamma function. The parameters ˛ and ˇ cane expressed in terms of the spike rate � and the coefficient ofariation CV via

= 1

CV2and ˇ = �

CV2. (A.5)

he hazard function of gamma processes decreases monotoni-ally for CV > 1 where �˛,ˇ(x)→∞ as x→0. In contrast, it increasesonotonically for CV < 1 with �˛,ˇ(x)→ˇ as x→∞ (Cox, 1962,

ee][p. 20).

.3. Lognormal processes

The inter-spike interval distribution of lognormal processeseads as

�,�(x) = 1

x�√

2e−(ln(x)−�)2

2�2 , (A.6)

here

= − log �− 12

ln(CV2 + 1) and � =√

log(CV2 + 1). (A.7)

ts hazard function begins always at zero, rises to a maximum andhen decreases to zero very slowly. This maximum does not haveclosed analytical expression and, hence, needs to be determinedumerically (Sweet, 1990, see][for more details).

ppendix B. Lower confidence bound for the maximalrder of correlation �

Here we demonstrate that under the CPP model � =min{m |m > ˛} − 1 is an approximate level-˛ lower confidence limit forhe maximal correlation order � (cf. Section 3.1.2).

For simplicity we assume that the test statistic Tm = m/�m isxactly standard normally distributed under the null-hypothesism0 (so that the p-value pm from Eq. (20) is exactly uniformly dis-

ributed on the unit interval). It has to be shown that whatever theorrelation structure � = (�1, �2, . . .) underlying the CPP, one has

�

(� ≤ �

)≥ 1− ˛.

ence Methods 208 (2012) 18–33 31

The first step is a basic fact about confidence regions. Let theobservation x be distributed as P� for some unknown parameter� ∈�. Furthermore, let A(�0) be the acceptance region of somelevel-˛ test for H0 : � = �0 versus H1 : � /= �0, for every �0 ∈�. ThenC =

{� ∈� | x ∈ A(�)

}is a level-˛ confidence region, i.e.

P�(C � �) = 1− ˛ for every � ∈�

(see Section 3.5 in Lehmann, 1959).Next, consider a parameter of interest ω = ω(�), that is, a real-

valued function of �. For every ω0 let A(ω0) denote the acceptanceregion of some level-˛ test of the hypothesis H0 : ω≤ω0 versusH1 : ω > ω0. Furthermore, define ω−(x) = inf

{ω | x ∈ A(ω)

}.

Then

P�(ω−(x) ≤ ω(�)) ≥ 1− ˛ for every � ∈�.

Proof: For every � ∈� it holds thatω−(x)≤ω(�)⇔∃ω≤ω(�) : x∈A(ω)⇔ x∈

⋃ω≤ω(�)A(ω) . Hence,

P�(ω−(x) ≤ ω(�)) = P�(x ∈⋃

ω≤ω(�)A(ω))≥ P�(x ∈ A(ω(�))) = 1− ˛.

Our present case corresponds to the special case where theparameters are � = � = (�1, �2, . . .), ω =min

{m |

∑k≥m�k = 0

}.

Then ω = � + 1, and in view of Definition (17) the proof is complete.

Appendix C. Simulating populations of point processeswith identical correlation structures

The non-Poissonian spike trains have been simulated with awarm-up time of min

{10/� · (1+ |1− CV |), 10

}. The parameters

of the source Poisson processes have been chosen such that forall target populations the pairwise coincidence probability cp wasequal to 0.05 and �peak of the binomial approximation of the cor-relation structure was equal to 3. That is, we chose �syn = 15 andRm = 5 ·� where � = 6, 2, 1 for the populations of size N = 15, 45, 90.Note that this value for Rm lies below the maximum of the hazardfunction of gamma processes with � = 6 only for CV≥0.4472 andhence more regular processes could not be generated.

In doing so, the resulting correlation structures are quite similarto each other. More precisely, with l1(fA1 (k), fA2 (k)) =

∑k|fA1 (k)−

fA2 (k)| measuring the distance between two amplitude distribu-tions, the correlation structures of processes with the same Nand inter-spike interval distribution differed on average betweenl1 = 0.0052 and l1 = 0.0062. Thereby, the value did not vary muchwith the CV. The mean distance across all correlation structureswas l1 = 0.0066. In order to obtain comparable Poisson processes wedetermined the true correlation structure for the non-Poissonianspike trains in each of the 100 trials and generated Poisson processwith the exact same number of spikes and higher-order correla-tions.

Appendix D. Fitting the estimated amplitude distribution

In Section 3.3 the same parameter setting has been used with asimulation time of 200 s. Let fA(k) = �k/

∑n�n denote the amplitude

distribution with correlation structure (�1, �2, . . . , �N) estimatedvia EDP. To this we fitted an amplitude distribution with two peaks

fAbinomial(k) =

� · �Rm· ı1,k +

∑2i=1ωi(1− �)B(k|�i

syn, �Rm

)

� · �Rm+

∑Nk=1

∑2i=1ωi(1− �)B(k|�i

syn, �Rm

). (D.1)

More precisely, we solved numerically:

(Rm, ω1, ω2, �1syn, �2

syn) = arg min∑

k

(fAbinomial(k)− fA(k)) (D.2)

respecting the following settings and constraints

3 urosci

i

i

v

woaii

R

A

A

A

B

B

B

B

B

B

B

B

B

B

C

C

CC

D

D

D

D

E

E

E

G

G

G

2 I.C.G. Reimer et al. / Journal of Ne

i) � : = 〈�i〉, CV : = 〈CVi〉ii) cp := cp, where cp is given by Eq. (5) and fA,

ii) � =∑

iωi · �i · (�i− �

Rm−cp(N−1))∑

iωi · �i · (�i− �

Rm−cp(N−1))+cp(N−1) · �

Rm

v) Rlowm ≤ Rm ≤ � ·

∑iωi · �i

syn · (�isyn−1)∑

iωi · �i

syn · cp(N−1), where Rlow

m =maxx

��,CV(x)

v)∑

iωi = 1

i) �isyn ≤ N

here �i and CVi denote the firing rate and coefficient of variationf the i-th neuron, respectively, and ��,CV(x) the hazard function oflognormal process with these averaged spike statistics. Property

ii) is a combination of Eq. (5) and fAbinomial. The second inequality in

v) follows from �≥0.

eferences

beles M, Bergman H, Margalit E, Vaadia E. Spatiotemporal firing patterns in thefrontal cortex of behaving monkeys. J Neurophysiol 1993;70(4):1629–38.

marasingham A, Chen TL, Geman S, Harrison MT, Sheinberg DL. Spike count relia-bility and the poisson hypothesis. J Neurosci 2006;26(3):801–9.

mari SI, Nakahara H, Wu S, Sakai Y. Synchronous firing and higher-order interac-tions in neuron pool. Neural Comput 2003;15(1):127–42.

aker SN, Gerstein GL. Improvements to the sensitivity of gravitational clusteringfor multiple neuron recordings. Neural Comput 2000;12(11):2597–620.

ar-Gad I, Ritov Y, Vaadia E, Bergman H. Failure in identification of overlappingspikes from multiple neuron activity causes artificial correlations. J NeurosciMethods 2001;107(1-2):1–13.

äuerle N, Grübel R. Multivariate counting processes: copulas and beyond. AstinBull 2005;35(2):379.

enucci A, Verschure PFMJ, Koenig P. Dynamical features of higher-order correlationevents: impact on cortical cells. Cogn Neurodyn 2007;1(1):53–69.

erger D, Warren D, Normann R, Arieli A, Gruen S. Spatially organized spike corre-lation in cat visual cortex. Neurocomputing 2007;70(10-12):2112–6.

eyer H, Schmidt J, Hinrichs O, Schmolke D. A statistical study of the interspike-interval distribution of cortical neurons. Acta Biol Med Ger 1975;34(3):409–17.

ohte SM, Spekreijse H, Roelfsema PR. The effects of pair-wise and higher ordercorrelations on the firing rate of a post-synaptic neuron. Neural Comput2000;12(1):153–79.

rette R. Generation of correlated spike trains. Neural Comput 2009;21(1):188–215.

rown EN, Barbieri R, Ventura V, Kass RE, Frank LM. The time-rescaling theo-rem and its application to neural spike train data analysis. Neural Comput2002;14(2):325–46.

urns BD, Webb AC. The spontaneous activity of neurons in the cat’s cerebral cortex.Proc R Soc Lond B: Biol Sci 1976;194(1115):211–23.

âteau H, Reyes AD. Relation between single neuron and population spiking statis-tics and effects on network activity. Phys Rev Lett 2006;96(5):058101.

ohen MR, Kohn A. Measuring and interpreting neuronal correlations. Nat Neurosci2011;14(7):811–9.

ox DR. Renewal theory. Methuen & Co. Ltd; 1962.ox DR, Isham V. Point processes. In: Monographs on applied probability and statis-

tics. London: Chapman & Hall; 1980.eger M, Helias M, Boucsein C, Rotter S. Statistical properties of superimposed

stationary spike trains. J Comput Neurosci 2011, Epub ahead of print.eger M, Helias M, Cardanobile S, Atay FM, Rotter S. Nonequilibrium dynamics

of stochastic point processes with refractoriness. Phys Rev E: Stat Nonlin SoftMatter Phys 2010;82(2 Pt 1):021129.

evroye L. Non-uniform random variate generation. New York: Springer-Verlag;1986.

iesmann M, Gewaltig MO, Aertsen A. Stable propagation of synchronous spikingin cortical neural networks. Nature 1999;402(6761):529–33.

cker AS, Berens P, Keliris GA, Bethge M, Logothetis NK, Tolias AS. Decorrelatedneuronal firing in cortical microcircuits. Science 2010;327(5965):584–7.

hm W, Staude B, Rotter S. Decomposition of neuronal assembly activity via empir-ical de-Poissonization. Electr J Stat 2007;1:473–95.

hm W, Staude B, Rotter S. Decomposition of neuronal assembly activ-ity via empirical de-Poissonization. In: Frontiers in computationalneuroscience. conference abstract: bernstein symposium 2008; 2008.,http://dx.doi.org/10.3389/conf.neuro.10.2008.01.042.

anmor E, Segev R, Schneidman E. Sparse low-order interaction network underliesa highly correlated and learnable neural population code. Proc Natl Acad Sci U

S A 2011;108(23):9679–84.

erstein GL. Searching for significance in spatio-temporal firing patterns. Acta Neu-robiol Exp (Warsz) 2004;64(2):203–7.

rün S. Data-driven significance estimation for precise spike correlation. J Neuro-physiol 2009;101(3):1126–40.

ence Methods 208 (2012) 18–33

Grün S, Diesmann M, Aertsen A. Unitary events in multiple single-neuron spikingactivity. I. Detection and significance. Neural Comput 2002a;14(1):43–80.

Grün S, Diesmann M, Aertsen A. Unitary events in multiple single-neuron spikingactivity. II. Nonstationary data. Neural Comput 2002b;14(1):81–119.

Gutnisky DA, Josic K. Generation of spatiotemporally correlated spike trains andlocal field potentials using a multivariate autoregressive process. J Neurophysiol2010;103(5):2912–30.

Haider B, Krause MR, Duque A, Yu Y, Touryan J, Mazer JA, et al. Synaptic and networkmechanisms of sparse and reliable visual cortical activity during nonclassicalreceptive field stimulation. Neuron 2010;65(1):107–21.

Hromádka T, Deweese MR, Zador AM. Sparse representation of sounds in the unanes-thetized auditory cortex. PLoS Biol 2008;6(1).

Johnson DH, Goodman IN. Jointly Poisson processes; 2009. ArXiv:0911.2524.Kass RE, Ventura V. A spike-train probability model. Neural Comput

2001;13(8):1713–20.Kohn A, Zandvakili A, Smith MA. Correlations and brain states: from electrophysiol-

ogy to functional imaging. Curr Opin Neurobiol 2009;19(4):434–8.Krumin M, Reutsky I, Shoham S. Correlation-based analysis and generation of mul-

tiple spike trains using hawkes models with an exogenous input. Front ComputNeurosci 2010:4.

Krumin M, Shoham S. Generation of spike trains with controlled auto- and cross-correlation functions. Neural Comput 2009;21(6):1642–64.

Kuffler SW, Fitzhugh R, Barlow HB. Maintained activity in the cat’s retina in lightand darkness. J Gen Physiol 1957;40(5):683–702.

Kuhn A, Aertsen A, Rotter S. Higher-order statistics of input ensembles and theresponse of simple model neurons. Neural Comput 2003;15(1):67–101.

Lee AK, Manns ID, Sakmann B, Brecht M. Whole-cell recordings in freely movingrats. Neuron 2006;51(4):399–407.

Lehmann E. Testing statistical hypotheses. New York: Wiley; 1959.Levine MW. The distribution of the intervals between neural impulses in the main-

tained discharges of retinal ganglion cells. Biol Cybern 1991;65(6):459–67.Lewis PAW, Shedler GS. Simulation of nonhomogeneous poisson processes by thin-

ning. Naval Res Logistics Quart 1979;26(3):403–13.Lindner B. Superposition of many independent spike trains is generally not a poisson

process. Phys Rev E 2006;73(2):022901.Lopes-dos Santos V, Conde-Ocazionez S, Nicolelis MAL, Ribeiro ST, Tort ABL.

Neuronal assembly detection and cell membership specification by principalcomponent analysis. PLoS One 2011;6(6):e20996.

Louis S, Borgelt C, Grün S. Generation and selection of surrogate methods for cor-relation analysis. In: Grün S, Rotter S, editors. Analysis of parallel spike trains.Springer series in computational neuroscience; 2010a. p. 359–82.

Louis S, Gerstein GL, Grün S, Diesmann M. Surrogate spike train generation throughdithering in operational time. Front Comput Neurosci 2010b;4:127.

Ly C, Tranchina D. Spike train statistics and dynamics with synaptic inputfrom any renewal process: A population density approach. Neural Comput2009;21(2):360–96.

Lyamzin DR, Macke JH, Lesica NA. Modeling population spike trains with specifiedtime-varying spike rates, trial-to-trial variability, and pairwise signal and noisecorrelations. Front Comput Neurosci 2010;4:144.

Macke JH, Berens P, Ecker AS, Tolias AS, Bethge M. Generating spike trains withspecified correlation coefficients. Neural Comput 2009;21(2):397–423.

Macke JH, Opper M, Bethge M. Common input explains higher-order correlationsand entropy in a simple model of neural population activity. Phys Rev Lett2011;106(20):208102.

Maimon G, Assad JA. Beyond poisson: increased spike-time regularity across primateparietal cortex. Neuron 2009;62(3):426–40.

Martignon L, Deco G, Laskey K, Diamond M, Freiwald W, Vaadia E. Neural coding:higher-order temporal patterns in the neurostatistics of cell assemblies. NeuralComput 2000;12(11):2621–53.

Martignon L, Hasseln HV, Gruen S, Aertsen A, Palm G. Detecting higher-orderinteractions among the spiking events in a group of neurons. Biol Cybern1995;73(1):69–81.

McKeegan DEF. Spontaneous and odour evoked activity in single avian olfactorybulb neurones. Brain Res 2002;929(1):48–58.

Minich J, Negyessy L, Procyk E, Barone P, Odry P, Bazso F. Statistical and stochasticalproperties of inter spike interval time series. In: Frontiers in Systems Neu-roscience. Conference Abstract: 12th Meeting of the Hungarian NeuroscienceSociety; 2009., http://dx.doi.org/10.3389/conf.neuro.01.2009.04.213.

Montani F, Ince RAA, Senatore R, Arabzadeh E, Diamond ME, Panzeri S. The impactof high-order interactions on the rate of synchronous discharge and informa-tion transmission in somatosensory cortex. Philos Transact A Math Phys Eng Sci2009;367(1901):3297–310.

Muller E, Buesing L, Schemmel J, Meier K. Spike-frequency adapting neuralensembles: beyond mean adaptation and renewal theories. Neural Comput2007;19(11):2958–3010.

Nawrot M, Boucsein C, Rodriguez-Molina V, Aertsen A, Grün S, Rotter S. Serial inter-val statistics of spontaneous activity in cortical neurons in vivo and in vitro.Neurocomputing 2007;70:1717–22.

Nawrot MP, Boucsein C, Molina VR, Riehle A, Aertsen A, Rotter S. Measure-ment of variability dynamics in cortical spike trains. J Neurosci Methods2008;169(2):374–90.

Niebur E. Generation of synthetic spike trains with defined pairwise correlations.Neural Comput 2007;19(7):1720–38.

Ohiorhenuan IE, Mechler F, Purpura KP, Schmid AM, Hu Q, Victor JD. Sparsecoding and high-order correlations in fine-scale cortical networks. Nature2010;466(7306):617–21.

urosci

O

O

P

P

P

P

P

R

R

R

S

S

S

I.C.G. Reimer et al. / Journal of Ne

nken A, Grünewälder S, Munk MHJ, Obermayer K. Analyzing short-term noisedependencies of spike-counts in macaque prefrontal cortex using copulas andthe flashlight transformation. PLoS Comput Biol 2009;5(11):e1000577.

nken A, Obermayer K. A Frank mixture copula family for modeling higher-ordercorrelations of neural spike counts. J Phys: Confer Series 2009;197:012019.

azienti A, Grün S. Robustness of the significance of spike synchrony with respectto sorting errors. J Comput Neurosci 2006;21(3):329–42.

erkel DH, Gerstein GL, Moore GP. Neuronal spike trains and stochastic point pro-cesses. I. The single spike train. Biophys J 1967;7(4):391–418.

ipa G, Wheeler DW, Singer W, Nikolic D. Neuroxidence: reliable and efficientanalysis of an excess or deficiency of joint-spike events. J Comput Neurosci2008;25(1):64–88.

once-Alvarez A, Kilavik B, Riehle A. Comparison of local measures of spike timeirregularity and relating variability to firing rate in motor cortical neurons. JComput Neurosci 2010;29:351–65.

rut Y, Vaadia E, Bergman H, Haalman I, Slovin H, Abeles M. Spatiotemporal struc-ture of cortical activity: properties and behavioral relevance. J Neurophysiol1998;79(6):2857–74.

enart A, de la Rocha J, Bartho P, Hollender L, Parga N, Reyes A, et al. The asyn-chronous state in cortical circuits. Science 2010;327(5965):587–90.

osenbaum R, Josic K. Mechanisms that modulate the transfer of spiking correla-tions. Neural Comput 2011;23(5):1261–305.

oudi Y, Nirenberg S, Latham PE. Pairwise maximum entropy models for studyinglarge biological systems: when they can work and when they can’t. PLoS ComputBiol 2009;5(5):e1000380.

chneidman E, Berry MJ, Segev R, Bialek W. Weak pairwise correlationsimply strongly correlated network states in a neural population. Nature2006;440(7087):1007–12.

himazaki H, Amari SI, Brown EN, Grün S. State-space analysis of time-varying

higher-order spike correlation for multiple neural spike train data. PLoS ComputBiol 2012;8(3):e1002385.

hinomoto S, Kim H, Shimokawa T, Matsuno N, Funahashi S, Shima K, et al. Relatingneuronal firing patterns to functional differentiation of cerebral cortex. PLoSComput Biol 2009;5(7):e1000433.

ence Methods 208 (2012) 18–33 33

Shlens J, Field GD, Gauthier JL, Grivich MI, Petrusca D, Sher A, et al. The struc-ture of multi-neuron firing patterns in primate retina. J Neurosci 2006;26(32):8254–66.

Softky WR, Koch C. The highly irregular firing of cortical cells is inconsis-tent with temporal integration of random epsps. J Neurosci 1993;13(1):334–50.

Staude B, Grün S, Rotter S. Higher order correlations and cumulants. In: Grün S, Rot-ter S, editors. Analysis of parallel spike trains. Springer series in computationalneuroscience; 2010a. p. 253–80.

Staude B, Grün S, Rotter S. Higher-order correlations in non-stationary parallel spiketrains: statistical modeling and inference. Front Comput Neurosci 2010b;4:16.

Staude B, Rotter S, Grün S. Detecting the existence of higher-order correlationsin multiple single-unit spike trains. In: Society for neuroscience abstractviewer/itinerary planner. vol. 103.9/AAA18; 2007.

Staude B, Rotter S, Grün S. CuBIC: cumulant based inference of higher-order correlations in massively parallel spike trains. J Comput Neurosci2010c;29(1–2):327–50.

Stein RB. A theoretical analysis of neuronal variability. Biophys J 1965;5:173–94.

Sweet AL. On the hazard rate of the lognormal distribution. IEEE Trans Reliab1990;39(3):325–8.

Tang A, Jackson D, Hobbs J, Chen W, Smith JL, Patel H, et al. A maximum entropymodel applied to spatial and temporal correlations from cortical networks invitro. J Neurosci 2008;28(2):505–18.

Tetzlaff T, Rotter S, Stark E, Abeles M, Aertsen A, Diesmann M. Dependence of neu-ronal correlations on filter characteristics and marginal spike train statistics.Neural Comput 2008;20(9):2133–84.

Tuckwell HC. Introduction to theoretical neurobiology. Vol. 2. Nonlinear andstochastic theories. Cambridge University Press; 1988.

Wolfe J, Houweling AR, Brecht M. Sparse and powerful cortical spikes. Curr OpinNeurobiol 2010;20(3):306–12.

Yu S, Yang H, Nakahara H, Santos GS, Nikolic D, Plenz D. Higher-orderinteractions characterized in cortical activity. J Neurosci 2011;31(48):17514–26.