Identification of the Volterra kernels ofnonlinear systems

S. Y. Fakhouri, M.Sc, Ph.D., Mem. I.E.E.E.

Indexing terms: Nonlinear systems, Algorithms, Simultation

Abstract: An algorithm is presented for the identification of the Volterra kernels of nonlinear systems usingcoloured Gaussian excitation. The solution is presented in terms of multidimensional Z-transforms of thesystem's kernels. The cases of white Gaussian and non-Gaussian inputs are also considered. The results of asimulation study are included to illustrate the validity of the algorithm.

1 Introduction

A large class of nonlinear systems can be characterised bythe functional Volterra series which was introduced byVolterra1 early in the twentieth century. Wiener2 was thefirst to use the Volterra representation in 1942 to analysethe response of a nonlinear circuit to noise. Later hereformulated the Volterra series representation into sums ofmultidimensional convolutions, called the G-functionals,3

which form an orthogonal set of functions. Severalmethods4'5'6 have been proposed for the identification ofthe Wiener kernels, most of which are modified versions ofLee and Schetsen's7 cross-correlation technique using whiteGaussian excitation. The use of various discrete-levelpseudorandom signals for the identification of isolatedVolterra kernels has been studied by many authors.8'9'10> n

However, in spite of the generality of the functional seriesexpansion of nonlinear systems, relatively few research-ers12' 13 have attempted to identify practical systems basedon this representation. This can be attributed to the for-midable amount of computation required and the diffi-culties associated with the identification of the system'skernels. Hence, the need arises for an efficient algorithmwhich greatly reduces the computational burden, reducesthe number of the parameters to be estimated, eliminatesthe restriction of using white Gaussian inputs as test signals,and presents the identified kernels in a mathematical formwhich easily yields itself for the design of a control strategy.

In the present study, nonlinear systems which can beexpanded in terms of Volterra series14 are considered. Byusing nonwhite Gaussian excitation, and representing theVolterra kernels in terms of their multidimensional pulsetransfer functions, the computational requirements aregreatly reduced. Moreover, the parameterisation of thesystem's kernels results in a reduction of the number ofparameters to be estimated and enables the incorporationof any a priori information about the process, e.g. the formof the kernels, to simplify the identification procedure. Thecase of white Gaussian excitation is also considered inSection 4. The method is extended in Section 5 to provideestimates of the system's parameters when the input is anon-Gaussian process. Simulated examples are presented inSection 6 to demonstrate the feasibility of the algorithm.

2 Problem formulation

For a continuous and bounded input u(f), the output

Paper 1002D, first received 4th May 1979 and in final form 8thSeptember 1980Dr. Fakhouri is with the Postgraduate School of Studies in ControlEngineering, University of Bradford, Bradford BD7 1DP, England

296

0143-7054/80/060296 + 09 $01-50/0

y(i) can be expressed in terms of the Volterra series as

= i r...u(t-tj)dtl ...dtj (1)

where hj(t1, . . . , tj) is termed the /th-order Volterrakernel. The Volterra kernels are symmetric or can be madesymmetric through the cyclic changing of the argumentsof the asymmetrical kernel

'ij sym\t\ > • • • > 0) ~ •• 2- hjasyVl > • • • , tj)]• J

(2)

where J = j \ and the summation15 2 j extends over all the/ possible permutations of the subscripts on the ts.

When the system is inherently discrete and the signalstake on nonzero values only, when

t = iT (3)

or when sampling takes place in the continuous system,an analogous equation can be written for the output ofthe system as

N m m m

I Z ••• Zfe, =0 k2 =0 kj = 0

hj(k\, . . . , kj)u(i — ki) . . .u(i — kj) ) (4)

In terms of multidimensional Z-transforms,14 eqn. 4 canbe rewritten as

Y(z) =N

N

= I Fj[Yj{z,,..., (5)

where F} is the operator by which a /th-dimensional Z-transform can be reduced to a one-dimensional Z-transform,and Yj(zx,. . . , Zj) is the output of the /th-order kernelgiven by

= Zj\ I . . . I x[fe,=0 fty=0

IEEPROC, Vol. 127, Pt. D, No. 6, NOVEMBER 1980

.,ii)\ (6)

where Zj[y>j(ii,.. . , I,-)] is the/th-dimensional Z-transform) is defined by»• • • > (/

Z •••Xi, =0 iy=O

^ (7)

The /th-dimensional kernel Hj(z1,.. . , z;) can be expressedas

I Z ••• Z ^f,...^1'*;1'0 i O

I Zt l = o i 2 = 0 iy = (8)

n, n2 nj . . .

Z r ? ^-» — | ~t<y ""It

/ • • " / &i i i'^l *2 ' • ••*?^ ^ ^ ^ • l » 2 * * " / *

I,=0l2=0 «; = °

If Hj(zi,. . . , Zy) is symmetrical, then

«! = n2 = . . . = rij = nh — h (Q\Dix...ij ~ °irOl...ij) \7)

ail...ij = aHil...ij)

where n denotes any permutations of the subscripts

Thus, a substantial reduction in the number of param-eters to be estimated can be gained by defining

(10)

and rewriting eqn. 8 in the form

n n

' , = 0 i a = ' i ' ; - ' J - « }

fi n n

Z Z - I *...••</i , = 0 i 2 = 0 i / = * / - ,

where the summation,15 (2'zj"'1 . . . zjli\ is taken over allthe possible combinations of the superscripts ix,.. ., ijon thezs.

Thus, the identification problem can be formulated asthe estimation of the parameters {0/} and {dj\ associatedwith the/th-dimensional pulse transfer function///(zj, . . . ,

3 Identification of the Volterra kernels

The identification algorithm developed in this Section isbased on a correlation analysis procedure to isolate theresponse of the highest-order kernel of the system, followedby a least-squares algorithm for the estimation of theparameters {j3y} and {dj} of the multidimensional pulsetransfer function defined by eqn. 11 for j =N. Once the

IEEPROC, Vol. 127, Pt. D, No. 6, NOVEMBER 1980

highest-order kernel is identified, its effect on the measureddata can be eliminated and the whole procedure is repeatedto identify sequentially the lower-order kernels.

3.1 Correlation analysis with coloured Gaussianexcitation

Let the input sequence to the process under investigationbe defined by

u(i) = x(i) + b 02)

where x(i) is a zero mean coloured Gaussian process and bis an arbitrary mean level. Define the correlation function

'//x1...xNy(al> • • • >°N)

1

N'

Z(-IN1

_2l + 1).. .x(i-OaN)]

= 777 Z • • • Z M ^ i > • • • , fcjv) Z <t>xx(.oai ~* i )N\ JV!fe,=0 fejv = O

•••<l>xx(aaN-kN) (13)

wherey'(i) =y(i) —E[y(i)] ,E is the expectation operator,

(N-2)/2, for N evenN' =

(N-l)/2, for A odd

and the summation15 2 ' is taken over all the combinations

(5)of the numbers ait. . . , otN, which are positive integers

ranging from 1 to N with a,- otj, V i¥=j, ( J indicatesthe number of combinations, and is the binomial coefficientgiven by

(N-2l)\2l\(14)

The proof of eqn. 13 is summarised in the Appendix. Notethat E[x(i-oaN_2Ul) ... x(i-oaN)] can be evaluatedfrom <t>xx(o), o = 0, 1, . . . , by using eqn. 60 and thesymmetry property of the autocorrelation function, i.e.

From eqns. 2 and 13,

'//x,...JcNy'(al ' • • • >

• • • L , n N sym[K\, . . . , K N ) q > x x ( O x K i ) . . .

k^—0 kj^ = 0

<l>xx(oN"kN) (15)

i.e. ijix XNy'(oi, • • •, oN) yields the response of thesymmetrised Afth-order kernel for an input

(16)

297

Hence, in order to get consistent estimates of the param-eters of the Nth-order kernel from ^Xl...xNy'(ai> • • • >°N)>4>xxip) must be a persistently exciting16 sequence which isnot a very restrictive condition.

3.2 Parameter estimation of the multidimensional pulsetransfer functions

The result of eqn. 15 can be used to estimate the param-eters of the jVth-dimensional pulse transfer functiondefined by eqn. 11, for/ =N. Thus, from eqns. 6, 11 and15, we have

+ • • • + (V..nn0j«cOi -n) . . .<!>xx{oN — n)

- • • •-dn...nntxi...xNy'(Ol ~n,. • • , ON - / l )

(17c)

Since (j>xx(p)^O for some values of o = kx to o = k2,then the computational burden can be greatly reduced by

xl'xl...xNy'(zli • • • >ziv)

or

_ ' l = 0 * 2 = * |

Z Z« ,=o «,= «

Z Z ••• Z« , = 0 i" 2 = i , iN='N-l

ilN=lN-l

<Pxx\?\ ) (17a)

setting (7i = a2 = . . . = oN = a. Hence, eqn. 17c can berewritten as

z zi , = 0 i7=il

Without loss of generality we can set dQ oo = l'O; ex-panding both sides of eqn. lib, and rearranging terms,we get

~ &Q.. .00<!>XX(P\ ) • • •

(o,. . . , a, a - 1)

(18)

N o t e t h a t ' by e v a l u a t i n 8 ^x,.. .«Ny'(a,. . . . a), for a =fc!| fci + 1 , . . . , ^2 . then tXl...xNyio - / , . . . , a - / ) ,/ = 1 , . . . , TV, for a = &! + / , . . . , k2 + / , is readily avail-able from i//x XMy'(a> • • • > a)> s m c e only a s n ^ m ^ m e

is involved. Similarly, y/Xl...XiVy'(ai ~7 , . . . , oN —j) canbe obtained from the estimated ^x,...jciVy'(ori, • • • , Opj).

By considering (M + 1) points of the sampled cross-and autocorrelograms, eqn. 18 yields the matrix equation

' x , . . . x N y - i / /Xl...xNy ik+M-n,. ..,

M-n)

Po ... 00

Po.. .01

^ 0 . . .01

e(k+M)

298 IEEPROC, Vol. 127, Pt. D, No. 6, NOVEMBER 1980

i.e.

4/ = $P + E (19 )

where E is an error term. Since all the elements in 4; and 9matrices are estimated, a least-squares estimate of theparameters j30...oo, • • • , 0n...nn.do...oi» • • • >dn...nn, canbe readily computed:

P = (iTiri9TV (20)

and the identification of the ./Vth-order kernel is complete.Once the Nth-dimensional pulse transfer function has

been identified, the output yN (/) can be computed from

- M * O . . . O I . P J V O \ - • • . ' . ' - I )

- . . .-dnnnyN(i-n,. .. ,i-n) (21)

or, alternatively, hN(kx, . . . , kN), for kx, k2, • • • , kN = 0,1,. . . , m, can be evaluated from H^{zx, . . . , zN) and theoutput yN(i) can be computed from

m m

- I ••• Ife,=O kN =

,..., kN)u(i -kx)

(22)

and a reduced system output y'yN-x (/) can be defined

-id) = y(f)-yN(i) (23)

Continuing the above procedure the (N — l)th kernel canbe identified by computing the (N — l)th correlationfunction

1(N-\)\

(N-l)'

+ I (-1)' I x(5T1)

°\

-x(i-aQJ] aNJ (24)

which reduces to

2- • • •fe,=0

(25)

By following a similar procedure to that of eqns. 18—20,the parameters of the (N — l)th-dimensional pulse transferfunction can be estimated.

Similarly, the /th-order kernel can be identified sequen-tially by computing

yy'iif) = y'(J) - X y\Q) (26)

m= I ••• I hjsym{kx,...,kj)x

fe,=O fey=0

<t>xx(°l - * l ) • • • <Pxx(.°j-kj) (27)

Note that if the output of the system is corrupted by anadditive noise process n(i), providing this is statisticallyi n d e p e n d e n t of x(i), t hen E[n(i)x(i — o1)...x(i — ay)] =0V ox, • • . , Oj, j = 1, . . . , N, and the results of eqns. 15,25 and 27 are unaffected and the estimates remain unbiased.

4 The case of white Gaussian excitation

If x(i), defined in eqn. 12, is a zero-mean Gaussian processwith a spectral density of £W per cycle, then its auto-correlation function is given by

<t>xx(o) = KS(a)

where 5 (a) is the unit impulse function.Substituting eqn. 28 in eqn. 15 yields

^.xNy

( ° \ , • • • , = K hNsym(ax

(28)

(29)

Hence, from eqns. 17, 28 and 29, we have

\jjXl...xNy'(a\ > • • • 5

-n) . . . 8(oN -n)

-do...oi{*Pxx...xNy'(°i>- • • >°N - 1 )

(30)

Since 5 (a) = 0, Va =£ 0, then, unlike the case of colouredGaussian input, it is essential to take into account thepoints of ^Xl...xNy'(°i > • • • > °N) for aU t n e possiblecombinations of ax,. . . , oN, for Oj = 0,. . . , M, andj = 1, . . . , N, in order to avoid the singularity of the matrix9 in eqn. 19. In matrix form eqn. 30 can be written as

Via = 9WP + E (3 1)

where

(32)

IEEPROC, Vol. 127, Pt. D, No. 6, NOVEMBER 1980 299

~KN0 . . . 0 I 0

0 KN...O \-*

I'• • • - K " I - * , , . . . xNy tl...xNy'®,---,0)

and

== [&O...OO&O...01 fin... nn I ^ 0 . . . 01 • • • &n... nn\

= [PT \Pl]The least-squares solution of eqn. 31 is given by

P = &Z*wTl*lvw

(34)

(35)

If the dimension of the matrix (9^9^) is large, the param-eters {dN} and {fix} can be estimated in two steps by re-writing eqn. 31 as

u / , , i ' w l 2

o [ *»„+

£ 2

i.e.

which yields the least-squares estimates of P2

and the estimates of P\ can be obtained from

Pi = ( 9 £ ? u , r 1 9 ' £ ( 4 ' u , 1 ~9wl2P2)

(36)

(37)

(38)

(39)

The identification of the rest of the kernels can be carriedout sequentially by following the procedure of the previousSection.

5 Extension to non-Gaussian inputs

The identification procedure outlined in the previousSections can be applied with some modifications even ifthe output is non-Gaussian, providing that it has a nonzeromean level. The development is based on the followingresult:

#iv["l«2 —\H[ul+u2 + ... + uN]

N-\

/AT \\N-l)

(40)

300

-n,. . . ,M-n)

(33)

where

HN[uxu2 . . .

and

m mZ ••• Z

fe,=0 fejy =

. ..uN(iN-kN)

N ( m m

Z ••• I /*y

( 4 1 )

for p = 1 , 2 , . . . , 7 V ( 4 2 )

Schetsen17 used the above result to identify the Volterrakernels of a continuous nonlinear process by defining

ut = S(t-Ti), i = 1 , . . . ,7V (43)

and substituting in eqn. 40 to yield

HN[uxu2 ...uN] = hN(t-Tlt...,t-TN) (44)

He also suggested the use of step inputs such that

ut =aU(t-Ti)

where U(t) is a step function, which yields

HN[Ulu2 ...uN] = — SN(t-Tlt...,t-TN)

t-T,

...dtN (45)

and the A^th-order kernel can be obtained from

tx,...,tN)hN(ti,...,tN) = (46)

aNN\ 3r, . . .3fjv

It was George18 who first suggested the identification of

IEEPROC, Vol. 127, Pt. D, No. 6, NOVEMBER 1980

an isolated second-order kernel from the observation

H2[uiU2] = h{ff2[ui+u2] ~H2[ux] — H2[u2]}

(47)

where uhj = 1, 2 is as defined by eqn. 43. He also suggestedthe use of multidimensional step inputs for the identification

However, it may not be practically feasible to identifysystems by means of very narrow pulses which approximateimpulses, nor is it advisable to use the differentiationapproach, particularly in the case of noise-corrupted data,as it yields biased estimates. A more efficient algorithmwhich eliminates the effect of noise combines correlationanalysis and the least-squares technique to estimate theparameters {j3y} and {dj},j = N, N — 1, . . . , 1, of the multi-dimensional transfer function defined by eqn. 11, is derivedbelow.

Fromeqns. 11 and 41 and setting Uj = u(ij),j = 1 , . . . ,N,

= 0o...oo«O'i). • -"O'JV)

+ • • - + 0n...nn"O"l ~ «) • • • " OW ~ «)

~ " 0 . . . 01 [yN\h > • • • > '*AT 1)

- • ••-dn...nnyN(h ~n, . . . ,iN-n) (48)

To reduce the number of measurements, only the points atix = i2 — • • • = z'jv = i are considered. Providing u(i) is anonzero mean sequence, define the correlation function

= Po...ooE[u(i-o)uN(i)]

~Nd0 0lE[u{i - o)yN{i,. . . , / , / -

(49)

By considering (M + 1) points at o = Mx, Mx + 1 , . . . ,Mi + M, eqn. 49 can be written in matrix form as

vj; = 9 •/» + E

where

(50)

(51)

0 1 2-

1-0-5 z-1

0-1 z"

0 z

1-0-7 z

1 -1 -3z -1 .0 -42z -2

Fig. 1 Third-order nonlinear system

IEEPROC, Vol. 127, Pt. D, No. 6, NOVEMBER 1980

E{u(i-Mx-r)uN(i-n)}

-E{u(i -Mi -r)yN(i,...,/, z - 1 ) } . . .

E{u(i -Mx -r)yN(i -n,...,i- n)}],

r = 0, 1 , . . . ,M (52)

9(r+ D is the (r + l)th row of the matrix 9 and P is the sameas defined by eqn. 19. The least-squares estimates of Pcan be readily computed from eqn. 50.

Note that the above algorithm requires the estimation ofE{u(i — a)HN[ui . . . uN]}, which can be evaluated from

E{u(i-o)HN[ui ...uN]}

N-l ,

+ K-D1 II-l IN \

\N-ljE{u{i-a (53)

which involves only the statistical properties of thesequences Uj, j = 1, . . . , N, but not their instantaneousvalues.

Once the parameters of the iVth order kernel are esti-mated, the rest of the kernels can be identified sequentiallyin a similar manner to that explained in Section 3.2.

Eqn. 53 implies that (2N — 1) measurements are requiredto estimate E[u(i — o)HN(ix, . . . , iN)]. However, if thisprocedure is carried out, then some of the measurementswill be redundant, since only a shift of time is involved insome of the measurements compared to others. Also, themeasurements which were carried out to identify the Mh-order kernel could be employed to identify the lower-orderkernels by subtracting the response of the identified higher-order kernels from the measured response of the TVth-ordersystem.

6 Simulation results

The identification procedure outlined above was used toidentify a third-order nonlinear system, shown in Fig. 1.The model was simulated on a PDP10 digital computerusing 2800 data points generated by recording the responseto a coloured Gaussian sequence A^{0-007, 0-513} whosesecond-order autocorrelation function is shown in Fig. 2.

0-3

-10

19

v •*

- 0 - 1 L

Fig. 2 Autocorrelation function of input Gaussian sequence,(t>xx{o)

301

In terms of multidimensional Z-transforms the pulsetransfer functions are given by

H3(z1,z2,z3) -

Hx(z) =Olz "1

l-O-OSz"1

(55)

(56)

(54)

8 Acknowledgment

The author acknowledges the support of the UK ScienceResearch 'Council in carrying out the research involved inthis work.

The model was simulated with K = — 10 and M= 29 ineqn. 19. The simulation and identification of the modelrequired l-5min of computer time. Inspection of theestimated system's parameters, summarised in Table 1,clearly demonstrates the effectiveness of the algorithm.

7 Conclusions

Identification of the Volterra kernels of nonlinear systemshas been investigated, and algorithms based on correlationanalysis using both Gaussian and non-Gaussian inputsequences have been derived. The excessive experimentationtime associated with the use of white Gaussian or non-Gaussian inputs will often dictate the use of colouredGaussian excitation, whenever this is possible, to identifythe system's kernels.

By presenting the solution in terms of the multi-dimensional pulse transfer functions, which in some casescan be realised in terms of cascaded linear and nonlinear ele-ments,14'19 the computational burden is greatly reduced.Also, the algorithm enables the utilisation of the structuralproperties of the Volterra kernels to simplify the com-putational procedure and to preserve the system structure.

Since the identification will normally be performed withthe aid of a digital computer, the Volterra kernels of non-linear continuous-time systems can be identified from theirsampled input/output records using any of the algorithmsdescribed above. A continuous-time model, if desired, canbe derived from the discrete one either by using tablesrelating multidimensional Laplace- and Z-transforms orany other analytical procedure.

The main advantages of this method are its generality, theexplicit relation it provides between the input and the out-put, the modest computational requirements, the relativesimplicity of the algorithms, and for the case of Gaussianinput the only necessary data for the characterisation ofthe nonlinear system is a record of the input and the sys-tem output sequences; tests involving multiple input signalsare avoided.

9 References

1 VOLTERRA, V.: Theory of functional' (Blackie, 1930)2 WIENER, N.: 'Response of a nonlinear device to noise'. Report

V-165, MIT Radiation Lab., April 19423 WIENER, N.: 'Non-linear problems in random theory' (Wiley,

1958)4 FRENCH, A.S., and BUTZ, E.G.: 'Measuring the Wiener kernels

of a nonlinear system using fast Fourier transform algorithm',Int. J. Control, 1973, 17, pp. 529-539

5 FRENCH, A.S.: 'Measuring the Wiener kernels of a nonlinearsystem by use of the fast Fourier transform and Walsh function'.Proceedings of the symposium on testing and identification ofnon-linear systems, Pasadena, California, March 1975, pp. 76-88

6 CHOIC, C, and WARREN, M.E.: 'Identification of nonlineardiscrete systems'. Proceedings of Southeaston '79 region 3conference, Atlanta, Ba., USA, April 1978, pp. 329-333

7 LEE, Y.W., and SCHETSEN, M.: 'Measurement of the Wienerkernels of a nonlinear system by cross-correlation', Int. J.Control, 1965, 2, pp. 237-259

8 HOOPER, R.J., and GYFTOPOULOS, E.P.: 'On the measure-ment of the characteristic kernels of a class of nonlinear systems'.Presented at the symposium on neutron, noise, waves and pulsepropagation, University of Florida, Gainesville, Fla., USA,Feb. 1966

9 REAM, N.: 'Nonlinear identification using inverse repeat m-sequences', Proc. IEE, 1970, 117, (1), pp. 213-218

10 BARKER, H.A., and DAVY, R.W.: 'Measurement of secondorder Volterra kernels using pseudorandom ternary signals',Int. J. Control 1978, 27, pp. 277-291

11 FAKHOURI, S.Y.: 'Identification of nonlinear systems'. Ph.D.thesis, Sheffield University, 1978

12 HUNG, G., and STARK, L.: The kernel identification method(1910-1977) — Review of theory, calculation, application, andinterpretation', Math. Biosci., 1977, 37, pp. 135-190

13 BILLINGS, S.A.: 'Identification of nonlinear systems - Asurvey. Research report 112, Department of Control Engineering,Sheffield University, 1980

14 ALPER, P.: 'Higher dimensional Z-transforms and nonlineardiscrete systems', Rev. A, 1964, 6, pp. 199-212

15 BEDROSIAN, E., and RICE, S.O.: 'The output properties ofVolterra systems (nonlinear systems with memory) driven byharmonic and Gaussian inputs', Proc. IEEE, 1971, 59, pp.1688-1707

16 EYKHOFF, P.: 'System identification' (Wiley, 1974)17 SCHETZEN, M.: 'Measurement of the kernel of a nonlinear

system of finite order', Int. J. Control, 1965, 2, pp. 251 (also'Corrigendum', ibid., pp. 408

Table 1 : Summary of identification results of third-order nonlinear system

Parameters

theoretical valuesH, [Z.,Z.,Z3)

estimates

Parameters

theoretical valuesH7 \Z1 , Z2) estimates

Parameters

theoretical valuesH [Z)1 estimates

0.,, (=^.,i>10

102

0-10-1026b i

0-1

0 09

"ooi '—' aooi ~ aoio aioo '

- 0 - 7

— 0-7003

Cn (=aii>

- 1 - 3-1-2913a !- 0 - 5

- 0-433

^on '—aon "~ aioi ~ a n o '

0-49

0-496

di2 (= a2 2 )

0-420-4082

-

:

diu <=ani )

-0-343

-0-367

-

-

-

-

302 IEE PROC, Vol. 127, Pt. D, No. 6, NOVEMBER 1980

18 GEORGE, D.A.: 'Continuous nonlinear systems'. Technicalreport 355, MIT Research Laboratory of Electronics, Cambridge,Ma., USA, 1959

19 SHANMUGAM, K.S., and LAL, M.: 'Analysis and synthesis of aclass of nonlinear systems', IEEE Trans., 1976, CAS-23, pp.17-25

20 SCHETZEN, M.: 'Average of the product of Gaussian variables'.MIT Research Laboratory of Electronics, quarterly progressreport 60, 1960, pp. 137-141

10 Appendix: Derivation of yXi XNy'{olr... ,aN)

Let us at first consider the case when the input to the non-linear system is a zero-mean Gaussian process, Le. b = 0 andu(i) = x(i). Define theiVth-order correlation function

N

where

MO

E[yN(i)x(i-Ol)...

N-l

+ Z E[y'j(i)x(i — o

= y(i)-E[y(i))

m

- I •m

y xk:=O

x(i ~ aN)]

i)...x(i-oN)] (57)

(58)

j = l,...,N (59)

and E is the expectation operator.The /th-order correlation function of a zero-mean

Gaussian process20 is given by

E [x (i! )x (i2) . . . JC (ij)] = 0 for / odd

i-aai).. .x(i-oaN_2l)] •

E[x(i-oaN_2Ul)...x(i-oaN)]

m m= Z • • • I hN(kl , . . . , k N ) x

fe,=0 fejv=O

r XX\ CJtj 1 / ' ' ' TXX \ Qlffl £y /

Z ( - D / + 1 Z ' x

AT!

N

Z = l

; N ( 0 ^ 0 " - CTa,) • • • X(i ~ OaN_2l)] '

F\x(i n ^ x(i a 1

where

N' =

N-2

N-l

for A even

for N odd

and

}(i)x(i - oi) . . . x(i - aN)]

= Z ••• Z hj(k1,...,kj)xfe,=0 kj = 0

(61)

for / even (60)

where the summation is over all the ways of dividing/ objects into pairs and <}>xx(i) is the second-order auto-correlation function of the sequence x(i).

From eqns. 57, 59 and 60 we have

E [y'N(J)x(i - a, ) . . . * ( / - aN)]

fe,=0••• I hN(klt...,kN)x

m rn N'

fe,=O kj = OZ(s)

hN(kl,...,kN)fe,=0 kN=o

N\

IEEPROC, Vol. 127, Pt. D, No. 6, NOVEMBER 1980

E{x(i-

N'

= y (— iz=i

E[y'j(i)>

E\x(i —

for/ = 1,. . .

°cN-2U^

:(i-oai).

,N-l

...x(i-oaN)}

• •x(.i~°aN.2i)]

• x(i-oaN)],

(62)

303

Substituting eqns. 61 and 62 in eqn. 57 yields

E [ / O X * -oi)...x(i- aN)]

m m

= X ... X h N ( k x , . . . , k N )fe, = o kN = o

X <Pxx(oat —kx) . . .((>xx(0aN

AM

N

+ X (-0/+1 X' -

E[y'N(i)x(i- oai)...x(i—oaN_2l)]

(AT-1) JV'

; = i

?WOX[x(i~aa

( - iy+i

**,)•).

X ' x

(a). .x(/

. .;t0•

-ac*Jv-2^>]

- a « N ) ] >

m m= X • • • X

= 0

N !

Ar-

+ X (-O'+1 X

E[x(i-aaN_2l + l ) . . . J f ( / - a

Define

'xt...xNy

= —(E [^'(

+ X (-

• ,ON)

cO ' -a i ) . . .xO —ojv)]

{E[y'(i)x(i-oat)...x(i-oaN_2l)]

(63)

(64)

Thus, from eqn. 63,

i m m

= 777 X • • • X M * i » • • • ,kN) X XN\ fe,=O

<Pxx(oai - k i ) . . . <t>xx(paN - kN) (65)

The above derivation can be extended to include the caseof nonzero-mean input by substituting

u(i) = x(i) + b (66)

in eqn. 59, and hence the first term in the right-hand sideof eqn. 57 yields

E[y'N(J)x(i - a , ) . . . x(i - oN)]

m m

= X ••• X h N ( k l , . . . , k N ) -h, = 0 hN = 0

N-l

I

- E{x(i -kai)... x(i - kaN_r)}) x

[i-al)...x(i-oN)\\\ (67)

where the . S' .is over all the combinations of aii, a2, . . . ,IN \\N-r)

&M-r> which are positive integers ranging from 1 to N — rwith at ¥= otj V / =£/.

From eqns. 61, 62 and 67,

E [y'N (j)x(i - a,) . . . x(i - oN)]

= 1 Z • • • X M * I , . - . , * J V ) Zfe, = O fejv = 0 ^ '

(aaw~^Ar)>xx(°aN

N

+ 1 ( -D1*1 ! x

N O X ' - Oa, ) • • • *(' - aaN_2

O*-aajv_2I+1).. .x(i-oaN)] (68)

which is the same relationship as that given by eqn. 61 forthe case of the zero-mean input.

Similarly, it can be proved that eqn. 62 holds for thecase of nonzero-mean input as well, and consequently\pXi XNy(oi, . . ., oN) defined by eqns. 64 and 65remains unchanged when the input has a nonzero-meanlevel.

304 IEEPROC, Vol. 127, Pt. D, No. 6, NOVEMBER 1980