Metaphors for time‐series analysis

This article was downloaded by: [The University of Manchester Library]On: 05 December 2014, At: 12:50Publisher: Taylor & FrancisInforma Ltd Registered in England and Wales Registered Number: 1072954Registered office: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK

International Journal ofMathematical Education in Scienceand TechnologyPublication details, including instructions for authors andsubscription information:http://www.tandfonline.com/loi/tmes20

Metaphors for time‐series analysisD.S.G. Pollock aa Queen Mary and Westfield College, University of London ,EnglandPublished online: 09 Jul 2006.

To cite this article: D.S.G. Pollock (1995) Metaphors for time‐series analysis, InternationalJournal of Mathematical Education in Science and Technology, 26:4, 501-515, DOI:10.1080/0020739950260404

To link to this article: http://dx.doi.org/10.1080/0020739950260404

PLEASE SCROLL DOWN FOR ARTICLE

Taylor & Francis makes every effort to ensure the accuracy of all the information(the “Content”) contained in the publications on our platform. However, Taylor& Francis, our agents, and our licensors make no representations or warrantieswhatsoever as to the accuracy, completeness, or suitability for any purposeof the Content. Any opinions and views expressed in this publication are theopinions and views of the authors, and are not the views of or endorsed by Taylor& Francis. The accuracy of the Content should not be relied upon and should beindependently verified with primary sources of information. Taylor and Francisshall not be liable for any losses, actions, claims, proceedings, demands, costs,expenses, damages, and other liabilities whatsoever or howsoever caused arisingdirectly or indirectly in connection with, in relation to or arising out of the use ofthe Content.

This article may be used for research, teaching, and private study purposes.Any substantial or systematic reproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in any form to anyone is expresslyforbidden. Terms & Conditions of access and use can be found at http://www.tandfonline.com/page/terms-and-conditions

http://www.tandfonline.com/loi/tmes20

http://www.tandfonline.com/action/showCitFormats?doi=10.1080/0020739950260404

http://dx.doi.org/10.1080/0020739950260404

http://www.tandfonline.com/page/terms-and-conditions

http://www.tandfonline.com/page/terms-and-conditions

INT. J. MATH. EDUC. SCI. TECHNOL., 1995, VOL. 26, NO. 4, 501-515

Metaphors for time-series analysis

by D. S. G. POLLOCKQueen Mary and Westfield College, University of London, England

(Received 14 October 1993)

This paper provides a coherent collection of metaphors and physical analogieswhich help to explain the nature of the processes depicted by autoregressiveintegrated moving average (ARIMA) models and which help in deriving theirforecast functions. ARIMA models are widely used in the social sciences; but,all too often, their predictions go awry. In seeking to determine whether majorerrors of prediction are due to the inapproriateness of the models or to a failureto apply them correctly, it is useful to bear in mind their physical analogies.

1. IntroductionMuch of the mathematical and statistical modelling in the social sciences is based

upon an analogy which compares society or the economy to a machine. Often themodels fail dramatically to predict untoward events, and sometimes they fail evento track gradually evolving trajectories of the variables in question.

There can be various conflicting responses in the face of such failure. Oneresponse is to declare that the analogy is faulty and that the models are inappropriate.Another response is to admit that the analogy may be weak but to enquire,nevertheless, whether it is a failure to fulfil the analogy adequately rather than itsinappropriateness per se which is at fault. This paper is written in the belief that agood way of understanding the strengths and limitations of the ARIMA(autoregressive integrated moving average) time-series models which are widelyapplied in the social sciences is to have in mind a full and precise physical analogy.In this way, much of the mystique of mathematical modelling can be dispelled.

An alternative approach to the derivation and the interpretation of mathematicalmodels in the social sciences is to root them deeply in the context of the specificproblems to which they are applied. Thus in econometrics, for example, dynamicmodels of economic behaviour are frequently derived by considering the costs andincentives faced by individual economic agents in their task of adjusting to aconstantly changing economic environment.

These two approaches to mathematical modelling should be seen as complemen-tary rather than opposed. A successful model must satisfy the criteria of bothapproaches. A model constructed according to an inappropriate physical analogywhich bears no relationship to the social context is bound to fail. Likewise a modelof dynamic social behaviour which ignores the basic principles of dynamics as theyapply to the physical world is unlikely to have much predictive power.

0020-739X/95 $1000 © 1995 Taylor & Francis Ltd.

Dow

nloa

ded

by [

The

Uni

vers

ity o

f M

anch

este

r L

ibra

ry]

at 1

2:50

05

Dec

embe

r 20

14

502 D. S. G. Pollock

2. The laws of motionThere are numerous physical systems governed by differential equations which

will serve as metaphors for ARIMA processes. The simplest system consists of abody travelling through a viscous medium. The body may be moving freely or it maybe connected elastically to a fixed point. Its motion may be inertial, or it may beimpelled by a constant force or by a series of impacts.

We should begin by considering the laws of motion of bodies in a free space.According to a well-known dictum

Every body continues in its state of rest or of uniform motion in a straight lineunless it is compelled to change that state by forces impressed upon it.

This is Newton's first law of motion [1]. The kinematic equation for the distancetravelled by a body moving with constant velocity in a straight line is

y = yo + vt (1)

where v is the uniform velocity and yo represents the initial position of the body attime t = 0. This is just a linear time trend; and such trends represent the most basicdevice for extrapolative forecasting in time-series analysis.

Newton's second law of motion asserts that

The change of motion is proportional to the motive forces impressed; and ismade in the direction of the straight line in which the force is impressed.

In modern language, this is expressed by saying that the acceleration of a body alonga straight line is proportional to the force which is applied in that direction. Thekinematic equation for the distance travelled under uniformly accelerated rectilinearmotion is:

y = yo + v0t + -at2, (2)

where VQ is the velocity at time t = 0 and a is the constant acceleration due to themotive force. This is just a quadratic in t. Such quadratic trends are the appropriateforecasting functions for many non-stationary time-series.

Perhaps the most fruitful metaphor is that of a particle suffering a series ofimpacts randomly spaced in time and of randomly varying magnitudes. Suchcircumstances are rather difficult to model mathematically; and so we are bound tosimplify the model. There are two routes to follow. The first is to employ a schemeof discrete time consisting of isolated instants separated by regular intervals. Thenonly the magnitude of the impacts is irregular. The problem here is that themechanics of physical systems is expressed almost exclusively in continuous time interms of differential equations, whereas a system of discrete time forces us to employdifference equations.

The alternative way of simplifying the model provides for an easy access toclassical mechanics, but it requires the development of continuous-time models ofstochastic processes. These are somewhat counter-intuitive and they present variousmathematical difficulties. Such difficulties have spurred the development of thenon-standard mathematical analysis which was associated, in the early part of thecentury, with the development of quantum mechanics and which is familiarnowadays from the theory of chaos and fractals and in the notion self-similarity.

We shall attempt to pursue concurrently both of these routes towards workable

Dow

nloa

ded

by [

The

Uni

vers

ity o

f M

anch

este

r L

ibra

ry]

at 1

2:50

05

Dec

embe

r 20

14

Time-series metaphors 503

metaphors. First we shall consider the metaphor of free motion and then we shallimpose the kind of elastic constraint which gives rise to a centripetal force and whichresults in harmonic motions.

3. Unconstrained stochastic motionWhen we think of a body travelling in a resistant medium under the influence

of intermittent impacts, we are thinking of Brownian motion. For a physical exampleof Brownian motion, one can imagine small particles, such as pollen grains, floatingon the surface of a viscuous liquid. The viscosity might be expected to bring theparticles to a halt quickly if they were in motion. However, if the particles are verylight and very small, then they will dart hither and thither on the surface of the liquidunder the impact of its molecules which are themselves in constant motion.

The appropriate way to model Brownian motion is to employ a Wiener process.A Wiener process Z(t) consists of an accumulation of independently distributedstochastic increments. The path of Z(t) is continuous almost everywhere anddifferentiable almost nowhere. If dZ(t) stands for the increment of the process inthe infinitesimal interval di, and if Z(a) is the value of the function at time a, thenthe value at time x > a is given by

Z(T) = Z(a) + dZ(t) (3)

Moreover, it is assumed that the change in the value of the function over any finiteinterval (a, T] is a random variable with a zero expectation:

£{Z(T)-Z(a)}=0 (4)

Let us write ds H dt = 0 whenever ds and dt represent non-overlapping intervals.Then the conditions affecting the increments may be expressed by writing

, . • (5,dt iids = dt

These conditions imply that the variance of the change over the interval (a, T] isproportional to the length of the interval. Thus

V{Z(x)-Z(a)}=\ \ E{dZ(s)dZ(t)}

= J o2dt = <J2(x-a)J t = a

(6)

The simple Wiener process has at least some of the features of Brownian motion;over long periods of time, the properties of the process are indeed very similar tothose which are observed experimentally.

The discrete-time counterpart of the Wiener process is the first-order randomwalk. Let {yt} be the random-walk sequence. Then its value at time t is obtainedfrom the previous value via the equation

yt = yt-\ + Et (7)

Dow

nloa

ded

by [

The

Uni

vers

ity o

f M

anch

este

r L

ibra

ry]

at 1

2:50

05

Dec

embe

r 20

14


1

0

-1

-2

-3

0 25 50 75 100Figure 1. A sequence generated by a white-noise process.

Here e, is an element of a white-noise sequence (Figure 1) of independently andidentically distributed random variables with

= 0 and V(e,) = (T2 for all t (8)

By a process of back-substitution, the following expression can be derived:

y, = yo+{e, + et-1 + . . . + £,}. (9)

This depicts yt as the sum of an initial value y0 and of an accumulation of stochasticincrements. If y0 has a fixed finite value, then the mean and the variance of yt aregiven by

E(yt)=y0 and V(yt) = tXo2 (10)

The second of these is the counterpart of the property of a Wiener process givenunder equation (6). There is no central tendency in the random-walk process; and,if its starting point is in the indefinite past rather than at time t = 0, then the meanand the variance are undefined.

To reduce the random walk (Figure 2) to a stationary stochastic process, it isnecessary only to take its first differences. Thus

yt = yt-i = et (H)

The values of a random walk, as the name implies, have a tendency to wanderhaphazardly. However, if the variance of the white-noise process is small, then thevalues of the stochastic increments will also be small and the random walk willwander slowly.

There is no better way of predicting the outcome of a random walk than to takethe most recently observed value and to extrapolate it indefinitely into the future.This is demonstrated by taking the expected values of the elements of the equation

yt+h = yt+h-i + Bt+h (12)

which represents the value which lies h periods ahead at time t. The expectations,

Dow

nloa

ded

by [

The

Uni

vers

ity o

f M

anch

este

r L

ibra

ry]

at 1

2:50

05

Dec

embe

r 20

14


0 25 50 75 100

Figure 2. A first-order random walk.

which are conditional upon the information of the set Jt = {yt,yt-i,---} containingobservations on the series up to time t, may be denoted as follows:

l f h>0(13)

In these terms, the predictions of the values of the random walk for h > 1 periodsahead and for one period ahead are given, respectively, by

(14)

50

-50

-100 -

-150 -

-200 -

0 25 50 75 100Figure 3. A second-order random walk formed by accumulating the values of the first-order

random walk of Figure 2.

Dow

nloa

ded

by [

The

Uni

vers

ity o

f M

anch

este

r L

ibra

ry]

at 1

2:50

05

Dec

embe

r 20

14


The first of these, which comes from equation (12), depends on the fact that•E'(e< + A|./<) = 0. The second, which comes from taking expectations in the equationyt+1 = Vt + £( +1, uses the fact that the value of yt is already known. The implicationof the two equations is that yt serves as the optimal predictor for all future valuesof the random walk.

4. Second-order processesA more detailed analysis reveals that there are some weak points in modelling

Brownian motion by an ordinary Wiener process. The difficulties correspondroughly to the problems which we have in finding a physical interpretation for theprocess. The essential characteristics of the Wiener process are (a) that almost allsample functions are continuous functions of t and (b) that almost all paths arenowhere differentiable in respect of t. Whereas the first property is a physicalnecessity for any body in motion, the second is impossible since Newton's laws implythat only particles with zero mass can move along routes which are nowheredifferentiable.

An improved model for the local behaviour of the Brownian path can be obtainedby assuming that it is the velocity or momentum of the particle rather than its locationwhich is the subject of a Wiener process or a random walk. The integral of the processrepresents the trajectory of the particle.

It simplifies matters if, at first, we assume that there is no resistance to the motionof the particle. Then, if V(t) denotes the Wiener process, we have the followingequation for the change in the momentum of the particle:

(a) = m\ dV( = p(x - a) (15)

Here the final equality invokes the physical law that a change in momentum is theproduct of a force p with the time T — a for which it acts. The product is describedas an impulse or an impact. In the present context, the value of p represents anaverage which is taken over the interval (a, T]. Nevertheless, it seems fair to describethe Wiener process as an 'impact process'.

The physical analogy helps us in visualizing the trajectory of the particle. If theparticle is already in motion and if its kinetic energy is large relative to the energyof the impacts, then its linear motion will be disturbed only slightly. In order topredict where the body might be in some future period, we simply extrapolate itslinear motion free from disturbances.

The discrete-time counterpart of the integrated Wiener process is a second-orderrandom walk whose values are formed by accumulating the values of a first-orderprocess (Figure 3). Thus, if {e,} and {vt} are, respectively, a white-noise sequenceand the corresponding cumulated sequence which constitutes a first-order randomwalk, then

yt = yt-\ + v,= y«-i + t),-i + e, (16)= 2y,-i-y,-2 + e,

defines the second-order random walk. Here the final expression is obtained bysetting vt-\ — yt-i ~yt-i in the second expression. It is clear that, to reduce thesequence {yt} to the stationary white-noise sequence, we must take first differencestwice in succession.

Dow

nloa

ded

by [

The

Uni

vers

ity o

f M

anch

este

r L

ibra

ry]

at 1

2:50

05

Dec

embe

r 20

14


The nature of a second-order process can be understood by recognizing that itrepresents a trend in which the slope—which is its first difference—follows a randomwalk. If the random walk wanders slowly, then the slope of this trend is liable tochange only gradually. Therefore, for extended periods, the second-order randomwalk may appear to follow a linear time trend.

To demonstrate that the forecast function for a second-order random walk is astraight line, we may take the expectations, which are conditional upon Jt, of theelements of the equation

(17)

For h periods ahead and for one period ahead, this gives

+ h\-ft) - y t + h\t= 2yt + h - i \ t - y t + h-2\t(18)

E(y, +11 Jt) = y, +11, = 2y, - yt -1

which together serve to define a simple iterative scheme. It is straightforward toconfirm that these difference equations have an analytic solution of the form

yt + h\t = a + Ph with a.=yt and fi = yt-yt-\ (19)

which generates a linear time trend.It is possible to define random walks of higher orders. Thus a third-order random

walk is formed by accumulating the values of a second-order process. A third-orderprocess can be expected to give rise to local quadratic trends; and the appropriateway of predicting its values is by quadratic extrapolation.

Before closing this section, we should consider the case where the particle whosemotion we are describing is travelling in a resistant medium. In a viscous fluid, theresistance to motion is directly proportional to velocity. For a body being driventhrough the fluid by a constant force p, the equation of motion would be

dvm— = p-cv (20)

where cv is the drag force. In discrete time, an analogous equation for the velocityof a particle driven by stochastic impacts is

v,-v,-\ = et-flv,-\ (21)

where /? e [0,1]. This becomes an ordinary random walk in v when [1 = 0, whereas,when P = 1, it becomes white noise. On setting <f) = 1 — /?, and rearranging theequation, we obtain

vt = (f>v,-x + et (22)

which is the equation of a first-order autoregressive process. On substituting forvi=yt— yt-1, this becomes

y, = (l+4>)y,-i-4>y,-2 + Bt (23)

This is the equation of a so-called integrated autoregressive (IAR) process (Figure4). The process may be envisaged as a first-order random walk which is driven byan autoregressive process instead of white noise.

To forecast the future position of a particle whose trajectory is described by anIAR process, we extrapolate its motion free from the disturbances of the forcing

Dow

nloa

ded

by [

The

Uni

vers

ity o

f M

anch

este

r L

ibra

ry]

at 1

2:50

05

Dec

embe

r 20

14


200

150

100

50

0 50 100 150

Figure 4. The sample trajectory and the forecast function of the IAR processy(t) = (1 + (j))y(t - 1) - <t>y(t - 2) + e(«) when ip - 09.

function but still subject to the resisting forces of the viscous medium. In thesecircumstances, the velocity of the particle will suffer an exponential decay which isgoverned by the equation

= <t>hv, (24)

(25)

The position of the particle at time t + h is forecast by the equation

where

a n d y = (26)

The long-term forecast is y, which represents the position where the particle willcome to rest when all of its kinetic energy is dissipated; and this may be more or lessremote from yt, depending upon the arresting forces of the medium.

5. Random walks with added noiseA stochastic trend of the random-walk variety may be elaborated by the addition

of an irregular component. A simple model consists of a first-order random walk withan added white-noise component. The model is specified by the equations

(27)

wherein v, and r\t are generated by two mutually independent white-noise processes.The noise t\t may be interpreted as an error of observation which has no effect uponthe trajectory of the particle.

Dow

nloa

ded

by [

The

Uni

vers

ity o

f M

anch

este

r L

ibra

ry]

at 1

2:50

05

Dec

embe

r 20

14


The equations combine to give

yt-yt-\ = Zt-£-\ + tit — rit-i(28)

= vt + rt,-rjt-i

The expression on the RHS can be reformulated to give

Vt+t1t-rit-i = e,-ne,-i (29)

where e, and e(-1 are elements of a white-noise sequence and /i is a parameter of anappropriate value. Thus, the combination of the random walk and white noise givesrise to the single equation

yt = yt-\ + et-iiEt-l (30)

The forecast for h steps ahead, which is obtained by taking expectations in the

equation yt + h=yt + h-\ + et + h~ H£t + h-\, is given by

E{yt+h\St)=yt+h\t = y't+h-\\t (31)

The forecast for one step ahead, which is obtained from the equationyt+\ =yt + Bt+i - /iEt, is

E(y, +11 Jt) = yt+I i (= yt - y-e,

yt\,-i) (32)

The result yt\t-\ —Vt-\ — fiet-\, which leads to the identity et = yt — $t\t-\ uponwhich the second equality (32) depends, reflects the fact that, if the information attime t—\ consists of the elements of the set J, -1 = {yt - \, y, _ 2,...} and the valueof n, then e,_i is a known quantity which is unaffected by the process of takingexpectations.

By applying a straightforward process of back-substitution to the final equation(32), it will be found that

yt+iu = (l - n)(yt + nyt-1 + . . . + ii'~ly\) + n'yo, (33)

= (1 - fi){y, + ny,-\ + fiyt-2 + .-.}

where the final expression stands for an infinite series. This is a so-calledexponentially-weighted moving average; and it is the basis of the widely-usedforecasting procedure known as exponential smoothing.

To form the one-step-ahead forecastyt+i\t in the manner indicated by the firstof the equations (33), an initial value yo is required. Equation (31) indicates that allthe succeeding forecastsyt + 2it,yt + 3it etc. have the same value as the one-step-aheadforecast.

When there are no errors of observation, the forecast values of a random walkare equal to the most recent observation on the process which is y,. When there areerrors of observation, the true value £( is, in effect, estimated by equation (33) whichcomprises the previous observations as well. These are discounted increasingly asthey recede into the past. The discount factor is determined by the value of ji whichlies in the interval [0,1). It can be shown that ji decreases monotonically as the ratioV{vt)IV{t]t) increases; and this is what our intuition might suggest.

Equation (30) is a simple example of an integrated moving-average (IMA) model.A more elaborate model of the autoregressive integrated moving-average (ARIMA)

Dow

nloa

ded

by [

The

Uni

vers

ity o

f M

anch

este

r L

ibra

ry]

at 1

2:50

05

Dec

embe

r 20

14


variety would be obtained by adding a white-noise component to the integratedautoregressive (IAR) model of the previous section. Such a model is of particularinterest since its forecast function corresponds to the device for extrapolativeforecasting which was developed independently by Holt [3] and Winters [5] viaarguments which were essentially intuitive.

6. Constrained stochastic motionAn idealized physical model of an oscillatory system consists of a weight of mass

m suspended from a helical spring of neglible mass which exerts a force proportionalto its extension. Let y be the displacement of the weight from its position of rest andlet h be Hooke's modulus which is the force exerted by the spring per unit ofextension. Then Newton's second law of motion gives the equation

m-A+hy = 0 (34)dt

This is an instance of a second-order differential equation. The solution is

) = 2pcos(a>nt-6) (35)

where con = (ft/zn)1'2 is the so-called natural frequency and p and 6 are constantsdetermined by the initial conditions. There is no damping or frictional force in thesystem and its motion is perpetual.

In a system which is subject to viscous damping, the resistance to the motion isproportional to its velocity. Then the differential equation becomes

d2v dv , ,„„m^ + c— + hy = 0 (36)

dt dt

The solution to the equation now becomes

\ = 2pexp(y')cos(cot-6) (37)where y < 0 is a damping factor directly related to the coefficient c. If the dampingis light, then the system will oscillate with a gradually decreasing amplitude. If thedamping is heavy, then the displaced weight will return to its position of rest withoutovershooting.

In contexts of electrical and mechanical engineering, it is natural to consider thebehaviour of damped system when its motion is driven by a sinusoidal forcingfunction. In such circumstances, the system will oscillate at the same frequency asthe forcing function, albeit with a phase displacement.

The amplitude of the motion will depend upon the natural frequency <un of theundamped system. If the damping is light and the natural frequency is close to thefrequency co of the forcing function, then the system will oscillate with a largeamplitude. In effect, it will absorb kinetic energy from the forcing function. If theforcing frequency differs widely from the natural frequency, then the system willrapidly dissipate energy and its oscillations will be small. The ratio of the amplitudeof y to that of the forcing function is described as the gain of the system.

The diagram which shows how systems behave under differing forcingfrequencies and with differing degrees of damping is called the frequency-response

Dow

nloa

ded

by [

The

Uni

vers

ity o

f M

anch

este

r L

ibra

ry]

at 1

2:50

05

Dec

embe

r 20

14


0.0 0.5 1.0 1.5 2.0 2.5Figure 5. The frequency response of a second-order system with various damping ratios.On the vertical axis is the gain in the amplitude of the sinusoidal motion. On the horizontalaxis is the relative frequency <w/a>n of the forcing function. The six curves, from the highest

to the lowest, correspond to the damping ratios f = 0 1 , 0-15, 0-2, 0-3, 0 4 , 0-6.

diagram (Figure 5). A similar diagram is used widely in the frequency-domainanalysis of time series where it corresponds to the spectral density function of asecond-order process.

Our purpose here is to find the discrete-time counterpart of the oscillatory systemand, at the same time, to consider how the system behaves when the forcing functionis a sequence of stochastic impulses.

To convert the differential equation of the undamped system (34) to thecorresponding difference equation, we replace the second derivative d2yldt2 byy, — 2yt-\ +yt-2, which is the second-order difference of y,. Also, we replace theterm in y within equation (34) by a corresponding term in yt-\. Finally, once theequation has been normalized by setting the coefficient of y, to unity, we may addto the RHS an element from the stochastic sequence {ej. The result is an equationin the form of

y t 2 t

(38),- \-yt- 2 + e,

where con is the natural frequency of the system.It is easy to see that equation (38) becomes the equation of a second-order random

walk in the case where con = 0. As we have already established, the second-orderrandom walk gives rise to trends which can remain virtually linear over considerableperiods. When con > 0, the equation will generate a quasi-cyclical stochastic sequencewherein the average period of a cycle is 2nl(on time units. A sequence generated byequation (38) when {e,} is a white-noise sequence is depicted in Figure 6.

Whereas there is little difficulty in understanding that an accumulation of purelyrandom disturbances can give rise to linear trend, there is often surprise at the factthat such disturbances can also generate cycles which are more or less regular. Anunderstanding of this phenomenon can be reached by considering a physical analogy.One such analogy, which is very apposite, was provided by Yule [4] whose article

Dow

nloa

ded

by [

The

Uni

vers

ity o

f M

anch

este

r L

ibra

ry]

at 1

2:50

05

Dec

embe

r 20

14


0 25 50 75 100Figure 6. A quasi-cyclical sequence generated by the equation y, = 2coscony,-1 — yt~2 + B,

when <on = 20°.

of 1927 introduced the concept of a second-order autoregressive process of whichequation (38) is a limiting case. Yule's purpose was to explain, in terms of randomcauses, a cycle of roughly 11 years which characterizes the Wolfer sunspot index.

Yule invited his readers to imagine a pendulum attached to a recording device.Any deviations from perfectly harmonic motion which might be recorded must bethe result of superimposed errors of observation which could be all but eliminatedif a long sequence of observations were subjected to a regression analysis.

The recording apparatus is left to itself and unfortunately boys get into theroom and start pelting the pendulum with peas, sometimes from one side andsometimes from the other. The motion is now affected not by superposedfluctuations but by true disturbances, and the effect on the graph will be of anentirely different kind. The graph will remain surprisingly smooth, butamplitude and phase will vary continuously.

The phenomenon described by Yule is due to the inertia of the pendulum. In theshort term, the impacts of the peas impart very little energy to the system comparedwith the sum of its kinetic and potential energies at any point in time. However, ontaking a longer view, we can see that, in the absence of clock weights, the system isdriven by the impacts alone.

The equation which Yule fitted to the sunspot index was a damped version ofequation (38) which takes the form

(39)p2yt - 2

where a>n is the natural frequency and p e (0,1) is the damping factor. This is theequation of a stationary or stable second-order autoregressive AR(2) process.

The parameter values estimated by Yule from the sunspot data were <$i = 1-343and $2 = 0-655. These imply estimates of &>n = 33-96 and p = 0-809 for the natural

Dow

nloa

ded

by [

The

Uni

vers

ity o

f M

anch

este

r L

ibra

ry]

at 1

2:50

05

Dec

embe

r 20

14


6

4

2

0

-2

- 4

-6

A

1 1

1

1

n

M U 1 |

1 1

s /

' A ^

0 20 40 60

Figure 7. The sample trajectory and the forecast function of the 2nd-order processy, = 2/?cos&>n>>,_ [ — p2y,-2 + B,, when a>n = 33° and p = 0-9. The dotted lines define a predic-

tion interval of plus and minus twice the root-mean-square error of the forecast.

frequency and the damping factor, respectively. When these parameters are used inconstructing a simulated series, the result is a trajectory which is considerably lessregular than that of the original sunspot series. Also, the forecast function which isimplied by these estimates converges quite rapidly to zero (Figure 7). To obtain amore regular series, the value of p must be set somewhat closer to unity.

It is a feature of the classical estimators of autoregressive models that they tendto overestimate considerably the degree of damping in lightly damped systems; and,nowadays, in electrical and mechanical engineering, where this feature can be asevere disadvantage, alternative estimators, such as the Burg estimator [5] are usuallypreferred. It may be true of the sunspot series that the overestimation of the degreeof damping is also due, in part, to the asymmetry of the waves. This was the opinionof Yule himself.

7. Flawed analogies in time-series analysisIn econometrics, it is a common practice to use an undamped system to model

the seasonal fluctuations in economic time series. The classical example of a seasonalARIMA model is one which was applied by Box and Jenkins [6] to a series of monthlyobservations on the number of airline passengers. The difference of the logarithmof the series, which we shall denote by ylt may be modelled by an equation in theform

^ = ^ - 1 2 + 6,-06,-12 (40)

which is a slightly simplified version of the equation of Box and Jenkins. This formcomes from combining the equations

(41)

Dow

nloa

ded

by [

The

Uni

vers

ity o

f M

anch

este

r L

ibra

ry]

at 1

2:50

05

Dec

embe

r 20

14


-5

-10 -

- 1 5

0 20 40 60Figure 8. The sample trajectory and the forecast function of the nonstationary 12th-order

process y, = y,- n + e,.

wherein v, and t]t are independent white-noise disturbances. These are comparableto the equations (27) which represent a random walk observed with error. In bothcases, the error of observation r\ has no effect upon the trajectory of £.

The forecasts of a seasonal series modelled by equation (40) are generated by asimple 12-order difference equation. The difference equation reproduces a patternof seasonal fluctuations repeatedly without any attenuation or modification.

In the case where 0 = 0, this pattern is provided by the twelve observations ofthe series which are most recent at the time of making the forecast. Theseobservations serve as the initial conditions for the difference equation. In the casewhere 0 =£ 0, the pattern of the annual cycle which is defined by the initial conditionsis based upon a weighted average of all previous cycles wherein the weights aregeometrically declining in the manner of equation (33). It is precisely the perpetualreproduction of a cyclical pattern which has shown considerable regularity withinthe sample that makes the seasonal ARIMA model so successful in the predictingof economic time series.

It is worth observing that, whereas their appearances are superficially similar, theseasonal economic series and the series generated by equation (40) are, fundamen-tally, of very different natures. Therefore, the physical analogy which we are temptedto apply in modelling a seasonal economic series is flawed. In the case of the seriesgenerated by an undamped stochastic differerence equation, there is no bound, inthe long run, on the amplitude of the cycles. Also there is a tendency for the phasesof the cycles to drift without limit. If the latter were a feature of the monthly timeseries of consumer expenditures, for example, then we could not expect the annualboom in sales to occur at a definite time of the year. In fact, it occurs invariably atChristmas time.

A clue to the disparity between the model and the economic reality is providedby the confidence intervals of the forecast function which are based on theroot-mean-square error of the forecast. These define a band which increases in widthin proportion to the lead time of the forecast (Figure 8). This suggests that, beyond

Dow

nloa

ded

by [

The

Uni

vers

ity o

f M

anch

este

r L

ibra

ry]

at 1

2:50

05

Dec

embe

r 20

14


a short distance into the future, the forecasts have very little reliability. In fact, theseasonal ARIMA model is so successful in economic applications precisely becausethe forecasts are effective for long periods.

It should be clear by now that the forecasting success of the seasonal ARIMAmodel is a fortuitous circumstance which is in spite of the model's inappropriatenessas a description of the processes generating the data. There are lessons to be learntfrom this contrary result. The first lesson is that, in statistical modelling, we oftensucceed in doing the right thing for the wrong reasons. Therefore we should remainflexible and opportunistic in our choice of models.

The second lesson is a more subtle one. It begins with a recognition that allstatistical models are only analogies for the empirical processes to which they areapplied. They should not be mistaken for the processes themselves. It follows thata good way of understanding the worth and the limitations of a model in anyapplication is to associate the model in advance with a complete and coherent analogysuch as an analogy drawn from the realm of the physical sciences. This may becompared with the new and tentative analogy which is implied by the modelcurrently under construction.

If we pursue the new analogy keenly, then we can find the points where it isweakest. The discovery of the weak points may suggest the reasons for the failureof the model. If the model is useful in spite of weak points in the analogy, then wehave been lucky. The knowledge that our good luck is fortuitous should prevent usfrom depending upon it unreasonably in other circumstances.

References[1] NEWTON, I., 1686, Philosophae Naturalis Principia Mathematica (The Mathematical

Principles of Natural Philosophy), translated into English by Andrew Motte 1729.Translation revised and supplied with an historical and explanatory appendix byFlorian Cajori 1934. Cambridge University Press.

[2] HOLT, C. C., 1957, O.N.R. Memorandum, No. 52, Carnegie Institute of Technology.[3] WINTERS, P. R., 1960, Management Sci., 6, 342.[4] YULE, G. U., 1927, Phil. Trans. F. Soc, 89, 1-64.[5] BURG, J. P., 1978, Modern Spectral Analysis (New York: IEEE Press) pp. 34-46.[6] Box, G. E. P., and JENKINS, G. M., 1976, Time Series Analysis: Forecasting and Control,

revised edition (San Francisco: Holden-Day).

Dow

nloa

ded

by [

The

Uni

vers

ity o

f M

anch

este

r L

ibra

ry]

at 1

2:50

05

Dec

embe

r 20

14

Documents

Metaphors for time‐series analysis