Looking Backwards to theFuture

Tony Lawrance

Department of Statistics

University of Warwick

1

2

First of all, sincere thanks for making this such a great day for me -(provisional remark…)

Especially – JohnTheodore

and thanks to the Statistics Department for ‘sponsoring’ the event

3

Looking backwards to the future – what does it mean ?

An excuse to briefly look back on an enjoyable time in statistics with a wish to alsolook forward to some more time in statistics… Will try and pin the talk on somesignificant and not so significant events in my statistics life

Nearly 40 years of statistics before Warwick – so some reminiscing here for the firsttime here may be acceptable…

In Warwick for just less 10 years – but very enjoyable ones

Most of my publications are now on the site ‘researchgate.net’

Diary of LifeMaths undergraduate in Leicester – graduated 1963‘Intimidated’ into statistics by Nageeb Rahman, a Cambridge PhD student of HenryDaniels – in that, I am the two-year elder ‘statistical brother’ of Phil Brown

Nageeb sent me in 1963 to Aberystwyth for an MSc (and then Phil Brown in 1965)because Dennis Lindley from Cambridge had started a Stats Department there in1960, with David Bartholomew, Mervyn Stone and Ann Mitchell (Dennis was inHarvard for half my year, but taught frequentist inference in the second term)

4

Carol? DonaldEast SylviaLutkins DavidBartholomew DennisLindley MervynStone AnnMitchell PeterKing Eileen?

GwynJones MikeSamworth PgslyGwynne GrahamPhipp ^ JeffWood ClivePayne ?Bambegye BasilSpringer ErylBasset RichdMorton

Department of Statistics, Aberystwyth 1963-64

5

The IBM 1620 Electronic Computer, Aberystwyth Stats Dept 1963

Out of bounds to MSc students

6

After MSc -Leicester October 1964 - started as a tutorial assistant 1 year -> assistant lecturer

Frank Downton, d 1986 ?Nageeb Rahman, d 90’s ?Mike Phillips – 1968-…Brian English – 1969-70?

Took 4 ‘summers’ to get a PhD, Stochastic Point Processes’, awarded in 1969.Started by Frank Downton giving me a sheet with a few references … To 7

Lightly supervised by Frank Downton, who almost immediately after my arrival backin Leicester moved to Birmingham, enticed by Henry Daniels

Never-the-less, Frank Downton had big influence encouraging me, researchconfidence building…

Another big influence in supporting my career was my external examiner David Cox

So this seems a good point to get a bit more technical

Diary of Life

7

(back to 6)

PhD and Point Processes…Time series of point events on the line – mainly Poisson and renewal processes at thetime – spatial or dependent interval versions had not been much considered

8

My first issue was what was meant by a ‘typical event’ to start an interval in astationary point process ?

I wrote to David Cox – good question, he said ! “We have avoided it in my justcompleted Methuen monograph with Peter Lewis” on ‘Series of Events’ – 1966

I went for dependent interval versions with stationarity and first studied Cox’s 1954Biometrika paper on ‘superposition of renewal processes’ or ‘pooled processes’

What was the inter-point distribution and dependency of this process ?

So after a while I investigated two ideas…

time

Process 1

Process 2

Superposition

(I hope my memory is correct !)

9

An Average Event – an interval beginning with an ‘average event’ in thestationary PP with intervals

1 2, ,...X X

has distribution 1

1limn

ii

P X x P X xn n

…a bit clunky

An Arbitrary Event - a more elegant approach follows from Khintchine’s (1955)work** on stationary input processes for queues**. This developed from ‘Palmdistributions’, referencing Palm (1943) , who introduced the idea of an intervalbeginning with ‘at least one point’ in a telephone queuing context

( , )N t t Thus, with the counting variable in a stationary point process, thedefinition of the distribution of an interval beginning with an arbitrary event is

lim ( , ) 0 | (0, ) 1)0P X x P N x N

It turned out that this definition mathematically connected the idea of an arbitraryevent with that of an arbitrary time, and involved length-biased sampling andforward and backward recurrence times – previously informal concepts for ageneral stationary point process

My thesis work also contained work on this arbitrary event approach and onparticular point processes…

(To 10)

10

Khintchine (1894-1959). Mathematical Methods of Queuing, 1955, English Eds,1960, 1969, Griffin

From the introduction…

(back to 9)

.

11

My First Seminar was 25 Feb 1970 at UMIST, Manchester, on ‘selectiveinteraction of point processes’, one of my PhD point processes

My Most Recent Seminar reconstructed part of my first seminar at the MauricePriestley memorial meeting, 18 December 2013…

The selective interaction model was introduced by the Dutch neurophysiologists TenHoopen and Reuver (1965, 1967) to explain multi-modal inter-spike distributions fordark firing of lateral geniculate neurons, observed by Bishop et al (1964)

The process can be explained as follows - you can see that I was rather keen ongraphics even in those distant days…

I explored it as an applied probability model. I really wish now that I had followed up onthe statistical aspects, contacting the experimenters, analysing their data, attempting tocollaborate, etc, and doing simulations – but there was little electronic computing andno internet, and Holland was a long way away

Diary of Life

(from my thesis)

The Selective Interaction Neuron Firing Point Process Model

Excitatory

Inhibitory

ObservedResponse

The model was justified empirically by a multi-modal distribution of times betweenthe responses’, in the ‘spike trains’ of observed neuron firings – convolutions ofexcitatory intervals

Poisson excitatory results by very detailed calculation – in my thesis

General results by appealing to the compound distribution structure of the observedresponse count, resulting in

stnryIntervalprocess

stnry stocpnt countprocess

This image cannot currently be displayed.

, ( )i II y

This image cannot currently be displayed.Selective interaction process

12

from Priestley meeting talk

( )

, ,1

( ) ( ) , 1 { ( ) 0} 1 { ( ) 0}, 0IN t

i iR E E I E I E i E i

i

N t N t with prob P N I P N I otherwise

Continued, (J Appl Prob papers 1970-71 &1979)

Excitatory

Inhibitory

Response

It follows

and approximately (?) via compound distribution results

sdevs

Compounding the exciting process intervals using the inhibitory process to get theinter-response distribution is more difficult…but I used arbitrary events

For more detailed results when the excitatory process is Poisson, see my 4 JAPpapers in the 70’s. No model fitting, no simulations – what a pity !

stationaryintervalprocess

stationarystoc ptcountprocess

( )EN t

, ( )i II y

( )RN tSelective interaction process

0{ ( )} Pr{ ( ) 1} ( )R E I E Iy

E N t N y y dy t

,var{ ( )} [ ( ) var( )]R E I I E IN t E t

( )

, ,1

( ) ( ) , 1 { ( ) 0} 1 { ( ) 0}, 0IN t

i iR E E I E I E i E i

i

N t N t with prob P N I P N I otherwise

,E I

14

1970 - Next move - the year 970/71 at the ‘IBM Thomas J Watson ResearchCenter’, New York, invited by Peter Lewis

Extended and consolidated PhD work by investigating branching Poisson processpoint models for computer failures, and co-organizing big point process conference

1972 – Returned to Leicester for 1 year – moved to Birmingham for years

1973-2004 My Birmingham Years

1970 – After PhD exam joined David Cox’s weekly PP journal club at IC from Leicester

met Valery Isham,Anthony Atkinsonat IC

Diary Life

52

Henry DanielsDavid WishartPaul DaviesPhil BertramRoger Holder

Frank DowntonMalcolm Faddy

Alan GirlingJohn CopasChris Jones

Richard AtkinsonFrank CritchleyPrakash Patil

Christmas Meal 1981/82PhilB? FrankD ? Chris Gray AJL AnnieM ChrisJ TriciaC

15

KamilaZ WolfgangB AlanG PrakashP SaidS MalcolmF RichardA

Birmingham Group (when MalcolmF moved back to NZ for second time, 2003)

16

1973 – Farewell Point Processes

Found research opportunities in hydrology (from teaching with Nath Kottegoda inCivil Engineering) after devising a course in hydrological time series for Bham MSc inHydrology

RSS Read Paper on the topic with Nath Kottegoda (Stochastic Modelling of Riverflow Time Series)

Teaching has influenced my ‘choice’ of research areas quite a bit but not thereverse

1973 – Hello Time Series – as it was moving into the nonlinear era

Time series started to move away in several directions from ‘Box-Jenkins’ linearGaussian models to be able to capture more statistically varied and complexbehaviour

Maurice Priestley, with non-stationary processes and spectraHowell Tong, with dynamical-statistical thresholdsRobert Engle, Clive Granger, with volatility, co-integrationPeter Lewis et al, with specified nonGaussian models, including discrete distributionmodels, simulation in operations research

1980-1990 Worked on non-Gaussian time series models with Peter Lewis, by thenat Naval Postgraduate School, Monterey, California (nice summers)

Examined Jane’s PhD on ‘dry’ rivers…

17

Peter Lewis, 1932-2011

18

1978-80 – Work started with nonGaussian solutions to linear time series models,exponential, mixed exponential, gamma

1980-87 - Then ways to formulate autoregression operation with nonGaussianvariables – in ways natural to the particular distribution, e.g. convolution andmultiplication, minimization

1989-90 – Non-reversibility, directionality, in nonGaussian linear time series

An early linear problem – it’s easy to set up …(so I describe here)

The AR(1) Innovation Problem

How to specify the error distribution for an AR(1) process with specified marginaldistribution

Gaver & Lewis made a start with the gamma distribution but could not explicitly obtainthe innovation distribution..…

1t t tX X

19

The AR(1) Innovation Problem – ‘epsilon for given X’

Solution easy in terms of Laplace transforms – Gaver & Lewis, from

Exponential( ) solution clear:0 with proby

( ) with proby1t

tE

Gamma solution –>( )k

X zz

Can you invert this Lapalce transform without serendipity ?

( )k

zz

z

‘Consider a shot noise process in continuous time’, of course…

1

i

NU

t ii

Y

(0,1)iU uniform1( log )N Poisson k

( )iY exponential

A compound Poisson distribution

( ) ( ) ( ), ( ) ( ) ( )X X X Xz z z z z z

1 , 0 1, , ??distbnt t t t tX X X D

20

Diary Life

1985 - RSS ‘read paper’ on nonlinear AR exponential variables, with Peter Lewis

1986 – ISI Tashkent - Very Sick ! (Time series directionality)

1986 - Began teaching inference in Bham - beginning of regression diagnostics

1986 – Seconded RSS vote of thanks at Cook’s 1986 local influence read paper, andshowed how it applied to regression transformation diagnostics

1988 - JASA paper on regression transformation local influence diagnostics

1988 Got chair in Bham (poster of inaugural lecture)

1989 – Papers on regression transformation score statistics

1991 - IMA Minnesota Robustness & Diagnostics workshop (photos Anthony, Frank)

1981-1991 Tim Davis PhD collaboration ‘Survival of Tyres’, Dunlop-Sumitomo-Ford1991 Tim Davis PhD

1995 - Regression diagnostics – Cook’s bivariate & conditional distance

1995 - 98 Engine mapping, with Tim Davis, Tim Holiday- PhD-1996

1992- Statistical aspects of chaos

1998- Chaos-based communications

(To 23, 24)

(To 21, 22)

Gary Brown PhD 1995

Technometrics paper 1998

Biometrika papers 1987, 1989-ACA

took over my research & publication (To 25)

21

(Back to 20)

(Back to 20)

23

A trip across Minnesota and Iowa with Anthony Atkinson and Frank Critchley toSpillville, Iowa, to visit Dvorak connections, 1991, on the workshop rest day…

(To 24)

Spillville, Iowa 1991

Dvorak’s ‘American Quartet’(String Quartet in F Major,op96) composed here in1893, also, String Quintet inE Flat Major,op97(sometimes called the‘Spillville Quintet’), and afterreturning to NY, hisHumoresque, No 7 in G FlatMajor

Back to 20

25

25

1992- 2010 Statistical aspects of chaos, leading to

Chaos-based communications

‘What got me started’… the Uniform Distribution Solution to the AR(1)Process – Bartlett’s last paper, probably (another case of the AR(1) innovation problem)

1

1 1 1, , 1, 2, ...,t t t t

iU U wp i k

k k k

Chaos – instabilities produced by a deterministic rule

CollaboratorsBala BalakrishnaAlexander BaranovskyTohru KhodaGan OhamaRodney WolfTheodore PapamarkouNancy SpencerAtsushi UchidaChibisi Chima-Okereke

26

1

1 1 1, , 1, 2, ...,t t t t

iU U wp i k

k k k

1 ( ) mod(1)t tU kU

The reverse of this model is the following chaotic and deterministic model

And, incidentally, there is a negatively correlated version reversing to

1 { (1 )}mod(1)t tU k U

deterministic rule called achaotic shift map ~ like cntscongruential random numbergenerator

It follows (and more) generally that deterministic chaotic processes have statisticalproperties, i.e., there are statistical properties of chaos

Such ideas prompted some electronic engineers to have the idea of ‘communicatingwith chaos’ – instead of communicating with sinusoidal radio waves

Where is the chaos from this model?

Transmit onebit b=+/- 1

ChaoticSpreading

Signal

1

n

i iX

( )

1,2,...,

ib X

i n

Channel Noise

1

n

i i

Received Signal

Also available incoherent case

( )

1,2,...,

i i iR b X

i n

1

n

i iX

Decoder

bit =b̂

A particular chaos communication system using a chaotic map is

Chaos Shift Keying (CSK) – ‘Coherent’ Case -simplest

Exact theory for bit error rate of such a system, Lawrance & Ohama (IEEE, 2002)

2

( 1)

1( )

( ) ( )

n id

i

x c

xBER N f x dx

1( )i iX X

(estimate b)

28

Performance of CSKAssessed by bit error rate (BER)

Depends on statistical aspects of the system as well as the dynamics, according toprevious formula

Worst; IID Gaussian

Logistic map

Shift map

Best:circular map andtheoretical lower bound

Area has moved on from chaotic-map and electronic circuitry chaos to laser-chaos communication; this is still a research area but with several experimentaldemonstrations and US military applicationsCurrent work with Atsushi Uchida and Chibisi Chima-Okereke

Different types of chaoticspreading, compared toIID Gaussian

Optimum circular mapspreading: Ji Yao, TPapamarkou

-> Police mergers

29

A Brief Diversion - In the Press…

Police Mergers 2006 – the misuse of statistics

Total Score by Force (excluding London)

p is significant to the 0.01 level

R2 = 0.58090.00

10.00

20.00

30.00

40.00

50.00

60.00

70.00

80.00

0 1000 2000 3000 4000 5000 6000 7000 8000 9000

Force Size (Officer Strength)

To

talS

co

re

Line equates

to an average

score of 3score = 3

63

Government O’Connor Report said : This strongly suggests that forces with over4,000 officers (or 6,000 total staff) tend to meet the standard across the range ofservices measures in that they demonstrate good reactive capability with a clearmeasure of proactive capacity…’.

Charles Clarke.Home Secretary

4000

30

What this plot shows to me is:

Rather rough upward scatter of points

Least squares line is misleading because of extremes

Large variability at each force size – very important

Line at 63 shows most forces ‘fail’ – artifact of scoring and choice of ‘3’

Meaningless statistical elaborations of p-value and R-squared due to automatic use ofsoftware

No justification of 4,000 figure

Total Score by Force (excluding London)

p is significant to the 0.01 level

R2 = 0.58090.00

10.00

20.00

30.00

40.00

50.00

60.00

70.00

80.00

0 1000 2000 3000 4000 5000 6000 7000 8000 9000

Force Size (Officer Strength)

To

tal

Sco

re

Line equatesto an averagescore of 3score = 3

63

What I said about the O’Connor Report:

31

Another example of rubbish in the O’Connor report

Overall Trend for Protective Services

For c e S i z e (Smallest f rom lef t )

Sco

re

Ser ious &Or ganised

Publ ic Or der

Major Cr ime

Roads Pol icingCivi l Contingencies

Cr i tical Incidents

CT &DE

What I said about this plot ‘This is an almost perfect example of how not to present agraph - no scales on either axis, no data plotted to justify the lines drawn. It is almostimpossible to obtain any critical understanding from it, except that it is intended toprove that score for protective capability increases with force size’

32

What was said in the House of Commons:

MP David Davis: ….Frankly, the best that I can do is to repeat to the House thecoruscating opinion of Professor Lawrance, a professor of statistics at WarwickUniversity…

MP Adrian Baily: …I rather regret the attempt by the University of Warwick torubbish the statistical basis and the credibility of that report. It has a goodpedigree and I shall make my judgement on the balance of professional policeopinion, rather than on the opinion of university professors in Warwick…

Another newspaper appearance…

33

A Publication in ‘The Sun’… - 14th October 2013

34

A Publication in ‘The Sun’… - 14th October 2013

A MATHS professor has told The Sun bills are so complicated even he can’t understand them. Tony Lawrance,right, of Warwick University said : “They’re absurdly over-complicated. Most professors would find them difficultto understand – the public doesn’t stand a chance.’’

35

Bala BalakrishnaGan OhamaRachel HilliamYi YaoTheodore PapamarkouChibisi Chima-OkerekeAtsushi Uchida

Chaos-based Communications 2001 – 2014 - ??

Current work|: laser-chaos-based communications

(laser = light amplification by stimulated emission of radiation)

Key laser features of laser-based communication

Collaborators:

A message is hidden in a segment of the chaotic laser sequence - steganography,rather than cryptography when a message is visible but has to be decoded

75%

With BalaBalakrishna,CochinUniversityKerala

1. Lasers can produce chaotic waveswhich look stochastic – (use semi-conductor laser with optical feedback)

2. Lasers producing chaotic behaviourcan be synchronized by a trigger signal

36

Current Work-1: Laser-based Chaos Communication

Experimental data via collaboration with Atsushi Uchida, Saitama University,Tokyo, and analysis collaboration with Chibisi Chima-Okereke ofActiveAnalytics, Bristol

Each set of data consists of three time series of 10m values

Experiment set up to probe chaos shift-keying system of communication using semi-conductor lasers with optical feedback and transmission though 60m fibre optic cable

andbinarymessage

binarymessage b

37

Experimental setup not quite so simple as it may have seemed…

38

Some Experimental Results

5004003002001000

0.3

0.2

0.1

0.0

-0.1

-0.2

-0.3

-0.4

-0.5

-0.6

Time Index - 5m

ad

jDrv

_w

Op

tN

se

_1

Adjusted Received and Synchronized Laser Signals (5,000,001:1,000,500)

0.30.20.10.0-0.1-0.2-0.3-0.4-0.5-0.6-0.7-0.8

12

10

8

6

4

2

0

Intensity

De

nsit

y

Intensity of Drive Laser

0.30.20.10.0-0.1-0.2-0.3-0.4-0.5-0.6-0.7-0.8

16

14

12

10

8

6

4

2

0

Intensity

De

nsit

y

Adjusted Optical Noise

Example of laser synchronization

Is Optical NoiseIndependent ?

Drive laser Optical Noise

Based on post-processing for instrument effects – Noise not Gaussian

39

0.23

5

0.22

5

0.21

5

0.20

5

0.19

5

0.18

5

0.17

5

0.16

5

0.15

5

0.14

5

0.13

5

0.12

5

0.11

5

0.10

5

0.09

5

0.08

5

0.07

5

0.06

5

0.05

5

0.04

5

0.03

5

0.02

5

0.01

5

0.00

5

-0.005

-0.015

-0.025

-0.035

-0.045

-0.055

-0.065

-0.075

-0.085

-0.095

-0.105

-0.115

0.15

0.10

0.05

0.00

-0.05

-0.10

No

ise

Bo

xp

lots

Boxplots of Optical Noise versus Drive Signal Strength

Distribution of Optical Noise Conditional on Driver Signal Strength

N.B. BER v SNR plot under development, but initial work indicatesacceptable values can be obtained using range of SNR controlled byrange of spreading

40

Current Work-2:Volatility Modelling and Exploratory Graphics

Topic comes from teaching financial time series in the Financial Mathematicsmasters program

Financial time series ‘means’ volatility modelling

Volatility is changing conditional variance in a time series

Motivation – volatility models are routinely used without justification of the type ofvolatility structure existing in the data series

But it has not been clear how to reveal volatility structure

Attitude has been ‘fit the model you think will be ok and undertake somegeneral tests of its fit’ - but never obtain the empirical volatility and compareit with the model volatility

My attitude is ‘get an empirical version of the volatility function and choosea model which gives a good volatility fit, i.e. get the volatility right first’ -may be not the purest of likelihood approaches – but surely volatility is themost important aspect of volatility models !

1var( | )t tX X

1 1 , (0,1)( )t tX IIDt t tXX

The General Volatility Model to be used

41

FTSE100 Daily Data

4th Jan 2005 – 10th Feb 2011

Daily Adjusted Closing Values and Daily Returns

01/01/201101/01/201001/01/200901/01/200801/01/200701/01/200601/01/2005

8000

7000

6000

5000

4000

10%

5%

0%

-5%

-10%

Daily Date

Retu

rns

FTSE

Valu

es

42

1 1ˆ ) ( | )t t t tcalculate x smo x

1 2

211 1

2

ˆ1 ( 1) ( ) / ( | )n

t t tt

n x smo x

1 1( ) ( | )t tx smo x

1 1 , (0,1)( )t tX IIDt t tXX

Volatility Graphics

Based on the general volatility model for returns

1 volatility functiontX

Graphics Steps

(unscaled individual volatilities) (smoothed unscaled individual volatilities)

scaling gives standardized innovations

Smoothed & scaled i-volatilities give empirical version of volatility function1( )tx

Journal of the Royal StatisticalSociety, Series C, Applied Statistics(2013) 62, Part 5, pp. 669-686

( nearly constantwith returns)

43

75310-1-3-5-7

12

10

8

6

4

2

1

0

Previous Return

Vo

lati

lity

Scaled Individual Volatilities and Their Smooth

Empirical volatilityfunction

44

75310-1-3-5-7

12

10

8

6

4

2

1

0

Previous Return

Vo

lati

lity

Bootstrapping the Volatility Function

That’s Enough, except…

(see 20013 JRSS’C’ paper for more details)

45

The one nice thing about getting olderis that younger people follow you…

46

47

Many thanks