Ulf Grenander�mathemati al tools for pra ti al problems�Georg Lindgren11Mathemati al Statisti sLund University23rd Nordi Conferen eonMathemati al Statisti sVoss, June 15, 2010Lindgren - georg�maths.lth.se Ulf Grenander

Career � I◮ Born 1923 in Västervik, Sweden◮ Undergraduate studies in Uppsala and Sto kholm 1942-1946◮ PhD thesis: Sto hasti Pro esses and Statisti al Inferen e

Arkiv för Matematik 17, 1950Lindgren - georg�maths.lth.se Ulf Grenander

Career � II◮ Early en ounter with omputers � BARK (1947�1951) andConny Palm at KTH, Sto kholm◮ Tea hing at KTH � sto hasti pro esses and inferen e, Hilbertspa e theory � �Best tea her ever� (KJÅ, Sto hasti ControlTheory)◮ US/Sto kholm tour � Chi ago, Berkeley, Brown university(1951�1958)◮ Professor in Insuran e mathemati s and mathemati alstatisti s in Sto kholm after Harald Cramér (1958�1966)◮ Professor in Applied Mathemati s at Brown University (1966�)◮ S ienti� dire tor of the Swedish Institute of AppliedMathemati s, ITM (1971�1973)Lindgren - georg�maths.lth.se Ulf Grenander

Always ahead◮ Constant sour e of ideas for the future of applied mathemati sand statisti s ...◮ ... and of inspiration/en ouragement for young people◮ Often abstra t, always on rete; not always so immediate,great impa t but with some time delay

Lindgren - georg�maths.lth.se Ulf Grenander

Publi ations◮ More than 15 books

◮ Statisti al analysis of stationary time series, 1957◮ Toeplitz forms and their appli ations, 1958◮ Probabilities on algebrai stru tures, 1963◮ Abstra t inferen e. 1981◮ Hands: A pattern theoreti study of biologi al shapes, 1990◮ ...◮ Pattern theory, 2007

◮ Almost 100 resear h arti les◮ Representation of knowledge in omplex systems, JRSS B,1994

◮ A onversation with Ulf Grenander, Statisti al S ien e, 2006Lindgren - georg�maths.lth.se Ulf Grenander

Impa t?◮ Does itation ounts and bibliometry really measure impa t?

Lindgren - georg�maths.lth.se Ulf Grenander

Three main fo i◮ a general theory for statisti al inferen e in sto hasti pro esses◮ tea hing and training in applied mathemati s◮ pattern theory

Lindgren - georg�maths.lth.se Ulf Grenander

Sto hasti pro esses and statisti al inferen e, 1950◮ Sto hasti pro esses as obje ts for lassi al statisti al inferen e◮ Whi h mean fun tion of the Ornstein-Uhlenbe k pro ess is themost likely?H0: red urve = m0H1: bla k urve = m1

0 10 20 30 40 50 60 70 80 90 100−4

−3

−2

−1

0

1

2

3

4

Lindgren - georg�maths.lth.se Ulf Grenander

Compute the �likelihood� of m0 and m1 against m ≡ 0◮ Observe the Gaussian pro ess X (t) with mean m(t) and known ovarian e fun tion r(s, t) = Cov(X (s),X (t)) over a ≤ t ≤ b◮ Assume X (t) is ontinuous so all information on thedistribution an be obtained from asequen e of OBSERVABLES,whi h are hosen as yk =

∫ ba hk(t)X (t) dt◮ Sin e X (t) is Gaussian, all yk are normal variables◮ Choose the test fun tions hk (t) so that the yk be omeindependent and Var(y1) ≥ Var(y2) ≥ . . .

◮ How is that possible?Lindgren - georg�maths.lth.se Ulf Grenander

A mathemati al fa t and its statisti al onsequen e◮ If r(s, t) is a ontinuous ovarian e fun tion, then there exists(eigen)fun tions hk (t) and (eigen)values 1 ≥ 2 ≥ . . . withthe property

∫ ba r(s, t) hk(s) ds = khk (t)∫ ba hj (t)hk(t) dt =

{ 0 if j 6= k1 if j = k◮ Then yk =

∫ ba hk (t)X (t) dthas the desired properties!Lindgren - georg�maths.lth.se Ulf Grenander

Properties of yk◮ Sin e X (t) = m(t) + X̃ (t) where X̃ (t) has mean 0,yk =

∫ ba hk(t)m(t) dt +

∫ ba hk (t) X̃ (t) dtwhere the �rst term ak , is a number, depending on the truemean value fun tion, and the se ond term is normal (0, k )regardless of what hypothesis is true◮ The hypotheses an be expressed as yk ∈ N(ak , k ),independent, H0 : ak = ak0 =

∫ ba hk (t)m0(t) dtH0 : ak = ak1 =

∫ ba hk (t)m1(t) dtLindgren - georg�maths.lth.se Ulf Grenander

The Likelihood ratio test ...◮ The Likelihood ratio between H1 and H0 based on y1 is

ℓ(y1) =e−(y1−a11)2/2 1e−(y1−a01)2/2 1 = ey1×(a11−a01)/ 1−(a211−a201)/2 1 = eZ1with Z1 ∈ N(±(a11 − a01)2/2 1, (a11 − a01)2/ 1), ± hosena ording to if H1 or H0 is true.

◮ With independent y1, . . . , yn, . . ., n ≤ ∞

ℓ(y1, . . . , yn) = ePn1 Zkwhere ∑n1 Zk isN (± n∑1 (a1k − a0k)2/2 k , n∑1 (a1k − a0k)2/ k)Lindgren - georg�maths.lth.se Ulf Grenander

... reje ts H0 if ∑Zk is large◮ Expressed in the hypotheti al mean fun tions, withf (t) =

∞∑1 hk(t) a1k − a0k k ,the test quantity is∞∑1 Zk =

∫ b1 f (t) (X (t) − m0(t) + m1(t)2 ) dt◮ Proofs use all of ontemporary probability theory � but alsoexpressed in useful languageLindgren - georg�maths.lth.se Ulf Grenander

Impa t !◮ The example is the �mat hed �lter� developed for radardete tion during WW II◮ Important monograph:Grenander & Rosenblatt: Statisti al analysis of stationary timeseries, Wiley 1957◮ Set a standard for modern textbooks in signal pro essing,ele tri al engineering � great impa t on Sto hasti ontroltheory◮ From the prefa e: ... to dire t his [the statisti ians℄ attentionto an approa h to time series analysis that is essentiallydi�erent from most of the te hniques used by time seriesanalysts in the past.Lindgren - georg�maths.lth.se Ulf Grenander

Professor in Sto kholm 1958 � 1966◮ Statisti s and probability edu ation - relation betweenmathemati s, mathemati al statisti s, and statisti s◮ Abstra t inferen e for a on rete problem. Identi� ation ofwritten symbols, pattern re ognition motivated:

◮ Probabilities on algebrai stru tures, A & W, 1963◮ Abstra t interen e, Wiley, 1981

Lindgren - georg�maths.lth.se Ulf Grenander

Summer s hool 1967Summer s hool, Kall, 1967 on Statisti al inferen e for sto hasti pro esses. Tea hers: Cramér, Grenander, Åström, Zetterberg

Lindgren - georg�maths.lth.se Ulf Grenander

Applied mathemati s◮ Shift in fo us � Institute of Applied Mathemati s, ITM,1971�2001◮ Real resear h proje ts in applied mathemati s in wide sense;university/industry◮ Stipends◮ Seed money for future major ooperation◮ No ommer ial interests◮ Repla ed 2001 by the Fraunhofer-Chalmers Centre forIndustrial Mathemati s

Lindgren - georg�maths.lth.se Ulf Grenander

The mental base� The purpose of applied mathemati s is to aid in thesolution of real-world problems, using mathemati al tools� The purpose of applying mathemati s is to gainunderstanding about the subje t matter in question, notto prove theorems� It may take a hange in generation before themathemati al ommunity has ome to regard the omputer as a natural extension of analyti pro edure,but we an speed it up by in luding more omputers ien e in our general edu ation� Computational probabilityLindgren - georg�maths.lth.se Ulf Grenander

Institute of Applied Mathemati s, ITMUtdrag ur - Avtal mellan svenska staten o h Stiftelsen tillämpadmatematik angående gemensam �nansiering av viss forskning o hutve klingsverksamhet� Styrelsen för tekniska utve kling, STU, o h Stiftelsen tillämpadmatematik, i det följande kallad stiftelsen, åtar sig att under tiden 1 juli1971 � 30 juni 1974 gemensamt �nansiera forskning o h utve kling inomtillämpad matematik samt praktisk användning av dess resultat. ...� STU åtar sig att ... medverka i �nansieringen av den avtaladeverksamheten genom att tillskjuta400.000 kronor under budgetåret 1971/72,400.000 kronor under budgetåret 1972/73 o h400.000 kronor under budgetåret 1973/74.Stiftelsen åtar sig å sin sida att ... medverka i �nansieringen av denavtalade verksamheten genom att tillskjuta lägst 1.500.000 kronor.Lindgren - georg�maths.lth.se Ulf Grenander

Pattern theory - a visionary (!) resear h program 196x �◮ Started in the 1960's from interest in handwritinginterpretation (?)◮ Major resear h group at Brown, in luding Stuart Geman andDavid Mumford◮ Most ited paper: Grenander & Miller, Representation ofknowledge in omplex systems, JRSS B, 1994

◮ image restoration◮ pattern re ognition◮ image understandingneeds basi elements � regular stu tures � to represent theobje t of interest

◮ Build a model for the world, observed through data � trueBayesianLindgren - georg�maths.lth.se Ulf Grenander

A medi al image example◮ Obje t of interest: the brain � is there a pathologi al deviationfrom normal shape or size?◮ Grenander & Miller: Computational anatomy � an emergingdis ipline, 1998

Lindgren - georg�maths.lth.se Ulf Grenander

The statisti al modelDeformation in X - and Y - oordinates are de�ned by sto hasti di�erential equations∇2X (x , y) = W x(x , y) + dx∇2Y (x , y) = W y (x , y) + dywhere W x ,W y are white noise pro esses, and dx , dy are onstants.Hypotheses:H0 : dx = dy = 0 �no systemati deformation�H+ : dx , dy > 0 �systemati dilatation�H− : dx , dy < 0 �systemati ontra tion�Lindgren - georg�maths.lth.se Ulf Grenander

Modelling◮ The obje t for modelling is a system/stru ture of

◮ numbers?◮ fun tions?◮ stru tures!

◮ The goal for modelling is understandingLindgren - georg�maths.lth.se Ulf Grenander

To be ontinued ...

Lindgren - georg�maths.lth.se Ulf Grenander