Remembering Gene Golub: From Chicago to Stanford and ... · Remembering Gene Golub: From Chicago to Stanford and Around the World Feb. 29, 1932–Nov. 16, 2007 Greg Fasshauer Department

Remembering Gene Golub:From Chicago to Stanford and Around the World

Feb. 29, 1932–Nov. 16, 2007

Greg Fasshauer

Department of Applied MathematicsIllinois Institute of Technology

Feb. 29, 2008

[email protected] Applied Math, IIT Feb. 29, 2008

http://math.iit.edu/~fass

Gene Golub, 02/29/1932–11/16/2007


Gene.avi


Gene Golub, 02/29/1932–11/16/2007



Gene Golub, 02/29/1932–11/16/2007



Gene Golub, 02/29/1932–11/16/2007



Gene Golub, 02/29/1932–11/16/2007



Gene Golub, 02/29/1932–11/16/2007



Gene Golub, 02/29/1932–11/16/2007 One Big Family


Simeon Poisson

Michel Chasles Gustav Dirichlet

Rudolf LipschitzH.A. Newton

E.H. Moore

George D. Birkhoff

Hassler Whitney

Herbert Robbins

Herbert Wilf

Fan Chung Graham

Robert Ellis

Carl Friedrich Gauss

Christian Gerling

Julius Plücker

Felix Klein

Frank Cole

Eric Bell

Howard Robertson

Abraham Taub

Gene Golub

Carl Louis Lindemann

David Hilbert

Richard Courant

Joseph B. Keller

Charles Tier

Erhard Schmidt

Salomon Bochner

Samuel Karlin

Larry Schumaker

Greg Fasshauer

George Papanicolaou

Russel Caflisch

Xiaofan Li

Georg Hamel

Michael Sadowsky

Eli Sternberg

Morton Gurtin

Warren Edelstein

Friedrich Bessel

Heinrich Scherk

Ernst Kummer

Hermann Schwarz

Leopold Fejer

Steven Gaal

Patrick Ahern

Art Lubin


http://news-service.stanford.edu/news/2008/january9/karlin-010908.html

http://news-service.stanford.edu/news/2007/november28/golub-112807.html

Gene Golub, 02/29/1932–11/16/2007 One Big Family


Sam Karlin

Simeon Poisson

Michel Chasles Gustav Dirichlet

Rudolf LipschitzH.A. Newton

E.H. Moore

George D. Birkhoff

Hassler Whitney

Herbert Robbins

Herbert Wilf

Fan Chung Graham

Robert Ellis

Carl Friedrich Gauss

Christian Gerling

Julius Plücker

Felix Klein

Frank Cole

Eric Bell

Howard Robertson

Abraham Taub

Gene Golub

Carl Louis Lindemann

David Hilbert

Richard Courant

Joseph B. Keller

Charles Tier

Erhard Schmidt

Salomon Bochner

Samuel Karlin

Larry Schumaker

Greg Fasshauer

George Papanicolaou

Russel Caflisch

Xiaofan Li

Georg Hamel

Michael Sadowsky

Eli Sternberg

Morton Gurtin

Warren Edelstein

Friedrich Bessel

Heinrich Scherk

Ernst Kummer

Hermann Schwarz

Leopold Fejer

Steven Gaal

Patrick Ahern

Art Lubin



http://news-service.stanford.edu/news/2007/november28/golub-112807.html

Gene Golub, 02/29/1932–11/16/2007 Gene’s Children


Daniel Boley (1981) — 2Eric Grosse (1981)Stephen Nash (1982)Mark Kent (1989)Raymond Tuminaro (1990)Hongyuan Zha (1993) — 7Oliver Ernst (1995)Xiaowei Zhan (1997)Tong Zhang (1999)Nhat Nguyen (2000)Urmi Bhattacharya Holz (2002)James Lambers (2003)Yong Sun (2003)Sou-Cheng Choi (2006)Lek-Heng Lim (2007)

Richard Bartels (1968) — 10Michael Jenkins (1969)Lyle Smith (1969)George Ramos (1970)Richard Brent (1971) — 19Michael Saunders (1972) — 5John Palmer (1974)Richard Underwood (1975)Dianne O’Leary (1976) — 16John Lewis (1976)Margaret Wright (1976)Michael Heath (1978) — 10Franklin Luk (1978) — 4Michael Overton (1979) — 4Petter Bjørstad (1980) — 9




Daniel Boley (1981) — 2Eric Grosse (1981)Stephen Nash (1982)

Richard Bartels (1968) — 10Michael Jenkins (1969)Lyle Smith (1969)George Ramos (1970)Richard Brent (1971) — 19Michael Saunders (1972) — 5John Palmer (1974)Richard Underwood (1975)Dianne O’Leary (1976) — 16John Lewis (1976)Margaret Wright (1976)Michael Heath (1978) — 10Franklin Luk (1978) — 4Michael Overton (1979) — 4Petter Bjørstad (1980) — 9





Gene Golub, 02/29/1932–11/16/2007 A Few Things From Gene’s CV

Went to Haugan Elementary and Theodore Roosevelt High in theAlbany Park Neighborhood of Chicago

Wright Junior College, University of Chicago (junior year), UIUC(senior year): B.S. Mathematics (1953), M.A. MathematicalStatistics (1954)1959, Ph.D. in Mathematics from UIUC: "The Use of ChebyshevMatrix Polynomials in the Iterative Solution of Linear EquationsCompared to the Method of Successive Overrelaxation"Fletcher Jones Professor of Computer Science at Stanford30 Ph.D. students, for a total of more than 100 descendantsISI highly cited researcher with 315 publicationsCV from 2006: 18 books, 172 journal papers, 62 conference papers, 47 TRs

MathSciNet: 10 books, 240 journal papers, 46 proceedings

Foreword to special 75th Birthday issue of ETNA:"Besides the book, we find that Gene has around 20 other papers with 100 ormore citations, and 100 papers with 20 or more citations! His next hundredpapers after that each have between 6 and 20 citations; and on and on it goes."


http://maps.google.com/maps?ie=UTF8&oe=UTF-8&q=haugan+elementray+school&near=Chicago,+IL&fb=1&cid=0,0,14514311529635895564&ll=41.965457,-87.717698&spn=0.009669,0.023389&z=16&layer=c&cbll=41.964749,-87.714599

http://genealogy.math.ndsu.nodak.edu/id.php?id=39145



Went to Haugan Elementary and Theodore Roosevelt High in theAlbany Park Neighborhood of ChicagoWright Junior College, University of Chicago (junior year), UIUC(senior year): B.S. Mathematics (1953), M.A. MathematicalStatistics (1954)1959, Ph.D. in Mathematics from UIUC: "The Use of ChebyshevMatrix Polynomials in the Iterative Solution of Linear EquationsCompared to the Method of Successive Overrelaxation"

Fletcher Jones Professor of Computer Science at Stanford30 Ph.D. students, for a total of more than 100 descendantsISI highly cited researcher with 315 publicationsCV from 2006: 18 books, 172 journal papers, 62 conference papers, 47 TRs








Went to Haugan Elementary and Theodore Roosevelt High in theAlbany Park Neighborhood of ChicagoWright Junior College, University of Chicago (junior year), UIUC(senior year): B.S. Mathematics (1953), M.A. MathematicalStatistics (1954)1959, Ph.D. in Mathematics from UIUC: "The Use of ChebyshevMatrix Polynomials in the Iterative Solution of Linear EquationsCompared to the Method of Successive Overrelaxation"Fletcher Jones Professor of Computer Science at Stanford30 Ph.D. students, for a total of more than 100 descendants

ISI highly cited researcher with 315 publicationsCV from 2006: 18 books, 172 journal papers, 62 conference papers, 47 TRs








Went to Haugan Elementary and Theodore Roosevelt High in theAlbany Park Neighborhood of ChicagoWright Junior College, University of Chicago (junior year), UIUC(senior year): B.S. Mathematics (1953), M.A. MathematicalStatistics (1954)1959, Ph.D. in Mathematics from UIUC: "The Use of ChebyshevMatrix Polynomials in the Iterative Solution of Linear EquationsCompared to the Method of Successive Overrelaxation"Fletcher Jones Professor of Computer Science at Stanford30 Ph.D. students, for a total of more than 100 descendantsISI highly cited researcher with 315 publicationsCV from 2006: 18 books, 172 journal papers, 62 conference papers, 47 TRs








Matrix Computations (editions 1–3) has 17745 citations on GoogleScholar

Founder of Stanford’s Scientific Computing and ComputationalMathematics program (now Institute for Computational and Mathematical Engineering)

Member of National Academy of Sciences and other academies9 honorary degrees (including Doctor of Laws, U of Dundee, 1987)Founder of 2 SIAM Journals:

J. Scientific (and Statistical) Computing, 1980J. Matrix Analysis and Applications, 1988

President of SIAM (1985-1987)Founder of NA-NET and NA-Digest electronic newsletterCo-Founder of ICIAM (International Council for Industrial andApplied Mathematics)




Matrix Computations (editions 1–3) has 17745 citations on GoogleScholar

Founder of Stanford’s Scientific Computing and ComputationalMathematics program (now Institute for Computational and Mathematical Engineering)







Matrix Computations (editions 1–3) has 17745 citations on GoogleScholarFounder of Stanford’s Scientific Computing and ComputationalMathematics program (now Institute for Computational and Mathematical Engineering)








Member of National Academy of Sciences and other academies9 honorary degrees (including Doctor of Laws, U of Dundee, 1987)

Founder of 2 SIAM Journals:J. Scientific (and Statistical) Computing, 1980J. Matrix Analysis and Applications, 1988








President of SIAM (1985-1987)

Founder of NA-NET and NA-Digest electronic newsletterCo-Founder of ICIAM (International Council for Industrial andApplied Mathematics)










Gene Golub, 02/29/1932–11/16/2007 Remembering Gene

Remembering Gene Golub Around the World at least 30 similargatherings today!Gene Golub Memorial Blog at Stanford with over 200 entries

Obituary in the Chicago Tribune

Obituary in the New York Times

Obituary in Nature


http://cs.nyu.edu/overton/genearoundtheworld/

http://genehgolub.blogspot.com/2007/11/gene-h-golub-1932-2007.html


Gene Golub, 02/29/1932–11/16/2007 The SVD








Eugenio Beltrami (1873)

Camille Jordan (1874)

James Sylvester (1889)

Léon Autonne (1915)

Erhard Schmidt (1907), forintegral operators

Émile Picard (1910),introduces valeurs singulières



Beltrami worked with bilinear forms

f (x ,y) = xT Ay

with A a real n × n matrix.

Using the transformations

x = Uξ, y = Vη

leads tof (x ,y) = ξT Ση,

whereΣ = UT AV .

Beltrami showed that if U and V are orthogonal matrices, then theresulting matrix Σ is diagonal with its diagonal elements σi satisfying

det(AT A− σ2I) = 0.





f (x ,y) = xT Ay

with A a real n × n matrix. Using the transformations

x = Uξ, y = Vη


whereΣ = UT AV .







f (x ,y) = xT Ay

with A a real n × n matrix. Using the transformations

x = Uξ, y = Vη


whereΣ = UT AV .






TheoremLet A be a complex m × n matrix. A has a singular valuedecomposition of the form

A = UΣV ∗,

where Σ is a uniquely determined m × n (real) diagonal matrix, U is anm ×m unitary matrix, and V is an n × n unitary matrix.




Why is the SVD so fundamental?

The fact that U and V are unitary (orthogonal) isfundamental for geometric insights .

The fact that Σ is diagonal provides answers to importantquestions in linear algebra

number of non-zero singular values, r = rank(A)

range(A) = range(U(:,1 : r)), null(A) = range(V (:, r + 1, : n))

The SVD is stable, i.e., small changes in A will cause only smallchanges in the SVD (in fact, this is the most stable matrixdecomposition method).

The SVD is optimal in the sense that it provides thebest low-rank approximations of A .

Thanks to Gene there are efficient and stable algorithms tocompute the SVD.






































































How Gene got going on the SVD

A 1959/60 lecture by Cornelius Lanczos in London caught Gene’s attention. Lanczosshowed that the SVD was useful.

Much later, in 1963, Ben Rosen talked at Stanford about computing pseudo-inverses viaprojections. At the end of the talk George Forsythe got up and said,"Well, will somebody please figure out how to compute the pseudo-inverse of a matrix?!"Gene saw this as his marching orders.Gene asked Peter Businger (an RA) to compute the eigenvalues of

H =

[0 A

AT 0

]The absolute values of the eigenvalues of this matrix are indeed the singular values of A.Peter used an eigenvalue solver and they saw zeros on the diagonal of the tridiagonalmatrix that was generated.From David Young’s work and his own work Gene knew that this matrix could be reorderedto get a bidiagonal matrix.In the summer of 1963, while visiting Boing in Seattle, Gene figured out how to use left andright orthogonal transformations to bidiagonalize H. This led to the 1965 paper withWilliam Kahan.In 1970 Gene wrote another paper with Christian Reinsch that has become the goldstandard for computing the SVD.





A 1959/60 lecture by Cornelius Lanczos in London caught Gene’s attention. Lanczosshowed that the SVD was useful.Much later, in 1963, Ben Rosen talked at Stanford about computing pseudo-inverses viaprojections. At the end of the talk George Forsythe got up and said,

"Well, will somebody please figure out how to compute the pseudo-inverse of a matrix?!"Gene saw this as his marching orders.Gene asked Peter Businger (an RA) to compute the eigenvalues of

H =

[0 A

AT 0






A 1959/60 lecture by Cornelius Lanczos in London caught Gene’s attention. Lanczosshowed that the SVD was useful.Much later, in 1963, Ben Rosen talked at Stanford about computing pseudo-inverses viaprojections. At the end of the talk George Forsythe got up and said,"Well, will somebody please figure out how to compute the pseudo-inverse of a matrix?!"

Gene saw this as his marching orders.Gene asked Peter Businger (an RA) to compute the eigenvalues of

H =

[0 A

AT 0






A 1959/60 lecture by Cornelius Lanczos in London caught Gene’s attention. Lanczosshowed that the SVD was useful.Much later, in 1963, Ben Rosen talked at Stanford about computing pseudo-inverses viaprojections. At the end of the talk George Forsythe got up and said,"Well, will somebody please figure out how to compute the pseudo-inverse of a matrix?!"Gene saw this as his marching orders.

Gene asked Peter Businger (an RA) to compute the eigenvalues of

H =

[0 A

AT 0






A 1959/60 lecture by Cornelius Lanczos in London caught Gene’s attention. Lanczosshowed that the SVD was useful.Much later, in 1963, Ben Rosen talked at Stanford about computing pseudo-inverses viaprojections. At the end of the talk George Forsythe got up and said,"Well, will somebody please figure out how to compute the pseudo-inverse of a matrix?!"Gene saw this as his marching orders.Gene asked Peter Businger (an RA) to compute the eigenvalues of

H =

[0 A

AT 0

]

The absolute values of the eigenvalues of this matrix are indeed the singular values of A.Peter used an eigenvalue solver and they saw zeros on the diagonal of the tridiagonalmatrix that was generated.From David Young’s work and his own work Gene knew that this matrix could be reorderedto get a bidiagonal matrix.In the summer of 1963, while visiting Boing in Seattle, Gene figured out how to use left andright orthogonal transformations to bidiagonalize H. This led to the 1965 paper withWilliam Kahan.In 1970 Gene wrote another paper with Christian Reinsch that has become the goldstandard for computing the SVD.






H =

[0 A

AT 0

]The absolute values of the eigenvalues of this matrix are indeed the singular values of A.

Peter used an eigenvalue solver and they saw zeros on the diagonal of the tridiagonalmatrix that was generated.From David Young’s work and his own work Gene knew that this matrix could be reorderedto get a bidiagonal matrix.In the summer of 1963, while visiting Boing in Seattle, Gene figured out how to use left andright orthogonal transformations to bidiagonalize H. This led to the 1965 paper withWilliam Kahan.In 1970 Gene wrote another paper with Christian Reinsch that has become the goldstandard for computing the SVD.






H =

[0 A

AT 0

]The absolute values of the eigenvalues of this matrix are indeed the singular values of A.Peter used an eigenvalue solver and they saw zeros on the diagonal of the tridiagonalmatrix that was generated.

From David Young’s work and his own work Gene knew that this matrix could be reorderedto get a bidiagonal matrix.In the summer of 1963, while visiting Boing in Seattle, Gene figured out how to use left andright orthogonal transformations to bidiagonalize H. This led to the 1965 paper withWilliam Kahan.In 1970 Gene wrote another paper with Christian Reinsch that has become the goldstandard for computing the SVD.






H =

[0 A

AT 0

]The absolute values of the eigenvalues of this matrix are indeed the singular values of A.Peter used an eigenvalue solver and they saw zeros on the diagonal of the tridiagonalmatrix that was generated.From David Young’s work and his own work Gene knew that this matrix could be reorderedto get a bidiagonal matrix.

In the summer of 1963, while visiting Boing in Seattle, Gene figured out how to use left andright orthogonal transformations to bidiagonalize H. This led to the 1965 paper withWilliam Kahan.In 1970 Gene wrote another paper with Christian Reinsch that has become the goldstandard for computing the SVD.






H =

[0 A

AT 0

]The absolute values of the eigenvalues of this matrix are indeed the singular values of A.Peter used an eigenvalue solver and they saw zeros on the diagonal of the tridiagonalmatrix that was generated.From David Young’s work and his own work Gene knew that this matrix could be reorderedto get a bidiagonal matrix.In the summer of 1963, while visiting Boing in Seattle, Gene figured out how to use left andright orthogonal transformations to bidiagonalize H.

This led to the 1965 paper withWilliam Kahan.In 1970 Gene wrote another paper with Christian Reinsch that has become the goldstandard for computing the SVD.






H =

[0 A

AT 0

]The absolute values of the eigenvalues of this matrix are indeed the singular values of A.Peter used an eigenvalue solver and they saw zeros on the diagonal of the tridiagonalmatrix that was generated.From David Young’s work and his own work Gene knew that this matrix could be reorderedto get a bidiagonal matrix.In the summer of 1963, while visiting Boing in Seattle, Gene figured out how to use left andright orthogonal transformations to bidiagonalize H. This led to the 1965 paper withWilliam Kahan.

In 1970 Gene wrote another paper with Christian Reinsch that has become the goldstandard for computing the SVD.






H =

[0 A

AT 0





Applications of the SVD

Also known as principal component analysis (PCA), (discrete)Karhunen-Loève (KL) transformation, Hotelling transform, or properorthogonal decomposition (POD)

Data compressionNoise filteringRegularization of inverse problems

TomographyImage deblurringSeismology

Information retrieval and data mining (latent semantic analysis)Bioinformatics and computational biology

ImmunologyMolecular dynamicsMicroarray data analysis



Gene Golub, 02/29/1932–11/16/2007 Least Squares




Using the SVD to solve ‖Ax − b‖2 → minAssume A ∈ Rm×n, m ≥ n, and rank(A) = n.

Compute reduced SVDA = UΣV T ,

with U ∈ Rm×n, Σ ∈ Rn×n, and V ∈ Rn×n.




Using the SVD to solve ‖Ax − b‖2 → minAssume A ∈ Rm×n, m ≥ n, and rank(A) = n.Compute reduced SVD

A = UΣV T ,

with U ∈ Rm×n, Σ ∈ Rn×n, and V ∈ Rn×n.





A = UΣV T ,

with U ∈ Rm×n, Σ ∈ Rn×n, and V ∈ Rn×n.Use normal equations

AT Ax = AT b ⇐⇒ (V ΣT UT )(UΣV T )x = V ΣUT b





A = UΣV T ,


AT Ax = AT b ⇐⇒ (V ΣT︸︷︷︸=Σ

UT )(U︸︷︷︸=I

ΣV T )x = V ΣUT b





A = UΣV T ,



UT )(U︸︷︷︸=I

ΣV T )x = V ΣUT b

⇐⇒ V Σ2V T x = V ΣUT b





A = UΣV T ,



UT )(U︸︷︷︸=I

ΣV T )x = V ΣUT b

⇐⇒ V Σ2V T x = V ΣUT b(Σ−1V T )×⇐⇒ ΣV T x = UT b.





A = UΣV T ,



UT )(U︸︷︷︸=I

ΣV T )x = V ΣUT b


The least squares solution is given by

x = V Σ−1UT b.





A = UΣV T ,



UT )(U︸︷︷︸=I

ΣV T )x = V ΣUT b


The least squares solution is given by

x = V Σ−1UT︸︷︷︸=A+, pseudoinverse

b.



Gene Golub, 02/29/1932–11/16/2007 Other Areas Gene Contributed To

Gene’s Milestone Papers I

Iterative Methods for Linear SystemsChebyshev semi-iterative methods, successive over-relaxationiterative methods, and second-order Richardson iterative methods,Parts I and II, with R. S. Varga (1961)A generalized conjugate gradient method for non-symmetricsystems of linear equations, with P. Concus (1975)A generalized conjugate gradient method for the numerical solutionof elliptic partial differential equations, with P. Concus, D. O’Leary(1976)Hermitian and Skew-Hermitian splitting methods for non-Hermitianpositive definite linear systems, with Z.-Z. Bai, M. K. Ng (2001)

Solution of Least Squares ProblemsNumerical methods for solving linear least squares problems (1965)Singular value decomposition and least squares solutions, withC. Reinsch (1970)




Gene’s Milestone Papers II

The differentiation of pseudo-inverses and non-linear least squaresproblems whose variables separate, with V. Pereyra (1973)Generalized cross-validation as a method for choosing a good ridgeparameter, with M. Heath, G. Wahba (1979)An analysis of the total least squares problem, with C. Van Loan(1980)

Matrix Factorizations and ApplicationsCalculating the singular values and pseudo-inverse of a matrix, withW. Kahan (1965)The simplex method of linear programming using LUdecomposition, with R. Bartels (1969)On direct methods for solving Poisson’s equation, withB. L. Buzbee, C. W. Nielson (1970)Numerical methods for computing angles between linearsubspaces, with A. Björck (1973)Methods for modifying matrix factorizations, with P. Gill, W. Murray,M. Saunders (1974)




Gene’s Milestone Papers III

Orthogonal Polynomials and QuadratureCalculation of Gauss quadrature rules, with J. Welsch (1969)Matrices, moments and quadrature, with G. Meurant (1994)Computation of Gauss-Kronrod quadrature rules, with D. Calvetti,W. Gragg, L. Reichel (2000)

Eigenvalue ProblemsSome modified matrix eigenvalue problems (1973)Ill-conditioned eigensystems and the computation of the Jordancanonical form, with J. Wilkinson (1976)The block Lanczos method for computing eigenvalues, withR. Underwood (1977)The numerically stable reconstruction of a Jacobi matrix fromspectral data, with C. de Boor (1978)Adaptive methods for the computation of pagerank, with S. Kamvar,T. Haveliwala (2003)



Appendix References

References I


Gene H. Golub and Charles Van Loan.Matrix Computations.Johns Hopkins University Press, 3rd edition, 1996.

Raymond H. Chan, Chen Greif, and DianneO’Leary (eds.).Milestones in Matrix Computation: The SelectedWorks of Gene H. Golub with Commentaries.Oxford University Press, 2007.


Appendix References

References II

E. Beltrami.Sulle funzioni bilineari.Giornale di Matematiche ad Uso degli Studenti Della Università Italiane 11(1873), 98–106.English translation by D. Boley.Dept. of Computer Science, Univ. of Minnesota, TR 90-37, 1990.

Gene H. Golub.Numerical methods for solving linear least-squares problems.Numer. Math. 7 (1965), 206–216.

Gene H. Golub and William Kahan.Calculating the singular values and pseudo-inverse of a matrix.J. Soc. Indust. Appl. Math. Ser. B Numer. Anal. 2 (1965), 205–224.

Gene H. Golub and Christian Reinsch.Singular value decomposition and least squares solutions.Numer. Math. 14 (1970), 403–420.



Appendix References

References III

Chen Greif.Gene H. Golub Biography.in Milestones in Matrix Computation.Oxford University Press, 2007.also available online at http://fds.oup.com/www.oup.co.uk/pdf/0-19-920681-3.pdf

Nicholas J. Higham.An Interview with Gene Golub.University of Manchester, MIMS EPrint 2008.8.also available online at http://eprints.ma.man.ac.uk/1024/

G. W. Stewart.On the early history of the singular value decomposition.SIAM Rev. 35 (1993), 551–566.


http://fds.oup.com/www.oup.co.uk/pdf/0-19-920681-3.pdf

http://eprints.ma.man.ac.uk/1024/


Personal Memories



Personal Memories



Personal Memories

Would anyone like to share personal memories ofGene?



Gene Golub, 02/29/1932–11/16/2007 Obituaries

Return





Return




Return

OBITUARY

Gene H. Golub (1932–2007)Mathematician and godfather of numerical analysis.

A century ago, matrices and the techniques for their manipulation — linear algebra — were a backwater of mathematics. Today, they are the foundation not just of the mathematical field of numerical analysis, but also of computational science and engineering, and have become indispensable for anyone who wants to get numerical results from a computer. The pre-eminent figure in matrix computations over the past 50 years, Gene Golub, died on 16 November.

Golub was born in Chicago on 29 February 1932, to Jewish parents from Latvia and the Ukraine. His childhood was not affluent, but he was a good student. After two years at a junior college, he transferred to the University of Illinois at Urbana-Champaign, achieving his doctorate there in 1959. At the time, Illinois, with the first of its ‘ILLIAC’ supercomputers, was a great centre of computing, and Golub showed his affection for his Alma Mater by endowing a chair there 50 years later. Rumour has it that the funds for the gift came from Google stock acquired in exchange for some advice on linear algebra. Google’s PageRank search technology starts from a matrix computation — an eigenvalue problem with dimensions in the billions. Hardly surprising, Golub would have said: everything is linear algebra.

He came to believe that in his twenties, as he realized that new methods of orthogonal-matrix factorization introduced by Wallace Givens and Alston Householder offered the right mathematical recipe for solving all kinds of problems. In particular, Golub focused on the idea known as singular value decomposition, SVD, which systematically isolates the dominant components of a linear process. Together with William Kahan and Christian Reinsch, he invented the now-standard SVD algorithms, and showed scientists, engineers and statisticians how these algorithms could be used in areas such as the least-squares method to find the best fit to a curve; in optimization problems and control theory; and for the determination of crucial matrix parameters such as their norms, ranks and condition numbers. In later years he drove a car with the licence plate ‘PROF SVD’.

Golub found his way to Stanford University in 1962, eventually becoming the senior professor in its formidable computer science department. In 45 productive years there, he advanced matrix computations in areas as diverse as geodesy, data mining and quantum chromodynamics. The dimension of what

was considered a ‘big’ matrix grew from 100 to 1,000,000 in the same period, and Golub was among the first to develop the iterative algorithms that make problems involving such huge matrices tractable.

As the new methods came in, older ideas such as gaussian elimination (essentially, the way one is taught to solve a system of simultaneous equations in school, by eliminating the variables one by one) became a smaller part of a new and greater enterprise. Along with the new algorithms came a new world of software for solving mathematical problems, such as EISPACK, LAPACK and MATLAB. Golub’s book Matrix Computations, co-authored with Charles Van Loan of Cornell University, became a bestseller and the definitive textbook of the field. Honours flowed in, including membership of the US National Academies.

As a servant of the wider scientific community, Golub did as much as anybody to make the Society for Industrial and Applied Mathematics (SIAM) the organization it is today. He served it in various capacities, among them as president (1985–87). He also founded and edited two of the society’s journals, the SIAM Journal on Scientific and Statistical Computing and SIAM Journal on Matrix Analysis and Applications. It was his proposal that led to the quadrennial International Congresses on Industrial and Applied Mathematics.

But this impressive list of achievements misses the truly extraordinary aspect of this complex man: the scale of his devotion to people. Golub was a bachelor for most of his life, and his colleagues were his family. No family ever had a more loving, attentive or exasperating father. As he liked to say, “Every numerical analyst has a second home at Stanford”. Countless colleagues enjoyed a glass of wine at his home there, and hundreds of them stayed over for a night or even a month at his invitation. How did he remember all our birthdays and reading tastes and children’s names?

Golub could not spend a day without other people. He would eat dinner with them, talk matrices with them, organize conferences with them, write papers and books with them, argue academic politics with them — an endless dance of interactions, plans and projects. Anywhere in the world, a numerical analyst knows who is meant by ‘Gene’. About 250 of them were his co-authors. They knew that it would fall to them to do most of the writing; but Golub saw the connections, knew the literature, and made the paper happen.

He seemed almost to have invented e-mail. As early as 1981, his office computer was set up to beep the moment a message arrived. His personal address list evolved into the worldwide database of numerical analysts, and his notes to friends became the Numerical Analysis Digest. This newsletter, one of the first e-bulletins, is now sent to some 8,000 recipients weekly. He could not sit still. As he left us in Oxford last September after an extended sabbatical visit, having spent much of the preceding months talking with the graduate students in the common room — to which he had donated $1,000 for a biscuit fund — he mentioned that he had three trips to China planned for the upcoming year.

Gene Golub was restless and never entirely happy. He was a demanding friend; behind his back, we all had Gene stories to tell. It was a huge back: Gene was big, dominating any room he was in, and grew more impressive and imposing with the years. Graduate students around the world admired and loved him, and he bought them all dinner when he got the chance. His unexpected death, in Stanford in between speaking at a conference in Hong Kong and flying to Zurich for his eleventh honorary degree, has left the world of numerical analysis orphaned and reverberating. Lloyd N. TrefethenLloyd N. Trefethen is in the Oxford University Computing Laboratory, Oxford OX1 3QD, UK. e-mail: [email protected]

962

NATURE|Vol 450|13 December 2007NEWS & VIEWS



v2

v1

K1.0 K0.5 0 0.5 1.0

K0.8

K0.6

K0.4

K0.2

0.2

0.4

0.6

0.8

s1u1s2u2

K2 K1 0 1 2

K2

K1

1

2

Figure: ON bases for row space and column space of A.

Return




TheoremThe m × n matrix A can be decomposed into a sum of r rank-onematrices:

A =r∑

j=1

σjujv∗j .

Moreover, the best 2-norm approximation of rank ν (0 ≤ ν ≤ r ) to A isgiven by

Aν =ν∑

j=1

σjujv∗j .

In fact,‖A− Aν‖2 = σν+1.

Return



Sam Karlin, 06/08/1924–12/18/2007



Sam Karlin, 06/08/1924–12/18/2007

Born June 8, 1924, in Yanovo, Poland; died Dec.18, 2007B.S. (1944) and M.S. (1945) from IITPh.D. 1947 at Princeton with Salomon BochnerCaltech 1948-1956, then Stanford41 Ph.D. students, 540 descendants10 books, more than 450 papersChose a different field of interest every seven years : game theory, analysis, mathematical

statistics, total positivity, approximation theory, probability and random processes,

mathematical economics, inventory theory, population genetics, bioinformatics and

biomolecular sequence analysis.

Member: American Academy of Arts and Sciences, NationalAcademy of Sciences, American Philosophical SocietyPresident of Institute of Mathematical Statistics (IMS), 1978-79.National Medal of Science 1989IIT Professional Achievement Award (attempted last year, Samdelayed to 2008), Lifetime Achievement Award (this year?)Stanford News Report, Paper about Sam Karlin.

Return




http://news-service.stanford.edu/news/2008/january16/gifs/Karlin_paper.pdf


Sam Karlin, 06/08/1924–12/18/2007


Return


Documents

Remembering Gene Golub: From Chicago to Stanford and ... · Remembering Gene Golub: From Chicago to Stanford and Around the World Feb. 29, 1932–Nov. 16, 2007 Greg Fasshauer Department