Vorobyev’s Method of Moments in AppliedMathematics - Part I (Self-adjoint Operators)
MORE seminar, May 4, 2015
MORE seminar Jan Papež Vorobyev: Method of Moments
Yu. V. Vorobyev: Method ofMoments in Applied Math-ematics; Translated fromthe Russian original (1958)by Bernard Seckler; Gordonand Breach Science Publish-ers, New York, 1965.
Outline
Part I introduction, self-adjoint operatorsPart II generalizations, classificationPart III moment problem for completely continuous (compact)
operatorsPart IV partial realization, examples, mathematical physics,
miscellaneous, . . .
Other references
J. Liesen, Z. Strakoš:Krylov Subspace Methods, Principles and Analysis, Chapter 3.Oxford University Press, (2013).
J. Málek and Z. Strakoš:Preconditioning and the Conjugate Gradient Method in theContext of Solving PDEs.SIAM Spotlight Series, SIAM, Philadelphia, (2015).
MORE seminar Jan Papež Vorobyev: Method of Moments
Outline of Part I
1 Problem of moments in Hilbert space
2 Method of moments for self-adjoint operators
MORE seminar Jan Papež Vorobyev: Method of Moments
I. Preliminaries
• We consider a (complete) Hilbert space H with the innerproduct (·, ·) and the induced norm ‖ · ‖ ;
• operators act on H , i.e.,
A,An : H −→ H ;
• (strong) convergence of operators in H, An → A means
limn→∞
‖Anx − Ax‖ = 0 for all x ∈ H ;
• for a subspace Hn ⊂ H , En denotes the orthogonalprojection onto Hn .
MORE seminar Jan Papež Vorobyev: Method of Moments
I. Problem of moments in Hilbert space
Let z0, z1, . . . , zn be n + 1 linearly independent elements of Hilbertspace H . Consider the subspace Hn generated by all possiblelinear combinations of z0, z1, . . . , zn−1 and construct a linearoperator An defined on Hn such that
z1 = Anz0,
z2 = Anz1,...
zn−1 = Anzn−2,
Enzn = Anzn−1,
where Enzn is the projection of zn onto Hn .
MORE seminar Jan Papež Vorobyev: Method of Moments
I. Eigenvalues of operator An
Since Enzn is an element of Hn , we can find numbersα0, α1, . . . , αn−1 such that
Enzn = −α0z0 − α1z1 − · · · − αn−1zn−1 .
Using the definition of An ,
Pn(An)z0 := (Ann + αn−1A
n−1n + · · ·+ α0I ) z0 = 0 .
It can be shown that the eigenvalues of An are the roots of the(monic) polynomial
Pn(λ) = λn + αn−1λn−1 + · · ·+ α0 .
MORE seminar Jan Papež Vorobyev: Method of Moments
I. Approximation of bounded linear operators
Let A be a bounded linear operator in Hilbert space H .Choosing an element z0 , we first form a sequence of elementsz1, . . . , zn, . . .
z0, z1 = Az0, z2 = Az1 = A2z0, . . . , zn = Azn−1 = Anz0, . . .
For the present z1, . . . , zn are assumed to be linearly independent.By solving the moment problem we determine a sequence ofoperators An defined on the sequence of nested subspaces Hn
such thatz1 = Az0 = Anz0,
z2 = A2z0 = (An)2z0,...
zn−1 = An−1z0 = (An)n−1z0,Enzn = EnA
nz0 = (An)nz0.
MORE seminar Jan Papež Vorobyev: Method of Moments
I. Approximation of bounded linear operators
Using the projection En onto Hn we can write for the operatorsconstructed above (here we need the linearity of A )
An = En AEn .
The orthogonality of En gives
‖An‖ ≤ ‖A‖ .
Matching moments propertyThere holds
(A`nz0, z0) = (A`z0, z0) , ` = 0, 1, . . . , n − 1, n .
MORE seminar Jan Papež Vorobyev: Method of Moments
I. Convergence An → A
Given z0 , consider the subspace Hz as the closure, w.r.t. thenorm ‖ · ‖ , of the linear manifold
{x ∈ H | x = Q(A)z0 , Q(λ) is an arbitrary polynomial}
Then A can be considered to be an operator on Hz .
Theorem II, p. 21If A is a bounded linear operator and An a sequence of solutionsof the problem of moments, then the sequence An convergesstrongly to A in the subspace Hz .
MORE seminar Jan Papež Vorobyev: Method of Moments
I. Comments
Linear (in)dependence of z0, z1, . . . , zn, . . .
If zn is linearly dependent on z0, z1, . . . , zn−1 , then Hz = Hn
and An coincides with A on Hz .
Krylov subspacesThe choice of the elements z0, z1, . . . , zn, . . . as above givesKrylov subspaces that are closely connected with the application(described, e.g. by partial differential equations).
Method of moments falls within the general framework ofthe Galerkin’s methodThe approximate operator
An = En AEn .
MORE seminar Jan Papež Vorobyev: Method of Moments
Section 2
Method of moments for self-adjointoperators
MORE seminar Jan Papež Vorobyev: Method of Moments
II. Self-adjoint operators
Let A be a bounded self-adjoint operator, i.e.
(Ax , y) = (x ,Ay) for all x , y ∈ H .
Consequently (Ax , x) is a real number and the eigenvalues of Aare also real.Bounds of a self-adjoint operator
m (x , x) ≤ (Ax , x) ≤ M (x , x) for all x ∈ H .
MORE seminar Jan Papež Vorobyev: Method of Moments
II. Matching moments property for self-adjointoperators
The problem of moments for a self-adjoint operator A and anelement z0 yields the sequence of (nested) subspaces Hn and theself-adjoint operators An = EnAEn that matches the first 2nmoments of the operator A , i.e.
(A`nz0, z0) = (A`z0, z0) , ` = 0, 1, . . . , n, . . . , 2n − 1 .
MORE seminar Jan Papež Vorobyev: Method of Moments
II. Outline
• spectral representation of self-adjoint operators
• Stieltjes’ moment problem
• Gauss-Christoffel quadrature
• determining the spectrum of An
• orthogonal polynomials
• Lanczos algorithm, Conjugate Gradient Method
MORE seminar Jan Papež Vorobyev: Method of Moments
II. Finite-dimensional self-adjoint operator
Consider a self-adjoint operator An in some n-dimensional spaceHn and assume for the moment that it has distinct eigenvalues
λ1 < λ2 < . . . < λn .
The corresponding eigenelements (eigenvectors) uk are mutuallyorthogonal, i.e.
(ui , uk) = 0 i 6= k .
We consider eigenelements uk normalized with ‖uk‖ = 1 , so that{uk} form an orthonormal basis of Hn and
x = (x , u1) u1 + (x , u2) u2 + . . .+ (x , un) un , x ∈ Hn
MORE seminar Jan Papež Vorobyev: Method of Moments
II. Spectral function, n-dimensional case
The spectral function E(n)λ of An is the family of projections
E(n)λ x = 0 λ < λ1
E(n)λ x = (x , u1) u1 λ1 ≤ λ < λ2· · ·E(n)λ x = (x , u1) u1 + . . .+ (x , un−1) un−1 λn−1 ≤ λ < λn
E(n)λ x = x λn ≤ λ
Denoting the value of the jump by ∆kE(n)λ x ≡ (x , uk) uk ,
k = 1, . . . , n, we can write
Anx = λ1(x , u1) u1 + λ2(x , u2) u2 + . . .+ λn(x , un) un
=n∑
k=1
λk∆kE(n)λ x
MORE seminar Jan Papež Vorobyev: Method of Moments
II. Spectral function, general case
The spectral representation for any bounded self-adjoint operator A
Ax =
∫ M
mλdEλx ,
where the spectral function Eλ of A represents a family oforthogonal projections which is
• non-decreasing, i.e., if µ > ν , then the subspace onto whichEµ projects contains the subspace into which Eν projects;
• Em = 0, EM = I ;• Eλ is right continuous, i.e. lim
λ′→λ+Eλ′ = Eλ .
The values of λ where Eλ increases by jumps represent theeigenvalues of A .
MORE seminar Jan Papež Vorobyev: Method of Moments
II. Convergence of spectral functions
Let An be the solution of problem of moments for A, z0 and E(n)λ
be the spectral function of An .
As a result of (strong) convergence An → A in Hz , we have
Theorem IX, p. 61If A is a bounded self-adjoint operator and An a sequence ofsolutions of the problem of moments, then the sequence of spectralfunctions E(n)λ of the operators An converges strongly to thespectral function of A
E(n)λ → Eλin the subspace Hz for all λ not belonging to the discrete spectrumof A .
MORE seminar Jan Papež Vorobyev: Method of Moments
II. Distribution function
The spectral function Eλ and z0 determine the real function(Eλz0, z0) that satisfies• (Eλz0, z0) = 0 for λ < m,• (Eλz0, z0) = ‖z0‖2 for M ≤ λ,• (Eλz0, z0) is non-decreasing .
The function (Eλz0, z0) can be considered as a distribution functionfor a Riemann-Stieltjes integral.
Hereafter, we will for simplicity of notation consider normalizeddistribution functions, i.e., we consider
‖z0‖ = 1 .
MORE seminar Jan Papež Vorobyev: Method of Moments
II. Simplified Stieltjes’ moment problem
Given a distribution function ω(λ) on [m,M] , find a distributionfunction ω(n)(λ) , with n distinct points of increase in [m,M] ,such that the first 2n moments are matched, i.e. such that∫ M
mλ`dω(λ) =
∫ M
mλ`dω(n)(λ) , ` = 0, 1, . . . , 2n − 1 .
Equivalent formulation: find an n-node (mechanical) quadraturewith positive weights for the given Riemann-Stieltjes integral withdistribution function ω(λ) , that has algebraic degree of exactness2n − 1 , i.e. that is exact for polynomials of degree at most 2n − 1 .
[Liesen & Strakoš, §3.1]
MORE seminar Jan Papež Vorobyev: Method of Moments
II. Solution of Stieltjes’ moment problem
Gauss-Christoffel quadratureThe n-node (interpolatory) quadrature that has algebraic degree ofexactness 2n − 1 .The (distinct) quadrature nodes are given as the roots of the n-thmonic orthogonal polynomial w.r.t. the Riemann-Stieltjes integral∫ M
mf (λ)dω(λ) .
[Liesen & Strakoš, §3.2]Properties of orthogonal polynomials and the relation to continuedfractions: [Liesen & Strakoš, §3.3]
MORE seminar Jan Papež Vorobyev: Method of Moments
II. Orthogonal basis of Hn
Using Lanczos method of successive orthogonalization we constructan orthogonal basis of the subspace Hn = span{z0, z1, . . . , zn−1} :
p0 = z0 ,
p1 = Ap0 − a0p0 , a0 is such that (p0, p1) = 0p2 = Ap1 − a1p1 − b0p0 ,
. . .
pk+1 = (A− ak I ) pk − bk−1pk−1 .
Operator A is self-adjoint ⇒ three-term recurrence .
We can writepk = Pk(A) z0 ,
where Pk(λ) is a monic polynomial of degree k .
MORE seminar Jan Papež Vorobyev: Method of Moments
II. Orthogonal basis of Hn
Since An is the solution of problem of moments, we have
Pn(An) z0 = EnPn(A) z0 = Enpn .
From the orthogonality of pn to each of the elements p0, . . . , pn−1 ,
Enpn = 0 .
The spectrum of An is equal to the roots of Pn(λ) .
MORE seminar Jan Papež Vorobyev: Method of Moments
II. Orthogonal polynomials
Using the spectral function Eλ of the operator A ,
pk = Pk(A) z0 =
∫ M
mPk(λ)dEλz0 .
The orthogonality of the elements pk gives (for i 6= k)
0 = (pi , pk) =
∫ M
mPi (λ)Pk(λ) d(Eλz0, z0) ,
Therefore the (monic) polynomials Pk(λ) are orthogonal overinterval [m,M] w.r.t. the distribution function (Eλz0, z0) .
The roots of the polynomial Pn(λ) are the nodes of theGauss-Christoffel quadrature for solving the Stieltjes momentproblem with distribution function (Eλz0, z0) .
MORE seminar Jan Papež Vorobyev: Method of Moments
II. Lanczos algorithm
Denote symbolically Qn = (q1, . . . , qn) , where the columnsq1, . . . , qn form an orthonormal basis of Hn determined by theLanczos process
AQn = QnTn + δn+1qn+1eTn ,
with q1 = z0 (we assume ‖z0‖ = 1 ) . We get
A`n = QnT`nQ∗n , ` = 0, 1, . . .
and the moment matching property reads
eT1 T`ne1 = q∗1A
`q1 = (A`z0, z0) , ` = 0, 1, . . . , 2n − 1 .
[Liesen & Strakoš, §3.5, §3.7]
MORE seminar Jan Papež Vorobyev: Method of Moments
II. Lanczos algorithm, Jacobi matrix
The matrix Tn of the orthogonalization coefficients has the form
Tn =
γ1 δ2
δ2. . . . . .. . . . . . . . .
. . . . . . δnδn γn
,
where δj > 0 . Eigenvalues of Tn are equal to the eigenvalues ofAn .
Properties of Jacobi matrices: [Liesen & Strakoš, §3.4]
MORE seminar Jan Papež Vorobyev: Method of Moments
II. Conjugate Gradient Method
Let A be self-adjoint positive definite and the right-hand side f anelement of Hz . We are solving
Ax = f , i.e. x = A−1f =
∫ M
mλ−1 dEλf .
Approximation using the spectral function E(n)λ
xn =
∫ M
mλ−1 dE(n)λ f =
∫ M
mλ−1 dE(n)λ fn = A−1
n fn ,
where fn denotes the projection of f on Hn . Since all theeigenvalues of An lie in the interval [m,M] with m > 0 , theapproximation xn is well-defined for any n .
MORE seminar Jan Papež Vorobyev: Method of Moments
II. Conjugate Gradient Method
The approximation xn = A−1n fn can be computed by the Conjugate
Gradient Method (CG) by [Hestenes & Stiefel] .
The matrix representation of CG using the Jacobi matrix Tn andthe orthonormal basis Qn of Hn is
Tn yn = ‖f ‖ e1 , xn = Qnyn ∈ Hn .
Provided that f ∈ Hz , convergence An → A in Hz givesxn → x .
MORE seminar Jan Papež Vorobyev: Method of Moments
II. Model reductions matching 2n moments
Given a self-adjoint operator A and z0 ∈ H :
Vorobyev’s problem of momentsA : Hz −→ Hz , An : Hn −→ Hn , dim(Hn) = n .
Gauss-Christoffel quadraturefrom the distribution function ω(λ) to the distribution functionω(n)(λ) with n points of increase.
Lanczos algorithmfrom A, z0 to Tn, e1 , Tn ∈ Rn×n .
Conjugate Gradient Methodfrom Ax = f to Tn yn = ‖f ‖ e1 (for positive definite self-adjointoperator A ).
MORE seminar Jan Papež Vorobyev: Method of Moments
Questions for the next talks
• Can we match more than n + 1 moments if A is notself-adjoint?
• Can we consider an oblique (i.e. not-orthogonal) projection En ?• Do eigenvalues of An converge to eigenvalues of A ?• Can we generalize Gauss-Christoffel quadrature for non positivedefinite linear functionals?
• . . .
MORE seminar Jan Papež Vorobyev: Method of Moments
Thank you for your attention!
MORE seminar Jan Papež Vorobyev: Method of Moments