Embed Size (px)
Citation preview
Tracking the trajectory in finite precision CG computations
Tomas Gergelits <[email protected]> Zdenek Strakos <[email protected]>
Faculty of Mathematics and Physics, Charles University in Prague
Introduction
It is known that the behaviour of the conjugate gradient method (CG) can be strongly affected by theinfluence of the rounding errors. However, whereas the CG convergence rate may substantially differ infinite precision and exact arithmetic, we observe that the trajectories of approximations as well as thecorresponding Krylov subspaces are very similar.
Ax = b, A ∈ FN×N HPD, b ∈ FN , F is R or C
The essence of the CG method
I CG is a projection method which minimizes the energy norm of the error
xk ∈ x0 +Kk(A, r0), rk ⊥ Kk(A, r0) = span{r0,Ar0,A2r0, . . . ,Ak−1r0}, k = 1, 2, . . .
‖x − xk‖A = min {‖x − y‖A : y ∈ x0 +Kk(A, r0)}I CG is conforming with the Galerkin approximation
‖∇(u− u(k)h )‖2 = ‖∇(u− uh)‖2 + ‖x − xk‖2
A
I CG is a matrix formulation of the Gauss-Christoffel quadrature
A, r0/‖r0‖
Tk , e1
ω(λ),
∫ ∞0
f (λ) dω(λ)
ω(k)(λ),∑k
i=1 ω(k)i f (λ
(k)i )
first 2k moments matched
⇒Nonlinearity of CG and of its convergence behaviour.
Convergence behaviour in FP arithmetic
I Short recurrences =⇒ loss of orthogonality among the computed residual vectors.
Scheme of the backward-like analysis in [3]:
FN FN(k)
A, FP Lanczos → Tkthe first k steps
A(k), EXACT Lanczos → Tk
I Eigenvalues of A(k) are tightly clustered around the eigenvalues of A.
I In numerical experiments, A(k) can be replaced (with small inaccuracy) by an “universal” A withsufficiently many eigenvalues in the tight clusters around the eigenvalues of A (see [4]).
⇒ The rate of convergence for FP and exact CG typically substantially differs.
Composite polynomial convergence bounds in FP arithmetic [7]
I Assuming exact arithmetic, in case of m largeoutlying eigenvalues we have (see [1] or [2])
‖x − xk‖A‖x − x0‖A
≤ 2
(√κm(A)− 1√κm(A) + 1
)k−m
, k = m,m + 1, . . .
κm(A) = λN−m/λ1 effective condition number.
I In practical CG computations, effects of roundingerrors must be taken into account in all considerationsconcerning the rate of convergence. 0 20 40 60 80 100 120 140 160
10−15
10−10
10−5
100
iteration number
rela
tive
A−
norm
of t
he e
rror
exact CGFP CGcomp. bound
Delay of convergence & rank deficiency
In principle, delay of convergence of the energy norm of the error is determined by the rank-deficiency of thecomputed Krylov subspace [5].
0 10 20 30 40 50
10−15
10−10
10−5
100
delay ofconvergence
iteration number
rela
tive
A−
norm
of t
he e
rror
exact arithmeticfinite precision arithmeticloss of orthogonality
0 10 20 30 40 500
5
10
15
20
25
30
35
40
45
50
iteration number
rank
of t
he c
ompu
ted
Kry
lov
subs
pace
rank−deficiency
FP CG computationexact CG computation
0 5 10 15 20 25
10−15
10−10
10−5
100
rank of the computed Krylov subspace
rela
tive
A−
norm
of t
he e
rror
shifted FP CGexact CG
I Rank deficiency: k − k , where k = rank(Kk(A, r0)) is the numerical rank of the computed Krylov subspace.
I Threshold criterion: 10−1 (correspondence to a significant loss of orthogonality)
Correspondence among computed and exact approximation vectors
I xk ∈ FN : k-th approximation generated by FP CG
I xk ∈ FN : k-th approximation generated by exact CG
I Observation:
‖xk−xk‖∞‖x−xk‖∞
� 1
Distance between the exact and shifted FPapproximations is small in comparison with theactual size of error. 0 5 10 15 20 25
10−15
10−10
10−5
100
max
imum
nor
m
rank of the computed Krylov subspace
‖x− xk‖∞‖x− xk‖∞‖xk − xk‖∞‖xk−xk‖∞‖x−xk‖∞
Trajectory of approximations xk generated by FP CG computations follows closely the trajectoryof the exact CG approximations xk with a delay given by the rank-deficiency of the computedKrylov subspace.
Ax = b
xk
xk
x
FN
exact computation
finite precision computation
xk
xkx
x0
FNAx = b
exact computation
finite precision computation
delay at the k-th step
Correspondence among computed and exact Krylov subspaces
Comparison of (numerical) ranks of subspaces
IKk(A, r0): k-th Krylov subspace generated by FP CG
IKk(A, r0): k-th Krylov subspace generated by exact CG ( rank(Kk(A, r0)) = k )
k 30 52 103 154 205 256 307 358 409 460 511 562 613 664
k = rank(Kk(A, r0)) 30 52 81 87 95 100 106 112 115 120 125 128 131 132
rank(Kk(A, r0) ∪ Kk(A, r0)) 30 52 81 87 95 100 107 113 116 121 126 129 131 132
The distance between subspaces
I Principal angles between k-dimensional subspace Kk(A, r0) and the k-dimensional restriction of thesubspace Kk(A, r0) which corresponds to its numerical rank:
0 ≤ θ1 ≤ θ2 ≤ . . . ≤ θk ≤ π/2
I Distance:distance
(Kk(A, r0), restricted(Kk(A, r0))
)= sin(θk)
0 100 200 300 400 500 600 70010
−8
10−6
10−4
10−2
100
dist
ance
bet
wee
n co
rres
pond
ing
subs
pace
s
iteration number
sin(θk)sin(θk−1)sin(θk−2)sin(θk−3)
Data: Bcsstk04 from MatrixMarket database
Concluding remarks & work in progress
Krylov subspaces are in general sensitive to small perturbations of the matrix A. The observed “stability” (orinertia) of computed Krylov subspace represents a very remarkable phenomenon which needs to be studied.
Acknowledgement
This work has been supported by the ERC-CZ project LL1202, by the GACR grants 201/13-06684S and201/09/0917 and by the GAUK grant 695612.
Bibliography
Axelsson, O. (1976). A class of iterative methods for finite element equations. Comput. Methods Appl. Mech. Engrg. 9, 123–137.
Jennings, A. (1977). Influence of the eigenvalue spectrum on the convergence rate of the conjugate gradient method. J. Inst. Math. Appl. 20, 61–72.
Greenbaum, A. (1989). Behaviour of slightly perturbed Lanczos and conjugate-gradient recurrences. Linear Algebra and its Applications 113, 7–63.
Greenbaum, A. and Strakos, Z. (1992). Predicting the Behavior of Finite Precision Lanczos and Conjugate Gradient Computations. SIAM J. Matrix Anal. Appl. 13, 121–137.
Paige, C. C. and Strakos, Z. (1999). Correspondence between exact arithmetic and finite precision behaviour of Krylov space methods. In XIV. Householder Symposium, University of British Columbia, 250–253.
Liesen, J. and Strakos, Z. (2012). Krylov Subspace Methods – Principles and Analysis. Oxford University Press
Gergelits, T. and Strakos, Z. (2013). Composite convergence bounds based on Chebyshev polynomials and finite precision conjugate gradient computations. Numerical Algorithms, DOI: 10.1007/s11075-013-9713-z.
http://more.karlin.mff.cuni.cz
