Lecture 10: Miscellaneous

Applications

1

1. Linear integral operators

Let L2 = L2[a, b], the Lebesgue square integrable functions on

the finite interval [a, b]. Let K(s, t) be an L2–kernel on

a ≤ s, t,≤ b,

i.e.,

∫ b

a

∫ b

a

|K(s, t)|2ds dt exists and is finite.

Consider the two operators T1, T2 ∈ B(L2, L2) defined by

(T1x)(s) =

∫ b

a

K(s, t)x(t)dt , a ≤ s ≤ b ,

(T2x)(s) = x(s) −∫ b

a

K(s, t)x(t)dt , a ≤ s ≤ b ,

called Fredholm integral operators of the first kind and the

second kind, respectively. Then

(a) R(T2) is closed,

(b) R(T1) is nonclosed unless it is finite dimensional.

2

The Fredholm integral equation of the 2nd kind

x(s) − λ

∫ b

a

K(s, t) x(t) dt = y(s) , a ≤ s ≤ b , (1)

is also written as(I − λK)x = y ,

where λ and all functions are complex, and [a, b] is a bounded

interval.

We need the following facts from the Fredholm theory of integral

equations. For any λ, K as above

(a) (I − λK) ∈ B(L2, L2) ,

(b) (I − λK)∗ = I − λK∗ , where K∗(s, t) = K(t, s) .

(c) The null spaces N(I − λK) and N(I − λK∗) have equal finite

dimensions,

dim N(I − λK) = dim N(I − λK∗) = n(λ) , say . (2)

3

Fredholm (cont’d)

(d) A scalar λ is called a regular value of K if n(λ) = 0, in

which case the operator I − λK has an inverse

(I − λK)−1 ∈ B(L2, L2) written as

(I − λK)−1 = I + λR , (3)

where R = R(s, t; λ) is an L2–kernel called the resolvent of K.

(e) A scalar λ is called an eigenvalue of K if n(λ) > 0, in which

case any nonzero x ∈ N(I − λK) is called an eigenfunction of K

corresponding to λ.

For any λ and, in particular, for any eigenvalue λ, both range

spaces R(I − λK) and R(I − λK∗) are closed and,

R(I − λK) = N(I − λK∗)⊥ , R(I − λK∗) = N(I − λK)⊥ . (4)

4

Fredholm (cont’d)

(f) If λ is a regular value of K then (1) has, for any y ∈ L2, a

unique solution given by

x = (I + λR)y ,

or,

x(s) = y(s) + λ

∫ b

a

R(s, t, λ) y(t) dt , a ≤ s ≤ b . (5)

(g) If λ is an eigenvalue of K then (1) is consistent if and only if

y is orthogonal to every u ∈ N(I − λK∗), in which case the

general solution of (1) is

x = x0 +

n(λ)∑

i=1

cixi , ci arbitrary scalars ,

x0 a particular solution, {x1, . . . ,xn(λ)} a basis of N(I − λK).

5

Pseudo resolvents

Let λ be an eigenvalue of K. Following Hurwitz, an L2–kernel

R = R(s, t, λ) is called a pseudo resolvent of K if for any

y ∈ R(I − λK), the function

x(s) = y(s) + λ

∫ b

a

R(s, t, λ) y(t) dt (5)

is a solution of (1).

Hurwitz constructed a pseudo resolvent as follows.

Let λ0 be an eigenvalue of K, and let {x1, . . . ,xn} and

{u1, . . . ,un} be o.n. bases of N(I − λ0K) and N(I − λ0K∗)

respectively. Then λ0 is a regular value of the kernel

K0(s, t) = K(s, t) − 1

λ0

n∑

i=1

ui(s) xi(t) , (6)

6

Pseudo resolvents (cont’d)

The eigenvalue λ0 is a regular value of

K0(s, t) = K(s, t) − 1

λ0

n∑

i=1

ui(s) xi(t) , (6)

written for short as K0 = K − 1

λ0

n∑

i=1

uix∗i

and the resolvent R0 of K0 is a pseudo resolvent of K, satisfying

(I + λ0R0)(I − λ0K)x = x , for all x ∈ R(I − λ0K∗)

(I − λ0K)(I + λ0R0)y = y , for all y ∈ R(I − λ0K) (7)

(I + λ0R0)ui = xi , i = 1, . . . , n .

If R is a pseudo resolvent of K, then I + λR is a {1}–inverse of

I − λK. As with {1}–inverses, the pseudo resolvent is not unique.

7

Characterization of pseudo resolvents

The pseudo resolvent is not unique: For R0,ui,xi as above, and

any scalars cij , the kernel R0 +∑n

i,j,=1 cijxiu∗j is a pseudo

resolvent of K.

Theorem (Hurwitz). Let K be an L2–kernel, λ0 be an eigenvalue

of K and {x1, . . . ,xn} and {u1, . . . ,un} be orthonormal bases of

N(I − λ0K) and N(I − λ0K∗) respectively. An L2–kernel R is a

pseudo resolvent of K if and only if

R = K + λ0KR − 1

λ0

n∑

i=1

βiu∗i , (8a)

R = K + λ0RK − 1

λ0

n∑

i=1

xiα∗i , (8b)

where αi, βi ∈ L2 satisfy

〈αi,xj〉 = δij , 〈βi,uj〉 = δij , i, j = 1, . . . , n . (9)

8

Characterization (cont’d)

Here KR stands for the kernel

KR(s, t) =

∫ b

a

K(s, u)R(u, t) du

If λ is a regular value of K then (8a)–(8b) reduce to

R = K + λKR , R = K + λRK ,

which uniquely determines the resolvent R(s, t, λ).

9

Degenerate kernels

A kernel K(s, t) is called degenerate if it is a finite sum of

products of L2 functions, as follows:

K(s, t) =m∑

i=1

fi(s) gi(t) . (10)

Degenerate kernels are convenient because they reduce the integral

equation (1) to a finite system of linear equations. Also, any

L2–kernel can be approximated, arbitrarily close, by a degenerate

kernel.

Let K(s, t) be given by (10). Then

(a) The scalar λ is an eigenvalue of (10) if and only if 1/λ is an

eigenvalue of the m × m matrix

B = [bij ] , where bij =

∫ b

a

fj(s) gi(s) ds .

10

Degenerate kernels (cont’d)

(b) Any eigenfunction of K [K∗] corresponding to an eigenvalue λ

[λ] is a linear combination of the m functions f1, . . . , fm

[g1, . . . , gm].

(c) If λ is a regular value of (10), then the resolvent at λ is

R(s, t, ; λ) =

det

0... f1(s) · · · fm(s)

· · · · · · · · · · · · · · ·

−g1(t)...

...... I − λB

−gm(t)...

det(I − λB).

11

Example

Consider the equation

x(s) − λ

∫ 1

−1

(1 + 3st) x(t) dt = y(s) (11)

with K(s, t) = 1 + 3st. The resolvent is

R(s, t; λ) =1 + 3st

1 − 2λ.

K has a single eigenvalue λ = 12 and an o.n. basis of N(I − 1

2K) is

{x1(s) =

1√2, x2(s) =

√3√2

s

}

which, by symmetry, is also an orthonormal basis of N(I − 12K∗).

12

Example (cont’d)

From (6) we get

K0(s, t) = K(s, t) − 1

λ0

∑ui(s) xi(t)

= (1 + 3st) − 2

(1√2

1√2

+

√3√2s

√3√2t

)

= 0 ,

and the resolvent of K0(s, t) is therefore

R0(s, t; λ) = 0 .

(a) If λ 6= 12 , then for each y ∈ L2[−1, 1] equation (11) has a unique

solution,

x(s) = y(s) + λ

∫ 1

−1

1 + 3st

1 − 2λy(t) dt .

13

Example (cont’d)

(b) If λ = 12 , then (11) is consistent if and only if

∫ 1

−1

y(t) dt = 0 ,

∫ 1

−1

t y(t) dt = 0 ,

in which case the general solution is

x(s) = y(s) + c1 + c2s , c1, c2 arbitrary .

14

2. Linear systems theory

Systems modeled by linear differential equations call for symbolic

computation of generalized inverses for matrices whose elements are

rational functions.

As example, consider the homogeneous system

A(D)x(t) = 0 (1)

where x(t) : [0−,∞) → Rn, D :=d

dt,

A(D) = AqDq + · · · + A1D + A0 , (2)

and Ai ∈ Rm×n , i = 0, 1, . . . , q. Let L denote the Laplace

transform, and let x̂(s) = L(x(t)). The system (1) transforms to

A(s)x̂(s) = b̂(s) ,

allowing algebraic solution.

15

Linear systems theory (cont’d)

Theorem (Jones, Karampetakis and Pugh) The system (1)

has a solution if and only if

A(s)A(s)†b̂(s) = b̂(s) (3)

in which case the general solution is

x(t) = L−1(x̂(s)) = L−1{

A(s)†b̂(s) + (In − A(s)†A(s))y(s)}

(4)

where y(s) ∈ Rn(s) is arbitrary. �

16

3. Tchebycheff approximation

A Tchebycheff approximate solution of the system

Ax = b (1)

is a vector x minimizing the Tchebycheff norm

‖r‖∞ = maxi=1,...,m

{|ri|}

of the residual vector

r = b− Ax . (2)

Let A ∈ C(n+1)×nn and b ∈ Cn+1 be such that (1) is inconsistent.

Then (1) has a unique Tchebycheff approximate solution given by

x = A†(b + r) , (3)

17

Tchebycheff approximation (cont’d)

where the residual r = [ri] is

ri =

n+1∑j=1

|(PN(A∗)b)j |2

n+1∑j=1

|(PN(A∗)b)j |

(PN(A∗)b)i

|(PN(A∗)b)i|, i ∈ 1, n + 1 . (4)

Proof. From

r(x) − b = −Ax ∈ R(A)

it follows that any residual r satisfies

PN(A∗)r = PN(A∗)b

or equivalently

〈PN(A∗)b, r〉 = 〈b, PN(A∗)b〉 , (5)

since dimN(A∗) = 1 and b 6∈ R(A).

18

Tchebycheff approximation (cont’d)

Equation (5) represents the hyperplane of residuals. A routine

computation now shows, that among all residuals r satisfying (5)

there is a unique residual of minimum Tchebycheff norm

given by (4), from which (3) follows since N(A) = {0}.

19

4. Interval linear programming

For two vectors u = (ui),v = (vi) ∈ Rm let

u ≤ v

denote the fact that ui ≤ vi for i ∈ 1, m. A linear programming

problem of the form

maximize {cT x : a ≤ Ax ≤ b} , (1)

with given a,b ∈ Rm; c ∈ Rn; A ∈ Rm×n, is called an interval

linear program and denoted by IP (a,b, c, A) or simply by IP .

The IP (1) is consistent (also feasible) if the set

F = {x ∈ Rn : a ≤ Ax ≤ b} 6= ∅ (2)

in which case the elements of F are the feasible solutions of

IP (a,b, c, A).

20

Interval linear programming (cont’d)

A consistent IP (a,b, c, A) is bounded if

max {cT x : x ∈ F}

is finite, in which case the optimal solutions of IP (a,b, c, A) are

its feasible solutions x0 which satisfy

cT x0 = max{cT x : x ∈ F} .

Lemma. Let a,b ∈ Rm; c ∈ Rn; A ∈ Rm×n be such that

IP (a,b, c, A) is consistent. Then IP (a,b, c, A) is bounded if and

only if

c ∈ N(A)⊥ . (3)

Proof. F = F + N(A), etc. �

21

Interval linear programming (cont’d)

Let η : Rm × R

m × Rm → R

m be defined for u,v,w ∈ Rm by

ηi =

ui if wi < 0,

vi if wi > 0,

λiui + (1 − λi)vi where 0 ≤ λi ≤ 1 , if wi = 0

(4)

Theorem. Let a,b ∈ Rm; c ∈ R

n; A ∈ Rm×nm (full row-rank) be

such that IP (a,b, c, A) is consistent and bounded, and let A(1) be

any {1}–inverse of A. Then the general optimal solution of

IP (a,b, c, A) is

x = A(1)η(a,b, A(1)Tc) + y , y ∈ N(A) . (5)

Proof. For u = Ax, the problem (1) is

max {cT A(1)u : a ≤ u ≤ b} , etc.

Note: The rank assumption is a severe restriction of usefulness.

22

5. Nonlinear least squares solutions

Let f : Rn → R

m, and let

Jf (x) =

(∂fi(x)

∂xj

).

If the Newton method

x+ := x − Jf (x)† f(x)

converges to x∞, plus 2 more if ’s, then

Jf (x∞)† f(x∞) = 0

and x∞ is a stationary point of ‖f(x)‖2.

A Maple code for a Newton method using the Moore–Penrose

inverse of the Jacobi matrix is available, contact the instructor or

see http://benisrael.net/Newton-MP.pdf

23