lecture 1b (SI 507)n 1 = b n 1 A n 1n x n A n 1n 1 and so on... This is referred to as back substitution algorithm |Similar algorithm works for lower-triangular matrices Harsha Hutridurga

lecture 1b (SI 507)

Harsha Hutridurga

IIT Bombay

Harsha Hutridurga (IIT Bombay) Lecture 1b (SI 507) 1 / 11

systems of linear equations

♣ Notation:

Mm,n(R) := set of all matrices of size m× n

A = (Aij)1≤i≤m, 1≤j≤n ∈Mm,n(R)

b = (bi)1≤i≤m ∈ Rm

♣ objective:

Given a matrix A ∈Mm,n(R) and a vector b ∈ Rm,find a vector x ∈ Rn such that

Ax = b

♣ m equations and n unknowns:

A11x1 + · · ·+A1nxn = b1

A21x1 + · · ·+A2nxn = b2...

......

Am1x1 + · · ·+Amnxn = bmHarsha Hutridurga (IIT Bombay) Lecture 1b (SI 507) 2 / 11

why linear algebraic systems?

♣ Solving linear systems is of paramount importance in applications:

I mechanical systemsI atmospheric sciencesI data analysis

♣ History of solving these systems: gauss, jacobi, etc...

♣ Advent of computers revolutionised this field in mid-20th century


relevant questions

Given A and b, find x such that

Ax = b

♣ Does there exist a x ∈ Rn such that the above relation holds?

♣ If it exists, is it unique?

♣ How does one go about solving for x ∈ Rn?

♣ How practical is that approach for finding x?

categories

♣ m = n square linear system

♣ m < n underdetermined system

♣ m > n overdetermined system


underdetermined systems (m < n)

♣ More unknowns than equations

♣ do not expect uniqueness

♣ Such systems have

I either no solution (inconsistent)

I or infinitely many solutions (dependent)

♣ Example (inconsistent system):(1 1 11 1 1

) x1x2x3

=

(10

)♣ Example (dependent system):(

1 1 11 1 2

) x1x2x3

=

(13

)Harsha Hutridurga (IIT Bombay) Lecture 1b (SI 507) 5 / 11

overdetermined systems (m > n)

♣ More equations than unknowns

♣ Less possibility of existence of solution.

♣ Solutions may exist only when the number of linearly independentequations doesn’t exceed the number of unknowns

♣ Example (no solution): 1 32 71 −1

( x1x2

)=

123

♣ Example (solution exists): 1 1

1 24 8

( x1x2

)=

328


square system (m = n)

♣ same number of equations as the unknowns

♣ Either of the following holds true:

I If A is invertible, then there exists a unique solution x

I If A is not invertible, then

F either there is no solution

F or there are infinitely many solutions

A is not invertible (Ker(A) 6= {0})

♣ b /∈Range(A) i.e. @y ∈ Rn such that Ay = b (no solution)

♣ b ∈Range(A) i.e. ∃y ∈ Rn such that Ay = b. Then

x = y + c r with c ∈ R and r ∈ Ker(A) \ {0}

solves Ax = b (infinitely many solutions)


A is invertible

♣ The unique solution is given by

x = A−1b

♣ matrix inversion is prohibitively expensive

cramer’s formula

♣ Explicit formula for the unknown vector x

♣ Formula dates back to 1750

♣ Elegant formula

xi =det (Ai)

det (A)for i = 1, . . . , n

Ai is a matrix obtained by replacing ith column of A by b

♣ Formula deceptively simple

♣ expensive as it involves computation of determinants


special instances

♣ If A is a diagonal matrix, i.e.

Aij =

{Aii 6= 0 i = j = 1, . . . , n

0 i 6= j; i, j = 1, . . . , n

then the solution is

xi =biAii

for i = 1, . . . , n.

♣ If A is a unitary matrix, i.e. A−1 = A>, then the solution is

x = A>b i.e. xi =

n∑j=1

Aji bj for i = 1, . . . , n

♣ Examples: for θ ∈ [0, π],

A =

(cos θ − sin θsin θ cos θ

)A =

cos θ 0 − sin θ

0 1 0

sin θ 0 cos θ


one more special instance

♣ If A is an upper-triangular matrix, i.e.

Aij =

{Aij i ≤ j

0 i > j

then the solution is

xi =

bi −N∑

j=i+1

aijxj

aiifor i = N,N − 1, . . . , 2, 1

i.e. xn =bnAnn

, xn−1 =bn−1 −An−1n xn

An−1n−1and so on...

This is referred to as back substitution algorithm

♣ Similar algorithm works for lower-triangular matrices


thanks for your attention


Documents

lecture 1b (SI 507)n 1 = b n 1 A n 1n x n A n 1n 1 and so on... This is referred to as back substitution algorithm |Similar algorithm works for lower-triangular matrices Harsha Hutridurga