Linear Algebra. Session 11 - Texas A&M Universityroquesol/Math_304_Fall_2018... · 2018-11-15 · Session 11. Abstract Linear Algebra II Least Squares Problems Orthogonal Sets Least

Abstract Linear Algebra II

Linear Algebra. Session 11

Dr. Marco A Roque Sol

11 / 06 / 2018

Dr. Marco A Roque Sol Linear Algebra. Session 11

Abstract Linear Algebra IILeast Squares ProblemsOrthogonal Sets

Least Squares Problems.

Least Squares Problems

Let’s consider the Overdetermined system of linear equations:x + 2y = 3

3x + 2y = 5x + y = 2.09

⇒

x + 2y = 3−4y = −4−y = −0.09

Now, assume that a solution (x0, y0) does exist in fact but thesystem is not quite accurate, namely, there may be some errors inthe right-hand sides (rounding errors for instance).

Problem

Find a good approximation of (x0, y0)






3x + 2y = 5x + y = 2.09

⇒

x + 2y = 3−4y = −4−y = −0.09


Problem






Let’s consider

the Overdetermined system of linear equations:x + 2y = 3

3x + 2y = 5x + y = 2.09

⇒

x + 2y = 3−4y = −4−y = −0.09


Problem






Let’s consider the Overdetermined system

of linear equations:x + 2y = 3

3x + 2y = 5x + y = 2.09

⇒

x + 2y = 3−4y = −4−y = −0.09


Problem






Let’s consider the Overdetermined system of linear equations:

x + 2y = 3

3x + 2y = 5x + y = 2.09

⇒

x + 2y = 3−4y = −4−y = −0.09


Problem







3x + 2y = 5x + y = 2.09

⇒

x + 2y = 3−4y = −4−y = −0.09


Problem







3x + 2y = 5x + y = 2.09

⇒

x + 2y = 3−4y = −4−y = −0.09


Problem







3x + 2y = 5x + y = 2.09

⇒

x + 2y = 3−4y = −4−y = −0.09

Now,

assume that a solution (x0, y0) does exist in fact but thesystem is not quite accurate, namely, there may be some errors inthe right-hand sides (rounding errors for instance).

Problem







3x + 2y = 5x + y = 2.09

⇒

x + 2y = 3−4y = −4−y = −0.09

Now, assume that

a solution (x0, y0) does exist in fact but thesystem is not quite accurate, namely, there may be some errors inthe right-hand sides (rounding errors for instance).

Problem







3x + 2y = 5x + y = 2.09

⇒

x + 2y = 3−4y = −4−y = −0.09

Now, assume that a solution (x0, y0)

does exist in fact but thesystem is not quite accurate, namely, there may be some errors inthe right-hand sides (rounding errors for instance).

Problem







3x + 2y = 5x + y = 2.09

⇒

x + 2y = 3−4y = −4−y = −0.09

Now, assume that a solution (x0, y0) does exist in fact

but thesystem is not quite accurate, namely, there may be some errors inthe right-hand sides (rounding errors for instance).

Problem







3x + 2y = 5x + y = 2.09

⇒

x + 2y = 3−4y = −4−y = −0.09

Now, assume that a solution (x0, y0) does exist in fact but thesystem

is not quite accurate, namely, there may be some errors inthe right-hand sides (rounding errors for instance).

Problem







3x + 2y = 5x + y = 2.09

⇒

x + 2y = 3−4y = −4−y = −0.09

Now, assume that a solution (x0, y0) does exist in fact but thesystem is not quite accurate,

namely, there may be some errors inthe right-hand sides (rounding errors for instance).

Problem







3x + 2y = 5x + y = 2.09

⇒

x + 2y = 3−4y = −4−y = −0.09

Now, assume that a solution (x0, y0) does exist in fact but thesystem is not quite accurate, namely,

there may be some errors inthe right-hand sides (rounding errors for instance).

Problem







3x + 2y = 5x + y = 2.09

⇒

x + 2y = 3−4y = −4−y = −0.09

Now, assume that a solution (x0, y0) does exist in fact but thesystem is not quite accurate, namely, there may be some errors

inthe right-hand sides (rounding errors for instance).

Problem







3x + 2y = 5x + y = 2.09

⇒

x + 2y = 3−4y = −4−y = −0.09

Now, assume that a solution (x0, y0) does exist in fact but thesystem is not quite accurate, namely, there may be some errors inthe right-hand sides

(rounding errors for instance).

Problem







3x + 2y = 5x + y = 2.09

⇒

x + 2y = 3−4y = −4−y = −0.09


Problem







3x + 2y = 5x + y = 2.09

⇒

x + 2y = 3−4y = −4−y = −0.09


Problem







3x + 2y = 5x + y = 2.09

⇒

x + 2y = 3−4y = −4−y = −0.09


Problem

Find

a good approximation of (x0, y0)






3x + 2y = 5x + y = 2.09

⇒

x + 2y = 3−4y = −4−y = −0.09


Problem

Find a good approximation

of (x0, y0)






3x + 2y = 5x + y = 2.09

⇒

x + 2y = 3−4y = −4−y = −0.09


Problem





One approach is the least squares fit. Namely, we look for a pair(x , y) that minimize the sum

(x + 2y − 3)2 + (3x + 2y − 5)2 + (x + y − 2.09)2

Least squares solution

System of linear equations:a11x1 + a12x2 + · · ·+ a1nxn = b1a21x1 + a22x2 + · · ·+ a2nxn = b2

...am1x1 + am2x2 + · · ·+ amnxn = bm

For any x ∈ R define a residual r(x) = b− Ax




One approach

is the least squares fit. Namely, we look for a pair(x , y) that minimize the sum

(x + 2y − 3)2 + (3x + 2y − 5)2 + (x + y − 2.09)2



...am1x1 + am2x2 + · · ·+ amnxn = bm





One approach is the least squares fit.

Namely, we look for a pair(x , y) that minimize the sum

(x + 2y − 3)2 + (3x + 2y − 5)2 + (x + y − 2.09)2



...am1x1 + am2x2 + · · ·+ amnxn = bm





One approach is the least squares fit. Namely,

we look for a pair(x , y) that minimize the sum

(x + 2y − 3)2 + (3x + 2y − 5)2 + (x + y − 2.09)2



...am1x1 + am2x2 + · · ·+ amnxn = bm





One approach is the least squares fit. Namely, we look for

a pair(x , y) that minimize the sum

(x + 2y − 3)2 + (3x + 2y − 5)2 + (x + y − 2.09)2



...am1x1 + am2x2 + · · ·+ amnxn = bm





One approach is the least squares fit. Namely, we look for a pair(x , y)

that minimize the sum

(x + 2y − 3)2 + (3x + 2y − 5)2 + (x + y − 2.09)2



...am1x1 + am2x2 + · · ·+ amnxn = bm





One approach is the least squares fit. Namely, we look for a pair(x , y) that minimize

the sum

(x + 2y − 3)2 + (3x + 2y − 5)2 + (x + y − 2.09)2



...am1x1 + am2x2 + · · ·+ amnxn = bm






(x + 2y − 3)2 + (3x + 2y − 5)2 + (x + y − 2.09)2



...am1x1 + am2x2 + · · ·+ amnxn = bm






(x + 2y − 3)2 + (3x + 2y − 5)2 + (x + y − 2.09)2



...am1x1 + am2x2 + · · ·+ amnxn = bm






(x + 2y − 3)2 + (3x + 2y − 5)2 + (x + y − 2.09)2



...am1x1 + am2x2 + · · ·+ amnxn = bm






(x + 2y − 3)2 + (3x + 2y − 5)2 + (x + y − 2.09)2


System of linear equations:

a11x1 + a12x2 + · · ·+ a1nxn = b1a21x1 + a22x2 + · · ·+ a2nxn = b2

...am1x1 + am2x2 + · · ·+ amnxn = bm






(x + 2y − 3)2 + (3x + 2y − 5)2 + (x + y − 2.09)2



...am1x1 + am2x2 + · · ·+ amnxn = bm






(x + 2y − 3)2 + (3x + 2y − 5)2 + (x + y − 2.09)2



...am1x1 + am2x2 + · · ·+ amnxn = bm

For any

x ∈ R define a residual r(x) = b− Ax





(x + 2y − 3)2 + (3x + 2y − 5)2 + (x + y − 2.09)2



...am1x1 + am2x2 + · · ·+ amnxn = bm

For any x ∈ R

define a residual r(x) = b− Ax





(x + 2y − 3)2 + (3x + 2y − 5)2 + (x + y − 2.09)2



...am1x1 + am2x2 + · · ·+ amnxn = bm

For any x ∈ R define a residual

r(x) = b− Ax





(x + 2y − 3)2 + (3x + 2y − 5)2 + (x + y − 2.09)2



...am1x1 + am2x2 + · · ·+ amnxn = bm





The least squares solution x to the system is the one thatminimizes ||r(x)|| (or, equivalently, ||r(x)||2 ).

||r(x)||2 = (m∑i=1

(ai1x1 + ai2x2 + · · ·+ ainxn − bi )2

Let A be an m × n matrix and let b ∈ Rn

Theorem

A vector x̂ is a least squares solution of the system Ax if and onlyif it is a solution of the associated normal system ATAx = ATb




The least squares solution

x to the system is the one thatminimizes ||r(x)|| (or, equivalently, ||r(x)||2 ).

||r(x)||2 = (m∑i=1

(ai1x1 + ai2x2 + · · ·+ ainxn − bi )2


Theorem





The least squares solution x

to the system is the one thatminimizes ||r(x)|| (or, equivalently, ||r(x)||2 ).

||r(x)||2 = (m∑i=1

(ai1x1 + ai2x2 + · · ·+ ainxn − bi )2


Theorem





The least squares solution x to the system

is the one thatminimizes ||r(x)|| (or, equivalently, ||r(x)||2 ).

||r(x)||2 = (m∑i=1

(ai1x1 + ai2x2 + · · ·+ ainxn − bi )2


Theorem





The least squares solution x to the system is the one

thatminimizes ||r(x)|| (or, equivalently, ||r(x)||2 ).

||r(x)||2 = (m∑i=1

(ai1x1 + ai2x2 + · · ·+ ainxn − bi )2


Theorem





The least squares solution x to the system is the one thatminimizes ||r(x)||

(or, equivalently, ||r(x)||2 ).

||r(x)||2 = (m∑i=1

(ai1x1 + ai2x2 + · · ·+ ainxn − bi )2


Theorem





The least squares solution x to the system is the one thatminimizes ||r(x)|| (or, equivalently,

||r(x)||2 ).

||r(x)||2 = (m∑i=1

(ai1x1 + ai2x2 + · · ·+ ainxn − bi )2


Theorem






||r(x)||2 = (m∑i=1

(ai1x1 + ai2x2 + · · ·+ ainxn − bi )2


Theorem






||r(x)||2 = (m∑i=1

(ai1x1 + ai2x2 + · · ·+ ainxn − bi )2


Theorem






||r(x)||2 = (m∑i=1

(ai1x1 + ai2x2 + · · ·+ ainxn − bi )2

Let A

be an m × n matrix and let b ∈ Rn

Theorem






||r(x)||2 = (m∑i=1

(ai1x1 + ai2x2 + · · ·+ ainxn − bi )2

Let A be an

m × n matrix and let b ∈ Rn

Theorem






||r(x)||2 = (m∑i=1

(ai1x1 + ai2x2 + · · ·+ ainxn − bi )2

Let A be an m × n matrix and

let b ∈ Rn

Theorem






||r(x)||2 = (m∑i=1

(ai1x1 + ai2x2 + · · ·+ ainxn − bi )2

Let A be an m × n matrix and let

b ∈ Rn

Theorem






||r(x)||2 = (m∑i=1

(ai1x1 + ai2x2 + · · ·+ ainxn − bi )2


Theorem






||r(x)||2 = (m∑i=1

(ai1x1 + ai2x2 + · · ·+ ainxn − bi )2


Theorem






||r(x)||2 = (m∑i=1

(ai1x1 + ai2x2 + · · ·+ ainxn − bi )2


Theorem

A vector

x̂ is a least squares solution of the system Ax if and onlyif it is a solution of the associated normal system ATAx = ATb





||r(x)||2 = (m∑i=1

(ai1x1 + ai2x2 + · · ·+ ainxn − bi )2


Theorem

A vector x̂

is a least squares solution of the system Ax if and onlyif it is a solution of the associated normal system ATAx = ATb





||r(x)||2 = (m∑i=1

(ai1x1 + ai2x2 + · · ·+ ainxn − bi )2


Theorem

A vector x̂ is a least squares solution

of the system Ax if and onlyif it is a solution of the associated normal system ATAx = ATb





||r(x)||2 = (m∑i=1

(ai1x1 + ai2x2 + · · ·+ ainxn − bi )2


Theorem

A vector x̂ is a least squares solution of the system

Ax if and onlyif it is a solution of the associated normal system ATAx = ATb





||r(x)||2 = (m∑i=1

(ai1x1 + ai2x2 + · · ·+ ainxn − bi )2


Theorem

A vector x̂ is a least squares solution of the system Ax

if and onlyif it is a solution of the associated normal system ATAx = ATb





||r(x)||2 = (m∑i=1

(ai1x1 + ai2x2 + · · ·+ ainxn − bi )2


Theorem

A vector x̂ is a least squares solution of the system Ax if and only

if it is a solution of the associated normal system ATAx = ATb





||r(x)||2 = (m∑i=1

(ai1x1 + ai2x2 + · · ·+ ainxn − bi )2


Theorem

A vector x̂ is a least squares solution of the system Ax if and onlyif it is a solution

of the associated normal system ATAx = ATb





||r(x)||2 = (m∑i=1

(ai1x1 + ai2x2 + · · ·+ ainxn − bi )2


Theorem

A vector x̂ is a least squares solution of the system Ax if and onlyif it is a solution of the associated

normal system ATAx = ATb





||r(x)||2 = (m∑i=1

(ai1x1 + ai2x2 + · · ·+ ainxn − bi )2


Theorem

A vector x̂ is a least squares solution of the system Ax if and onlyif it is a solution of the associated normal system

ATAx = ATb





||r(x)||2 = (m∑i=1

(ai1x1 + ai2x2 + · · ·+ ainxn − bi )2


Theorem





proof

Ax is an arbitrary vector in R(A), the column space of A. Hencethe length of r(x) = b− Ax is minimal if Ax is the orthogonalprojection of b onto R(A) that is, if r(x) is orthogonal to R(A).We know that row space⊥ = Nullspace for any matrix. Inparticular, R(A)⊥ = N(A) the nullspace of the transpose matrix ofA. Thus, x̂ is a least squares solution if and only if

AT r(x) = 0 ⇐⇒ AT (b− Ax) = 0 ⇐⇒ ATAx = ATb �




proof






proof

Ax

is an arbitrary vector in R(A), the column space of A. Hencethe length of r(x) = b− Ax is minimal if Ax is the orthogonalprojection of b onto R(A) that is, if r(x) is orthogonal to R(A).We know that row space⊥ = Nullspace for any matrix. Inparticular, R(A)⊥ = N(A) the nullspace of the transpose matrix ofA. Thus, x̂ is a least squares solution if and only if





proof

Ax is an arbitrary vector

in R(A), the column space of A. Hencethe length of r(x) = b− Ax is minimal if Ax is the orthogonalprojection of b onto R(A) that is, if r(x) is orthogonal to R(A).We know that row space⊥ = Nullspace for any matrix. Inparticular, R(A)⊥ = N(A) the nullspace of the transpose matrix ofA. Thus, x̂ is a least squares solution if and only if





proof

Ax is an arbitrary vector in R(A),

the column space of A. Hencethe length of r(x) = b− Ax is minimal if Ax is the orthogonalprojection of b onto R(A) that is, if r(x) is orthogonal to R(A).We know that row space⊥ = Nullspace for any matrix. Inparticular, R(A)⊥ = N(A) the nullspace of the transpose matrix ofA. Thus, x̂ is a least squares solution if and only if





proof

Ax is an arbitrary vector in R(A), the column space

of A. Hencethe length of r(x) = b− Ax is minimal if Ax is the orthogonalprojection of b onto R(A) that is, if r(x) is orthogonal to R(A).We know that row space⊥ = Nullspace for any matrix. Inparticular, R(A)⊥ = N(A) the nullspace of the transpose matrix ofA. Thus, x̂ is a least squares solution if and only if





proof

Ax is an arbitrary vector in R(A), the column space of A. Hence

the length of r(x) = b− Ax is minimal if Ax is the orthogonalprojection of b onto R(A) that is, if r(x) is orthogonal to R(A).We know that row space⊥ = Nullspace for any matrix. Inparticular, R(A)⊥ = N(A) the nullspace of the transpose matrix ofA. Thus, x̂ is a least squares solution if and only if





proof

Ax is an arbitrary vector in R(A), the column space of A. Hencethe length

of r(x) = b− Ax is minimal if Ax is the orthogonalprojection of b onto R(A) that is, if r(x) is orthogonal to R(A).We know that row space⊥ = Nullspace for any matrix. Inparticular, R(A)⊥ = N(A) the nullspace of the transpose matrix ofA. Thus, x̂ is a least squares solution if and only if





proof

Ax is an arbitrary vector in R(A), the column space of A. Hencethe length of r(x) = b− Ax

is minimal if Ax is the orthogonalprojection of b onto R(A) that is, if r(x) is orthogonal to R(A).We know that row space⊥ = Nullspace for any matrix. Inparticular, R(A)⊥ = N(A) the nullspace of the transpose matrix ofA. Thus, x̂ is a least squares solution if and only if





proof

Ax is an arbitrary vector in R(A), the column space of A. Hencethe length of r(x) = b− Ax is minimal

if Ax is the orthogonalprojection of b onto R(A) that is, if r(x) is orthogonal to R(A).We know that row space⊥ = Nullspace for any matrix. Inparticular, R(A)⊥ = N(A) the nullspace of the transpose matrix ofA. Thus, x̂ is a least squares solution if and only if





proof

Ax is an arbitrary vector in R(A), the column space of A. Hencethe length of r(x) = b− Ax is minimal if Ax

is the orthogonalprojection of b onto R(A) that is, if r(x) is orthogonal to R(A).We know that row space⊥ = Nullspace for any matrix. Inparticular, R(A)⊥ = N(A) the nullspace of the transpose matrix ofA. Thus, x̂ is a least squares solution if and only if





proof

Ax is an arbitrary vector in R(A), the column space of A. Hencethe length of r(x) = b− Ax is minimal if Ax is the

orthogonalprojection of b onto R(A) that is, if r(x) is orthogonal to R(A).We know that row space⊥ = Nullspace for any matrix. Inparticular, R(A)⊥ = N(A) the nullspace of the transpose matrix ofA. Thus, x̂ is a least squares solution if and only if





proof

Ax is an arbitrary vector in R(A), the column space of A. Hencethe length of r(x) = b− Ax is minimal if Ax is the orthogonalprojection

of b onto R(A) that is, if r(x) is orthogonal to R(A).We know that row space⊥ = Nullspace for any matrix. Inparticular, R(A)⊥ = N(A) the nullspace of the transpose matrix ofA. Thus, x̂ is a least squares solution if and only if





proof

Ax is an arbitrary vector in R(A), the column space of A. Hencethe length of r(x) = b− Ax is minimal if Ax is the orthogonalprojection of b

onto R(A) that is, if r(x) is orthogonal to R(A).We know that row space⊥ = Nullspace for any matrix. Inparticular, R(A)⊥ = N(A) the nullspace of the transpose matrix ofA. Thus, x̂ is a least squares solution if and only if





proof

Ax is an arbitrary vector in R(A), the column space of A. Hencethe length of r(x) = b− Ax is minimal if Ax is the orthogonalprojection of b onto R(A)

that is, if r(x) is orthogonal to R(A).We know that row space⊥ = Nullspace for any matrix. Inparticular, R(A)⊥ = N(A) the nullspace of the transpose matrix ofA. Thus, x̂ is a least squares solution if and only if





proof

Ax is an arbitrary vector in R(A), the column space of A. Hencethe length of r(x) = b− Ax is minimal if Ax is the orthogonalprojection of b onto R(A) that is,

if r(x) is orthogonal to R(A).We know that row space⊥ = Nullspace for any matrix. Inparticular, R(A)⊥ = N(A) the nullspace of the transpose matrix ofA. Thus, x̂ is a least squares solution if and only if





proof

Ax is an arbitrary vector in R(A), the column space of A. Hencethe length of r(x) = b− Ax is minimal if Ax is the orthogonalprojection of b onto R(A) that is, if r(x)

is orthogonal to R(A).We know that row space⊥ = Nullspace for any matrix. Inparticular, R(A)⊥ = N(A) the nullspace of the transpose matrix ofA. Thus, x̂ is a least squares solution if and only if





proof

Ax is an arbitrary vector in R(A), the column space of A. Hencethe length of r(x) = b− Ax is minimal if Ax is the orthogonalprojection of b onto R(A) that is, if r(x) is orthogonal

to R(A).We know that row space⊥ = Nullspace for any matrix. Inparticular, R(A)⊥ = N(A) the nullspace of the transpose matrix ofA. Thus, x̂ is a least squares solution if and only if





proof

Ax is an arbitrary vector in R(A), the column space of A. Hencethe length of r(x) = b− Ax is minimal if Ax is the orthogonalprojection of b onto R(A) that is, if r(x) is orthogonal to R(A).

We know that row space⊥ = Nullspace for any matrix. Inparticular, R(A)⊥ = N(A) the nullspace of the transpose matrix ofA. Thus, x̂ is a least squares solution if and only if





proof

Ax is an arbitrary vector in R(A), the column space of A. Hencethe length of r(x) = b− Ax is minimal if Ax is the orthogonalprojection of b onto R(A) that is, if r(x) is orthogonal to R(A).We know

that row space⊥ = Nullspace for any matrix. Inparticular, R(A)⊥ = N(A) the nullspace of the transpose matrix ofA. Thus, x̂ is a least squares solution if and only if





proof

Ax is an arbitrary vector in R(A), the column space of A. Hencethe length of r(x) = b− Ax is minimal if Ax is the orthogonalprojection of b onto R(A) that is, if r(x) is orthogonal to R(A).We know that row space⊥ =

Nullspace for any matrix. Inparticular, R(A)⊥ = N(A) the nullspace of the transpose matrix ofA. Thus, x̂ is a least squares solution if and only if





proof

Ax is an arbitrary vector in R(A), the column space of A. Hencethe length of r(x) = b− Ax is minimal if Ax is the orthogonalprojection of b onto R(A) that is, if r(x) is orthogonal to R(A).We know that row space⊥ = Nullspace

for any matrix. Inparticular, R(A)⊥ = N(A) the nullspace of the transpose matrix ofA. Thus, x̂ is a least squares solution if and only if





proof

Ax is an arbitrary vector in R(A), the column space of A. Hencethe length of r(x) = b− Ax is minimal if Ax is the orthogonalprojection of b onto R(A) that is, if r(x) is orthogonal to R(A).We know that row space⊥ = Nullspace for any matrix.

Inparticular, R(A)⊥ = N(A) the nullspace of the transpose matrix ofA. Thus, x̂ is a least squares solution if and only if





proof

Ax is an arbitrary vector in R(A), the column space of A. Hencethe length of r(x) = b− Ax is minimal if Ax is the orthogonalprojection of b onto R(A) that is, if r(x) is orthogonal to R(A).We know that row space⊥ = Nullspace for any matrix. Inparticular, R(A)⊥ = N(A)

the nullspace of the transpose matrix ofA. Thus, x̂ is a least squares solution if and only if





proof

Ax is an arbitrary vector in R(A), the column space of A. Hencethe length of r(x) = b− Ax is minimal if Ax is the orthogonalprojection of b onto R(A) that is, if r(x) is orthogonal to R(A).We know that row space⊥ = Nullspace for any matrix. Inparticular, R(A)⊥ = N(A) the nullspace

of the transpose matrix ofA. Thus, x̂ is a least squares solution if and only if





proof

Ax is an arbitrary vector in R(A), the column space of A. Hencethe length of r(x) = b− Ax is minimal if Ax is the orthogonalprojection of b onto R(A) that is, if r(x) is orthogonal to R(A).We know that row space⊥ = Nullspace for any matrix. Inparticular, R(A)⊥ = N(A) the nullspace of the transpose matrix

ofA. Thus, x̂ is a least squares solution if and only if





proof

Ax is an arbitrary vector in R(A), the column space of A. Hencethe length of r(x) = b− Ax is minimal if Ax is the orthogonalprojection of b onto R(A) that is, if r(x) is orthogonal to R(A).We know that row space⊥ = Nullspace for any matrix. Inparticular, R(A)⊥ = N(A) the nullspace of the transpose matrix ofA.

Thus, x̂ is a least squares solution if and only if





proof

Ax is an arbitrary vector in R(A), the column space of A. Hencethe length of r(x) = b− Ax is minimal if Ax is the orthogonalprojection of b onto R(A) that is, if r(x) is orthogonal to R(A).We know that row space⊥ = Nullspace for any matrix. Inparticular, R(A)⊥ = N(A) the nullspace of the transpose matrix ofA. Thus,

x̂ is a least squares solution if and only if





proof

Ax is an arbitrary vector in R(A), the column space of A. Hencethe length of r(x) = b− Ax is minimal if Ax is the orthogonalprojection of b onto R(A) that is, if r(x) is orthogonal to R(A).We know that row space⊥ = Nullspace for any matrix. Inparticular, R(A)⊥ = N(A) the nullspace of the transpose matrix ofA. Thus, x̂ is

a least squares solution if and only if





proof

Ax is an arbitrary vector in R(A), the column space of A. Hencethe length of r(x) = b− Ax is minimal if Ax is the orthogonalprojection of b onto R(A) that is, if r(x) is orthogonal to R(A).We know that row space⊥ = Nullspace for any matrix. Inparticular, R(A)⊥ = N(A) the nullspace of the transpose matrix ofA. Thus, x̂ is a least squares solution

if and only if





proof






proof


AT r(x) = 0 ⇐⇒

AT (b− Ax) = 0 ⇐⇒ ATAx = ATb �




proof


AT r(x) = 0 ⇐⇒ AT (b− Ax) = 0 ⇐⇒

ATAx = ATb �




proof






Corollary

The normal system ATAx = ATb is always consistent.

Example 11.15

Find the least squares solution tox + 2y = 3

3x + 2y = 5x + y = 2.09

SolutionIn matrix notation, the system can be written as




Corollary


Example 11.15


3x + 2y = 5x + y = 2.09





Corollary

The normal system

ATAx = ATb is always consistent.

Example 11.15


3x + 2y = 5x + y = 2.09





Corollary

The normal system ATAx = ATb

is always consistent.

Example 11.15


3x + 2y = 5x + y = 2.09





Corollary


Example 11.15


3x + 2y = 5x + y = 2.09





Corollary


Example 11.15


3x + 2y = 5x + y = 2.09





Corollary


Example 11.15


3x + 2y = 5x + y = 2.09





Corollary


Example 11.15

Find

the least squares solution tox + 2y = 3

3x + 2y = 5x + y = 2.09





Corollary


Example 11.15

Find the least squares solution to

x + 2y = 3

3x + 2y = 5x + y = 2.09





Corollary


Example 11.15


3x + 2y = 5x + y = 2.09





Corollary


Example 11.15


3x + 2y = 5x + y = 2.09

SolutionIn matrix notation,

the system can be written as




Corollary


Example 11.15


3x + 2y = 5x + y = 2.09

SolutionIn matrix notation, the system

can be written as




Corollary


Example 11.15


3x + 2y = 5x + y = 2.09





1 23 21 1

(xy

)=

35

2.09

and the normal system is

(1 3 12 2 1

) 1 23 21 1

(xy

)=

(1 3 12 2 1

) 35

2.09

⇒(

11 99 9

)(xy

)=

(20.0918.09

)⇒{

x = 1y = 1.01




1 23 21 1

(xy

)=

35

2.09


(1 3 12 2 1

) 1 23 21 1

(xy

)=

(1 3 12 2 1

) 35

2.09

⇒(

11 99 9

)(xy

)=

(20.0918.09

)⇒{

x = 1y = 1.01




1 23 21 1

(xy

)=

35

2.09


(1 3 12 2 1

) 1 23 21 1

(xy

)=

(1 3 12 2 1

) 35

2.09

⇒(

11 99 9

)(xy

)=

(20.0918.09

)⇒{

x = 1y = 1.01




1 23 21 1

(xy

)=

35

2.09


(1 3 12 2 1

) 1 23 21 1

(xy

)=

(1 3 12 2 1

) 35

2.09

⇒(

11 99 9

)(xy

)=

(20.0918.09

)⇒{

x = 1y = 1.01




1 23 21 1

(xy

)=

35

2.09

and

the normal system is

(1 3 12 2 1

) 1 23 21 1

(xy

)=

(1 3 12 2 1

) 35

2.09

⇒(

11 99 9

)(xy

)=

(20.0918.09

)⇒{

x = 1y = 1.01




1 23 21 1

(xy

)=

35

2.09


(1 3 12 2 1

) 1 23 21 1

(xy

)=

(1 3 12 2 1

) 35

2.09

⇒(

11 99 9

)(xy

)=

(20.0918.09

)⇒{

x = 1y = 1.01




1 23 21 1

(xy

)=

35

2.09


(1 3 12 2 1

) 1 23 21 1

(xy

)=

(1 3 12 2 1

) 35

2.09

⇒(

11 99 9

)(xy

)=

(20.0918.09

)⇒{

x = 1y = 1.01




1 23 21 1

(xy

)=

35

2.09


(1 3 12 2 1

) 1 23 21 1

(xy

)=

(1 3 12 2 1

) 35

2.09

⇒

(11 99 9

)(xy

)=

(20.0918.09

)⇒{

x = 1y = 1.01




1 23 21 1

(xy

)=

35

2.09


(1 3 12 2 1

) 1 23 21 1

(xy

)=

(1 3 12 2 1

) 35

2.09

⇒(

11 99 9

)(xy

)=

(20.0918.09

)⇒{

x = 1y = 1.01




1 23 21 1

(xy

)=

35

2.09


(1 3 12 2 1

) 1 23 21 1

(xy

)=

(1 3 12 2 1

) 35

2.09

⇒(

11 99 9

)(xy

)=

(20.0918.09

)

⇒{

x = 1y = 1.01




1 23 21 1

(xy

)=

35

2.09


(1 3 12 2 1

) 1 23 21 1

(xy

)=

(1 3 12 2 1

) 35

2.09

⇒(

11 99 9

)(xy

)=

(20.0918.09

)⇒{

x = 1y = 1.01




Example 11.16Find the constant function that is the least squares fit to thefollowing data

x 0 1 2 3

f(x) 1 0 1 2

Solution

f (x) = c ⇒

c = 1c = 0c = 1c = 2

⇒

1012

c ⇒




Example 11.16

Find the constant function that is the least squares fit to thefollowing data

x 0 1 2 3

f(x) 1 0 1 2

Solution

f (x) = c ⇒

c = 1c = 0c = 1c = 2

⇒

1012

c ⇒




Example 11.16Find

the constant function that is the least squares fit to thefollowing data

x 0 1 2 3

f(x) 1 0 1 2

Solution

f (x) = c ⇒

c = 1c = 0c = 1c = 2

⇒

1012

c ⇒




Example 11.16Find the constant function

that is the least squares fit to thefollowing data

x 0 1 2 3

f(x) 1 0 1 2

Solution

f (x) = c ⇒

c = 1c = 0c = 1c = 2

⇒

1012

c ⇒




Example 11.16Find the constant function that is

the least squares fit to thefollowing data

x 0 1 2 3

f(x) 1 0 1 2

Solution

f (x) = c ⇒

c = 1c = 0c = 1c = 2

⇒

1012

c ⇒




Example 11.16Find the constant function that is the least squares fit

to thefollowing data

x 0 1 2 3

f(x) 1 0 1 2

Solution

f (x) = c ⇒

c = 1c = 0c = 1c = 2

⇒

1012

c ⇒





x 0 1 2 3

f(x) 1 0 1 2

Solution

f (x) = c ⇒

c = 1c = 0c = 1c = 2

⇒

1012

c ⇒





x 0 1 2 3

f(x) 1 0 1 2

Solution

f (x) = c ⇒

c = 1c = 0c = 1c = 2

⇒

1012

c ⇒





x 0 1 2 3

f(x) 1 0 1 2

Solution

f (x) = c ⇒

c = 1c = 0c = 1c = 2

⇒

1012

c ⇒





x 0 1 2 3

f(x) 1 0 1 2

Solution

f (x) = c ⇒

c = 1c = 0c = 1c = 2

⇒

1012

c ⇒





x 0 1 2 3

f(x) 1 0 1 2

Solution

f (x) = c ⇒

c = 1c = 0c = 1c = 2

⇒

1012

c ⇒





x 0 1 2 3

f(x) 1 0 1 2

Solution

f (x) = c ⇒

c = 1c = 0c = 1c = 2

⇒

1012

c ⇒





x 0 1 2 3

f(x) 1 0 1 2

Solution

f (x) = c ⇒

c = 1c = 0c = 1c = 2

⇒

1012

c ⇒




Then, the normal system is

(1 1 1 1

) 1111

c =(

1 1 1 1)

1012

c = 1

4(1 + 0 + 1 + 2) = 1 (mean arithmetic value)

Thus, the constant function is

f (x) = 1




Then,

the normal system is

(1 1 1 1

) 1111

c =(

1 1 1 1)

1012

c = 1



f (x) = 1





(1 1 1 1

) 1111

c =(

1 1 1 1)

1012

c = 1



f (x) = 1





(1 1 1 1

)

1111

c =(

1 1 1 1)

1012

c = 1



f (x) = 1





(1 1 1 1

) 1111

c =(

1 1 1 1)

1012

c = 1



f (x) = 1





(1 1 1 1

) 1111

c =

(1 1 1 1

) 1012

c = 1



f (x) = 1





(1 1 1 1

) 1111

c =(

1 1 1 1)

1012

c = 1



f (x) = 1





(1 1 1 1

) 1111

c =(

1 1 1 1)

1012

c = 14(1 + 0 + 1 + 2) = 1 (mean arithmetic value)


f (x) = 1





(1 1 1 1

) 1111

c =(

1 1 1 1)

1012

c = 1

4(1 + 0 + 1 + 2) = 1

(mean arithmetic value)


f (x) = 1





(1 1 1 1

) 1111

c =(

1 1 1 1)

1012

c = 1



f (x) = 1





(1 1 1 1

) 1111

c =(

1 1 1 1)

1012

c = 1


Thus,

the constant function is

f (x) = 1





(1 1 1 1

) 1111

c =(

1 1 1 1)

1012

c = 1



f (x) = 1





(1 1 1 1

) 1111

c =(

1 1 1 1)

1012

c = 1



f (x) = 1




Example 11.17

Find the linear polynomial function that is the least squares fit tothe following data

x 0 1 2 3

f(x) 1 0 1 2

Solution

f (x) = c1+c2x ⇒

c1 = 1

c1 + c2 = 0c1 + 2c2 = 1c1 + 3c2 = 2

⇒

1 01 11 21 3

(c1c2

)=

1012




Example 11.17


x 0 1 2 3

f(x) 1 0 1 2

Solution

f (x) = c1+c2x ⇒

c1 = 1

c1 + c2 = 0c1 + 2c2 = 1c1 + 3c2 = 2

⇒

1 01 11 21 3

(c1c2

)=

1012




Example 11.17

Find

the linear polynomial function that is the least squares fit tothe following data

x 0 1 2 3

f(x) 1 0 1 2

Solution

f (x) = c1+c2x ⇒

c1 = 1

c1 + c2 = 0c1 + 2c2 = 1c1 + 3c2 = 2

⇒

1 01 11 21 3

(c1c2

)=

1012




Example 11.17

Find the linear polynomial function

that is the least squares fit tothe following data

x 0 1 2 3

f(x) 1 0 1 2

Solution

f (x) = c1+c2x ⇒

c1 = 1

c1 + c2 = 0c1 + 2c2 = 1c1 + 3c2 = 2

⇒

1 01 11 21 3

(c1c2

)=

1012




Example 11.17

Find the linear polynomial function that is

the least squares fit tothe following data

x 0 1 2 3

f(x) 1 0 1 2

Solution

f (x) = c1+c2x ⇒

c1 = 1

c1 + c2 = 0c1 + 2c2 = 1c1 + 3c2 = 2

⇒

1 01 11 21 3

(c1c2

)=

1012




Example 11.17

Find the linear polynomial function that is the least squares fit

tothe following data

x 0 1 2 3

f(x) 1 0 1 2

Solution

f (x) = c1+c2x ⇒

c1 = 1

c1 + c2 = 0c1 + 2c2 = 1c1 + 3c2 = 2

⇒

1 01 11 21 3

(c1c2

)=

1012




Example 11.17


x 0 1 2 3

f(x) 1 0 1 2

Solution

f (x) = c1+c2x ⇒

c1 = 1

c1 + c2 = 0c1 + 2c2 = 1c1 + 3c2 = 2

⇒

1 01 11 21 3

(c1c2

)=

1012




Example 11.17


x 0 1 2 3

f(x) 1 0 1 2

Solution

f (x) = c1+c2x ⇒

c1 = 1

c1 + c2 = 0c1 + 2c2 = 1c1 + 3c2 = 2

⇒

1 01 11 21 3

(c1c2

)=

1012




Example 11.17


x 0 1 2 3

f(x) 1 0 1 2

Solution

f (x) = c1+c2x ⇒

c1 = 1

c1 + c2 = 0c1 + 2c2 = 1c1 + 3c2 = 2

⇒

1 01 11 21 3

(c1c2

)=

1012




Example 11.17


x 0 1 2 3

f(x) 1 0 1 2

Solution

f (x) = c1+c2x ⇒

c1 = 1

c1 + c2 = 0c1 + 2c2 = 1c1 + 3c2 = 2

⇒

1 01 11 21 3

(c1c2

)=

1012




Example 11.17


x 0 1 2 3

f(x) 1 0 1 2

Solution

f (x) = c1+c2x ⇒

c1 = 1

c1 + c2 = 0c1 + 2c2 = 1c1 + 3c2 = 2

⇒

1 01 11 21 3

(c1c2

)=

1012




Example 11.17


x 0 1 2 3

f(x) 1 0 1 2

Solution

f (x) = c1+c2x ⇒

c1 = 1

c1 + c2 = 0c1 + 2c2 = 1c1 + 3c2 = 2

⇒

1 01 11 21 3

(c1c2

)=

1012




Example 11.17


x 0 1 2 3

f(x) 1 0 1 2

Solution

f (x) = c1+c2x ⇒

c1 = 1

c1 + c2 = 0c1 + 2c2 = 1c1 + 3c2 = 2

⇒

1 01 11 21 3

(c1c2

)=

1012




Example 11.17


x 0 1 2 3

f(x) 1 0 1 2

Solution

f (x) = c1+c2x ⇒

c1 = 1

c1 + c2 = 0c1 + 2c2 = 1c1 + 3c2 = 2

⇒

1 01 11 21 3

(c1c2

)=

1012




Then, the nomal system is

(1 1 1 10 1 2 3

) 1 01 11 21 3

(c1c2

)=

(1 1 1 10 1 2 3

) 1012

(

4 66 14

) (c1c2

)=

(48

)⇒{

c1 = 0.4c2 = 0.4

Thus, the linear function is

f (x) = 0.4 + 0.4x




Then,

the nomal system is

(1 1 1 10 1 2 3

) 1 01 11 21 3

(c1c2

)=

(1 1 1 10 1 2 3

) 1012

(

4 66 14

) (c1c2

)=

(48

)⇒{

c1 = 0.4c2 = 0.4


f (x) = 0.4 + 0.4x





(1 1 1 10 1 2 3

) 1 01 11 21 3

(c1c2

)=

(1 1 1 10 1 2 3

) 1012

(

4 66 14

) (c1c2

)=

(48

)⇒{

c1 = 0.4c2 = 0.4


f (x) = 0.4 + 0.4x





(1 1 1 10 1 2 3

)

1 01 11 21 3

(c1c2

)=

(1 1 1 10 1 2 3

) 1012

(

4 66 14

) (c1c2

)=

(48

)⇒{

c1 = 0.4c2 = 0.4


f (x) = 0.4 + 0.4x





(1 1 1 10 1 2 3

) 1 01 11 21 3

(c1c2

)=

(1 1 1 10 1 2 3

) 1012

(

4 66 14

) (c1c2

)=

(48

)⇒{

c1 = 0.4c2 = 0.4


f (x) = 0.4 + 0.4x





(1 1 1 10 1 2 3

) 1 01 11 21 3

(c1c2

)=

(1 1 1 10 1 2 3

) 1012

(

4 66 14

) (c1c2

)=

(48

)⇒{

c1 = 0.4c2 = 0.4


f (x) = 0.4 + 0.4x





(1 1 1 10 1 2 3

) 1 01 11 21 3

(c1c2

)=

(1 1 1 10 1 2 3

)

1012

(

4 66 14

) (c1c2

)=

(48

)⇒{

c1 = 0.4c2 = 0.4


f (x) = 0.4 + 0.4x





(1 1 1 10 1 2 3

) 1 01 11 21 3

(c1c2

)=

(1 1 1 10 1 2 3

) 1012

(4 66 14

) (c1c2

)=

(48

)⇒{

c1 = 0.4c2 = 0.4


f (x) = 0.4 + 0.4x





(1 1 1 10 1 2 3

) 1 01 11 21 3

(c1c2

)=

(1 1 1 10 1 2 3

) 1012

(

4 66 14

)

(c1c2

)=

(48

)⇒{

c1 = 0.4c2 = 0.4


f (x) = 0.4 + 0.4x





(1 1 1 10 1 2 3

) 1 01 11 21 3

(c1c2

)=

(1 1 1 10 1 2 3

) 1012

(

4 66 14

) (c1c2

)=

(48

)⇒{

c1 = 0.4c2 = 0.4


f (x) = 0.4 + 0.4x





(1 1 1 10 1 2 3

) 1 01 11 21 3

(c1c2

)=

(1 1 1 10 1 2 3

) 1012

(

4 66 14

) (c1c2

)=

(48

)

⇒{

c1 = 0.4c2 = 0.4


f (x) = 0.4 + 0.4x





(1 1 1 10 1 2 3

) 1 01 11 21 3

(c1c2

)=

(1 1 1 10 1 2 3

) 1012

(

4 66 14

) (c1c2

)=

(48

)⇒{

c1 = 0.4c2 = 0.4


f (x) = 0.4 + 0.4x





(1 1 1 10 1 2 3

) 1 01 11 21 3

(c1c2

)=

(1 1 1 10 1 2 3

) 1012

(

4 66 14

) (c1c2

)=

(48

)⇒{

c1 = 0.4c2 = 0.4

Thus,

the linear function is

f (x) = 0.4 + 0.4x





(1 1 1 10 1 2 3

) 1 01 11 21 3

(c1c2

)=

(1 1 1 10 1 2 3

) 1012

(

4 66 14

) (c1c2

)=

(48

)⇒{

c1 = 0.4c2 = 0.4


f (x) = 0.4 + 0.4x





(1 1 1 10 1 2 3

) 1 01 11 21 3

(c1c2

)=

(1 1 1 10 1 2 3

) 1012

(

4 66 14

) (c1c2

)=

(48

)⇒{

c1 = 0.4c2 = 0.4


f (x) = 0.4 + 0.4x




Example 11.18

Find the quadratic polynomial function that is the least squares fitto the following data

x 0 1 2 3

f(x) 1 0 1 2

Solution

f (x) = c1 + c2x + c3x2 ⇒

c1 = 1

c1 + c2 + c3 = 0c1 + 2c2 + 4c3 = 1c1 + 3c2 + 9c3 = 2

⇒




Example 11.18


x 0 1 2 3

f(x) 1 0 1 2

Solution

f (x) = c1 + c2x + c3x2 ⇒

c1 = 1

c1 + c2 + c3 = 0c1 + 2c2 + 4c3 = 1c1 + 3c2 + 9c3 = 2

⇒




Example 11.18

Find

the quadratic polynomial function that is the least squares fitto the following data

x 0 1 2 3

f(x) 1 0 1 2

Solution

f (x) = c1 + c2x + c3x2 ⇒

c1 = 1

c1 + c2 + c3 = 0c1 + 2c2 + 4c3 = 1c1 + 3c2 + 9c3 = 2

⇒




Example 11.18

Find the quadratic polynomial function

that is the least squares fitto the following data

x 0 1 2 3

f(x) 1 0 1 2

Solution

f (x) = c1 + c2x + c3x2 ⇒

c1 = 1

c1 + c2 + c3 = 0c1 + 2c2 + 4c3 = 1c1 + 3c2 + 9c3 = 2

⇒




Example 11.18

Find the quadratic polynomial function that is

the least squares fitto the following data

x 0 1 2 3

f(x) 1 0 1 2

Solution

f (x) = c1 + c2x + c3x2 ⇒

c1 = 1

c1 + c2 + c3 = 0c1 + 2c2 + 4c3 = 1c1 + 3c2 + 9c3 = 2

⇒




Example 11.18

Find the quadratic polynomial function that is the least squares fit

to the following data

x 0 1 2 3

f(x) 1 0 1 2

Solution

f (x) = c1 + c2x + c3x2 ⇒

c1 = 1

c1 + c2 + c3 = 0c1 + 2c2 + 4c3 = 1c1 + 3c2 + 9c3 = 2

⇒




Example 11.18


x 0 1 2 3

f(x) 1 0 1 2

Solution

f (x) = c1 + c2x + c3x2 ⇒

c1 = 1

c1 + c2 + c3 = 0c1 + 2c2 + 4c3 = 1c1 + 3c2 + 9c3 = 2

⇒




Example 11.18


x 0 1 2 3

f(x) 1 0 1 2

Solution

f (x) = c1 + c2x + c3x2 ⇒

c1 = 1

c1 + c2 + c3 = 0c1 + 2c2 + 4c3 = 1c1 + 3c2 + 9c3 = 2

⇒




Example 11.18


x 0 1 2 3

f(x) 1 0 1 2

Solution

f (x) = c1 + c2x + c3x2 ⇒

c1 = 1

c1 + c2 + c3 = 0c1 + 2c2 + 4c3 = 1c1 + 3c2 + 9c3 = 2

⇒




Example 11.18


x 0 1 2 3

f(x) 1 0 1 2

Solution

f (x) = c1 + c2x + c3x2 ⇒

c1 = 1

c1 + c2 + c3 = 0c1 + 2c2 + 4c3 = 1c1 + 3c2 + 9c3 = 2

⇒




Example 11.18


x 0 1 2 3

f(x) 1 0 1 2

Solution

f (x) = c1 + c2x + c3x2 ⇒

c1 = 1

c1 + c2 + c3 = 0c1 + 2c2 + 4c3 = 1c1 + 3c2 + 9c3 = 2

⇒




1 0 01 1 11 2 41 3 9

c1

c2c3

=

1012

⇒Then, the nomal system is

1 1 1 10 1 2 30 1 4 9

1 0 01 1 11 2 41 3 9

c1

c2c3

=

1 1 1 10 1 2 30 1 4 9

1012




1 0 01 1 11 2 41 3 9

c1c2c3

=

1012


1 1 1 10 1 2 30 1 4 9

1 0 01 1 11 2 41 3 9

c1

c2c3

=

1 1 1 10 1 2 30 1 4 9

1012




1 0 01 1 11 2 41 3 9

c1

c2c3

=

1012


1 1 1 10 1 2 30 1 4 9

1 0 01 1 11 2 41 3 9

c1

c2c3

=

1 1 1 10 1 2 30 1 4 9

1012




1 0 01 1 11 2 41 3 9

c1

c2c3

=

1012

⇒


1 1 1 10 1 2 30 1 4 9

1 0 01 1 11 2 41 3 9

c1

c2c3

=

1 1 1 10 1 2 30 1 4 9

1012




1 0 01 1 11 2 41 3 9

c1

c2c3

=

1012

⇒Then,

the nomal system is

1 1 1 10 1 2 30 1 4 9

1 0 01 1 11 2 41 3 9

c1

c2c3

=

1 1 1 10 1 2 30 1 4 9

1012




1 0 01 1 11 2 41 3 9

c1

c2c3

=

1012


1 1 1 10 1 2 30 1 4 9

1 0 01 1 11 2 41 3 9

c1

c2c3

=

1 1 1 10 1 2 30 1 4 9

1012




1 0 01 1 11 2 41 3 9

c1

c2c3

=

1012


1 1 1 10 1 2 30 1 4 9

1 0 01 1 11 2 41 3 9

c1

c2c3

=

1 1 1 10 1 2 30 1 4 9

1012




1 0 01 1 11 2 41 3 9

c1

c2c3

=

1012


1 1 1 10 1 2 30 1 4 9

1 0 01 1 11 2 41 3 9

c1c2c3

=

1 1 1 10 1 2 30 1 4 9

1012




1 0 01 1 11 2 41 3 9

c1

c2c3

=

1012


1 1 1 10 1 2 30 1 4 9

1 0 01 1 11 2 41 3 9

c1

c2c3

=

1 1 1 10 1 2 30 1 4 9

1012




1 0 01 1 11 2 41 3 9

c1

c2c3

=

1012


1 1 1 10 1 2 30 1 4 9

1 0 01 1 11 2 41 3 9

c1

c2c3

=

1 1 1 10 1 2 30 1 4 9

1012




1 0 01 1 11 2 41 3 9

c1

c2c3

=

1012


1 1 1 10 1 2 30 1 4 9

1 0 01 1 11 2 41 3 9

c1

c2c3

=

1 1 1 10 1 2 30 1 4 9

1012




4 6 146 14 36

14 36 98

c1c2c3

=

48

22

⇒

c1 = 0.9c2 = −1.1c3 = 0.5

Thus, the quadratic function is

f (x) = 0.9− 1.1x + 0.5x2




4 6 146 14 36

14 36 98

c1c2c3

=

48

22

⇒

c1 = 0.9c2 = −1.1c3 = 0.5


f (x) = 0.9− 1.1x + 0.5x2




4 6 146 14 36

14 36 98

c1c2c3

=

48

22

⇒

c1 = 0.9c2 = −1.1c3 = 0.5


f (x) = 0.9− 1.1x + 0.5x2




4 6 146 14 36

14 36 98

c1c2c3

=

48

22

⇒

c1 = 0.9c2 = −1.1c3 = 0.5


f (x) = 0.9− 1.1x + 0.5x2




4 6 146 14 36

14 36 98

c1c2c3

=

48

22

⇒

c1 = 0.9c2 = −1.1c3 = 0.5


f (x) = 0.9− 1.1x + 0.5x2




4 6 146 14 36

14 36 98

c1c2c3

=

48

22

⇒

c1 = 0.9c2 = −1.1c3 = 0.5

Thus,

the quadratic function is

f (x) = 0.9− 1.1x + 0.5x2




4 6 146 14 36

14 36 98

c1c2c3

=

48

22

⇒

c1 = 0.9c2 = −1.1c3 = 0.5


f (x) = 0.9− 1.1x + 0.5x2




4 6 146 14 36

14 36 98

c1c2c3

=

48

22

⇒

c1 = 0.9c2 = −1.1c3 = 0.5


f (x) = 0.9− 1.1x + 0.5x2



Orthogonal Sets.

Orthogonal sets

Let < ·, · > denote the scalar product in Rn

Definition

Nonzero vectors v1, v2, · · · , vk ∈ Rn form an orthogonal set ifthey are orthogonal to each other: < vi , vj >= 0 for all i 6= j .

If, in addition, all vectors are of unit length, vi , v1, v2, · · · , vk iscalled an orthonormal set.

For instance, The standard basise1 = (1, 0, 0, ..., 0), e2 = (0, 1, 0, ..., 0), · · · , en = (0, 0, 0, ..., 1). Itis an orthonormal set.



Orthogonal Sets.

Orthogonal sets


Definition






Orthogonal Sets.

Orthogonal sets

Let

< ·, · > denote the scalar product in Rn

Definition






Orthogonal Sets.

Orthogonal sets

Let < ·, · > denote

the scalar product in Rn

Definition






Orthogonal Sets.

Orthogonal sets

Let < ·, · > denote the scalar product

in Rn

Definition






Orthogonal Sets.

Orthogonal sets


Definition






Orthogonal Sets.

Orthogonal sets


Definition






Orthogonal Sets.

Orthogonal sets


Definition

Nonzero vectors

v1, v2, · · · , vk ∈ Rn form an orthogonal set ifthey are orthogonal to each other: < vi , vj >= 0 for all i 6= j .





Orthogonal Sets.

Orthogonal sets


Definition

Nonzero vectors v1, v2, · · · , vk ∈ Rn

form an orthogonal set ifthey are orthogonal to each other: < vi , vj >= 0 for all i 6= j .





Orthogonal Sets.

Orthogonal sets


Definition

Nonzero vectors v1, v2, · · · , vk ∈ Rn form an orthogonal set

ifthey are orthogonal to each other: < vi , vj >= 0 for all i 6= j .





Orthogonal Sets.

Orthogonal sets


Definition

Nonzero vectors v1, v2, · · · , vk ∈ Rn form an orthogonal set ifthey are orthogonal

to each other: < vi , vj >= 0 for all i 6= j .





Orthogonal Sets.

Orthogonal sets


Definition

Nonzero vectors v1, v2, · · · , vk ∈ Rn form an orthogonal set ifthey are orthogonal to each other:

< vi , vj >= 0 for all i 6= j .





Orthogonal Sets.

Orthogonal sets


Definition

Nonzero vectors v1, v2, · · · , vk ∈ Rn form an orthogonal set ifthey are orthogonal to each other: < vi , vj >= 0

for all i 6= j .





Orthogonal Sets.

Orthogonal sets


Definition






Orthogonal Sets.

Orthogonal sets


Definition


If,

in addition, all vectors are of unit length, vi , v1, v2, · · · , vk iscalled an orthonormal set.




Orthogonal Sets.

Orthogonal sets


Definition


If, in addition,

all vectors are of unit length, vi , v1, v2, · · · , vk iscalled an orthonormal set.




Orthogonal Sets.

Orthogonal sets


Definition


If, in addition, all vectors are

of unit length, vi , v1, v2, · · · , vk iscalled an orthonormal set.




Orthogonal Sets.

Orthogonal sets


Definition


If, in addition, all vectors are of unit length,

vi , v1, v2, · · · , vk iscalled an orthonormal set.




Orthogonal Sets.

Orthogonal sets


Definition


If, in addition, all vectors are of unit length, vi , v1, v2, · · · , vk

iscalled an orthonormal set.




Orthogonal Sets.

Orthogonal sets


Definition


If, in addition, all vectors are of unit length, vi , v1, v2, · · · , vk iscalled

an orthonormal set.




Orthogonal Sets.

Orthogonal sets


Definition






Orthogonal Sets.

Orthogonal sets


Definition



For instance,

The standard basise1 = (1, 0, 0, ..., 0), e2 = (0, 1, 0, ..., 0), · · · , en = (0, 0, 0, ..., 1). Itis an orthonormal set.



Orthogonal Sets.

Orthogonal sets


Definition



For instance, The standard basis

e1 = (1, 0, 0, ..., 0), e2 = (0, 1, 0, ..., 0), · · · , en = (0, 0, 0, ..., 1). Itis an orthonormal set.



Orthogonal Sets.

Orthogonal sets


Definition



For instance, The standard basise1 = (1, 0, 0, ..., 0), e2 = (0, 1, 0, ..., 0), · · · , en = (0, 0, 0, ..., 1).

Itis an orthonormal set.



Orthogonal Sets.

Orthogonal sets


Definition






Orthogonal Sets.

Orthonormal bases

Suppose v1, v2, · · · , vn is an orthonormal basis for Rn (i.e., it is abasis and an orthonormal set).

Theorem

Let x = x1v1 + x2v2 + · · ·+ xnvn and y = y1v1 + y2v2 + · · ·+ ynvnwhere xi , y1 ∈ R

i) < x, y >=∑n

i=i xiyi

i) ||x|| =√∑n

i=i xiyi



Orthogonal Sets.

Orthonormal bases


Theorem


i) < x, y >=∑n

i=i xiyi

i) ||x|| =√∑n

i=i xiyi



Orthogonal Sets.

Orthonormal bases

Suppose

v1, v2, · · · , vn is an orthonormal basis for Rn (i.e., it is abasis and an orthonormal set).

Theorem


i) < x, y >=∑n

i=i xiyi

i) ||x|| =√∑n

i=i xiyi



Orthogonal Sets.

Orthonormal bases

Suppose v1, v2, · · · , vn

is an orthonormal basis for Rn (i.e., it is abasis and an orthonormal set).

Theorem


i) < x, y >=∑n

i=i xiyi

i) ||x|| =√∑n

i=i xiyi



Orthogonal Sets.

Orthonormal bases

Suppose v1, v2, · · · , vn is an orthonormal basis

for Rn (i.e., it is abasis and an orthonormal set).

Theorem


i) < x, y >=∑n

i=i xiyi

i) ||x|| =√∑n

i=i xiyi



Orthogonal Sets.

Orthonormal bases

Suppose v1, v2, · · · , vn is an orthonormal basis for Rn

(i.e., it is abasis and an orthonormal set).

Theorem


i) < x, y >=∑n

i=i xiyi

i) ||x|| =√∑n

i=i xiyi



Orthogonal Sets.

Orthonormal bases

Suppose v1, v2, · · · , vn is an orthonormal basis for Rn (i.e., it is abasis and

an orthonormal set).

Theorem


i) < x, y >=∑n

i=i xiyi

i) ||x|| =√∑n

i=i xiyi



Orthogonal Sets.

Orthonormal bases


Theorem


i) < x, y >=∑n

i=i xiyi

i) ||x|| =√∑n

i=i xiyi



Orthogonal Sets.

Orthonormal bases


Theorem


i) < x, y >=∑n

i=i xiyi

i) ||x|| =√∑n

i=i xiyi



Orthogonal Sets.

Orthonormal bases


Theorem

Let

x = x1v1 + x2v2 + · · ·+ xnvn and y = y1v1 + y2v2 + · · ·+ ynvnwhere xi , y1 ∈ R

i) < x, y >=∑n

i=i xiyi

i) ||x|| =√∑n

i=i xiyi



Orthogonal Sets.

Orthonormal bases


Theorem

Let x = x1v1 + x2v2 + · · ·+ xnvn and

y = y1v1 + y2v2 + · · ·+ ynvnwhere xi , y1 ∈ R

i) < x, y >=∑n

i=i xiyi

i) ||x|| =√∑n

i=i xiyi



Orthogonal Sets.

Orthonormal bases


Theorem

Let x = x1v1 + x2v2 + · · ·+ xnvn and y = y1v1 + y2v2 + · · ·+ ynvn

where xi , y1 ∈ R

i) < x, y >=∑n

i=i xiyi

i) ||x|| =√∑n

i=i xiyi



Orthogonal Sets.

Orthonormal bases


Theorem

Let x = x1v1 + x2v2 + · · ·+ xnvn and y = y1v1 + y2v2 + · · ·+ ynvnwhere

xi , y1 ∈ R

i) < x, y >=∑n

i=i xiyi

i) ||x|| =√∑n

i=i xiyi



Orthogonal Sets.

Orthonormal bases


Theorem


i) < x, y >=∑n

i=i xiyi

i) ||x|| =√∑n

i=i xiyi



Orthogonal Sets.

Orthonormal bases


Theorem


i) < x, y >=

∑ni=i xiyi

i) ||x|| =√∑n

i=i xiyi



Orthogonal Sets.

Orthonormal bases


Theorem


i) < x, y >=∑n

i=i xiyi

i) ||x|| =√∑n

i=i xiyi



Orthogonal Sets.

Orthonormal bases


Theorem


i) < x, y >=∑n

i=i xiyi

i) ||x|| =

√∑ni=i xiyi



Orthogonal Sets.

Orthonormal bases


Theorem


i) < x, y >=∑n

i=i xiyi

i) ||x|| =√∑n

i=i xiyi



Orthogonal Sets.

proof

i)

< x, y >=

⟨n∑i=i

xivi ,n∑j=i

yjvj

⟩=

n∑i=i

xi

⟨vi ,

n∑j=i

vj

⟩=

n∑i=i

xi

n∑j=i

yj 〈vi , vj〉 =n∑i=i

xiyi

ii) follows from i) when y = x �



Orthogonal Sets.

proof

i)

< x, y >=

⟨n∑i=i

xivi ,n∑j=i

yjvj

⟩=

n∑i=i

xi

⟨vi ,

n∑j=i

vj

⟩=

n∑i=i

xi

n∑j=i


xiyi




Orthogonal Sets.

proof

i)

< x, y >=

⟨n∑i=i

xivi ,n∑j=i

yjvj

⟩=

n∑i=i

xi

⟨vi ,

n∑j=i

vj

⟩=

n∑i=i

xi

n∑j=i


xiyi




Orthogonal Sets.

proof

i)

< x, y >=

⟨n∑i=i

xivi ,n∑j=i

yjvj

⟩=

n∑i=i

xi

⟨vi ,

n∑j=i

vj

⟩=

n∑i=i

xi

n∑j=i


xiyi




Orthogonal Sets.

proof

i)

< x, y >=

⟨n∑i=i

xivi ,n∑j=i

yjvj

⟩=

n∑i=i

xi

⟨vi ,

n∑j=i

vj

⟩=

n∑i=i

xi

n∑j=i


xiyi




Orthogonal Sets.

proof

i)

< x, y >=

⟨n∑i=i

xivi ,n∑j=i

yjvj

⟩=

n∑i=i

xi

⟨vi ,

n∑j=i

vj

⟩=

n∑i=i

xi

n∑j=i


xiyi




Orthogonal Sets.

proof

i)

< x, y >=

⟨n∑i=i

xivi ,n∑j=i

yjvj

⟩=

n∑i=i

xi

⟨vi ,

n∑j=i

vj

⟩=

n∑i=i

xi

n∑j=i

yj 〈vi , vj〉 =

n∑i=i

xiyi




Orthogonal Sets.

proof

i)

< x, y >=

⟨n∑i=i

xivi ,n∑j=i

yjvj

⟩=

n∑i=i

xi

⟨vi ,

n∑j=i

vj

⟩=

n∑i=i

xi

n∑j=i


xiyi




Orthogonal Sets.

proof

i)

< x, y >=

⟨n∑i=i

xivi ,n∑j=i

yjvj

⟩=

n∑i=i

xi

⟨vi ,

n∑j=i

vj

⟩=

n∑i=i

xi

n∑j=i


xiyi

ii) follows

from i) when y = x �



Orthogonal Sets.

proof

i)

< x, y >=

⟨n∑i=i

xivi ,n∑j=i

yjvj

⟩=

n∑i=i

xi

⟨vi ,

n∑j=i

vj

⟩=

n∑i=i

xi

n∑j=i


xiyi

ii) follows from i)

when y = x �



Orthogonal Sets.

proof

i)

< x, y >=

⟨n∑i=i

xivi ,n∑j=i

yjvj

⟩=

n∑i=i

xi

⟨vi ,

n∑j=i

vj

⟩=

n∑i=i

xi

n∑j=i


xiyi




Orthogonal Sets.

Suppose V is a subspace of Rn. Let p be the orthogonal projectionof a vector x ∈ Rn onto V.

If V is a one-dimensional subspace spanned by a v, thenp = <x,v>

<v,v>v

If V admits an orthogonal basis v1, v2, · · · , vk , then

p =< x, v1 >

< v1, v1 >v1 +

< x, v2 >

< v2, v2 >v2 + ... +

< x, vk >

< vk , vk >vk

Indeed, < p, vi >=∑k

j=i<x,vj><vj ,vj>

< vj , vi >= <x,vi><vi ,vi>

< vi , vi >=

< x, vi >⇒ < x− p, vi >= 0⇒ (x− p)⊥vi ⇒ (x− p)⊥V.



Orthogonal Sets.

Suppose V

is a subspace of Rn. Let p be the orthogonal projectionof a vector x ∈ Rn onto V.


<v,v>v


p =< x, v1 >

< v1, v1 >v1 +

< x, v2 >

< v2, v2 >v2 + ... +

< x, vk >

< vk , vk >vk


j=i<x,vj><vj ,vj>


< vi , vi >=

< x, vi >⇒ < x− p, vi >= 0⇒ (x− p)⊥vi ⇒ (x− p)⊥V.



Orthogonal Sets.

Suppose V is a subspace

of Rn. Let p be the orthogonal projectionof a vector x ∈ Rn onto V.


<v,v>v


p =< x, v1 >

< v1, v1 >v1 +

< x, v2 >

< v2, v2 >v2 + ... +

< x, vk >

< vk , vk >vk


j=i<x,vj><vj ,vj>


< vi , vi >=

< x, vi >⇒ < x− p, vi >= 0⇒ (x− p)⊥vi ⇒ (x− p)⊥V.



Orthogonal Sets.

Suppose V is a subspace of Rn.

Let p be the orthogonal projectionof a vector x ∈ Rn onto V.


<v,v>v


p =< x, v1 >

< v1, v1 >v1 +

< x, v2 >

< v2, v2 >v2 + ... +

< x, vk >

< vk , vk >vk


j=i<x,vj><vj ,vj>


< vi , vi >=

< x, vi >⇒ < x− p, vi >= 0⇒ (x− p)⊥vi ⇒ (x− p)⊥V.



Orthogonal Sets.

Suppose V is a subspace of Rn. Let p be

the orthogonal projectionof a vector x ∈ Rn onto V.


<v,v>v


p =< x, v1 >

< v1, v1 >v1 +

< x, v2 >

< v2, v2 >v2 + ... +

< x, vk >

< vk , vk >vk


j=i<x,vj><vj ,vj>


< vi , vi >=

< x, vi >⇒ < x− p, vi >= 0⇒ (x− p)⊥vi ⇒ (x− p)⊥V.



Orthogonal Sets.

Suppose V is a subspace of Rn. Let p be the orthogonal projection

of a vector x ∈ Rn onto V.


<v,v>v


p =< x, v1 >

< v1, v1 >v1 +

< x, v2 >

< v2, v2 >v2 + ... +

< x, vk >

< vk , vk >vk


j=i<x,vj><vj ,vj>


< vi , vi >=

< x, vi >⇒ < x− p, vi >= 0⇒ (x− p)⊥vi ⇒ (x− p)⊥V.



Orthogonal Sets.

Suppose V is a subspace of Rn. Let p be the orthogonal projectionof a vector x ∈ Rn

onto V.


<v,v>v


p =< x, v1 >

< v1, v1 >v1 +

< x, v2 >

< v2, v2 >v2 + ... +

< x, vk >

< vk , vk >vk


j=i<x,vj><vj ,vj>


< vi , vi >=

< x, vi >⇒ < x− p, vi >= 0⇒ (x− p)⊥vi ⇒ (x− p)⊥V.



Orthogonal Sets.



<v,v>v


p =< x, v1 >

< v1, v1 >v1 +

< x, v2 >

< v2, v2 >v2 + ... +

< x, vk >

< vk , vk >vk


j=i<x,vj><vj ,vj>


< vi , vi >=

< x, vi >⇒ < x− p, vi >= 0⇒ (x− p)⊥vi ⇒ (x− p)⊥V.



Orthogonal Sets.


If V

is a one-dimensional subspace spanned by a v, thenp = <x,v>

<v,v>v


p =< x, v1 >

< v1, v1 >v1 +

< x, v2 >

< v2, v2 >v2 + ... +

< x, vk >

< vk , vk >vk


j=i<x,vj><vj ,vj>


< vi , vi >=

< x, vi >⇒ < x− p, vi >= 0⇒ (x− p)⊥vi ⇒ (x− p)⊥V.



Orthogonal Sets.


If V is a one-dimensional subspace

spanned by a v, thenp = <x,v>

<v,v>v


p =< x, v1 >

< v1, v1 >v1 +

< x, v2 >

< v2, v2 >v2 + ... +

< x, vk >

< vk , vk >vk


j=i<x,vj><vj ,vj>


< vi , vi >=

< x, vi >⇒ < x− p, vi >= 0⇒ (x− p)⊥vi ⇒ (x− p)⊥V.



Orthogonal Sets.


If V is a one-dimensional subspace spanned by a v, then

p = <x,v><v,v>v


p =< x, v1 >

< v1, v1 >v1 +

< x, v2 >

< v2, v2 >v2 + ... +

< x, vk >

< vk , vk >vk


j=i<x,vj><vj ,vj>


< vi , vi >=

< x, vi >⇒ < x− p, vi >= 0⇒ (x− p)⊥vi ⇒ (x− p)⊥V.



Orthogonal Sets.


If V is a one-dimensional subspace spanned by a v, thenp =

<x,v><v,v>v


p =< x, v1 >

< v1, v1 >v1 +

< x, v2 >

< v2, v2 >v2 + ... +

< x, vk >

< vk , vk >vk


j=i<x,vj><vj ,vj>


< vi , vi >=

< x, vi >⇒ < x− p, vi >= 0⇒ (x− p)⊥vi ⇒ (x− p)⊥V.



Orthogonal Sets.



<v,v>v


p =< x, v1 >

< v1, v1 >v1 +

< x, v2 >

< v2, v2 >v2 + ... +

< x, vk >

< vk , vk >vk


j=i<x,vj><vj ,vj>


< vi , vi >=

< x, vi >⇒ < x− p, vi >= 0⇒ (x− p)⊥vi ⇒ (x− p)⊥V.



Orthogonal Sets.



<v,v>v

If V

admits an orthogonal basis v1, v2, · · · , vk , then

p =< x, v1 >

< v1, v1 >v1 +

< x, v2 >

< v2, v2 >v2 + ... +

< x, vk >

< vk , vk >vk


j=i<x,vj><vj ,vj>


< vi , vi >=

< x, vi >⇒ < x− p, vi >= 0⇒ (x− p)⊥vi ⇒ (x− p)⊥V.



Orthogonal Sets.



<v,v>v

If V admits an

orthogonal basis v1, v2, · · · , vk , then

p =< x, v1 >

< v1, v1 >v1 +

< x, v2 >

< v2, v2 >v2 + ... +

< x, vk >

< vk , vk >vk


j=i<x,vj><vj ,vj>


< vi , vi >=

< x, vi >⇒ < x− p, vi >= 0⇒ (x− p)⊥vi ⇒ (x− p)⊥V.



Orthogonal Sets.



<v,v>v

If V admits an orthogonal basis

v1, v2, · · · , vk , then

p =< x, v1 >

< v1, v1 >v1 +

< x, v2 >

< v2, v2 >v2 + ... +

< x, vk >

< vk , vk >vk


j=i<x,vj><vj ,vj>


< vi , vi >=

< x, vi >⇒ < x− p, vi >= 0⇒ (x− p)⊥vi ⇒ (x− p)⊥V.



Orthogonal Sets.



<v,v>v


p =< x, v1 >

< v1, v1 >v1 +

< x, v2 >

< v2, v2 >v2 + ... +

< x, vk >

< vk , vk >vk


j=i<x,vj><vj ,vj>


< vi , vi >=

< x, vi >⇒ < x− p, vi >= 0⇒ (x− p)⊥vi ⇒ (x− p)⊥V.



Orthogonal Sets.



<v,v>v


p =< x, v1 >

< v1, v1 >v1 +

< x, v2 >

< v2, v2 >v2 + ... +

< x, vk >

< vk , vk >vk


j=i<x,vj><vj ,vj>


< vi , vi >=

< x, vi >⇒ < x− p, vi >= 0⇒ (x− p)⊥vi ⇒ (x− p)⊥V.



Orthogonal Sets.



<v,v>v


p =< x, v1 >

< v1, v1 >v1 +

< x, v2 >

< v2, v2 >v2 + ... +

< x, vk >

< vk , vk >vk

Indeed,

< p, vi >=∑k

j=i<x,vj><vj ,vj>


< vi , vi >=

< x, vi >⇒ < x− p, vi >= 0⇒ (x− p)⊥vi ⇒ (x− p)⊥V.



Orthogonal Sets.



<v,v>v


p =< x, v1 >

< v1, v1 >v1 +

< x, v2 >

< v2, v2 >v2 + ... +

< x, vk >

< vk , vk >vk

Indeed, < p, vi >=

∑kj=i

<x,vj><vj ,vj>


< vi , vi >=

< x, vi >⇒ < x− p, vi >= 0⇒ (x− p)⊥vi ⇒ (x− p)⊥V.



Orthogonal Sets.



<v,v>v


p =< x, v1 >

< v1, v1 >v1 +

< x, v2 >

< v2, v2 >v2 + ... +

< x, vk >

< vk , vk >vk


j=i<x,vj><vj ,vj>

< vj , vi >=

<x,vi><vi ,vi>

< vi , vi >=

< x, vi >⇒ < x− p, vi >= 0⇒ (x− p)⊥vi ⇒ (x− p)⊥V.



Orthogonal Sets.



<v,v>v


p =< x, v1 >

< v1, v1 >v1 +

< x, v2 >

< v2, v2 >v2 + ... +

< x, vk >

< vk , vk >vk


j=i<x,vj><vj ,vj>


< vi , vi >=

< x, vi >⇒ < x− p, vi >= 0⇒ (x− p)⊥vi ⇒ (x− p)⊥V.



Orthogonal Sets.



<v,v>v


p =< x, v1 >

< v1, v1 >v1 +

< x, v2 >

< v2, v2 >v2 + ... +

< x, vk >

< vk , vk >vk


j=i<x,vj><vj ,vj>


< vi , vi >=

< x, vi >⇒

< x− p, vi >= 0⇒ (x− p)⊥vi ⇒ (x− p)⊥V.



Orthogonal Sets.



<v,v>v


p =< x, v1 >

< v1, v1 >v1 +

< x, v2 >

< v2, v2 >v2 + ... +

< x, vk >

< vk , vk >vk


j=i<x,vj><vj ,vj>


< vi , vi >=

< x, vi >⇒ < x− p, vi >= 0⇒

(x− p)⊥vi ⇒ (x− p)⊥V.



Orthogonal Sets.



<v,v>v


p =< x, v1 >

< v1, v1 >v1 +

< x, v2 >

< v2, v2 >v2 + ... +

< x, vk >

< vk , vk >vk


j=i<x,vj><vj ,vj>


< vi , vi >=

< x, vi >⇒ < x− p, vi >= 0⇒ (x− p)⊥vi ⇒

(x− p)⊥V.



Orthogonal Sets.



<v,v>v


p =< x, v1 >

< v1, v1 >v1 +

< x, v2 >

< v2, v2 >v2 + ... +

< x, vk >

< vk , vk >vk


j=i<x,vj><vj ,vj>


< vi , vi >=

< x, vi >⇒ < x− p, vi >= 0⇒ (x− p)⊥vi ⇒ (x− p)⊥V.



Orthogonal Sets.

Coordinates relative to an orthogonal basis

Theorem

If v1, v2, · · · , vn is an orthogonal basis for Rn, then

x =< x, v1 >

< v1, v1 >v1 +

< x, v2 >

< v2, v2 >v2 + ... +

< x, vn >

< vn, vn >vn

for any vector x ∈ Rn

Corollary

If v1, v2, · · · , vn is an orthonormal basis for Rn, then

z =< x, v1 > v1+ < x, v2 > v2 + ...+ < x, vn > vn

for any vector x ∈ Rn.



Orthogonal Sets.


Theorem


x =< x, v1 >

< v1, v1 >v1 +

< x, v2 >

< v2, v2 >v2 + ... +

< x, vn >

< vn, vn >vn


Corollary


z =< x, v1 > v1+ < x, v2 > v2 + ...+ < x, vn > vn




Orthogonal Sets.


Theorem


x =< x, v1 >

< v1, v1 >v1 +

< x, v2 >

< v2, v2 >v2 + ... +

< x, vn >

< vn, vn >vn


Corollary


z =< x, v1 > v1+ < x, v2 > v2 + ...+ < x, vn > vn




Orthogonal Sets.


Theorem

If

v1, v2, · · · , vn is an orthogonal basis for Rn, then

x =< x, v1 >

< v1, v1 >v1 +

< x, v2 >

< v2, v2 >v2 + ... +

< x, vn >

< vn, vn >vn


Corollary


z =< x, v1 > v1+ < x, v2 > v2 + ...+ < x, vn > vn




Orthogonal Sets.


Theorem

If v1, v2, · · · , vn

is an orthogonal basis for Rn, then

x =< x, v1 >

< v1, v1 >v1 +

< x, v2 >

< v2, v2 >v2 + ... +

< x, vn >

< vn, vn >vn


Corollary


z =< x, v1 > v1+ < x, v2 > v2 + ...+ < x, vn > vn




Orthogonal Sets.


Theorem

If v1, v2, · · · , vn is an orthogonal basis

for Rn, then

x =< x, v1 >

< v1, v1 >v1 +

< x, v2 >

< v2, v2 >v2 + ... +

< x, vn >

< vn, vn >vn


Corollary


z =< x, v1 > v1+ < x, v2 > v2 + ...+ < x, vn > vn




Orthogonal Sets.


Theorem

If v1, v2, · · · , vn is an orthogonal basis for Rn,

then

x =< x, v1 >

< v1, v1 >v1 +

< x, v2 >

< v2, v2 >v2 + ... +

< x, vn >

< vn, vn >vn


Corollary


z =< x, v1 > v1+ < x, v2 > v2 + ...+ < x, vn > vn




Orthogonal Sets.


Theorem


x =< x, v1 >

< v1, v1 >v1 +

< x, v2 >

< v2, v2 >v2 + ... +

< x, vn >

< vn, vn >vn


Corollary


z =< x, v1 > v1+ < x, v2 > v2 + ...+ < x, vn > vn




Orthogonal Sets.


Theorem


x =< x, v1 >

< v1, v1 >v1 +

< x, v2 >

< v2, v2 >v2 + ... +

< x, vn >

< vn, vn >vn


Corollary


z =< x, v1 > v1+ < x, v2 > v2 + ...+ < x, vn > vn




Orthogonal Sets.


Theorem


x =< x, v1 >

< v1, v1 >v1 +

< x, v2 >

< v2, v2 >v2 + ... +

< x, vn >

< vn, vn >vn

for any vector

x ∈ Rn

Corollary


z =< x, v1 > v1+ < x, v2 > v2 + ...+ < x, vn > vn




Orthogonal Sets.


Theorem


x =< x, v1 >

< v1, v1 >v1 +

< x, v2 >

< v2, v2 >v2 + ... +

< x, vn >

< vn, vn >vn


Corollary


z =< x, v1 > v1+ < x, v2 > v2 + ...+ < x, vn > vn




Orthogonal Sets.


Theorem


x =< x, v1 >

< v1, v1 >v1 +

< x, v2 >

< v2, v2 >v2 + ... +

< x, vn >

< vn, vn >vn


Corollary


z =< x, v1 > v1+ < x, v2 > v2 + ...+ < x, vn > vn




Orthogonal Sets.


Theorem


x =< x, v1 >

< v1, v1 >v1 +

< x, v2 >

< v2, v2 >v2 + ... +

< x, vn >

< vn, vn >vn


Corollary

If

v1, v2, · · · , vn is an orthonormal basis for Rn, then

z =< x, v1 > v1+ < x, v2 > v2 + ...+ < x, vn > vn




Orthogonal Sets.


Theorem


x =< x, v1 >

< v1, v1 >v1 +

< x, v2 >

< v2, v2 >v2 + ... +

< x, vn >

< vn, vn >vn


Corollary

If v1, v2, · · · , vn

is an orthonormal basis for Rn, then

z =< x, v1 > v1+ < x, v2 > v2 + ...+ < x, vn > vn




Orthogonal Sets.


Theorem


x =< x, v1 >

< v1, v1 >v1 +

< x, v2 >

< v2, v2 >v2 + ... +

< x, vn >

< vn, vn >vn


Corollary

If v1, v2, · · · , vn is an orthonormal basis

for Rn, then

z =< x, v1 > v1+ < x, v2 > v2 + ...+ < x, vn > vn




Orthogonal Sets.


Theorem


x =< x, v1 >

< v1, v1 >v1 +

< x, v2 >

< v2, v2 >v2 + ... +

< x, vn >

< vn, vn >vn


Corollary

If v1, v2, · · · , vn is an orthonormal basis for Rn,

then

z =< x, v1 > v1+ < x, v2 > v2 + ...+ < x, vn > vn




Orthogonal Sets.


Theorem


x =< x, v1 >

< v1, v1 >v1 +

< x, v2 >

< v2, v2 >v2 + ... +

< x, vn >

< vn, vn >vn


Corollary


z =< x, v1 > v1+ < x, v2 > v2 + ...+ < x, vn > vn




Orthogonal Sets.


Theorem


x =< x, v1 >

< v1, v1 >v1 +

< x, v2 >

< v2, v2 >v2 + ... +

< x, vn >

< vn, vn >vn


Corollary


z =< x, v1 > v1+ < x, v2 > v2 + ...+ < x, vn > vn




Orthogonal Sets.


Theorem


x =< x, v1 >

< v1, v1 >v1 +

< x, v2 >

< v2, v2 >v2 + ... +

< x, vn >

< vn, vn >vn


Corollary


z =< x, v1 > v1+ < x, v2 > v2 + ...+ < x, vn > vn

for any vector

x ∈ Rn.



Orthogonal Sets.


Theorem


x =< x, v1 >

< v1, v1 >v1 +

< x, v2 >

< v2, v2 >v2 + ... +

< x, vn >

< vn, vn >vn


Corollary


z =< x, v1 > v1+ < x, v2 > v2 + ...+ < x, vn > vn



Documents

Linear Algebra. Session 11 - Texas A&M Universityroquesol/Math_304_Fall_2018... · 2018-11-15 · Session 11. Abstract Linear Algebra II Least Squares Problems Orthogonal Sets Least