Upload
jacob-fitzgerald
View
229
Download
0
Embed Size (px)
DESCRIPTION
Preview We will only focus on one application – The Method of Least Squares.
Citation preview
MAT 2401Linear Algebra
2.5 Applications of Matrix Operations
http://myhome.spu.edu/lauw
HW Written Homework
Preview We will only focus on one
application – The Method of Least Squares.
Linear RegressionSuppose that a scientist has reason to believe that 2 quantities x and y are related linearly, that is,
y=mx+b.The scientist performs an experiment and collect data points (x1,y1),…,(xn,yn).
Linear Regressiony
x
,i ix y y mx b
ie
i i ie y mx b
Goals Find a line y=mx+b that minimize
the sum of the squares of the errors ei.
Use y=mx+b to estimate the function values.
Linear Regressiony
x
,i ix y y mx b
ie
i i ie y mx b
Matrix Equation
i i ie y mx b
Matrix Equation
1 11
2 22
11 Let , , ,
1 nn n
y exy ex b
Y X A Em
xy e
Matrix Form of Linear RegressionFor the linear regression model , the coefficients of the least squares regression line are given by
A= (XTX)-1XTYand the sum of squared error is
ETE1 11
2 22
11 Let , , ,
1 nn n
y exy ex b
Y X A Em
xy e
Plan… Computational Example HW Why the formula is correct? Very
Educational; Focus on the Ideas
Example 1Find the least squares regression line for the points (1,1), (2,2), (3,4), and (5,6).
1 11
2 22
1
11 Let , , ,
1
Then,
nn n
T T
y exy ex b
Y X A Em
xy e
A X X X Y
Example 1
Example 1
9 27 7
y x
Why? Give you some ideas why the
formula actually work.
Recall Q: How to find the minimum of a
function f(x)? A:
Recall Q: How to find the minimum of a
function f(x)? A: Q: How to find the minimum of a
function f(x,y)?
Recall: Sigma Notation A “compact” notation for sums to
avoid “…”30
2 2 2 2 2
1
1 2 3 30k
k
Recall: Sigma Notation
1 21
n
i ni
x x x x
Final value (upper limit)
Initial value (lower limit)Index
Recall: Linear Property 1
1 2
1 2
1
1
5
5
5 5 5
5
n
ii
n
ii
n
n
x x x
x
x
xx x
Recall: Linear Property 2
1 1 2 2
1 2 1
1
1 1
2
n n
n
n
i ii
n n
i ii i
n
x y x y x y
x x x y y y
x y
x y
Why? Let g(b,m) be the function of the
sum of the squared errors. We can find the critical point by
solving the equations0 and 0g g
b m
Why? Let g(b,m) be the function of the
sum of the squared error. We can find the critical point by
solving the equations
It can be shown that the critical point is a minimum (skip)
0 and 0g gb m
Why?
0 and 0g gb m
2
1
2
1
2
1
( , )n
ii
n
i ii
n
i ii
g b m e
y mx b
y mx b