Upload
lecong
View
222
Download
0
Embed Size (px)
Citation preview
7172017 httpnumericalmethodsengusfedu 1
Linear Regression
Major All Engineering Majors
Authors Autar Kaw Luke Snyder
httpnumericalmethodsengusfeduTransforming Numerical Methods Education for STEM
Undergraduates
Linear Regression
httpnumericalmethodsengusfedu
httpnumericalmethodsengusfedu 3
What is RegressionWhat is regression Given n data points
best fit )(xfy = to the data
Residual at each point is
)(xfy =
Figure Basic model for regression
)()()( 2211 nn yxyxyx
)( iii xfyE minus=
y
x
)( 11 yx
)( nn yx)( ii yx
)( iii xfyE minus=
httpnumericalmethodsengusfedu 4
Linear Regression-Criterion1Given n data points best fit xaay 10 += to the data
Does minimizingsum=
n
iiE
1work as a criterion
x
xaay 10 +=)( 11 yx
)( 22 yx)( 33 yx
)( nn yx
)( ii yx
iii xaayE 10 minusminus=
y
Figure Linear regression of y vs x data showing residuals at a typical point xi
)()()( 2211 nn yxyxyx
httpnumericalmethodsengusfedu 5
Example for Criterion1
x y
20 40
30 60
20 60
30 80
Example Given the data points (24) (36) (26) and (38) best fit the data to a straight line using Criterion1
Figure Data points for y vs x data
Table Data Points
0
2
4
6
8
10
0 1 2 3 4
y
x
Minimizesum=
n
iiE
1
httpnumericalmethodsengusfedu 6
Linear Regression-Criteria1
04
1=sum
=iiE
x y ypredicted E = y - ypredicted
20 40 40 00
30 60 80 -20
20 60 40 20
30 80 80 00
Table Residuals at each point for regression model y=4x minus 4
Figure Regression curve y=4x minus 4 and y vs x data
0
2
4
6
8
10
0 1 2 3 4
y
x
Using y=4x minus 4 as the regression curve
httpnumericalmethodsengusfedu 7
Linear Regression-Criterion1
x y ypredicted E = y - ypredicted
20 40 60 -20
30 60 60 00
20 60 60 00
30 80 60 20
04
1=sum
=iiE
0
2
4
6
8
10
0 1 2 3 4
y
x
Table Residuals at each point for regression model y=6
Figure Regression curve y=6 and y vs x data
Using y=6 as a regression curve
httpnumericalmethodsengusfedu 8
Linear Regression ndash Criterion 1
04
1=sum
=iiE for both regression models of y=4x-4 and y=6
The sum of the residuals is minimized in this case it is zero but the regression model is not unique
Hence the criterion of minimizing the sum of the residuals is a bad criterion
0
2
4
6
8
10
0 1 2 3 4
y
x
httpnumericalmethodsengusfedu 9
Linear Regression-Criterion1
04
1=sum
=iiE
x y ypredicted E = y - ypredicted
20 40 40 00
30 60 80 -20
20 60 40 20
30 80 80 00
Table Residuals at each point for regression model y=4x minus 4
Figure Regression curve y=4x-4 and y vs x data
0
2
4
6
8
10
0 1 2 3 4
y
x
Using y=4x minus 4 as the regression curve
httpnumericalmethodsengusfedu 10
Linear Regression-Criterion2
x
)( 11 yx
)( 22 yx)( 33 yx
)( nn yx
)( ii yx
iii xaayE 10 minusminus=
y
Figure Linear regression of y vs x data showing residuals at a typical point xi
Will minimizing ||1sum=
n
iiE work any better
xaay 10 +=
httpnumericalmethodsengusfedu 11
Example for Criterion2
x y20 40
30 60
20 60
30 80
Example Given the data points (24) (36) (26) and (38) best fit the data to a straight line using Criterion2
Figure Data points for y vs x data
Table Data Points
0
2
4
6
8
10
0 1 2 3 4
y
x
Minimizesum=
n
iiE
1||
httpnumericalmethodsengusfedu 12
Linear Regression-Criterion2
4||4
1=sum
=iiE
x y ypredicted E = y - ypredicted
20 40 40 00
30 60 80 -20
20 60 40 20
30 80 80 00
Table Residuals at each point for regression model y=4x minus 4
Figure Regression curve y= y=4x minus 4 and y vs xdata
0
2
4
6
8
10
0 1 2 3 4
y
x
Using y=4x minus 4 as the regression curve
httpnumericalmethodsengusfedu 13
Linear Regression-Criterion2
x y ypredicted E = y - ypredicted
20 40 60 -20
30 60 60 00
20 60 60 00
30 80 60 20
4||4
1=sum
=iiE
0
2
4
6
8
10
0 1 2 3 4
y
x
Table Residuals at each point for regression model y=6
Figure Regression curve y=6 and y vs x data
Using y=6 as a regression curve
httpnumericalmethodsengusfedu 14
Linear Regression-Criterion2for both regression models of y=4x minus 4 and y=6
The sum of the absolute residuals has been made as small as possible that is 4 but the regression model is not unique
Hence the criterion of minimizing the sum of the absolute value of the residuals is also a bad criterion
44
1=sum
=iiE
httpnumericalmethodsengusfedu 15
Least Squares CriterionThe least squares criterion minimizes the sum of the square of the
residuals in the model and also produces a unique line
( )2
110
1
2 sum minusminus=sum===
n
iii
n
iir xaayES
x
11 yx
22 yx
33 yx
nnyx
iiyx
y
Figure Linear regression of y vs x data showing residuals at a typical point xi
xaay 10 +=
iii xaayE 10 minusminus=
httpnumericalmethodsengusfedu16
Finding Constants of Linear Model( )
2
110
1
2 sum minusminus=sum===
n
iii
n
iir xaayESMinimize the sum of the square of the residuals
To find
( )( ) 0121
100
=minusminusminusminus=partpart sum
=
n
iii
r xaayaS
( )( ) 021
101
=minusminusminusminus=partpart sum
=
n
iiii
r xxaayaS
giving
i
n
iii
n
ii
n
ixyxaxa sumsumsum
===
=+1
2
11
10
0a and 1a we minimize with respect to 1a 0aandrS
sumsumsum===
=+n
iii
n
i
n
iyxaa
111
10
httpnumericalmethodsengusfedu17
Finding Constants of Linear Model0aSolving for
2
11
2
1111
minus
minus=
sumsum
sumsumsum
==
===
n
ii
n
ii
n
ii
n
ii
n
iii
xxn
yxyxna
and
2
11
2
1111
2
0
minus
minus=
sumsum
sumsumsumsum
==
====
n
ii
n
ii
n
iii
n
ii
n
ii
n
ii
xxn
yxxyxa
1aand directly yields
xaya 10 minus=
httpnumericalmethodsengusfedu18
Example 1The torque T needed to turn the torsion spring of a mousetrap through an angle is given below
Angle θ Torque T
Radians N-m
0698132 0188224
0959931 0209138
1134464 0230052
1570796 0250965
1919862 0313707
Table Torque vs Angle for a torsional spring
Find the constants for the model given byθ21 kkT +=
Figure Data points for Torque vs Angle data
01
02
03
04
05 1 15 2
θ (radians)
Torq
ue (N
-m)
httpnumericalmethodsengusfedu19
Example 1 cont
1a
The following table shows the summations needed for the calculations of the constants in the regression model
θ 2θ θTRadians N-m Radians2 N-m-Radians
0698132 0188224 0487388 0131405
0959931 0209138 0921468 0200758
1134464 0230052 12870 0260986
1570796 0250965 24674 0394215
1919862 0313707 36859 0602274
62831 11921 88491 15896
Table Tabulation of data for calculation of important
sum=
=5
1i
5=nUsing equations described for
25
1
5
1
2
5
1
5
1
5
12
minus
minus=
sumsum
sumsumsum
==
===
ii
ii
ii
ii
iii
n
TTnk
θθ
θθ
( ) ( )( )( ) ( )228316849185
1921128316589615
minus
minus=
21060919 minustimes= N-mrad
summations
0aTand with
httpnumericalmethodsengusfedu20
Example 1 cont
n
TT i
isum==
5
1_
Use the average torque and average angle to calculate 1k
_
2
_
1 θkTk minus=
ni
isum==
5
1_
θθ
519211
=
11038422 minustimes=
528316
=
25661=
Using
)25661)(1060919(1038422 21 minusminus timesminustimes=11017671 minustimes= N-m
httpnumericalmethodsengusfedu21
Example 1 Results
Figure Linear regression of Torque versus Angle data
Using linear regression a trend line is found from the data
Can you find the energy in the spring if it is twisted from 0 to 180 degrees
httpnumericalmethodsengusfedu22
Linear Regression (special case)
Given
best fit
to the data
)( )()( 2211 nn yxyx yx
xay 1=
httpnumericalmethodsengusfedu23
Linear Regression (special case cont)
Is this correct
2
11
2
1111
minus
minus=
sumsum
sumsumsum
==
===
n
ii
n
ii
n
ii
n
ii
n
iii
xxn
yxyxna
xay 1=
httpnumericalmethodsengusfedu24
x
11 yx
ii xax 1
nnyx
iiyx
iii xay 1minus=ε
y
Linear Regression (special case cont)
httpnumericalmethodsengusfedu25
Linear Regression (special case cont)
iii xay 1minus=ε
sum=
=n
iirS
1
2ε
( )2
11sum
=
minus=n
iii xay
Residual at each data point
Sum of square of residuals
httpnumericalmethodsengusfedu26
Linear Regression (special case cont)
Differentiate with respect to
gives
( )( )sum=
minusminus=n
iiii
r xxaydadS
11
1
2
( )sum=
+minus=n
iiii xaxy
1
2122
01
=dadSr
sum
sum
=
== n
ii
n
iii
x
yxa
1
2
11
1a
httpnumericalmethodsengusfedu27
Linear Regression (special case cont)
sum
sum
=
== n
ii
n
iii
x
yxa
1
2
11
( )sum=
+minus=n
iiii
r xaxydadS
1
21
1
22
021
22
1
2
gt= sum=
n
ii
r xda
Sd
sum
sum
=
== n
ii
n
iii
x
yxa
1
2
11
Does this value of a1 correspond to a local minima or local maxima
Yes it corresponds to a local minima
httpnumericalmethodsengusfedu28
Linear Regression (special case cont)
Is this local minima of an absolute minimum of rS rS
1a
rS
httpnumericalmethodsengusfedu29
Example 2
Strain Stress
() (MPa)
0 0
0183 306
036 612
05324 917
0702 1223
0867 1529
10244 1835
11774 2140
1329 2446
1479 2752
15 2767
156 2896
To find the longitudinal modulus of composite the following data is collected Find the longitudinal modulus Table Stress vs Strain data
E using the regression modelεσ E= and the sum of the square of the
00E+00
10E+09
20E+09
30E+09
0 0005 001 0015 002
Strain ε (mm)
Stre
ss σ
(Pa)
residuals
Figure Data points for Stress vs Strain data