y=a+bx

y=a+bx

n

1i

2ii

n

1i

2ii bxaybxayq

Sum of squares of errors

0)bxay(20a

qii

0)bxay(x20b

qiii

ii

i2ii

i

yx

y

b

a

xx

xn

Linear Regression: Method of Least Squares

The Method of Least Squares is a procedure to determine the best fit line to data; the proof uses simple calculus and linear algebra. The basic problem is to find the best fit straight line y = a + bx given that, for n ϵ {1,…,N}, the pairs (xn; yn) are observed.

The form of the fitted curve is

y intercept

slope

x y

-5 -2

2 4

7 3.5

5.42

5.5

34

478

218

1

b

a

a=1.188 b=0.484

y=1.188+0.484x

Example 1:

Find a 1st order polynomial y=a+bx for the values given in the Table.

3

1i

2222i

3

1ii

78725x

4725x

3n

3

1iii

3

1ii

5.425.3x74x2)2(x)5(yx

5.55.342y

5.42

5.5

b

a

784

43

-6 -4 -2 0 2 4 6 8-2

-1

0

1

2

3

4

5

6

7

x value

y va

lue

Data point

Fitted curve

clc;clearx=[-5,2,4];y=[-2,4,3.5];p=polyfit(x,y,1)x1=-5:0.01:7;yx=polyval(p,x1);plot(x,y,'or',x1,yx,'b')xlabel('x value')ylabel ('y value')

x y

0 200

3 230

5 240

8 270

10 290

y=a+bx

ii

i2ii

i

yx

y

b

a

xx

xn

6950

1230

b

a

19826

265

6950

1230

526

26198

314

1

b

a

y=200.13 + 8.82x

Example 2:

-2 0 2 4 6 8 10 12180

200

220

240

260

280

300

320

x value

y va

lue

clc;clearx=[0,3,5,8,10];y=[200,230,240,270,290];p=polyfit(x,y,1)x1=-1:0.01:12;yx=polyval(p,x1);plot(x,y,'or',x1,yx,'b')xlabel('x value')ylabel ('y value')

Data point

Fitted curve

Method of Least Squares:

Tensile tests were performed for a composite material having a crack in order to calculate the fracture toughness. Obtain a linear relationship between the breaking load F and crack length a.

495.1828.0*5.845.0*4.935.0*1.94.0*25.95.0*10yx

98.128.045.035.04.05.0y

98.4285.84.91.925.910x

25.465.84.91.925.910x

5n

ii

i

222222i

i

495.1898.1

aa

98.42825.4625.465

1

2

bxa)x(y

ii

i2ii

i

yxy

ba

xxxn

Method of Least Squares

ii

i

1

22ii

i

yxy

aa

xxxn

21 aFa)F(a

Slope

Intercept

Slope Intercept

495.1898.1

525.4625.4698.428

25.46*25.4698.428*51

aa

1

2


1542.0a0301.1a

1

2

0301.1F1542.0)F(a

with Visual Basic:

mls.txt

5

10,0.5

9.25,0.4

9.1,0.35

9.4,0.45

8.5,0.28

with Matlab: clc;clearx=[10,9.25,9.1,9.4,8.5];y=[0.5,0.4,0.35,0.45,0.28];p=polyfit(x,y,1)F=8:0.01:12;a=polyval(p,x1);plot(x,y,'or‘,F,a,'b')xlabel('x value')ylabel ('y value')


The change in the interior temperature of an oven with respet to time is given in the Figure. It is desired to model the relationship between the temperature (T) and time (t) by a first order polynomial as T=c1t+c2. Determine the coefficients c1 and c2.

T (°C)

t (min.)0 5 10 15175

204200

212

ii

i

1

22ii

i

yxy

cc

xxxn

21 ctc)t(T

Slope

Intercept

Slope Intercept

6200212*15200*10204*5175*0yx

791212200204175y

350151050x

30151050x

4n

ii

i

22222i

i

6200791

cc

35030304

1

2

6200791

43030350

30*30350*41

cc

1

2

14.2c7.181c

1

2

7.181t14.2)t(T


With Visual Basic:

mls.txt

4

0,175

5,204

10,200

15,212

with Matlab:

clc;clearx=[0,5,10,15];y=[175,204,200,212];p=polyfit(x,y,1)t=0:0.01:15;T=polyval(p,x1);plot(x,y,'or',t,T,'b')xlabel('x value')ylabel ('y value')

Documents

y=a+bx