Linear Algebra, Principal Component Analysis and their Chemometrics Applications

Linear Algebra, Principal Component Analysis and

their Chemometrics Applications

Linear algebra is the language of chemometrics. One can not expect to truly understand most chemometric techniques without a basic understanding of linear algebra

Linear Algebra

Vector

A vector is a mathematical quantity that is completely described by its magnitude and direction

x1

y1

x

y

P

Vector

A vector is a mathematical quantity that is completely described by its magnitude and direction

x1

y1

x

y

P

P =

x1

y1

MATLAB Notation

Length of a Vector

x1

y1

x

y

PP = x1

2 + y12

x = [x1, x2, …, xn]

x M= ( xi2 ) 0.5

i=1

n

Normal Vector

u =xx

u =1

Normalized vector

Mean Centered Vector

x1

x2

xn

…x = mx = M

xi

i=1

n

nmcx =

x1 - mx

…

x2 - mx

xn - mx

0015100

x = mx = 1

-1-1040-1-1

mcx =

0+20+21+25+21+20+20+2

y =

2237322

=mx = 3

0

0.2

0.4

0.6

0.8

1

1.2

400 450 500 550 600Wavelength (nm)

Ab

so

rba

nc

e

00.20.40.60.8

11.21.4

400 450 500 550 600Wavelength (nm)

Ab

so

rba

nc

e

-0.4

-0.2

0

0.2

0.4

0.6

400 450 500 550 600

Wavelength (nm)

Ab

so

rba

nc

e

Mean centeredMean centered

?

The length of a mean centered vector is proportional to the standard deviation of its elements

y1

y2

yn

…y = y* =

y1 - my

…

y2 - my

yn - myy* ≈ syi

A set of p vectors [x1, x2, …, xp] with same dimension n is linearly independent if the expression:

ci xi = 0Mi=1

p

holds only when all p coefficients ci are zero

Linear Independent Vectors

x1

x2

x3c1x1

c2x2

A vector space spanned by a set of p linearly independent vectors (x1, x2, …, xp) with the same dimension n is the set of all vectors that are linear combinations of the p vectors that span the space

Vector Space

Basis setA set of n vectors of dimension n which are linearly independent is called a basis of an n-dimensional vector space. There can be several bases of the same vector space

A coordinate space can be thought of as being constructed from n basis vectors of unit length which originate from a common point and which are mutually perpendicular

Coordinate Space

Q = P = x1

y1

x1

y1

x

y

P

x1

y1

Q

Vector Multiplication by a Scalar

0

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0.45

400 450 500 550

Wavelength (nm)

Ab

so

rba

nc

e

x = 1.19 y = 2.38

x

y

y = 2 x

Addition of Vectorsx1

x2

xn

…

y1

y2

yn

…x + y = + =

x1 + y1

…

x2 + y2

xn + yn

x + y

x1

x2

y1

y2y

x

x1 + y1

x2 + y2

x1cx

…ax + ay = + =

x2cx

xncx

y1cy

…

y2cy

yncy

x1cx + y1cy

…

x2cx + y2cy

xncx + yncy

Component 1 Component 2 mixture

0

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

400 420 440 460 480 500 520 540 560

Wavelength (nm)

Ab

so

rba

nc

e

axay

ax + ay

Subtraction of Vectorsx1

x2

xn

…

y1

y2

yn

…x - y = - =

x1 - y1

…

x2 - y2

xn - yn

x - y

x1

x2

y1

y2y

x

Inner Product (Dot Product)x1

x2

xn

…x . x = xTx = [x1 x2 … xn] = x12 + x2

2 + … +xn2

= x2

x . y = xTy = x y cos

The cosine of the angle of two vectors is equal to the dot product between the normalized vectors:

x . y

x ycos =

y

xx . y = x y

y

xx . y = - x y

y

x x . y = 0

Two vectors x and y are orthogonal when their scalar product is zero

x . y = 0 and x y = 1=

Two vectors x and y are orthonormal