Econ 205 - Slides from Lecture 6

Joel Sobel

August 30, 2010

Econ 205 Sobel

Linear Algebra Basics

Definitionn-dimensional Euclidean Space:

Rn = R× R× · · · × R× R

If X and Y are sets, then

X × Y ≡ {(x , y) | x ∈ X , y ∈ Y }

soRn = {(x1, x2, . . . , xn) | xi ∈ R,∀ i = 1, 2, . . . , n}

Econ 205 Sobel

Interpretations:

I Calculus: x ∈ Rn as a list of n real numbers; x = (x1, . . . , xn).

I Linear algebra x ∈ Rn is column vector. When we do this wewrite

x =

x1x2...xn

.

I Sometimes an element in Rn is best thought of as a direction.

Econ 205 Sobel

DefinitionThe zero element:

0 =

00...0

Definition (Vector Addition)

For x, y ∈ Rn we have

x + y =

x1 + y1x2 + y2

...xn + yn

Vector addition is commutative

x + y = y + xEcon 205 Sobel

Definition (Scalar Multiplication)

For x ∈ Rn, and a ∈ R we have

ax =

ax1ax2

...axn

In other words every element of x gets multiplied by a.

Econ 205 Sobel

Vector Spaces

Definition (Vector Space)

A vector space is a set V in which the operation of addition andmultiplication by a scalar make sense, and in which operations arecommutative and associative. In addition:0 + v = v (additive identity)for each v there is a −v (additive inverse)

A subset of a vector space that is itself a vector space is called asubspace.Euclidean Spaces (Rn) are the leading example of vector spaces.We will need to talk about subsets of Euclidean Spaces that have alinear structure (they contain 0, and if x and y are in the set, thenso is x + y and all scalar multiples of x and y).

Econ 205 Sobel

Matrices

DefinitionAn m × n matrix is an element of Mm×n written as in the form

A =

α11 α12 · · · α1n

α21 α22 · · · α2n...

......

αm1 αm2 · · · αmn

= [αij ]

where m denotes the number of rows and n denotes the number ofcolumns.

Note An m × n matrix is just of a collection of nm numbersorganized in a particular way. Hence we can think of a matrix asan element of Rm×n. The extra notation Mm×n makes it possibleto distinguish the way that the numbers are organized.

Econ 205 Sobel

Note Vectors are just a special case of matrices. e.g.

x =

x1x2...xn

∈Mn×1

This notation emphasizes that we think of a vector with ncomponents as a matrix with n rows and 1 column.

Econ 205 Sobel

A2×3

=

(0 1 56 0 2

)

Econ 205 Sobel

DefinitionThe transpose of a matrix A, is denoted At . To get the transposeof a matrix, we let the first row of the original matrix become thefirst column of the new (transposed) matrix.

At =

α11 α21 · · · α1n

α12 α22 · · · α2n...

......

α1m α2m · · · αnm

= [αji ]

DefinitionA matrix A is symmetric if A = At .

So we can see that if A ∈Mm×n, then At ∈Mn×m.

Econ 205 Sobel

Continuing the example, we see that

At

3×2=

0 61 05 2

Econ 205 Sobel

Matrix Algebra

[Addition of Matrices] If

Am×n

=

α11 α12 · · · α1n

α21 α22 · · · α2n...

......

αm1 αm2 · · · αmn

= [αij ]

and

Bm×n

=

β11 β12 · · · β1nβ21 β22 · · · β2n

......

...βm1 βn2 · · · βmn

= [βij ]

then

Econ 205 Sobel

A + Bm×n

= Dm×n

=

α11 + β11 α12 + β12 · · · α1n + β1nα21 + β21 α22 + β22 · · · α2n + β2n

......

...αm1 + βm1 αm2 + βm2 · · · αmn + βmn

= [δij ] = [αij+βij ]

A + B︸ ︷︷ ︸m×n

=

α11 + β11 α12 + β12 · · · α1n + β1nα21 + β21 α22 + β22 · · · α2n + β2n

......

...αm1 + βm1 αm2 + βm2 · · · αmn + βmn

= [αij+βij ]

Econ 205 Sobel

Definition (Multiplication of Matrices)

If Am×k

and Bk×n

are given, then we define

Am×k· Bk×n

= Cm×n

= [cij ]

such that

cij ≡k∑

l=1

ailblj

so note above that the only index being summed over is l.

Econ 205 Sobel

Let

A2×3

=

(0 1 56 0 2

)and

B3×2

=

0 31 02 3

Then

A2×3· B3×2︸ ︷︷ ︸

2×2

=

(0 1 56 0 2

)·

0 31 02 3

=

((0× 0) + (1× 1) + (5× 2), (0× 3) + (1× 0) + (5× 3)(6× 0) + (0× 1) + (2× 2), (6× 3) + (0× 0) + (2× 3)

)=

(11 154 24

)

Econ 205 Sobel

In general:A · B 6= B · A

For example

A2×3· B3×46= B

3×4· A2×3

(The product on the right is not defined.)

DefinitionAny matrix which has the same number of rows as columns isknown as a square matrix, and is denoted A

n×n.

Econ 205 Sobel

DefinitionThere is a special square matrix known as the identity matrix. Anymatrix multiplied by this identity matrix gives back the originalmatrix. The Identity matrix is denoted In and is equal to

Inn×n

==

1 0 . . . 00 1 . . . 0...

. . ....

0 . . . 0 1

.

Econ 205 Sobel

DefinitionA square matrix is called a diagonal matrix if aij = 0 wheneveri 6= j .

DefinitionA square matrix is called an upper triangular matrix (resp. lowertriangular if aij = 0 whenever i > j (resp. i < j).

Diagonal matrices are easy to deal with. Triangular matrixes arealso somewhat tractable. You’ll see that for many applications youcan replace an arbitrary square matrix with a related diagonalmatrix.

Econ 205 Sobel

For any matrix Am×n

we have the results that

Am×n· In = A

m×n

andIm · A

m×n= A

m×n

Econ 205 Sobel

DefinitionWe say a matrix A

n×nis invertible or non-singular if ∃ B

n×nsuch that

An×n· Bn×n︸ ︷︷ ︸

n×n

= Bn×n· An×n︸ ︷︷ ︸

n×n

= In

If A is invertible, we denote it’s inverse as A−1.So we get

A(n×n)

· A−1

(n×n)︸ ︷︷ ︸n×n

= A−1

n×n· An×n︸ ︷︷ ︸

n×n

= In

A square matrix that is not invertible is called singular.

Econ 205 Sobel

DefinitionThe determinant of a matrix A (written detA = |A|) is definedinductively.

n = 1 A(1×1)

detA = |A| ≡ a11

n ≥ 2 A(n×n)

detA =| A |≡

a11|A−11| − a12|A−12|+ a13|A−13| − · · · ± a1n|A−1n|

where A−1j is the matrix formed by deleting the firstrow and jth column of A.Note A−1j is an (n− 1)× (n− 1) dimensional matrix.

Econ 205 Sobel

Examples

If

A2×2

= [aij ] =

(a11 a12a21 a22

)=⇒ detA = a11a22 − a12a21

Econ 205 Sobel

If

A3×3

= [aij ] =

a11 a12 a13a21 a22 a23a31 a32 a33

=⇒ detA = a11

∣∣∣∣ a22 a23a32 a33

∣∣∣∣− a12

∣∣∣∣ a21 a23a31 a33

∣∣∣∣+ a13

∣∣∣∣ a21 a22a31 a32

∣∣∣∣

Econ 205 Sobel

Determinant

The determinant is useful primarily because of the following result:

TheoremA matrix is invertible if and only if its determinant 6= 0.

Econ 205 Sobel

DefinitionThe adjoint of a matrix A

n×n(adj (A)) is the n × n matrix with

entry ij equal toadj A = (−1)i+j detA−ij

where adj A is the adjoint of A.

DefinitionThe Inverse of a matrix A

n×nis defined as

A−1 =1

detA· adj A

where adj A is the adjoint of A.

Econ 205 Sobel

If A is a (2× 2) matrix and invertible then

A−1 =1

detA·(

a22 −a12−a21 a11

)

Econ 205 Sobel

Inner Product

Definition (Inner Product)

If x, y ∈Mn×1, then the inner product (or dot product or scalarproduct) is given by

xty = x1y1 + x2y2 + · · ·+ xnyn

=n∑

i=1

xiyi

Note that xty = ytx. We will have reason to use this conceptwhen we do calculus, and will write x · y =

∑ni=1 xiyi .

Econ 205 Sobel

Distance

Definition (Euclidean Distance)

d(x, y) = ||x− y||

where

||z|| =√z21 + z22 + · · ·+ z2n

=

√√√√ n∑i=1

z2i

Under the Euclidean metric, the distance between two points is thelength of the line segment connecting the points. We call ||z||,which is the distance between 0 and z the norm of z.Notice that ||z||2 = z · z.

Econ 205 Sobel

Inner Product Properties

When x · y = 0 we say that x and y are orthogonal/at rightangles/perpendicular.Two vectors are perpendicular if and only if their inner product iszero. This fact follows rather easily from “The Law of Cosines.”The law of cosines states that if a triangle has sides A,B, and Cand the angle θ opposite the side c , then

c2 = a2 + b2 − 2ab cos(θ),

where a, b, and c are the lengths of A, B, and C respectively. Thismeans that:

(x− y) · (x− y) = x · x + y · y− 2||x||||x|| cos(θ),

where θ is the angle between x and y.

Econ 205 Sobel

If you multiply everything out you get the identity:

||x||||y|| cos(θ) = xty. (1)

1. The inner product of two non-zero vectors is zero if and onlyif the cosine of the angle between them is zero.

2. Since the absolute value of the cosine is less than or equal toone,

||x||||y|| ≥| xty | .

Econ 205 Sobel

Systems of Linear Equations

Consider the system of m equations in n variables:

y1 = α11x1 + α12x2 + · · ·+ α1nxn

y2 = α21x1 + α22x2 + · · ·+ α2nxn...

yi = αi1x1 + αi2x2 + · · ·+ αinxn...

ym = αm1x1 + αm2x2 + · · ·+ αmnxn

Here the variables are the xj .

Econ 205 Sobel

This can be written as

y(m×1)

= A(m×n)

· x(n×1)︸ ︷︷ ︸

(m×1)

where

y(m×1)

=

y1y2...ym

, x(n×1)

=

x1x2...xn

Am×n

=

α11 α12 · · · α1n

α21 α22 · · · α2n...

......

αm1 αm2 · · · αmn

= [αij ]

or, putting it all together

Econ 205 Sobel

y(m×1)

= A(m×n)

· x(n×1)︸ ︷︷ ︸

(m×1)y1y2...ym

=

α11 α12 · · · α1n

α21 α22 · · · α2n...

......

αm1 αm2 · · · αmn

·

x1x2...xn

Econ 205 Sobel

Facts about solutions to linear equations

DefinitionA system of equations of the form Ax = 0 is called a homogeneoussystem of equations.

A homogeneous system always has a solution (x = 0). The solutionwill not be unique if there are more unknowns than equations or ifthere are as many equations as unknowns and A is singular.

TheoremWhen A is square, the system Ax = y has a unique solution if andonly if A is nonsingular.

If defined, x = A−1y.

If not, then there is a nonzero z such that Az = 0. This meansthat if you can find one solution to Ax = y you can find infinitelymany.

Econ 205 Sobel

General Theory

Using “elementary row operations” reduce general system ofequations to canonical form that permits you to identify solutionset easily.

Econ 205 Sobel