Spring 2011 Anant R. Shastriars/MA106/slides-I.pdf · Linear Algebra MA106 Prescribed text books: I 1. H. Anton, Elementary linear algebra with applications (8th Edition), John Wiley

MA106-Linear AlgebraSpring 2011

Anant R. Shastri

Department of MathematicsIndian Institute of Technology, Bombay

January 19, 2011

ARS (IITB) MA106-Linear Algebra January 19, 2011 2 / 59

—————————

Linear Algebra MA106

Prescribed text books:

I 1. H. Anton, Elementary linear algebra with applications (8th Edition), JohnWiley (1995).

I 2. G. Strang, Linear algebra and its applications (4th Edition), Thom-son(2006).

I 3. S. Kumaresan, Linear algebra - A Geometric approach, Prentice Hall ofIndia (2000).

I 4. E. Kreyszig, Advanced engineering mathematics (8th Edition), John Wiley(1999).


Overview of the Course

I This is the second course in mathematics for you in IIT.

I In the first course itself you have studied some basic properties of functionssuch as:

I continuity,I differentiability,I and integration, etc.

I These were real valued functions of one or two real variables.

I In this course we shall get into the realm of unlimited number of variables.

I On the other hand, in the beginning, it is prudent to keep this discussionAS SIMPLE AS POSSIBLE.

I Simplest of functions which have sufficiently interesting properties (notconstants!) are linear functions. Therefore, we shall start our study with thatof Linear Algebra.














I continuity,

I differentiability,I and integration, etc.









I continuity,I differentiability,

I and integration, etc.







































I On the other hand, in the beginning, it is prudent to keep this discussion

AS SIMPLE AS POSSIBLE.





















Introduction

To begin with Linear Algebra brings unified approach to certain topics which arefamiliar to you such as

I coordinate geometry

I vector algebra.

Apart from playing a very crucial role in the basic understanding of the calculus ofseveral variables, Linear Algebra has its own importance with applications inalmost all scientific studies.


Introduction



I vector algebra.



Introduction



I vector algebra.



Introduction



I vector algebra.



Introduction



I vector algebra.



Introduction

Let us lay a proper foundation of Linear Algebra with basic concepts such as

I vector spaces,

I linear transformations,

I linear dependence,

I dimension,

I matrices and determinants etc..At the end, some illustrations of applications to geometric problems will begiven with a brief study of

I conic sections and quadratic surfaces


Introduction


I vector spaces,



I dimension,




Introduction


I vector spaces,



I dimension,




Introduction


I vector spaces,



I dimension,




Introduction


I vector spaces,



I dimension,

I matrices and determinants etc..

At the end, some illustrations of applications to geometric problems will begiven with a brief study of



Introduction


I vector spaces,



I dimension,




Introduction


I vector spaces,



I dimension,




Introduction

We shall also give an elementary proof of

Fundamental Theorem of Algebra(FTA)using linear algebra that you learn in this course, thus achieving a kind of poeticjustice.For FTA has to be used in a non trivial fashion in Linear algebra.


Introduction

We shall also give an elementary proof ofFundamental Theorem of Algebra(FTA)

using linear algebra that you learn in this course, thus achieving a kind of poeticjustice.

For FTA has to be used in a non trivial fashion in Linear algebra.


Introduction

We shall also give an elementary proof ofFundamental Theorem of Algebra(FTA)

using linear algebra that you learn in this course, thus achieving a kind of poeticjustice.For FTA has to be used in a non trivial fashion in Linear algebra.


The Cartesian Coordinate Space

Today let us get familiar with our ‘working space’.

I Thanks to Rene Descartes, (1596-1650) of the fame cogito ergo sum

I The Cartesian coordinate space Rnis the totality of all ordered n-tuples

x := (x1, x2, . . . , xn) of real numbers xi ∈ R.I It is denoted by R× · · · ×R (n factors) or, in short, by Rn

.

I ‘n-vectors’.







.

I ‘n-vectors’.






x := (x1, x2, . . . , xn) of real numbers xi ∈ R.

I It is denoted by R× · · · ×R (n factors) or, in short, by Rn.

I ‘n-vectors’.







.

I ‘n-vectors’.







.

I ‘n-vectors’.







.

I ‘n-vectors’.



I For n = 1 you see that R1is nothing but the real line and a real number can

also be referred to as a 1-vector, simply denoted bywrite x , r , t, etc.

I Similarly for n = 2 and 3, we can avail of the familiar notation (x , y) ∈ R2

and (x , y , z) ∈ R3etc., respectively.



I For n = 1 you see that R1is nothing but the real line and a real number can

also be referred to as a 1-vector, simply denoted bywrite x , r , t, etc.

I Similarly for n = 2 and 3, we can avail of the familiar notation (x , y) ∈ R2

and (x , y , z) ∈ R3etc., respectively.


Cartesian Coordinate Space

I For a point x = (x1, . . . , xn) we refer to xi as the i th coordinate of x.

I For each i = 1, 2, . . . , n, let us denote byπi : Rn −→ R, πi ((x1, . . . , xn)) = πi (x1, . . . , xn) = xi .It is called the i th coordinate function or the i th coordinate projection.

I Observation A function f : A→ Rnis completely determined by the

functions fi = πi ◦ f called the i th component function of f , viz.,f (a) = (f1(a), . . . , fn(a)), ∀ a ∈ A.

I Thus one can expect that the study of functions f : A −→ Rncan be

effectively reduced to the study of functions fi : A −→ R, n of them innumber, and perhaps to the study of their inter-relations.

I Soon we shall see that this is in fact so.




I For each i = 1, 2, . . . , n, let us denote byπi : Rn −→ R, πi ((x1, . . . , xn)) = πi (x1, . . . , xn) = xi .

It is called the i th coordinate function or the i th coordinate projection.











































Cartesian Coordiante Space

I For n < m we can think of Rnas a subset of Rm

in various ways.

I One such standard way is to identify it with the set of points y of Rmwhose

last m − n coordinates are identically zero.

I Indeed, the ‘inclusion map’ (x1, . . . , xn) 7→ (x1, . . . , xn, 0, . . . 0) defines suchan identification.

I In particular, if n = 1, then we get the m coordinate axes of Rmfor

i = 1, 2, . . . ,m.

x 7→ (0, . . . , 0, x , 0, . . . , 0)

the variable x being put in the i th slot.




in various ways.





i = 1, 2, . . . ,m.

x 7→ (0, . . . , 0, x , 0, . . . , 0)





in various ways.





i = 1, 2, . . . ,m.

x 7→ (0, . . . , 0, x , 0, . . . , 0)





in various ways.





i = 1, 2, . . . ,m.

x 7→ (0, . . . , 0, x , 0, . . . , 0)




We presume that you are familiar with basic set theoretic concepts such as:

I domain co-domain of a function,

I image and inverse image of sets under functions,

I composition of functions,

I inverse of a function, etc..

A good reference for this is the I-volume of Apostol’s Calculus.



We presume that you are familiar with basic set theoretic concepts such as:

I domain co-domain of a function,

I image and inverse image of sets under functions,

I composition of functions,

I inverse of a function, etc..

A good reference for this is the I-volume of Apostol’s Calculus.



Example

I Letf (x , y) = (x2y , x + y); g(x , y) = (cos(x + y), sin(x/y)).

I Domain of f is the whole of R2, whereas that of g is R2 \ {(x , 0) : x ∈ R}.

I Neither f ◦ g nor g ◦ f is defined on the whole of R2.

I But if we restrict the domain of f to R2 \ {(x , y) : x + y = 0}, then g ◦ f isdefined.

I In that case we have, g ◦ f (x , y) = (cos(x2y + x + y), sin(x2y/x + y)).

I Can you determine the natural domain for f ◦ g and write down theexpression for it?



Example









Example









Example









Example









Example








Another Example

Example

Let N = {1, 2, . . . , n} be the set of first n natural numbers. Consider the cartesianproduct of n copies of N :

Nn = N × · · · × N

I How many elements does Nn has?

I Next consider the set S(n) of all sequences of length n with values in{1, 2, . . . , n}.

I Can you see any relation between Nn and S(n)? Further look at the setF (N,N) of all functions from N to N. Is there any relation between this setand the earlier two?

I Indeed, there are natural ways of getting one-to-one mappings of any one ofthese three sets into another.

I Let Σ(n) denote the subset of F (N,N) which are one-to-one. What does itcorrespond to in Nn and S(n)?


Another Example

Example


Nn = N × · · · × N







Another Example

Example


Nn = N × · · · × N







Another Example

Example


Nn = N × · · · × N







Another Example

Example


Nn = N × · · · × N







Another Example

Example


Nn = N × · · · × N







Some Open Exercises

I (i) Let ψ(x , y) = (x + y2, y + x2 + 2xy2 + y4). Is ψ a bijective map?

I (ii) (Here is an optional exercise which you can take up as a challenge. Donot consult the tutors for the solution, since they have been told not to helpyou with this problem. If you have solved it you may directly contact yourInstructor.)

Let f : R2 −→ R be a nonconstant continuous map. Then there exist−∞ ≤ a b. Hence or otherwise prove that(a) for every positive integer n, there exist at least n points which aremapped to the same point by f ;(b) if f is surjective then f −1(r) is infinite for all r ∈ R.(c) Finally construct a continuous map R2 −→ R to illustrate thatimprovement on this statement is not possible. Explicitly, find a continuous

function f : R2 −→ R such that f −1(−1) = {−1} and f −1(1) = {1}.


Some Open Exercises






Some Open Exercises



Let f : R2 −→ R be a nonconstant continuous map. Then there exist−∞ ≤ a b. Hence or otherwise prove that(a) for every positive integer n, there exist at least n points which aremapped to the same point by f ;(b) if f is surjective then f −1(r) is infinite for all r ∈ R.(c) Finally construct a continuous map R2 −→ R to illustrate thatimprovement on this statement is not possible. Explicitly, find a continuous



Some Open Exercises



Let f : R2 −→ R be a nonconstant continuous map. Then there exist−∞ ≤ a b. Hence or otherwise prove that(a) for every positive integer n, there exist at least n points which aremapped to the same point by f ;(b) if f is surjective then f −1(r) is infinite for all r ∈ R.(c) Finally construct a continuous map R2 −→ R to illustrate thatimprovement on this statement is not possible. Explicitly, find a continuous



Some Open Exercises



Let f : R2 −→ R be a nonconstant continuous map. Then there exist−∞ ≤ a b.

Hence or otherwise prove that(a) for every positive integer n, there exist at least n points which aremapped to the same point by f ;(b) if f is surjective then f −1(r) is infinite for all r ∈ R.(c) Finally construct a continuous map R2 −→ R to illustrate thatimprovement on this statement is not possible. Explicitly, find a continuous



Some Open Exercises



Let f : R2 −→ R be a nonconstant continuous map. Then there exist−∞ ≤ a b. Hence or otherwise prove that

(a) for every positive integer n, there exist at least n points which aremapped to the same point by f ;(b) if f is surjective then f −1(r) is infinite for all r ∈ R.(c) Finally construct a continuous map R2 −→ R to illustrate thatimprovement on this statement is not possible. Explicitly, find a continuous



Some Open Exercises



Let f : R2 −→ R be a nonconstant continuous map. Then there exist−∞ ≤ a b. Hence or otherwise prove that(a) for every positive integer n, there exist at least n points which aremapped to the same point by f ;

(b) if f is surjective then f −1(r) is infinite for all r ∈ R.(c) Finally construct a continuous map R2 −→ R to illustrate thatimprovement on this statement is not possible. Explicitly, find a continuous



Some Open Exercises






Algebraic Structure of Rn

Some of the algebraic structures of the real numbers induce similar structure onRn

.

The most important amongst them is the ‘sum’ or ‘addition.’ Given twon-vectors x = (x1, . . . , xn) and y = (y1, . . . , yn) we define their sum

x + y = (x1 + y1, . . . , xn + yn).

It is straightforward to verify that this addition obeys all the usual laws that theaddition of real numbers obey. In particular, observe that the zero element for thissum is the zero vector 0 = (0, . . . , 0) and the negative of (x1, . . . , xn) is(−x1, . . . ,−xn).What about ‘multiplication’?

xy := (x1y1, . . . , xnyn)??

Scalar multiplication: for each real number α and a vector x, we define

αx = (αx1, . . . , αxn).




.The most important amongst them is the ‘sum’ or ‘addition.’ Given twon-vectors x = (x1, . . . , xn) and y = (y1, . . . , yn) we define their sum

x + y = (x1 + y1, . . . , xn + yn).


xy := (x1y1, . . . , xnyn)??


αx = (αx1, . . . , αxn).





x + y = (x1 + y1, . . . , xn + yn).

It is straightforward to verify that this addition obeys all the usual laws that theaddition of real numbers obey. In particular, observe that the zero element for thissum is the zero vector 0 = (0, . . . , 0) and the negative of (x1, . . . , xn) is(−x1, . . . ,−xn).

What about ‘multiplication’?

xy := (x1y1, . . . , xnyn)??


αx = (αx1, . . . , αxn).





x + y = (x1 + y1, . . . , xn + yn).


xy := (x1y1, . . . , xnyn)??


αx = (αx1, . . . , αxn).





x + y = (x1 + y1, . . . , xn + yn).


xy := (x1y1, . . . , xnyn)??


αx = (αx1, . . . , αxn).


Algebraic Structure on Rn

It is

I 1. associative: α(βx) = (αβ)x;

I 2. distributes with the sum: α(x + y) = αx + αy, and (α + β)x = αx + βx;

I 3. and has an identity element 1x = x.

In particular, we also observe that, if n = 1, then these two operations coincidewith the usual addition and multiplication.



It is







It is







It is







It is






The Geometry of Rn

Before starting of with any sort of geometry or calculus, we need one moreimportant ingredient viz., the distance function on Rn

.

For any two pointsx, y ∈ Rn

as above, we define the distance between x and y by

d(x, y) =

√√√√ n∑i=1

(xi − yi )2.

Indeed, the distance function is closely related to the dot product

(x, y) 7→ x · y :=∑

i

xiyi .

One defines the norm function by

‖x‖ := d(x, 0) =

√√√√ n∑i

x2i =√

x · x.

Then it follows that d(x, y) = ‖x− y‖. (Such a distance function is called a linearmetric.


The Geometry of Rn


. For any two pointsx, y ∈ Rn


d(x, y) =

√√√√ n∑i=1

(xi − yi )2.


(x, y) 7→ x · y :=∑

i

xiyi .


‖x‖ := d(x, 0) =

√√√√ n∑i

x2i =√

x · x.



The Geometry of Rn




d(x, y) =

√√√√ n∑i=1

(xi − yi )2.


(x, y) 7→ x · y :=∑

i

xiyi .


‖x‖ := d(x, 0) =

√√√√ n∑i

x2i =√

x · x.



The Geometry of Rn




d(x, y) =

√√√√ n∑i=1

(xi − yi )2.


(x, y) 7→ x · y :=∑

i

xiyi .


‖x‖ := d(x, 0) =

√√√√ n∑i

x2i =√

x · x.



The Geometry of Rn




d(x, y) =

√√√√ n∑i=1

(xi − yi )2.


(x, y) 7→ x · y :=∑

i

xiyi .


‖x‖ := d(x, 0) =

√√√√ n∑i

x2i =√

x · x.



Euclidean Distance

You are already familiar with various properties of this function such as

I (d1) symmetry: d(x, y) = d(y, x);

I (d2) triangle inequality: d(x, y) ≤ d(x,w) + d(w, y);

I (d3) positivity: d(x, y) ≥ 0 and = 0 if and only if x = y.Moreover, we also have

I (d4) homogeneity: d(αx, αy) = |α|d(x, y).


Euclidean Distance







Euclidean Distance







Euclidean Distance




I (d3) positivity: d(x, y) ≥ 0 and = 0 if and only if x = y.

Moreover, we also have



Euclidean Distance







Geometric Aspects of Rn

All these properties are standard and can be verified easily.

For instance, in the proof of triangle inequality, we can use the famousCauchy-Schwarz inequality:

|x · y| ≤ ‖x‖‖y‖ (1)

The space Rntogether with the two algebraic structures and the distance

function d is called the n-dimensional euclidean space. This is going to be ourwork-space for quite some time.You are quite familiar with the case n = 2 and n = 3. Most of the concepts youhave met in these two special cases have appropriate and obvious generalizationsfor all n. However some caution is necessary, particularly while relying on ‘pictures’.



All these properties are standard and can be verified easily.For instance, in the proof of triangle inequality, we can use the famousCauchy-Schwarz inequality:

|x · y| ≤ ‖x‖‖y‖ (1)






|x · y| ≤ ‖x‖‖y‖ (1)






|x · y| ≤ ‖x‖‖y‖ (1)




A non-Eclidean Example

Define

δ(x, y) =n∑

i=1

|xi − yi |

andD(x, y) = max {|xi − yi | : 1 ≤ i ≤ n}.

Show that δ,D satisfy properties (d1), (d2), (d3) above in place of d .


Another non-Euclidean Example

On the set of 64 squares of a chess board define the distance d from one squareto another to be the least number of knight moves required. Check that thisdistance function satisfy (d1), (d2), (d3). Determine the diameter of the chessboard with respect to this distance where diameter of a space is the supremum ofall values of d .


Coins in a SquareHere is an example to show that it is not advisable to rely too much on intuitionwhen dealing with higher dimensions. Consider a square of side 4 units and placefour coins of unit radius one in each corner so as to touch two of the sides. Ofcourse each coin touches other two coins. Now place a coin at the center of thesquare so as to touch all the four coins.

Notice that if you are a 2-dimensional bug outside the box,, even if the box is notthere, you cannot go and touch the central coin.








Coins in a Square-continued

Do this in the 3-dimensional space with 8-balls of radius 1. What is the radius ofthe central ball? Can you touch it from outside?

Do this inside a n-dimensional cube of side 4 units with 2n n-dimensional balls ofunit radius. For n = 2, . . . , 9 the central ball which is kept touching all the balls inthe corner lies inside the cube. The surprise is that for n > 9 the central ballcannot fit inside the cube. Prove this by showing that the radius of the centralball is

√n − 1.


Coins in a Square-continued

Do this in the 3-dimensional space with 8-balls of radius 1. What is the radius ofthe central ball? Can you touch it from outside?Do this inside a n-dimensional cube of side 4 units with 2n n-dimensional balls ofunit radius. For n = 2, . . . , 9 the central ball which is kept touching all the balls inthe corner lies inside the cube. The surprise is that for n > 9 the central ballcannot fit inside the cube. Prove this by showing that the radius of the centralball is

√n − 1.


Linear Maps: Basic Definitions and Examples

An important class of functions from Rnto Rm

are the so called lineartransformations. Though this word makes sense even in a larger context, we shallfirst study them on euclidean spaces.

Definition

A function f : Rn −→ Rmis said to be a linear map if

f (αx + βy) = αf (x) + βf (y), for all scalars α, β ∈ R and all vectors x, y ∈ Rn.

Equivalently, this can also be said in the following way:1. f (x + y) = f (x) + f (y) (additivity).2. f (αx) = αf (x)(homogeneity).


Linear Maps: Basic Definitions and Examples

An important class of functions from Rnto Rm

are the so called lineartransformations. Though this word makes sense even in a larger context, we shallfirst study them on euclidean spaces.

Definition

A function f : Rn −→ Rmis said to be a linear map if

f (αx + βy) = αf (x) + βf (y), for all scalars α, β ∈ R and all vectors x, y ∈ Rn.

Equivalently, this can also be said in the following way:1. f (x + y) = f (x) + f (y) (additivity).2. f (αx) = αf (x)(homogeneity).


Linear transformations

Exercise: Show that if f is a linear map then f (∑k

i=1 αixi ) =∑k

i=1 αi f (xi ).

Example

Examples of linear maps are easy to produce.

I The projection maps πi , the inclusion maps etc. which we have just seen arethe standard examples.

I The identity map, the zero map and multiplication by a fixed scalar α arealso linear.

I More generally, taking dot product with a fixed vector is a linear mapRn −→ R.

I Indeed, we shall soon see that any linear map Rn −→ R corresponds totaking dot product with a unique vector in Rn

.

I Finally, a map f : Rn −→ Rmis linear if and only if its component maps

fi : Rn −→ R are all linear. In this way, we will be able to determine all linearmaps Rn −→ Rm

.




i=1 αixi ) =∑k

i=1 αi f (xi ).

Example






.



.




i=1 αixi ) =∑k

i=1 αi f (xi ).

Example






.



.




i=1 αixi ) =∑k

i=1 αi f (xi ).

Example






.



.




i=1 αixi ) =∑k

i=1 αi f (xi ).

Example






.



.




i=1 αixi ) =∑k

i=1 αi f (xi ).

Example






.



.



ExampleI Distance travelled is a linear function of time when velocity is constant.

I So is the voltage as a function of resistance when the current is constant.

I The logarithm of the change in concentration in any first order chemicalreaction is a linear function of time.

I Non examples are even easier to produce. On R, |x |, xn(n ≥ 2), sin x etc. areeasily seen to be non linear maps.

I Gravitational force between two bodies is not a linear function of the distancebetween the two bodies.































Exercises:

I (i) Show that the projection on a line L passing through the origin defines a

linear map of R2to R2

and its image is equal to L.

I (ii) Show that rotation through a fixed angle θ is a linear map, R2 −→ R2.

I (iii) By a rigid motion of Rnwe mean a map f : Rn −→ Rn

such that

d(f (x), f (y)) = d(x , y). Show that a rigid motion of R3which fixes the

origin is a linear map.



Exercises:






such that





Exercises:






such that





Exercises:






such that




The Structure of Linear Maps

I Let L(n,m) denote the totality of all the linear maps from Rnto Rm

.

I The algebraic structure on Rminduces similar structure on L(n,m) also:

For α ∈ R, f , g ∈ L(n,m) we can define αf and f + g by the formula,

(αf )(x) = αf (x); (f + g)(x) = f (x) + g(x).

Moreover, given f ∈ L(n,m), g ∈ L(m, `) we can define the compositefunction g ◦ f and it is not difficult to verify that this is a linear map; thusg ◦ f ∈ L(n, `).However, we should be careful, while considering such new operations.For instance, if f , g ∈ L(n, 1) then we can define fg : Rn −→ R by theformula (fg)(x) = f (x)g(x), ∀ x ∈ Rn

.But can we say that fg ∈ L(n, 1)?




.



(αf )(x) = αf (x); (f + g)(x) = f (x) + g(x).






.



(αf )(x) = αf (x); (f + g)(x) = f (x) + g(x).






.



(αf )(x) = αf (x); (f + g)(x) = f (x) + g(x).

Moreover, given f ∈ L(n,m), g ∈ L(m, `) we can define the compositefunction g ◦ f and it is not difficult to verify that this is a linear map; thusg ◦ f ∈ L(n, `).

However, we should be careful, while considering such new operations.For instance, if f , g ∈ L(n, 1) then we can define fg : Rn −→ R by theformula (fg)(x) = f (x)g(x), ∀ x ∈ Rn





.



(αf )(x) = αf (x); (f + g)(x) = f (x) + g(x).

Moreover, given f ∈ L(n,m), g ∈ L(m, `) we can define the compositefunction g ◦ f and it is not difficult to verify that this is a linear map; thusg ◦ f ∈ L(n, `).However, we should be careful, while considering such new operations.

For instance, if f , g ∈ L(n, 1) then we can define fg : Rn −→ R by theformula (fg)(x) = f (x)g(x), ∀ x ∈ Rn





.



(αf )(x) = αf (x); (f + g)(x) = f (x) + g(x).


.

But can we say that fg ∈ L(n, 1)?




.



(αf )(x) = αf (x); (f + g)(x) = f (x) + g(x).




The Structure of Linear MapsThe structure of linear maps is indeed, very simple.

First of all consider the elements

zi = (0, . . . , 0, 1, 0, . . . , 0),

wherein we have put 1 in the i th-place.These elements, for i = 1, 2, . . . , n are called the standard basis elements for Rn

.It is easily seen that every element x := (x1, . . . , xn) ∈ Rn

can be written in aunique way, as a linear combination of z1, . . . zn, viz.,

x =n∑

i=1

xizi . (2)

Now given a linear map f : Rn −→ Rm, the linearity of f implies that

f (x) =∑

i

xi f (zi ). (3)

This means that to know the effect of f on any arbitrary element we have only toknow the n values f (z1), . . . , f (zn) ∈ Rm

.


The Structure of Linear MapsThe structure of linear maps is indeed, very simple.First of all consider the elements

zi = (0, . . . , 0, 1, 0, . . . , 0),

wherein we have put 1 in the i th-place.

These elements, for i = 1, 2, . . . , n are called the standard basis elements for Rn.

It is easily seen that every element x := (x1, . . . , xn) ∈ Rncan be written in a

unique way, as a linear combination of z1, . . . zn, viz.,

x =n∑

i=1

xizi . (2)


f (x) =∑

i

xi f (zi ). (3)


.



zi = (0, . . . , 0, 1, 0, . . . , 0),


.

It is easily seen that every element x := (x1, . . . , xn) ∈ Rncan be written in a

unique way, as a linear combination of z1, . . . zn, viz.,

x =n∑

i=1

xizi . (2)


f (x) =∑

i

xi f (zi ). (3)


.



zi = (0, . . . , 0, 1, 0, . . . , 0),




x =n∑

i=1

xizi . (2)


f (x) =∑

i

xi f (zi ). (3)


.



zi = (0, . . . , 0, 1, 0, . . . , 0),




x =n∑

i=1

xizi . (2)


f (x) =∑

i

xi f (zi ). (3)


.



zi = (0, . . . , 0, 1, 0, . . . , 0),




x =n∑

i=1

xizi . (2)


f (x) =∑

i

xi f (zi ). (3)


.ARS (IITB) MA106-Linear Algebra January 19, 2011 29 / 59


Conversely, suppose we are given any n vectors v1, . . . , vm ∈ Rm, in a particular

order.

Then we can define a (unique) linear map f which has its values on zi equal to vi

(taken in that order) by the formula (4).

f (x) =∑

i

xi f (zi ) :=∑

i

xivi . (4)



Conversely, suppose we are given any n vectors v1, . . . , vm ∈ Rm, in a particular

order.Then we can define a (unique) linear map f which has its values on zi equal to vi

(taken in that order) by the formula (4).

f (x) =∑

i

xi f (zi ) :=∑

i

xivi . (4)



Example

When m = 1 = n, a linear map f : R −→ R is determined by its value on z1 = 1.

In fact, we havef (r) = rf (1).

Conversely, multiplication by each fixed real number is a linear map. Therefore theonly linear maps R −→ R are those obtained by multiplication by a fixed scalar.



Example

When m = 1 = n, a linear map f : R −→ R is determined by its value on z1 = 1.In fact, we have

f (r) = rf (1).




Example

When m = 1 = n, a linear map f : R −→ R is determined by its value on z1 = 1.In fact, we have

f (r) = rf (1).




Example

Consider the case n ≥ 2, and m = 1.

Put u = (f (z1), . . . , f (zn)). Then we see that f (x) = u · x = x · u.On the other hand, we know that taking dot product with a fixed vector is a linearmap.Thus a linear map Rn −→ R is nothing but taking dot product with a fixedvector in Rn

.



Example

Consider the case n ≥ 2, and m = 1.Put u = (f (z1), . . . , f (zn)). Then we see that f (x) = u · x = x · u.

On the other hand, we know that taking dot product with a fixed vector is a linearmap.Thus a linear map Rn −→ R is nothing but taking dot product with a fixedvector in Rn

.



Example

Consider the case n ≥ 2, and m = 1.Put u = (f (z1), . . . , f (zn)). Then we see that f (x) = u · x = x · u.On the other hand, we know that taking dot product with a fixed vector is a linearmap.

Thus a linear map Rn −→ R is nothing but taking dot product with a fixedvector in Rn

.



Example

Consider the case n ≥ 2, and m = 1.Put u = (f (z1), . . . , f (zn)). Then we see that f (x) = u · x = x · u.On the other hand, we know that taking dot product with a fixed vector is a linearmap.Thus a linear map Rn −→ R is nothing but taking dot product with a fixedvector in Rn

.



Example

Consider the problem of solving a system of m linear equations in n variables:

a11x1 + a12x2 + · · · + a1nxn = b1

a21x1 + a22x2 + · · · + a2nxn = b2

· · · · · · · · ·am1x1 + am2x2 + · · · + amnxn = bm

(5)

A solution (x1, . . . , xn) can be thought of as an element of the euclidean space Rn.

Then the set of all solutions which satisfy the j th equation above is nothing butsome hyper plane Pj in Rn

.Solving the system means nothing but finding the intersection of all these planesviz., P1 ∩ · · · ∩ Pm. On the other hand, we also can think of the lhs of each ofthese equation as a map Ti : Rn −→ R.Then each Ti is easily seen to be a linearmap.Together, these m linear functions define one linear function T : Rn −→ Rm

such that T = (T1, . . . ,Tm). Solving the system of equations is now the same asdetermining points x ∈ Rn

such that T (x) = b(= (b1, . . . , bm)).



Example


a11x1 + a12x2 + · · · + a1nxn = b1

a21x1 + a22x2 + · · · + a2nxn = b2

· · · · · · · · ·am1x1 + am2x2 + · · · + amnxn = bm

(5)





such that T (x) = b(= (b1, . . . , bm)).



Example


a11x1 + a12x2 + · · · + a1nxn = b1

a21x1 + a22x2 + · · · + a2nxn = b2

· · · · · · · · ·am1x1 + am2x2 + · · · + amnxn = bm

(5)





such that T (x) = b(= (b1, . . . , bm)).



Example


a11x1 + a12x2 + · · · + a1nxn = b1

a21x1 + a22x2 + · · · + a2nxn = b2

· · · · · · · · ·am1x1 + am2x2 + · · · + amnxn = bm

(5)



.

Solving the system means nothing but finding the intersection of all these planesviz., P1 ∩ · · · ∩ Pm. On the other hand, we also can think of the lhs of each ofthese equation as a map Ti : Rn −→ R.Then each Ti is easily seen to be a linearmap.Together, these m linear functions define one linear function T : Rn −→ Rm


such that T (x) = b(= (b1, . . . , bm)).



Example


a11x1 + a12x2 + · · · + a1nxn = b1

a21x1 + a22x2 + · · · + a2nxn = b2

· · · · · · · · ·am1x1 + am2x2 + · · · + amnxn = bm

(5)



.Solving the system means nothing but finding the intersection of all these planesviz., P1 ∩ · · · ∩ Pm.

On the other hand, we also can think of the lhs of each ofthese equation as a map Ti : Rn −→ R.Then each Ti is easily seen to be a linearmap.Together, these m linear functions define one linear function T : Rn −→ Rm


such that T (x) = b(= (b1, . . . , bm)).



Example


a11x1 + a12x2 + · · · + a1nxn = b1

a21x1 + a22x2 + · · · + a2nxn = b2

· · · · · · · · ·am1x1 + am2x2 + · · · + amnxn = bm

(5)



.Solving the system means nothing but finding the intersection of all these planesviz., P1 ∩ · · · ∩ Pm. On the other hand, we also can think of the lhs of each ofthese equation as a map Ti : Rn −→ R.

Then each Ti is easily seen to be a linearmap.Together, these m linear functions define one linear function T : Rn −→ Rm


such that T (x) = b(= (b1, . . . , bm)).



Example


a11x1 + a12x2 + · · · + a1nxn = b1

a21x1 + a22x2 + · · · + a2nxn = b2

· · · · · · · · ·am1x1 + am2x2 + · · · + amnxn = bm

(5)



.Solving the system means nothing but finding the intersection of all these planesviz., P1 ∩ · · · ∩ Pm. On the other hand, we also can think of the lhs of each ofthese equation as a map Ti : Rn −→ R.Then each Ti is easily seen to be a linearmap.

Together, these m linear functions define one linear function T : Rn −→ Rm


such that T (x) = b(= (b1, . . . , bm)).



Example


a11x1 + a12x2 + · · · + a1nxn = b1

a21x1 + a22x2 + · · · + a2nxn = b2

· · · · · · · · ·am1x1 + am2x2 + · · · + amnxn = bm

(5)




such that T = (T1, . . . ,Tm).

Solving the system of equations is now the same asdetermining points x ∈ Rn

such that T (x) = b(= (b1, . . . , bm)).



Example


a11x1 + a12x2 + · · · + a1nxn = b1

a21x1 + a22x2 + · · · + a2nxn = b2

· · · · · · · · ·am1x1 + am2x2 + · · · + amnxn = bm

(5)





such that T (x) = b(= (b1, . . . , bm)).


Matrix Representation

These ideas can be put in a very concrete way in terms of matrices.

We have been writing elements of Rnin the form (x1, . . . , xn).

This is certainly a good practice if all the scripts are written along horizontal lines.However, what would happen if we use Chinese or Japanese?We would be writing them in columns! Certainly this does not cause any changein the mathematical aspects of the objects that are under study.We shall borrow this idea from the Chinese and for the sake of notationalconformity later on, we shall now itself agree to write any element of Rn

as acolumn of real numbers:

x1

x2

.

.

.xn



These ideas can be put in a very concrete way in terms of matrices.We have been writing elements of Rn

in the form (x1, . . . , xn).

This is certainly a good practice if all the scripts are written along horizontal lines.However, what would happen if we use Chinese or Japanese?We would be writing them in columns! Certainly this does not cause any changein the mathematical aspects of the objects that are under study.We shall borrow this idea from the Chinese and for the sake of notationalconformity later on, we shall now itself agree to write any element of Rn


x1

x2

.

.

.xn




in the form (x1, . . . , xn).This is certainly a good practice if all the scripts are written along horizontal lines.

However, what would happen if we use Chinese or Japanese?We would be writing them in columns! Certainly this does not cause any changein the mathematical aspects of the objects that are under study.We shall borrow this idea from the Chinese and for the sake of notationalconformity later on, we shall now itself agree to write any element of Rn


x1

x2

.

.

.xn




in the form (x1, . . . , xn).This is certainly a good practice if all the scripts are written along horizontal lines.However, what would happen if we use Chinese or Japanese?

We would be writing them in columns! Certainly this does not cause any changein the mathematical aspects of the objects that are under study.We shall borrow this idea from the Chinese and for the sake of notationalconformity later on, we shall now itself agree to write any element of Rn


x1

x2

.

.

.xn




in the form (x1, . . . , xn).This is certainly a good practice if all the scripts are written along horizontal lines.However, what would happen if we use Chinese or Japanese?We would be writing them in columns! Certainly this does not cause any changein the mathematical aspects of the objects that are under study.

We shall borrow this idea from the Chinese and for the sake of notationalconformity later on, we shall now itself agree to write any element of Rn


x1

x2

.

.

.xn




in the form (x1, . . . , xn).This is certainly a good practice if all the scripts are written along horizontal lines.However, what would happen if we use Chinese or Japanese?We would be writing them in columns! Certainly this does not cause any changein the mathematical aspects of the objects that are under study.We shall borrow this idea from the Chinese and for the sake of notationalconformity later on, we shall now itself agree to write any element of Rn


x1

x2

.

.

.xn


Matrix RepresentationAs we shall see soon, this convention will bring better conformity in notationrather than confusion.

However, we also see that this practice will also cost us a lot of space.So, as space saving device, let us also have another convention viz., (x1, . . . , xn)T

(read ‘x1, dotdotdot xn transpose’) to represent the above column. (We shall alsouse the notation (x1, . . . , xn)t .)The net effect is that from now onwards we will write an element of Rn

in theform (x1, . . . , xn)T and call it a column vector, in addition to our earlierconvention of referring to it as a row vector as and when it suits us. Wait amoment to see why all this fuss.In particular, we shall now onward use the notation

ei := zti := zT

i :=

0...1...0

.


Matrix RepresentationAs we shall see soon, this convention will bring better conformity in notationrather than confusion.However, we also see that this practice will also cost us a lot of space.

So, as space saving device, let us also have another convention viz., (x1, . . . , xn)T



ei := zti := zT

i :=

0...1...0

.


Matrix RepresentationAs we shall see soon, this convention will bring better conformity in notationrather than confusion.However, we also see that this practice will also cost us a lot of space.So, as space saving device, let us also have another convention viz., (x1, . . . , xn)T

(read ‘x1, dotdotdot xn transpose’) to represent the above column.

(We shall alsouse the notation (x1, . . . , xn)t .)The net effect is that from now onwards we will write an element of Rn


ei := zti := zT

i :=

0...1...0

.



(read ‘x1, dotdotdot xn transpose’) to represent the above column. (We shall alsouse the notation (x1, . . . , xn)t .)

The net effect is that from now onwards we will write an element of Rnin the

form (x1, . . . , xn)T and call it a column vector, in addition to our earlierconvention of referring to it as a row vector as and when it suits us. Wait amoment to see why all this fuss.In particular, we shall now onward use the notation

ei := zti := zT

i :=

0...1...0

.




in theform (x1, . . . , xn)T and call it a column vector, in addition to our earlierconvention of referring to it as a row vector as and when it suits us. Wait amoment to see why all this fuss.

In particular, we shall now onward use the notation

ei := zti := zT

i :=

0...1...0

.





ei := zti := zT

i :=

0...1...0

.



Given a linear map f : Rn −→ Rmwe have n column vectors (each of size m)

viz., f (e1), . . . , f (en).

We place them side by side to obtain an array of real numbers arranged in m rowsand n columns. For instance, if f (ej) = (f1j , f2j , . . . fmj)

T , then the array we obtainlooks like:

Mf :=

f11 f12 . . . f1n

f21 f22 . . . f2n

......

......

fm1 fm2 . . . fmn

This array is called a matrix with m rows and n columns and with real entriesfij ∈ R in the (i , j)th place.




viz., f (e1), . . . , f (en).We place them side by side to obtain an array of real numbers arranged in m rowsand n columns.

For instance, if f (ej) = (f1j , f2j , . . . fmj)T , then the array we obtain

looks like:

Mf :=

f11 f12 . . . f1n

f21 f22 . . . f2n

......

......

fm1 fm2 . . . fmn





viz., f (e1), . . . , f (en).We place them side by side to obtain an array of real numbers arranged in m rowsand n columns. For instance, if f (ej) = (f1j , f2j , . . . fmj)


Mf :=

f11 f12 . . . f1n

f21 f22 . . . f2n

......

......

fm1 fm2 . . . fmn





viz., f (e1), . . . , f (en).We place them side by side to obtain an array of real numbers arranged in m rowsand n columns. For instance, if f (ej) = (f1j , f2j , . . . fmj)


Mf :=

f11 f12 . . . f1n

f21 f22 . . . f2n

......

......

fm1 fm2 . . . fmn




We say the matrix Mf is of size m × n or simply that it is a m × n matrix. Wealso use the simplified notation ((fij)) for this matrix.

Clearly, two matrices like this will be treated as equal if and only if their sizes arethe same and the entries in the corresponding places are equal.In particular, if we have m = 1 we get row matrices which are the same as rowvectors. Similarly, if n = 1 we get column matrices which are the same as columnvectors.



We say the matrix Mf is of size m × n or simply that it is a m × n matrix. Wealso use the simplified notation ((fij)) for this matrix.Clearly, two matrices like this will be treated as equal if and only if their sizes arethe same and the entries in the corresponding places are equal.

In particular, if we have m = 1 we get row matrices which are the same as rowvectors. Similarly, if n = 1 we get column matrices which are the same as columnvectors.



We say the matrix Mf is of size m × n or simply that it is a m × n matrix. Wealso use the simplified notation ((fij)) for this matrix.Clearly, two matrices like this will be treated as equal if and only if their sizes arethe same and the entries in the corresponding places are equal.In particular, if we have m = 1 we get row matrices which are the same as rowvectors. Similarly, if n = 1 we get column matrices which are the same as columnvectors.



From what we have seen earlier, it follows that the correspondence

f 7→ Mf

is one-one from L(n,m) onto Mm,n, the space of all m × n matrices with realentries.

This is called the matrix representation of linear maps.The correspondence M : L(n,m) −→ Mm,n itself has the linearity property, viz.,

M(f +g) =Mf +Mg ; Mαf = αMf .

Later on we shall see various properties and usefulness of this representation.




f 7→ Mf

is one-one from L(n,m) onto Mm,n, the space of all m × n matrices with realentries. This is called the matrix representation of linear maps.

The correspondence M : L(n,m) −→ Mm,n itself has the linearity property, viz.,






f 7→ Mf

is one-one from L(n,m) onto Mm,n, the space of all m × n matrices with realentries. This is called the matrix representation of linear maps.The correspondence M : L(n,m) −→ Mm,n itself has the linearity property, viz.,





Example

Let us write down the matrix representation for some simple maps.

I (i) Since the identity map takes ei to ei it follows that MId = In whereI = ((δij)) is an n × n matrix with δij = 1 if i = j and = 0 otherwise.

In =

1

1...

1

= diag(1, 1, . . . , 1).

(We shall denote it by In, the suffix n to indicate the size of the matrix if thisis not clear from the context.)



Example

Let us write down the matrix representation for some simple maps.

I (i) Since the identity map takes ei to ei it follows that MId = In whereI = ((δij)) is an n × n matrix with δij = 1 if i = j and = 0 otherwise.

In =

1

1...

1

= diag(1, 1, . . . , 1).

(We shall denote it by In, the suffix n to indicate the size of the matrix if thisis not clear from the context.)



Example

I (ii) Clearly then M−Id = −I .

I (iii) The linear map T : R2 −→ R2which interchanges the two coordinates

is represented by [0 11 0

](6)

I (iv) Consider the scalar multiplication by α ∈ R on Rn. The matrix of this

linear map is the diagonal matrix diag(α, · · · , α).



Example




](6)





Example




](6)




Rotation through an angle

Example

The rotation through an angle θ about the origin in the plane is represented by[cos θ − sin θsin θ cos θ

]. (7)

First of all why it is a linear map?In order to see this, we must understand how the sum of two vectors in the planeis defined geometrically, viz., by the parallelogram.



Example


]. (7)

First of all why it is a linear map?

In order to see this, we must understand how the sum of two vectors in the planeis defined geometrically, viz., by the parallelogram.



Example


]. (7)

First of all why it is a linear map?In order to see this, we must understand how the sum of two vectors in the planeis defined geometrically, viz., by the parallelogram.



1

Q

P

O

P

x +xx

2y+y1y

yy1

2

1 2 1 x2

2

Now use the law of congruent triangles to see that under a rotation the entireparallelogram gets rotated.



1

Q

P

O

P

x +xx

2y+y1y

yy1

2

1 2 1 x2

2

Now use the law of congruent triangles to see that under a rotation the entireparallelogram gets rotated.


Rotation through an angleThe proof for the law for scaling is similar and indeed simpler.

1

1 +z

z

2z

z2

z2

1z

+ z21

z

a

a

a( )


Reflection in a line through origin

Example

Let us right down the matrix representation for the reflection through a line Lψpassing through the origin in the plane and making an angle ψ with the positivereal axis, 0 ≤ ψ < π. Let us denote this by Rψ.

Clearly e1 is mapped to a unit vector which makes an angle 2ψ with the positivereal axis whereas e2 is mapped to one which makes an angle 2ψ − π/2.Hence thematrix of reflection Rψ is [

cos 2ψ sin 2ψsin 2ψ − cos 2ψ

]. (8)

Thus you can see that reflection through the line Lψ is the composite of reflectionin the x−axis followed by the rotation through the angle 2ψ.If we denote the rotation through an angle θ by ρθ then we have

Rψ = ρ2ψ ◦ R0 (9)



Example

Let us right down the matrix representation for the reflection through a line Lψpassing through the origin in the plane and making an angle ψ with the positivereal axis, 0 ≤ ψ < π. Let us denote this by Rψ.Clearly e1 is mapped to a unit vector which makes an angle 2ψ with the positivereal axis whereas e2 is mapped to one which makes an angle 2ψ − π/2.

Hence thematrix of reflection Rψ is [


]. (8)


Rψ = ρ2ψ ◦ R0 (9)



Example

Let us right down the matrix representation for the reflection through a line Lψpassing through the origin in the plane and making an angle ψ with the positivereal axis, 0 ≤ ψ < π. Let us denote this by Rψ.Clearly e1 is mapped to a unit vector which makes an angle 2ψ with the positivereal axis whereas e2 is mapped to one which makes an angle 2ψ − π/2.Hence thematrix of reflection Rψ is [


]. (8)


Rψ = ρ2ψ ◦ R0 (9)



Example



]. (8)

Thus you can see that reflection through the line Lψ is the composite of reflectionin the x−axis followed by the rotation through the angle 2ψ.

If we denote the rotation through an angle θ by ρθ then we have

Rψ = ρ2ψ ◦ R0 (9)



Example



]. (8)


Rψ = ρ2ψ ◦ R0 (9)


Operations on Matrices

The set of all matrices of a fixed size say m × n, and with entries in the field R isdenoted by Mm,n(R) or by simply Mm,n when the field of entries is understood.

Certain algebraic operations of the field R induce similar operations on this spacealso, in an entry-wise fashion. Given two matrices A = ((aij)),B = ((bij)) ∈ Mm,n,we define A + B ∈ Mm,n to be the matrix whose (i , j)th entry is aij + bij , i.e.,

((aij)) + ((bij)) = ((aij + bij)).

Given a scalar α ∈ R, we can define the matrix αA ∈ Mm,n to be the matrixwhose (i , j)th entry is αaij .It is easily checked that the matrix O with all its entries zero is the zero elementfor the sum operation, the matrix ((−aij)) is the negative of A etc.Also verify that scalar multiplication distributes over the addition.



The set of all matrices of a fixed size say m × n, and with entries in the field R isdenoted by Mm,n(R) or by simply Mm,n when the field of entries is understood.Certain algebraic operations of the field R induce similar operations on this spacealso, in an entry-wise fashion. Given two matrices A = ((aij)),B = ((bij)) ∈ Mm,n,we define A + B ∈ Mm,n to be the matrix whose (i , j)th entry is aij + bij , i.e.,

((aij)) + ((bij)) = ((aij + bij)).





((aij)) + ((bij)) = ((aij + bij)).

Given a scalar α ∈ R, we can define the matrix αA ∈ Mm,n to be the matrixwhose (i , j)th entry is αaij .

It is easily checked that the matrix O with all its entries zero is the zero elementfor the sum operation, the matrix ((−aij)) is the negative of A etc.Also verify that scalar multiplication distributes over the addition.




((aij)) + ((bij)) = ((aij + bij)).

Given a scalar α ∈ R, we can define the matrix αA ∈ Mm,n to be the matrixwhose (i , j)th entry is αaij .It is easily checked that the matrix O with all its entries zero is the zero elementfor the sum operation, the matrix ((−aij)) is the negative of A etc.

Also verify that scalar multiplication distributes over the addition.




((aij)) + ((bij)) = ((aij + bij)).




Earlier we have identified Rnwith the set of all column vectors and hence with

M1,n(R). We now observe that this identification preserves the algebraic structureas well. The same is true of the identification of Rm

with Mm,1(R).The ‘transpose’ operation that we introduced earlier just for the convenience ofprinting acquires a wider meaning now. It identifies the space M1,n with Mn,1.

We observe that this operation can be easily extended for all matrices as follows:AT is the n ×m matrix whose (i , j)th entry is aji . This way we get the transposeoperation: T : Mm,n −→ Mn,m, by the formula A 7→ AT .(The functional notation would have been T (A), but we would like to stick toclassical notation. Other notations in practice are At and tA etc..) It is easilyverified that

(αA + βB)T = αAT + βBT

where α, β are scalars.This property of the transpose can be referred to as the linearity of the transpose.





with Mm,1(R).The ‘transpose’ operation that we introduced earlier just for the convenience ofprinting acquires a wider meaning now. It identifies the space M1,n with Mn,1.We observe that this operation can be easily extended for all matrices as follows:AT is the n ×m matrix whose (i , j)th entry is aji . This way we get the transposeoperation: T : Mm,n −→ Mn,m, by the formula A 7→ AT .

(The functional notation would have been T (A), but we would like to stick toclassical notation. Other notations in practice are At and tA etc..) It is easilyverified that







with Mm,1(R).The ‘transpose’ operation that we introduced earlier just for the convenience ofprinting acquires a wider meaning now. It identifies the space M1,n with Mn,1.We observe that this operation can be easily extended for all matrices as follows:AT is the n ×m matrix whose (i , j)th entry is aji . This way we get the transposeoperation: T : Mm,n −→ Mn,m, by the formula A 7→ AT .(The functional notation would have been T (A), but we would like to stick toclassical notation. Other notations in practice are At and tA etc..) It is easilyverified that




Operations on MatricesWhen we have functions f : Rn −→ Rm

, g : Rm −→ Rlwe can talk about the

composite g ◦ f .

We have also seen that if f and g are linear then so is g ◦ f .A natural question to ask is: what is the relation between the matrixrepresentations Mf =: A,Mg =: B and Mg◦f ?To determine this, we have to compute the value of (g ◦ f )(ei ).

(g ◦ f )(ej) = g(∑

i aijei )

=∑

i

aijg(ei )

=∑

i

aij

(∑k

bkiek

)

=∑

k

(∑i

bkiaij

)ek .

Thus if Mg◦f = C = ((ckj)) then we have

ckj =m∑

i=1

bkiaij (10)




composite g ◦ f . We have also seen that if f and g are linear then so is g ◦ f .

A natural question to ask is: what is the relation between the matrixrepresentations Mf =: A,Mg =: B and Mg◦f ?To determine this, we have to compute the value of (g ◦ f )(ei ).

(g ◦ f )(ej) = g(∑

i aijei )

=∑

i

aijg(ei )

=∑

i

aij

(∑k

bkiek

)

=∑

k

(∑i

bkiaij

)ek .


ckj =m∑

i=1

bkiaij (10)




composite g ◦ f . We have also seen that if f and g are linear then so is g ◦ f .A natural question to ask is: what is the relation between the matrixrepresentations Mf =: A,Mg =: B and Mg◦f ?

To determine this, we have to compute the value of (g ◦ f )(ei ).

(g ◦ f )(ej) = g(∑

i aijei )

=∑

i

aijg(ei )

=∑

i

aij

(∑k

bkiek

)

=∑

k

(∑i

bkiaij

)ek .


ckj =m∑

i=1

bkiaij (10)




composite g ◦ f . We have also seen that if f and g are linear then so is g ◦ f .A natural question to ask is: what is the relation between the matrixrepresentations Mf =: A,Mg =: B and Mg◦f ?To determine this, we have to compute the value of (g ◦ f )(ei ).

(g ◦ f )(ej) = g(∑

i aijei )

=∑

i

aijg(ei )

=∑

i

aij

(∑k

bkiek

)

=∑

k

(∑i

bkiaij

)ek .


ckj =m∑

i=1

bkiaij (10)




composite g ◦ f . We have also seen that if f and g are linear then so is g ◦ f .A natural question to ask is: what is the relation between the matrixrepresentations Mf =: A,Mg =: B and Mg◦f ?To determine this, we have to compute the value of (g ◦ f )(ei ).

(g ◦ f )(ej) = g(∑

i aijei )

=∑

i

aijg(ei )

=∑

i

aij

(∑k

bkiek

)

=∑

k

(∑i

bkiaij

)ek .


ckj =m∑

i=1

bkiaij (10)



Therefore we shall define the composite of B and A to be the `× n matrix C :

C = BA

where C = ((ckj)) with ckj given by (10).It is of prime importance to note that number of columns in B is the same as thenumber of rows in A in order that BA is defined. We shall implicitly assume thiswhile writing BA. It also follows that the number of rows and columns of BC arerespectively equal to that of B and A.



Basic Properties

I 1. Associativity: If A,B,C are matrices such that AB and BC are definedthen (AB)C and A(BC ) are also defined and are equal, A(BC ) = (AB)C .

I 2. Right and Left Distributivity: If A(B + C ) = AB + AC provided at leastone side is (and hence both sides are) defined. Similarly, we have(B + C )A = BA + CA, etc..Let In denote the n × n matrix whose (ij)th entry is equal to δij . (Recallδij = 1 if i = j and 0 otherwise. These are called Kronecker symbols.) Thematrix In is called the identity matrix.

I 3. Mutliplicative Identity For any A ∈ Mm,n,B ∈ Mn,k we have AIn = A andInB = B.

I 4.(AB)T = BT AT .

I 5. Let f : Rn −→ Rm, g : Rm −→ Rl

be linear maps. Then we have,Mg◦f =MgMf .



Basic Properties


I 2. Right and Left Distributivity: If A(B + C ) = AB + AC provided at leastone side is (and hence both sides are) defined. Similarly, we have(B + C )A = BA + CA, etc..

Let In denote the n × n matrix whose (ij)th entry is equal to δij . (Recallδij = 1 if i = j and 0 otherwise. These are called Kronecker symbols.) Thematrix In is called the identity matrix.


I 4.(AB)T = BT AT .





Basic Properties




I 4.(AB)T = BT AT .





Basic Properties




I 4.(AB)T = BT AT .





Basic Properties




I 4.(AB)T = BT AT .





Basic Properties




I 4.(AB)T = BT AT .





Remark

Consider the space M2,2. By writing the entries of successive rows one afteranother in a single row, we see that this is in one-to-one correspondence with thetotality of ordered 4-tuples of real numbers. Moreover this correspondencepreserves the two algebraic operations of taking sum and scalar product. Thus we

can actually think of the space of 2× 2 matrices with real entries as R4. Exactly

in a similar fashion, Mm,n can be thought of as Rm×n. Calculus on this space will

involve ‘mn variables’. Soon, we shall see the true meaning of the phrase ‘numberof variables’, the easiest way to understand the meaning of ‘dimension.’


Invertible transformations Matrices

DefinitionAny function f : X −→ Y is said to be invertible, if there exists g : Y −→ X suchthat g ◦ f = IdX and f ◦ g = IdY .

Here for any set Z , IdZ denotes the identity map of the set Z . The inverse ofa function if it exists is unique and is denoted by f −1.

DefinitionLet A,B be any two matrices We say B is a left-inverse of A and A is aright-inverse of B if we have BA = I . We say A is invertible if there is a leftinverse as well as a right inverse for A.













LemmaIf C ,B are respectively a left-inverse and a right-inverse of A then C = B. In thiscase, if C ′ and B ′ are any left-inverse and right-inverse of A, then we haveC ′ = C ,B ′ = B,

Proof: Let C ∈ Mp,q,A ∈ Mq,r ,B ∈ Mr ,s be such that CA = I , and AB = I .(This already implies p = r and q = s.) By associative law (CA)B is definedand C (AB) is also defined and we have C (AB) = (CA)B. Therefore

B = IB = (CA)B = C (AB) = CI = C .

Applying the same to B ′ in place of B we get B ′ = C . Similarly, B = C ′

Therefore, C = C ′ and B = B ′. ♠



DefinitionIn view of the above lemma, whenever a matrix has both left-inverse and aright-inverse, these two are equal and we call it merely the inverse of A anddenote it by A−1.

RemarkI Check that (A−1)−1 = A.

I Check that if A1,A2 are invertible then so is A1A2. What is its inverse?

I Clearly In, is invertible with I−1n = In, and

I diag (a1, a2, . . . , an) is invertible iff each ai 6= 0. Moreover, its inverse isdiag (a−1

1 , . . . , a−1n ).








1 , . . . , a−1n ).








1 , . . . , a−1n ).



Remark

I If fA, fB : Rn −→ Rnare the linear transformations defined by A,B = A−1

then it follows that fA ◦ fB = fAB = Id .

I Likewise fB ◦ fA = Id . Even the converse holds. viz., if a linear transformationf is invertible, then its associated matrix Mf is invertible with its inversecorresponding to the transformation f −1.

I It is easily checked that an invertible map is one-one and onto.

I If f : Rn −→ Rnis an invertible linear map, check that f −1 is linear.

I In general, we do not yet know how to determine whether a given matrix isinvertible.

I The study of invertible matrices is very important to us and we are going tospend some time on this soon.

I We shall first take up this below, for the case of matrices which are slightvariations of the identity matrix, called elementary matrices.



Remark











Remark











Remark











Remark











Remark










Elementary matricesConsider a square matrix Eij whose (i , j)th entry is 1 and all other entries are 0.

We deliberately do not mention the size of this matrix, because we would like touse the same notation in all possible sizes, the actual size being understood by thecontext.If we multiply a matrix A by Eij on the left, then what we get is a matrix whosei th row is equal to the j th row of A and all other rows are zero.In particular EijEij = 0, (i 6= j). It follows that for any number α and i 6= j

(I + αEij)(I − αEij) = I + αEij − αEij − α2EijEij = I . (11)

What happens if we take i = j in the above discussion?Clearly (11) is no longervalid. On the other hand consider

(I + αEii )(I + βEii ) = I + (α + β + αβ)Eii (12)

This equation tells us that if we want the RHS to be equal to I then we musthave α+ β + αβ = 0. Given any α we can solve for β iff 1 + α 6= 0. Thus we haveproved that I + αEii is invertible if α 6= −1.[Of course, this can be verified directly also: the matrix I +αEii is nothing but thediagonal matrix with all the diagonal entries equal to 1 except the ii th one whichis equal to 1 + α. Therefore, it is invertible iff 1 + α 6= 0. ]


Elementary matricesConsider a square matrix Eij whose (i , j)th entry is 1 and all other entries are 0.We deliberately do not mention the size of this matrix, because we would like touse the same notation in all possible sizes, the actual size being understood by thecontext.

If we multiply a matrix A by Eij on the left, then what we get is a matrix whosei th row is equal to the j th row of A and all other rows are zero.In particular EijEij = 0, (i 6= j). It follows that for any number α and i 6= j






Elementary matricesConsider a square matrix Eij whose (i , j)th entry is 1 and all other entries are 0.We deliberately do not mention the size of this matrix, because we would like touse the same notation in all possible sizes, the actual size being understood by thecontext.If we multiply a matrix A by Eij on the left, then what we get is a matrix whosei th row is equal to the j th row of A and all other rows are zero.

In particular EijEij = 0, (i 6= j). It follows that for any number α and i 6= j






Elementary matricesConsider a square matrix Eij whose (i , j)th entry is 1 and all other entries are 0.We deliberately do not mention the size of this matrix, because we would like touse the same notation in all possible sizes, the actual size being understood by thecontext.If we multiply a matrix A by Eij on the left, then what we get is a matrix whosei th row is equal to the j th row of A and all other rows are zero.In particular EijEij = 0, (i 6= j). It follows that for any number α and i 6= j


What happens if we take i = j in the above discussion?

Clearly (11) is no longervalid. On the other hand consider














This equation tells us that if we want the RHS to be equal to I then we musthave α+ β + αβ = 0.

Given any α we can solve for β iff 1 + α 6= 0. Thus we haveproved that I + αEii is invertible if α 6= −1.[Of course, this can be verified directly also: the matrix I +αEii is nothing but thediagonal matrix with all the diagonal entries equal to 1 except the ii th one whichis equal to 1 + α. Therefore, it is invertible iff 1 + α 6= 0. ]






This equation tells us that if we want the RHS to be equal to I then we musthave α+ β + αβ = 0. Given any α we can solve for β iff 1 + α 6= 0. Thus we haveproved that I + αEii is invertible if α 6= −1.

[Of course, this can be verified directly also: the matrix I +αEii is nothing but thediagonal matrix with all the diagonal entries equal to 1 except the ii th one whichis equal to 1 + α. Therefore, it is invertible iff 1 + α 6= 0. ]








Elementary matrices

We can play this game a little further.

Consider now I + Eij + Eji − Eii − Ejj . Usingsimilar method to the above check that this is also invertible.Now if we actually write down this matrix, this is nothing but the identity matrixafter interchanging the i th and j th rows.These are called transposition matrices.The linear maps corresponding to them merely interchange the i th and j th

coordinates (ei and ej) and does not disturb anything else.We shall denote them simply by Tij .


Elementary matrices

We can play this game a little further.Consider now I + Eij + Eji − Eii − Ejj . Usingsimilar method to the above check that this is also invertible.

Now if we actually write down this matrix, this is nothing but the identity matrixafter interchanging the i th and j th rows.These are called transposition matrices.The linear maps corresponding to them merely interchange the i th and j th



Elementary matrices

We can play this game a little further.Consider now I + Eij + Eji − Eii − Ejj . Usingsimilar method to the above check that this is also invertible.Now if we actually write down this matrix, this is nothing but the identity matrixafter interchanging the i th and j th rows.

These are called transposition matrices.The linear maps corresponding to them merely interchange the i th and j th



Elementary matrices

We can play this game a little further.Consider now I + Eij + Eji − Eii − Ejj . Usingsimilar method to the above check that this is also invertible.Now if we actually write down this matrix, this is nothing but the identity matrixafter interchanging the i th and j th rows.These are called transposition matrices.The linear maps corresponding to them merely interchange the i th and j th

coordinates (ei and ej) and does not disturb anything else.

We shall denote them simply by Tij .


Elementary matrices

We can play this game a little further.Consider now I + Eij + Eji − Eii − Ejj . Usingsimilar method to the above check that this is also invertible.Now if we actually write down this matrix, this is nothing but the identity matrixafter interchanging the i th and j th rows.These are called transposition matrices.The linear maps corresponding to them merely interchange the i th and j th



Elementary matrices

To sum up we have:

Theorem

The elementary matrices I + αEij (i 6= j), I + αEii , (α 6= −1) andTij = I + Eij + Eji − Eii − Ejj are all invertible with their respective inversesI − αEij , I − (α/1 + α)Eii and Tij .

The importance of elementary matrices stems out from the fact that the operationof multiplying a given matrix A on the left by one of them has simpleinterpretation in terms of a certain operation on the rows of A.


Elementary matrices

To sum up we have:

Theorem

The elementary matrices I + αEij (i 6= j), I + αEii , (α 6= −1) andTij = I + Eij + Eji − Eii − Ejj are all invertible with their respective inversesI − αEij , I − (α/1 + α)Eii and Tij .

The importance of elementary matrices stems out from the fact that the operationof multiplying a given matrix A on the left by one of them has simpleinterpretation in terms of a certain operation on the rows of A.


Permutation Matrices

DefinitionA permutation matrix is a square matrix which has all the entries in any given row(and column) equal to zero except one of them and this non zero entry is equal to1.

[0 11 0

] 0 1 01 0 00 0 1

1 0 00 0 10 1 0

etc. are easy examples.



DefinitionA permutation matrix is a square matrix which has all the entries in any given row(and column) equal to zero except one of them and this non zero entry is equal to1.

[0 11 0

] 0 1 01 0 00 0 1

1 0 00 0 10 1 0

etc. are easy examples.



In general given such a matrix, for each 1 ≤ i ≤ n consider the i th row. Look atthe place where the entry 1 occurs. Call it σ(i).

This way we get a map σ : N −→ N which is clearly a one-to-one mapping.Conversely, given a permutation σ : N −→ N, we define a matrix Pσ = ((pij)) bythe formula

pij =

{0 if j 6= σ(i)1 if j = σ(i)

Can you now see that a permutation matrix is obtained by merely shuffling therows of the identity matrix, or, by shuffling the columns?Check that they have another interesting property viz., PP t = P tP = In. Inparticular, they are invertible.[Proof: If Ri denotes the i th row of P and Cj denotes the j th column of P t , bydefinition of P t , it follows that Cj = R t

j . Since clearly, RiRtj = δij (the Kronecker

symbols), it follows that RiCj = δij which is the (i , j)th entry of In.]



In general given such a matrix, for each 1 ≤ i ≤ n consider the i th row. Look atthe place where the entry 1 occurs. Call it σ(i).This way we get a map σ : N −→ N which is clearly a one-to-one mapping.

Conversely, given a permutation σ : N −→ N, we define a matrix Pσ = ((pij)) bythe formula

pij =

{0 if j 6= σ(i)1 if j = σ(i)






In general given such a matrix, for each 1 ≤ i ≤ n consider the i th row. Look atthe place where the entry 1 occurs. Call it σ(i).This way we get a map σ : N −→ N which is clearly a one-to-one mapping.Conversely, given a permutation σ : N −→ N, we define a matrix Pσ = ((pij)) bythe formula

pij =

{0 if j 6= σ(i)1 if j = σ(i)







pij =

{0 if j 6= σ(i)1 if j = σ(i)

Can you now see that a permutation matrix is obtained by merely shuffling therows of the identity matrix, or, by shuffling the columns?

Check that they have another interesting property viz., PP t = P tP = In. Inparticular, they are invertible.[Proof: If Ri denotes the i th row of P and Cj denotes the j th column of P t , bydefinition of P t , it follows that Cj = R t






pij =

{0 if j 6= σ(i)1 if j = σ(i)

Documents

Spring 2011 Anant R. Shastriars/MA106/slides-I.pdf · Linear Algebra MA106 Prescribed text books: I 1. H. Anton, Elementary linear algebra with applications (8th Edition), John Wiley