Lesson 5Matrix Algebra and The Transpose

Math 20

September 28, 2007

Announcements

I Thomas Schelling at IOP (79 JFK Street), Wednesday 6pm

I Problem Set 2 is on the course web site. Due October 3

I Problem Sessions: Sundays 6–7 (SC 221), Tuesdays 1–2 (SC116)

I My office hours: Mondays 1–2, Tuesdays 3–4, Wednesdays1–3 (SC 323)

Remember the definition of matrix addition

DefinitionLet A = (aij)m×n and B = (bij)m×n be matrices. The sum of Aand B is the matrix C = (cij)m×n defined by

cij = aij + bij

That is, C is obtained by adding corresponding elements of A AndB.

We do not define A + B if A and B do not have the samedimension.

Remember the definition of matrix addition

DefinitionLet A = (aij)m×n and B = (bij)m×n be matrices. The sum of Aand B is the matrix C = (cij)m×n defined by

cij = aij + bij

That is, C is obtained by adding corresponding elements of A AndB.We do not define A + B if A and B do not have the samedimension.

Properties of Matrix Addition

RulesLet A, B, C, and D be m × n matrices.

(a) A + B = B + A

(so addition is commutative)

(b) A + (B + C) = (A + B) + C (so addition is associative)

(c) There is a unique m × n matrix O such that

A + O = A

for any m × n matrix A. The matrix O is called the m × nadditive identity matrix.

(d) For each m × n matrix A, there is a unique m × n matrix Dsuch that

A + D = O

(We write D = −A.) So additive inverses exist.

Properties of Matrix Addition

RulesLet A, B, C, and D be m × n matrices.

(a) A + B = B + A (so addition is commutative)

(b) A + (B + C) = (A + B) + C (so addition is associative)

(c) There is a unique m × n matrix O such that

A + O = A

for any m × n matrix A. The matrix O is called the m × nadditive identity matrix.

(d) For each m × n matrix A, there is a unique m × n matrix Dsuch that

A + D = O

(We write D = −A.) So additive inverses exist.

Properties of Matrix Addition

RulesLet A, B, C, and D be m × n matrices.

(a) A + B = B + A (so addition is commutative)

(b) A + (B + C) = (A + B) + C

(so addition is associative)

(c) There is a unique m × n matrix O such that

A + O = A

for any m × n matrix A. The matrix O is called the m × nadditive identity matrix.

(d) For each m × n matrix A, there is a unique m × n matrix Dsuch that

A + D = O

(We write D = −A.) So additive inverses exist.

Properties of Matrix Addition

RulesLet A, B, C, and D be m × n matrices.

(a) A + B = B + A (so addition is commutative)

(b) A + (B + C) = (A + B) + C (so addition is associative)

(c) There is a unique m × n matrix O such that

A + O = A

for any m × n matrix A. The matrix O is called the m × nadditive identity matrix.

(d) For each m × n matrix A, there is a unique m × n matrix Dsuch that

A + D = O

(We write D = −A.) So additive inverses exist.

Properties of Matrix Addition

RulesLet A, B, C, and D be m × n matrices.

(a) A + B = B + A (so addition is commutative)

(b) A + (B + C) = (A + B) + C (so addition is associative)

(c) There is a unique m × n matrix O such that

A + O = A

for any m × n matrix A. The matrix O is called the m × nadditive identity matrix.

(d) For each m × n matrix A, there is a unique m × n matrix Dsuch that

A + D = O

(We write D = −A.) So additive inverses exist.

Proof

Proof.Let’s prove (b). We need to show that every entry of A + (B + C)is equal to the corresponding entry of (A + B) + C.

Well,

[A + (B + C)]ij = aij + [B + C]ij = aij + (bij + cij)

= (aij + bij) + cij as real numbers

= [A + B]ij + cij = [(A + B) + C]ij .

Proof

Proof.Let’s prove (b). We need to show that every entry of A + (B + C)is equal to the corresponding entry of (A + B) + C. Well,

[A + (B + C)]ij = aij + [B + C]ij = aij + (bij + cij)

= (aij + bij) + cij as real numbers

= [A + B]ij + cij = [(A + B) + C]ij .

Remember the definition of matrix multiplication

DefinitionLet A = (aij)m×n and B = (bij)n×p. Then the matrix product ofA and B is the m × p matrix whose jth column is Abj . In otherwords, the (i , j)th entry of AB is the dot product of ith row of Aand the jth column of B. In symbols

(AB)ij =n∑

k=1

aikbkj .

Another way to look at it:

A(b1 b2 . . . bp

)=(Ab1 Ab2 . . . Abp

)

Remember the definition of matrix multiplication

DefinitionLet A = (aij)m×n and B = (bij)n×p. Then the matrix product ofA and B is the m × p matrix whose jth column is Abj . In otherwords, the (i , j)th entry of AB is the dot product of ith row of Aand the jth column of B. In symbols

(AB)ij =n∑

k=1

aikbkj .

Another way to look at it:

A(b1 b2 . . . bp

)=(Ab1 Ab2 . . . Abp

)

Examples

Let

A =

(1 2−1 2

)B =

(0 23 1

)

C =

(1 −1 22 3 −1

)D =

0 1 2 0−1 0 2 03 1 0 0

Compute

(a) (A + B)C

(b) AC + BC

(c) A(BC)

(d) (AB)C

(e) AC and CA

(f) AB and BA

Does matrix multiplication distribute?

Solution to (a) and (b)

(A + B)C =

(9 11 −28 7 1

)= AC + BC

In fact, this is true in general:

Distributive rules for matricesIf A, B, and C are of appropriate sizes, then

(A + B)C = AC + BC.

If A, B, and C are of appropriate sizes, then

A(B + C) = AB + AC

Does matrix multiplication distribute?

Solution to (a) and (b)

(A + B)C =

(9 11 −28 7 1

)= AC + BC

In fact, this is true in general:

Distributive rules for matricesIf A, B, and C are of appropriate sizes, then

(A + B)C = AC + BC.

If A, B, and C are of appropriate sizes, then

A(B + C) = AB + AC

Is matrix multiplication associative?

Solution to (c) and (d)

A(BC) =

(14 6 86 −6 12

)= (AB)C

In fact, this is true in general:

Associative rules for matrix multiplication

If A, B, and C are of appropriate sizes, then

(AB)C = A(BC)

Is matrix multiplication associative?

Solution to (c) and (d)

A(BC) =

(14 6 86 −6 12

)= (AB)C

In fact, this is true in general:

Associative rules for matrix multiplication

If A, B, and C are of appropriate sizes, then

(AB)C = A(BC)

Is matrix multiplication commutative?

Solution to (e)

AC =

(5 5 03 7 −4

), but CA is not even defined. We cannot

multiply a 2 × 3 matrix by a 2 × 2 matrix.

Solution to (f)

BA =

(−2 42 8

), but AB =

(6 46 0

).

So matrix multiplication is (usually) not commutative, even whenit’s possible to test.

Is matrix multiplication commutative?

Solution to (e)

AC =

(5 5 03 7 −4

),

but CA is not even defined. We cannot

multiply a 2 × 3 matrix by a 2 × 2 matrix.

Solution to (f)

BA =

(−2 42 8

), but AB =

(6 46 0

).

So matrix multiplication is (usually) not commutative, even whenit’s possible to test.

Is matrix multiplication commutative?

Solution to (e)

AC =

(5 5 03 7 −4

), but CA is not even defined. We cannot

multiply a 2 × 3 matrix by a 2 × 2 matrix.

Solution to (f)

BA =

(−2 42 8

), but AB =

(6 46 0

).

So matrix multiplication is (usually) not commutative, even whenit’s possible to test.

Is matrix multiplication commutative?

Solution to (e)

AC =

(5 5 03 7 −4

), but CA is not even defined. We cannot

multiply a 2 × 3 matrix by a 2 × 2 matrix.

Solution to (f)

BA =

(−2 42 8

), but AB =

(6 46 0

).

So matrix multiplication is (usually) not commutative, even whenit’s possible to test.

Is matrix multiplication commutative?

Solution to (e)

AC =

(5 5 03 7 −4

), but CA is not even defined. We cannot

multiply a 2 × 3 matrix by a 2 × 2 matrix.

Solution to (f)

BA =

(−2 42 8

),

but AB =

(6 46 0

).

So matrix multiplication is (usually) not commutative, even whenit’s possible to test.

Is matrix multiplication commutative?

Solution to (e)

AC =

(5 5 03 7 −4

), but CA is not even defined. We cannot

multiply a 2 × 3 matrix by a 2 × 2 matrix.

Solution to (f)

BA =

(−2 42 8

), but AB =

(6 46 0

).

So matrix multiplication is (usually) not commutative, even whenit’s possible to test.

Is matrix multiplication commutative?

Solution to (e)

AC =

(5 5 03 7 −4

), but CA is not even defined. We cannot

multiply a 2 × 3 matrix by a 2 × 2 matrix.

Solution to (f)

BA =

(−2 42 8

), but AB =

(6 46 0

).

So matrix multiplication is (usually) not commutative, even whenit’s possible to test.

Is matrix multiplication commutative?

Solution to (e)

AC =

(5 5 03 7 −4

), but CA is not even defined. We cannot

multiply a 2 × 3 matrix by a 2 × 2 matrix.

Solution to (f)

BA =

(−2 42 8

), but AB =

(6 46 0

).

So matrix multiplication is (usually) not commutative, even whenit’s possible to test.

The Transpose

But there is another operation on matrices, which is just flippingrows and columns.

DefinitionLet A = (aij)m×n be a matrix. The transpose of A is the matrixA′ = (aij)n×m whose (i , j)th entry is aji .

Example

Let A =

1 23 45 6

. Then

A′ =

(1 3 52 4 6

).

The Transpose

But there is another operation on matrices, which is just flippingrows and columns.

DefinitionLet A = (aij)m×n be a matrix. The transpose of A is the matrixA′ = (aij)n×m whose (i , j)th entry is aji .

Example

Let A =

1 23 45 6

. Then

A′ =

(1 3 52 4 6

).

The Transpose

But there is another operation on matrices, which is just flippingrows and columns.

DefinitionLet A = (aij)m×n be a matrix. The transpose of A is the matrixA′ = (aij)n×m whose (i , j)th entry is aji .

Example

Let A =

1 23 45 6

. Then

A′ =

(1 3 52 4 6

).

Examples

Let

A =

(1 2−1 2

)B =

(0 23 1

)

C =

(1 −1 22 3 −1

)D =

0 1 2 0−1 0 2 03 1 0 0

Compute

(a) (A′)′

(b) (A + B)′ and A′ + B′(c) (AC)′ and A′C′

(d) (AC)′ and C′A′

Solution to (a).

(A′)′ =

A

In fact, this is true in general.

Solution to (a).

(A′)′ = A

In fact, this is true in general.

Solution to (a).

(A′)′ = A

In fact, this is true in general.

Does the transpose distribute over addition?

Solution to (b).

(A + B)′ =

(1 24 3

)= A′ + B′

In fact, this is true in general.

Does the transpose distribute over addition?

Solution to (b).

(A + B)′ =

(1 24 3

)= A′ + B′

In fact, this is true in general.

Does the transpose distribute over multiplication?

Solution to (c) and (d).

(AC)′ =

5 35 70 −4

, but A′C′ is not defined.

On the other hand,

C′A′ =

5 35 70 −4

= (AC)′.

This is true in general.

Does the transpose distribute over multiplication?

Solution to (c) and (d).

(AC)′ =

5 35 70 −4

, but A′C′ is not defined. On the other hand,

C′A′ =

5 35 70 −4

= (AC)′.

This is true in general.

Does the transpose distribute over multiplication?

Solution to (c) and (d).

(AC)′ =

5 35 70 −4

, but A′C′ is not defined. On the other hand,

C′A′ =

5 35 70 −4

= (AC)′.

This is true in general.

Remember these properties (including which ones aren’t true)because they are the rules of the game in linear algebra!

Transpose and dot product

Notice:p · q = p′q

Also,

TheoremLet A be an n × n matrix and v, w vectors in Rn. Then

v · Aw = (A′v) ·w.

Proof.Remember v ·Aw is a scalar, so is equal to its own transpose. Then

v · Aw = (v′Aw)′ = (Aw)′(v′)′ = w′A′v = w · A′v = (A′v) ·w.

Transpose and dot product

Notice:p · q = p′q

Also,

TheoremLet A be an n × n matrix and v, w vectors in Rn. Then

v · Aw = (A′v) ·w.

Proof.Remember v ·Aw is a scalar, so is equal to its own transpose. Then

v · Aw = (v′Aw)′ = (Aw)′(v′)′ = w′A′v = w · A′v = (A′v) ·w.

Transpose and dot product

Notice:p · q = p′q

Also,

TheoremLet A be an n × n matrix and v, w vectors in Rn. Then

v · Aw = (A′v) ·w.

Proof.Remember v ·Aw is a scalar, so is equal to its own transpose.

Then

v · Aw = (v′Aw)′ = (Aw)′(v′)′ = w′A′v = w · A′v = (A′v) ·w.

Transpose and dot product

Notice:p · q = p′q

Also,

TheoremLet A be an n × n matrix and v, w vectors in Rn. Then

v · Aw = (A′v) ·w.

Proof.Remember v ·Aw is a scalar, so is equal to its own transpose. Then

v · Aw = (v′Aw)′ = (Aw)′(v′)′ = w′A′v = w · A′v = (A′v) ·w.

Example

Some special cases which are useful. Let

A =

(3 2−1 1

)e1 =

(10

)e2 =

(01

)

E11 =

(1 00 0

)E12 =

(0 10 0

)E21 =

(0 01 0

)E22 =

(0 00 1

)

D =

(2 00 −1

)I =

(1 00 1

)Compute

(a) Ae1 and Ae2

(b) AE11 and E11A

(c) AE12 and E12A

(d) AD and DA

Solution to (a)

Ae1 =

(3−1

)Ae2 =

(21

)

In general,

I Aei is the ith column of A.

I e′jA is the jth row of A.

Solution to (a)

Ae1 =

(3−1

)Ae2 =

(21

)In general,

I Aei is the ith column of A.

I e′jA is the jth row of A.

Solution to (b) and (c)

AE11 =

(3 0−1 0

)E11A =

(3 20 0

)AE12 =

(0 30 −1

)E12A =

(−1 10 0

)

In general,

I AEij has all zeros except for the jth column, which is the ithcolumn of A

I EijA has all zeros except for the ith row, which is the jth rowof A

Solution to (b) and (c)

AE11 =

(3 0−1 0

)E11A =

(3 20 0

)AE12 =

(0 30 −1

)E12A =

(−1 10 0

)In general,

I AEij has all zeros except for the jth column, which is the ithcolumn of A

I EijA has all zeros except for the ith row, which is the jth rowof A

Solution to (d)

DA =

(6 41 −1

)AD =

(6 −22 −1

)

In general,

I a diagonal matrix times a matrix scales every row of theright-hand matrix by the corresponding diagonal entries

I a matrix times a diagonal matrix scales every column of theleft-hand matrix by the corresponding diagonal entries.

I diagonal matrices do commute

Solution to (d)

DA =

(6 41 −1

)AD =

(6 −22 −1

)In general,

I a diagonal matrix times a matrix scales every row of theright-hand matrix by the corresponding diagonal entries

I a matrix times a diagonal matrix scales every column of theleft-hand matrix by the corresponding diagonal entries.

I diagonal matrices do commute