Lecture 1 - University of California, San Diegoweb.eng.ucsd.edu/.../Lec01_Linear_Algebra_1M.pdf · 2020. 1. 2. · Lecture1 Logistics andGoals Matrices andIntro toLinear Transfor-mations

Lecture 1

Logisticsand Goals

Matricesand Introto LinearTransfor-mationsContinuousLinearTransforma-tions

Eigeneverything

LTI Systems

Trace

Determinant

Rank

Lecture 1ECE 278 Mathematics for MS Comp Exam

ECE 278 Math for MS Exam- Winter 2019 Lecture 1 1

Lecture 1

Logisticsand Goals


Eigeneverything

LTI Systems

Trace

Determinant

Rank

Math Part of MS Exam

The link to the specific requirements for the math part of the MS exam is here.

This guideline is the primary resource for the math part of the MS exam—notECE 278.

If there is any question, the guideline supersedes the material in ECE 278.

This class is a supplement to the requirements listed.

Designed to cover and integrating part of the material.


http://ece.ucsd.edu/sites/ece.ucsd.edu/files/assets/msexams/MS%20Exam%20Information%20-%20Updated%2C%20Math.pdf

Lecture 1

Logisticsand Goals


Eigeneverything

LTI Systems

Trace

Determinant

Rank

How ECE 278 Fits in

ECE is a supplement to the requirements for the Math part of the MS exam.

ECE 278 is not a comprehensive treatment of all the mathematics that couldbe tested on the math part of the MS exam!

The basic rule is that any material in ECE 278 may be on the exam, but materialon the exam may not be covered in ECE 278.

In particular, ECE 278 does not cover any of the material from the UCSDsequence Math 20 A, Math 20 B, and Math 20 C.Only partially covers other material.

ECE 278 should not be used as a substitute for the material covered in theseclasses.

The goal of ECE 278 is to integrate several mathematical topics into a cohesivewhole that can be applied to a variety of engineering problems.

This integrated approach should aid your preparation for the math part of theMS comp exam.


Lecture 1

Logisticsand Goals


Eigeneverything

LTI Systems

Trace

Determinant

Rank

Material in Kreyszig useful for the math part of the MS Exam

One useful feature of the text is that it covers the three main areas tested in themath part of the MS exam: linear systems and algebra, calculus and differentialequations, and probability.

The following sections in the book provide a good starting point for these topics,but the material in the book is not sufficiently in depth.

Its strength is that all the (starting) material is in one place.

Linear Systems, Algebra and TransformationsAll of Chapter 7 and Chapter 8Chapter 11.1, 11.5-11.6, notes on signal space.Calculus and Differential EquationsAll of Chapter 1-4All of Chapter 6All of Chapter 9-10Chapter 12.1-12.3 (intro. to PDEs)This material provides reasonable coverage of differential equations and vectorcalculus.ProbabilityAll of Chapter 24

This coverage is not sufficient and will be supplemented with class notes. This isthe weakest part of the text. Do not assume that knowing the material in thetext is sufficient for the exam!


Lecture 1

Logisticsand Goals


Eigeneverything

LTI Systems

Trace

Determinant

Rank

Matrices

A matrix H is a doubly-indexed set {hij , i = 1, . . . ,n, j = 1, . . . ,m} of real orcomplex numbers conventionally interpreted as a two-dimensional array

H = [hij ] =

[h11 h12 h13h21 h22 h23h31 h32 h33

].The conjugate of A is the matrix H∗ = [h∗ij ].

The transpose of H is the matrix HT = [hji].

The conjugate transpose of H is the matrix HH = [h∗ji]. (Sometimes calledhermitian conjugate.)

A matrix with n = m is a square matrix.


Lecture 1

Logisticsand Goals


Eigeneverything

LTI Systems

Trace

Determinant

Rank

Matrices and Linear Transformations

A matrix can be viewed as a compact representation of a discrete lineartransformation that maps one sequence of numbers expressed by a columnvector x of length N into a different sequence of numbers expressed by a columnvector y of length M where

x =

x1x2...

xN

y = y1y2.

.

.yM

so that

y = Hxor written out y1y2.

.

.yM

= h11 h12 · · · h1Nh21 h22 · · · h2N.

.

....

. . ....

hM1 hM2 · · · hMN

x1x2...

xN

When vectors are defined using rows, the vector is to the left of the matrix


Lecture 1

Logisticsand Goals


Eigeneverything

LTI Systems

Trace

Determinant

Rank

Matrix Multiplication - Cascaded Transformations

A matrix product can represent a sequence of transformations

C︸︷︷︸m×p

= A︸︷︷︸m×n

B︸︷︷︸n×p

Columns of matrix to the left must equal row of matrix to the right, or it is not defined.

In general, AB 6= BA (even if they are square)

Two square matrices A and B of the same size commute if AB = BA.

A matrix A that commutes with AH is called a normal matrix.Then AAH = AH A.

Two matrices with a common set of eigenvectors commute.

For this case, the order in which the transformations are applied AB or BA doesnot affect the outcome.

Two matrices that do not commute do not have a common set of eigenvectorsorder in which the transformations are applied affects the outcome.


Lecture 1

Logisticsand Goals


Eigeneverything

LTI Systems

Trace

Determinant

Rank

Continuous Linear Transformations

Now consider the corresponding continuous linear transformation

y(t) =

∫ ∞−∞

h(t, τ )x(τ )dτ

This transformation is called a superposition integral

When h(t, τ ) = h(t− τ )

y(t) =

∫ ∞−∞

h(t− τ )x(τ )dτ

superposition integral reduces to a convolution, which is not as general.

This means that the general formalism of matrices can represent a discreteconvolution - used often in DSP.


Lecture 1

Logisticsand Goals


Eigeneverything

LTI Systems

Trace

Determinant

Rank

Eigenfunctions, Eigenvectors, and Eigenvalues

For linear transformations, when the output function is a scaled form of the inputfunction, then that input function is called an eigenfunction for a continuoustransformation or an eigenvector for a discrete transformation.

For a continuous transformation, we have

λψ(t) =

∫ ∞−∞

h(t, τ )ψ(τ )dτ

where λ is the eigenvalue, ψ(t) is the eigenfunction, and there may be more thanone eigenfunction.

For the discrete case we haveλx = Hx

We will discuss these concepts at length- this is only a first pass.


Lecture 1

Logisticsand Goals


Eigeneverything

LTI Systems

Trace

Determinant

Rank

Discrete Linear Transformations

A discrete linear transformation, represented by the transformation matrix A, canbe represented as a vector-matrix product

r = As.

When the length of the output vector r is equal to the length of the input vectors, the matrix A is a square matrix.

A discrete linear transformation corresponding to a matrix A is characterized bya finite set of eigenvectors {ej} and a corresponding finite set of eigenvalues{λj}, possibly complex, such that

Aej = λjej .

The eigenvalues are the zeros of det(A− λI) = 0, which is a polynomial in λ ofdegree n.


Lecture 1

Logisticsand Goals


Eigeneverything

LTI Systems

Trace

Determinant

Rank

Linear Time (or Shift) Invariant Systems

For a linear time-invariant system defined by a frequency response H(f ) , whichis the Fourier transform of the impulse response h(t) and exponential input eiωthas an output H(ω)eiωt

eiωt −→︸︷︷︸LT I system

H(ω)eiωt

This means that the complex exponential eiωt is an eigenfunction of any LTIsystem.

Because eiωt is an eigenfunction of any LTI system, all LTI systems share acommon set of eigenfunctions–namely the complex exponentials

This means that all LTI transformation commute and thus the order that theyare applied does not affect the outcome.

Power analysis tool for analysis because you can “re-arrange” blocks

Not all transformation commute and must check to see if they have a commonset of eigenvectors (functions) before asserting this


Lecture 1

Logisticsand Goals


Eigeneverything

LTI Systems

Trace

Determinant

Rank

Trace of a Matrix-1

The trace of a square matrix A is defined as the sum∑

nhnn of the diagonal

elements of A.

The trace is an inherent property of the discrete transformation represented bythe matrix and is independent of the choice of basis used to represent thattransformation.

We will show later that the trace is the sum of the eigenvalues of the matrix.

The trace operation has the following properties:

trace(cA)

= c traceA (1)

trace(

A + B)

= traceA + traceB (2)

trace(

AB)

= trace(

BA)

. (3)

Note the reverse order of the last expression.


Lecture 1

Logisticsand Goals


Eigeneverything

LTI Systems

Trace

Determinant

Rank

Trace of a Matrix-2

The trace of a square matrix that can be expressed as an outer product of twovectors xyH is equal to the inner product of the same two vectors yHx. This isgiven by

trace(xyH ) = yHx. (4)

The outer product is a column vector x of size (N × 1) multiplied by a rowvector yH of size (1×N) and gives a square N ×N matrix, which can representa transformation.

The inner product is a row vector yH of size (1×N) multiplied by a columnvector x of size (N × 1) and gives a number.

Note that if row vectors are used then forms are reversed


Lecture 1

Logisticsand Goals


Eigeneverything

LTI Systems

Trace

Determinant

Rank

Determinant of a Matrix

The determinant of a square matrix is an inherent property of the transformationrepresented by a matrix and is independent of the choice of the basis.

The determinant, denoted det(·), is defined by the Laplace recursion formula.

Let Aij be the (n− 1) by (n− 1) matrix obtained from the n by n matrix A bystriking out the ith row and the jth column. Then, for any fixed i,

det A =n∑

j=1

(−1)i+jaij det Aij , (5)

where aij is the element of A indexed by i and j.

A common method to determine the determinant is Cramer’s rule as discussed inSection 7.7.


Lecture 1

Logisticsand Goals


Eigeneverything

LTI Systems

Trace

Determinant

Rank

Properties of the Determinant

The determinant has the following properties for an n by n matrix:

det AB = det A det B (6)det(cA) = cn det A (7)

det A =1

det(

A−1) , (8)

provided that det A is nonzero where A−1 is the matrix inverse of A.

The trace and the determinant are the two important invariant scalar metricsdescribing the characteristics of a square matrix.


Lecture 1

Logisticsand Goals


Eigeneverything

LTI Systems

Trace

Determinant

Rank

Rank of a Matrix

A matrix whose determinant is nonzero is a full-rank matrix.

For this matrix, all the eigenvalues are nonzero, and if they are distinct, then theeigenvectors are orthogonal with the number equal to the size of the squarematrix.

The rank of a matrix is defined as the maximum number of linearly-independentrow vectors in the matrix.

Equivalently, the rank of a matrix is the maximum number oflinearly-independent column vectors in the matrix.

The rank of a matrix is equal to the size of the largest square submatrix with anonzero determinant.

When A is a full rank n× n matrix and B is has the same size then AB = Oimplies that B =O where O is the zero matrix.

When AB = O and neither A or B are the zero matrix, then neither A or B arefull rank. (See Chapter 7 Theorem 3)


Logistics and GoalsMatrices and Intro to Linear TransformationsContinuous Linear TransformationsEigen everythingLTI SystemsTraceDeterminantRank

Documents

Lecture 1 - University of California, San Diegoweb.eng.ucsd.edu/.../Lec01_Linear_Algebra_1M.pdf · 2020. 1. 2. · Lecture1 Logistics andGoals Matrices andIntro toLinear Transfor-mations