Enhancing Numerical Analysis Teaching with MATLAB€¦ · Enhancing Numerical Analysis Teaching with MATLAB Carola-Bibiane Schonlieb¨ Department for Applied Mathematics and Theoretical

Enhancing Numerical Analysis Teaching withMATLAB

Carola-Bibiane Schonlieb

Department for Applied Mathematics and Theoretical PhysicsUniversity of Cambridge, UK

MATLAB EXPO - October, 8th 2013

Schonlieb (DAMTP) MATLAB support for NA in Cambridge MATLAB EXPO - 8/10/2013 1 / 17

The idea . . .

Point of attack:Classical setup of the numerical analysis courses inCambridge does not require the students to haveprogramming skills but concentrates on teaching themconcepts and numerical methodologies as well asmathematical analysis.

MATLAB enhancement:Illustrate the concepts underlying numerical analysis, as wellas demonstrating the power of modern, advancedscientific-computing software, to accompany numericalanalysis instruction by computer demonstrations.


The Numerical Analysis (NA) Curriculum in Cambridge

NA Curriculum in Cambridge

Undergraduate TeachingPart IB (2nd year mathematics),Part II (3rd year mathematics).

Graduate TeachingPart III (master course, 4th year mathematics).



NA Curriculum in Cambridge

Undergraduate Teaching

Part IB (2nd year mathematics),Part II (3rd year mathematics).

Graduate TeachingPart III (master course, 4th year mathematics).



Part IB NA – Learning Outcomes

By the end of this course, you should:

understand the role of algorithms in numerical analysis;understand the role and basic theory (including orthogonalpolynomials and the Peano kernel theorem) of polynomialapproximation;understand multistep and Runge–Kutta methods for ordinarydifferential equations and the concepts of convergence, orderand stability;understand the theory of algorithms such as LU and QRfactorisation, and be able to apply them, for example to leastsquares calculations.



Part II NA – Learning Outcomes

By the end of this course, you should:

understand the solution of partial differential equations(Poisson, diffusion, advection, wave equation) by

finite differences (consistency, convergence, stability),spectral methods (Fourier expansion, FFT);

understand iterative techniques for linear equations;

understand the calculation of eigenvalues and eigenvectors;



The team behindDAMTP faculty:

Stephen Cowley

Arieh Iserles

Alexei Shadrin

CBS

DAMTP students:

Ben Champion

Francesca Balestrieri

Simon Morris

MathWorks representatives:

Tanya Morton

Coorous Mohtadi(MathWorks Cambridge).Schonlieb (DAMTP) MATLAB support for NA in Cambridge MATLAB EXPO - 8/10/2013 6 / 17

http://www.mathworks.co.uk/videos/university-of-cambridge-teaches-numerical-analysis-to-undergraduates-77423.html?type=shadow

http://www.mathworks.co.uk/videos/university-of-cambridge-students-use-their-perspectives-to-design-new-teaching-model-77422.html?type=shadow

MATLAB demos for NA courses

Outline of the project

MATLAB implementations of numerical algorithms embeddedin GUIs for straightforward use by students outside class.These are linked to the example sheets that the students have towork on by themselves.



Outline of the project (cont)

A “Numerical Analysis Webpage”, which includes all the coursematerial, i.e., lecture notes, example sheets, MATLAB demos andMATLAB codes. For this we took advantage of the Publishoption of MATLAB.



Outline of the project (cont)

A “Numerical Analysis Webpage”, which includes all the coursematerial, i.e., lecture notes, example sheets, MATLAB demos andMATLAB codes. For this we took advantage of the Publishoption of MATLAB.



A tour through the MATLAB material

http://www.maths.cam.ac.uk/undergrad/course/na/





Example 1: QR algorithm

Mathematical explanation

Dr C.-B. Schonlieb

Mathematical Tripos Part II: Michaelmas Term 2012

Numerical Analysis – Lecture 23 & 24

Revision 5.8 (Givens rotations) The notation Ω[i,j] denotes the following n× n matrix

Ω[i,j] =

1 . . .c s. . .− s c . . .↑ ↑ 1i j

, c2 + s2 = 1.

Generally, for any vector ak ∈ Rn, we can find a matrix Ω[i,j] such that

Ω[i,j]a =

1 . . .c s. . .− s c . . .↑ ↑ 1i j

a1k:

aik:ajk:

ank

=

a1k:r ←i:0 ←j:

ank

c = aik√a2ik+a2

jk

,

s =ajk√

a2ik+a2

jk

,

r =√

a2ik+a2

jk .

1) We can choose Ω[i,j] so that any prescribed element ajk in the j-th row of A = Ω[i,j] × A iszero.

2) The rows of A = Ω[i,j]×A are the same as the rows of A, except that the i-th and j-th rowsof the product are linear combinations of the i-th and j-th rows of A.

3) The columns of A = A×Ω[i,j]T are the same as the columns of A, except that the i-th andj-th columns of A are linear combinations of the i-th and j-th columns of A.

4) Ω[i,j] is an orthogonal matrix, thus A = Ω[i,j]AΩ[i,j]T inherits the eigenvalues of A.5) If A is symmetric, then so is A.

Method 5.9 (Transformation to an upper Hessenberg form) We replace A by A = SAS−1, where S is aproduct of Givens rotations Ω[i,j] chosen to annihilate subsubdiagonal elements aj,i−1 in the (i−1)-st column:

∗ ∗ ∗ ∗∗ ∗ ∗ ∗∗ ∗ ∗ ∗∗ ∗ ∗ ∗

Ω[2,3]×→

∗ ∗ ∗ ∗• • • •0 • • •∗ ∗ ∗ ∗

×Ω[2,3]T

→

∗ • • ∗∗ • • ∗0 • • ∗∗ • • ∗

Ω[2,4]×→

∗ ∗ ∗ ∗• • • •0 ∗ ∗ ∗0 • • •

×Ω[2,4]T

→

∗ • ∗ •∗ • ∗ •0 • ∗ •0 • ∗ •

Ω[3,4]×→

∗ ∗ ∗ ∗∗ ∗ ∗ ∗0 • • •0 0 • •

×Ω[3,4]T

→

∗ ∗ • •∗ ∗ • •0 ∗ • •0 0 • •

The •-elements have changed through a single transformation while the ∗-elements remained the same.

It is seen that every element that we have set to zero remains zero, and the final outcome is indeed anupper Hessenberg matrix. If A is symmetric then so will be the outcome of the calculation, hence it will betridiagonal. In general, the cost of this procedure is O(n3).

Alternatively, we can transform A to upper Hessenberg using Householder reflections, rather than Givensrotations. In that case we deal with a column at a time, taking u such that, with Hu = I − 2uuT /‖u‖2, thei-th column of B = HuB is consistent with the upper Hessenberg form. Such a u has its first i coordinatesvanishing, therefore B = BHT

u has the first i columns unchanged, and all new and old zeros (which are inthe first i columns) stay untouched.

∗ ∗ ∗ ∗ ∗∗ ∗ ∗ ∗ ∗∗ ∗ ∗ ∗ ∗∗ ∗ ∗ ∗ ∗∗ ∗ ∗ ∗ ∗

H1×→

∗ ∗ ∗ ∗ ∗• • • • •0 • • • •0 • • • •0 • • • •

×HT1→

∗ • • • •∗ • • • •0 • • • •0 • • • •0 • • • •

H2×→

∗ ∗ ∗ ∗ ∗∗ ∗ ∗ ∗ ∗0 • • • •0 0 • • •0 0 • • •

×HT2→

∗ ∗ • • •∗ ∗ • • •0 ∗ • • •0 0 • • •0 0 • • •

H3×→

∗ ∗ ∗ ∗ ∗∗ ∗ ∗ ∗ ∗0 ∗ ∗ ∗ ∗0 0 • • •0 0 0 • •

×HT3→

∗ ∗ ∗ • •∗ ∗ ∗ • •0 ∗ ∗ • •0 0 ∗ • •0 0 0 • •

Algorithm 5.10 (The QR algorithm) The “plain vanilla” version of the QR algorithm is as follows. Set A0 =A. For k = 0, 1, . . . calculate the QR factorization Ak = QkRk (here Qk is n× n orthogonal and Rk is n× nupper triangular) and set Ak+1 = RkQk.

45

NA-webpage & MATLAB GUI


http://www.damtp.cam.ac.uk/user/cbs31/Teaching_files/c23.pdf

http://www.maths.cam.ac.uk/undergrad/course/na/ii/qr_hex/qr_hex.php


Example 2: FFT


Dr C.-B. Schonlieb


Numerical Analysis – Lecture 4Algorithm 1.19 (The fast Fourier transform (FFT)) We assume that n is a power of 2, i.e. n = 2m = 2p,and for y ∈ Π2m, denote by

y(E) = y2jj∈Z and y(O) = y2j+1j∈Zthe even and odd portions of y, respectively. Note that y(E),y(O) ∈ Πm.

Suppose that we already know the inverse DFT of both ‘short’ sequences,

x(E) = F−1m y(E), x(O) = F−1m y(O).

It is then possible to assemble x = F−12my in a small number of operations. Since ω2m2m = 1, we obtain

ω22m = ωm, and

x` =

2m−1∑

j=0

ωj`2myj =

m−1∑

j=0

ω2j`2my2j +

m−1∑

j=0

ω(2j+1)`2m y2j+1

=

m−1∑

j=0

ωj`my

(E)j + ω`

2m

m−1∑

j=0

ωj`my

(O)j = x

(E)` + ω`

2mx(O)` , ` = 0, . . . ,m− 1.

Therefore, it costs just m products to evaluate the first half of x, provided that x(E) and x(O) are known.It actually costs nothing to evaluate the second half, since

ωj(m+`)m = ωj`

m, ωm+`2m = −ω`

2m ⇒ xm+` = x(E)` − ω`

2mx(O)` , l = 0, . . . ,m−1.

To execute FFT, we start from vectors of unit length and in each s-th stage, s = 1...p, assemble 2p−s

vectors of length 2s from vectors of length 2s−1: this costs 2p−s2s−1 = 2p−1 products. Altogether, thecost of FFT is p2p−1 = 1

2n log2 n products.

0 1 2 3 4 5 6 7

← 1 block of length 2p

0 2 4 6

1 3 5 7

← 2p−s blocks of length 2s

0 4

2 6

1 5

3 7

← 2p−1 blocks of length 2

0 4 2 6 1 5 3 7 ← 2p blocks of length 1

For n = 1024 = 210, say, the cost is ≈ 5 × 103 products, compared to ≈ 106 for naive matrix multiplica-tion! For n = 220 the respective numbers are ≈ 1.05 × 107 and ≈ 1.1 × 1012, which represents a savingby a factor of more than 105.

Matlab demo: Check out the online animation for computing the FFT at http://www.maths.cam.ac.uk/undergrad/course/na/ii/fft_gui/fft_gui.php and download the Matlab GUI fromthere to follow the computation of each single FFT term.

Example 1.20 Computation of FFT for n = 4 in general, and for the vector y = (1, 1,−1,−1) in particu-lar.

02 = 01 + 11 12 = 21 + i31 22 = 01 − 11 32 = 21 − i31

01 = 00 + 20 21 = 00 − 20

11 = 10 + 30 31 = 10 − 30

00 20 10 30

⇒

0 2 + 2i 0 2− 2i

0 2

0 2

+1 −1 +1 −1

7




http://www.maths.cam.ac.uk/undergrad/course/na/ii/fft_gui/fft_gui.php


Example 3: Stability of PDE discretisations


Dr C.-B. Schonlieb


Numerical Analysis – Lecture 6Definition 2.11 (Normal matrices) We say that a matrix A is normal if A = QDQT , where D isa (complex) diagonal matrix and Q is a unitary matrix (such that QQT = I). In other words, amatrix is normal if it has a complete set of orthonormal eigenvectors.

Examples of the real normal matrices, besides the familiar symmetric matrices (A = AT ),inlclude also the matrices which are skew-symmetric (A = −AT ), or more generally the matriceswith skew-symmetric off-diagonal part.

Proposition 2.12 If A is normal, then ‖A‖ = ρ(A).

Proof. For any complex matrix B, we have ‖B‖ = sup ‖Bx‖‖x‖ , and ifw be an eigenvector of B, then

‖Bw‖ = |λ|‖w‖. So, we deduce that

ρ(B) ≤ ‖B‖ ∀B ∈ Cn×n (and for every vector norm ‖ · ‖) .Next, let A be normal and recall that unitary matrices are isometries with respect to the Euclideannorm, i.e., ‖Qv‖ = ‖v‖ for any v. Therefore (for the Euclidean norm)

‖Av‖ = ‖QDQTv‖ = ‖DQTv‖ = ‖Du‖ ,whereu = QTv, hence ‖u‖ = ‖v‖. FinallyD is diagonal and similar toA, therefore diagD = σ(A)and ‖D‖ = ρ(A), hence

‖Av‖ = ‖Du‖ ≤ ‖D‖ ‖u‖ = ρ(A)‖v‖ .Thus ‖A‖ ≤ ρ(A), hence ‖A‖ = ρ(A).

Remark 2.13 More generally, one can prove that ‖A‖ = [ρ(AAT )]1/2, and the previous result canbe deduced from that formula.

Example 2.14 (Crank–Nicolson method for diffusion equation) Let

un+1m − unm = 1

2µ(un+1

m−1 − 2un+1m + un+1

m+1) + 12µ(unm−1 − 2unm + unm+1), m = 1...M.

Then un+1 = B−1Cun, where the matrices B and C are Toeplitz symmetric tridiagonal (TST),

un+1 = B−1Cun, B =

1 + µ − 12µ

− 12µ 1 + µ

. . .. . . . . . − 1

2µ− 1

2µ 1 + µ

, C =

1− µ 12µ

12µ 1− µ . . .

. . . . . . 12µ

12µ 1− µ

.

All M ×M TST matrices share the same eigenvectors, hence so does B−1C. Moreover, theseeigenvectors are orthogonal. Therefore, also A = B−1C is normal and its eigenvalues are

λk(A) =λk(C)

λk(B)=

(1− µ) + µ cosπkh

(1 + µ)− µ cosπkh=

1− 2µ sin2 12πkh

1 + 2µ sin2 12πkh

⇒ |λk(A)| ≤ 1, k = 1...M.

Consequently Crank–Nicolson is stable for all µ > 0. [Note: Similarly to the situation with stiffODEs, this does not mean that k = ∆t may be arbitrarily large, but that the only valid considera-tion in the choice of k = ∆t vs h = ∆x is accuracy.]

Matlab demo: Download the Matlab GUI for Stability of 1D PDEs from http://www.maths.cam.ac.uk/undergrad/course/na/ii/pde_stability/pde_stability.php and solvethe diffusion equation in the interval [0, 1] with the Euler method and with Crank–Nicolson. Seethe effect of unconditional stability!

11




http://www.maths.cam.ac.uk/undergrad/course/na/ii/pde_stability/pde_stability.php

Does it work?

Conclusion after the 1st year of implementation

Was it a success? Yes

. . . at least partially.

Clear enhancement of lectures by live demos⇒

MATLAB demos link theory to computations; student feedbackpositive.

Students did not use online material excessively⇒

Online interaction is not assessed; integrate the online materialmore into the course; in particular, through instructing students

and supervisors; amending example sheets takes care and time;change format of use (GUI)?


Does it work?


Was it a success? Yes . . . at least partially.







Does it work?









Does it work?









Does it work?

Why MATLAB?

MATLAB is a great software for a very intuitive way ofprogramming.

Easy and flexible way of visualising results.

It provides many inbuilt functions that facilitate computation.GUI’s easy to build.

The publish option in MATLAB connected the documentation ofthe GUI’s directly with the NA-webpage.


Does it work?

Why MATLAB?

MATLAB is a great software for a very intuitive way ofprogramming. One of our developer students only had very basicprogramming skills when she started the job!





Does it work?

Why MATLAB?






Does it work?

Why MATLAB?






Does it work?

Why MATLAB?






Does it work?

Effects so far

Improves teaching experience.

Gives fresh drive into the otherwise dry NA theory....

...Motivates students to explore NA further: One of our NAstudents did a summer internship on a NA topic in Cambridge.


Does it work?

Effects so far

Improves teaching experience.Gives fresh drive into the otherwise dry NA theory.

...

...

Motivates students to explore NA further: One of our NAstudents did a summer internship on a NA topic in Cambridge.


Does it work?

Effects so far

Improves teaching experience.Gives fresh drive into the otherwise dry NA theory.

...

...Motivates students to explore NA further: One of our NAstudents did a summer internship on a NA topic in Cambridge.


Does it work?

What’s the future?

Evaluation by student questionnaire.

GUI code is unreadable⇒ Have more understandable MATLABcode in the future to facilitate student learning from providedimplementations.Expand MATLAB material

by industrial/application examples (e.g. diffusion for medicalimaging applications – image denoising)

to Part III NA - this needs a more advanced student.


Does it work?


Evaluation by student questionnaire.GUI code is unreadable⇒ Have more understandable MATLABcode in the future to facilitate student learning from providedimplementations.

Expand MATLAB materialby industrial/application examples (e.g. diffusion for medicalimaging applications – image denoising)



Does it work?


Evaluation by student questionnaire.GUI code is unreadable⇒ Have more understandable MATLABcode in the future to facilitate student learning from providedimplementations.Expand MATLAB material

by industrial/application examples (e.g. diffusion for medicalimaging applications – image denoising)



Does it work?

The End

Thank you for your kind attention!

For more details seehttp://www.maths.cam.ac.uk/undergrad/course/na/

andhttp://www.damtp.cam.ac.uk/user/cbs31/Teaching.html

or write to: [email protected]



http://www.damtp.cam.ac.uk/user/cbs31/Teaching.html

[email protected]

Documents

Enhancing Numerical Analysis Teaching with MATLAB€¦ · Enhancing Numerical Analysis Teaching with MATLAB Carola-Bibiane Schonlieb¨ Department for Applied Mathematics and Theoretical