Slide 1
18-660: Numerical Methods for Engineering Design and Optimization
Xin Li Department of ECE
Carnegie Mellon University Pittsburgh, PA 15213
Slide 3
Numerical Computation
Numerical computation: approximation techniques and algorithms to numerically solve mathematical problems Not all mathematical problems have closed-form solutions
Mathematical examples
Solve nonlinear algebraic equations Find minimum of a nonlinear function Determine eigenvalues of a matrix
This course will encompass various aspects of numerical
methods and their applications Linear & nonlinear solver, nonlinear optimization, randomized
algorithm, etc.
Slide 4
Numerical Computation
Numerical computation is an important tool to solve many practical engineering problems
PageRank finds the eigenvector of a 1010-by-1010 matrix – the largest matrix computation in the world
Airlines use optimization algorithms to decide ticket prices, airplane assignments and fuel needs
Slide 5
Numerical Computation
Several additional real-world examples:
Car companies run computer simulations of car crashes to identify safety issues
Wall street runs statistical simulation tools to predict stock prices
Slide 6
Brief History of Numerical Computation
Before 1950s
Algorithms Linear interpolation Newton’s method Gaussian elimination
Tools
Hand calculation Formula handbook Data table
Carl Friedrich Gauss (1777-1855) German Mathematician
Slide 7
Brief History of Numerical Computation
After 1950s The invention of modern computers motivates the development
of a great number of new numerical algorithms
1E+04
1E+05
1E+06
1E+07
1E+08
1E+09
1E+10
1E+1119
71
1974
1979
1984
1986
1990
1994
1996
1999
2000
2002
2005
2005
2006
2006
2007
2008
Inst
ruct
ion
Per
Sec
ond
1E+05
1E+06
1E+07
1E+08
1E+09
1E+10
CP
U F
requ
ency
(Hz)
IPS Freq
Slide 8
Numerical Computation Problems
Ordinary and partial differential equations
Regression
Classification
Slide 9
Ordinary Differential Equation (ODE)
Transient analysis for electrical circuit
F1
Ω11
0
1
0
( ) ( ) ( )tvtudt
tdv−=
( )tu ( )tv
( ) ( ) ( )t
ttvtvdt
tdv∆
∆−−≈
t
v(t)
t1 t0 t2 t3 t4 t5 t6 t7
( ) ( ) ( ) ( )
( ) ( ) ( ) ( )
2212
1101
tvtut
tvtv
tvtut
tvtv
−=∆−
−=∆−
Algebraic equations
Slide 10
Partial Differential Equation (PDE)
Thermal analysis: heat conduction is governed by partial differential equation (PDE):
( ) ( )[ ] ( )tzyxptzyxTt
tzyxTCp ,,,,,,,,,+∇⋅⋅∇=
∂
∂⋅⋅ κρ
Material density (kg/m3)
Specific heat capacity (Joules/K-kg)
Thermal conductivity (Watts/K-m)
Power density of heat sources
(Watts/m3)
Slide 11
Linear Regression
Linear regression (response surface modeling)
f(x)
x
( ) baxxf +=
( )( )( )
baxxfbaxxfbaxxf
+=+=+=
33
22
11
Solve over-determined linear equation to find a and b (more equations than unknowns)
Slide 12
Linear Regression
Linear regression with regularization
Original image Recovered image
BA =α 1
2
2min αλαα
⋅+− BA
Least-squares error
L1-norm regularization
Slide 13
Classification
Support vector machine (SVM)
Feature x1
Feat
ure x
2
Margin
( ) ( )( )
<≥
+=BClassAClass
CXWXf T
0 0
SVM Coefficients Features
( )( )BClassXCXW
AClassXCXWW
T
T
0 0S.T.
min 2
2
∈<+∈≥+
L2-norm (maximize margin)
Solve convex quadratic programming to find W and C
Slide 14
Classification
Brain computer interface based on classification
Neural signal recording
Neural signal processing
Movement decoding
Feedback
Methods: filtering; signal subspace methods … Tool Boxes: MNE-Suite; FieldTrip; MaxFilter …
-0.5 0 0.5 1 1.5 2-2.5
-2
-1.5
-1
-0.5
0
0.5
1
1.5
2
Time (Sec.)
Ampl
itude
(Nor
mal
ized)
-0.5 0 0.5 1 1.5 2-2.5
-2
-1.5
-1
-0.5
0
0.5
1
1.5
2
Time (Sec.)
Ampl
itude
(Nor
mal
ized)
MEG EEG