Nonlinear Equations

How can we solve these equations?• Spring force: ! = # $

What is the displacement when ! = 2N?

! = 40 %/'

-

IF- = Kx o

x = I = -2N -0.05M (XI 0.05Mk 40 Nlm

How can we solve these equations?• Drag force: ! = 0.5 +( , - .) = /( .)

What is the velocity when! = 20N?

*! = 0.5 %-/'

=

⇐÷H←,TRA to ⇐6.5 mls

F- give → it -Ff → v --IFA ⇒ E- 6.3 m/s

0 . = /( .)−! = 0*! = 0.5 %-/'

Nonlinear Equations in 1D

Goal: Solve 0 $ = 0 for 0:ℛ → ℛ

Find the root (zero) of the nonlinear equation 0 .

Often called Root Finding

E-µ if ⇒FIZ -Of-(v)= 0

IT→tf@loQ6-3m1s0O0r7ooo.s=

numerical -iterative

Bisection method* Define interval that

ghastlier• interval :[a.b ]

fan)>O l a=o;b

KH ql to - lb-a 1=10

* Define midpoint:

is 1/1 f m-ts.az -softa)l0 l

B Bo ! beto * check the signs toA-0

Aiffca.fm)> o :

igneous tan:c:[miss.lt//sinmkqItesIffa).f(b)so | elsenew = Ia ,my

=

b. = M

Bisection method to = lb -al a- to

,

ti-lb-al-e.toi 2 2

ma l te tf -- tff ll l

! I,

ts = tf - tfi i

j i ! !

A=0 :

/its b-to

fritzto = 10 -

* every iteration , theinterval

is divided by 2 !

ConvergenceAn iterative method converges with rate 5 if:

lim.→0||2!"#||||2!||$

= +, 0 < + < ∞ 5 = 1: linear convergence

Linear convergence gains a constant number of accurate digits each step (and + < 1 matters!)

For example: Power Iteration=kjm4fItY¥7constant-8 → linear convergence

& Xz- Xi → constant f I → show convergenceX, = A Xz → C = at→ faster convergence as

a increases

ConvergenceAn iterative method converges with rate 5 if:

lim.→0||>.34||||>.||5

= +, 0 < + < ∞

5 = 1: linear convergence5 > 1: superlinear convergence5 = 2: quadratic convergence

Linear convergence gains a constant number of accurate digits each step (and + < 1 matters!)

Quadratic convergence doubles the number of accurate digits in each step (however it only starts making sense once ||>.|| is small (and + does not matter much)

E'"" ""

l L re 2

Convergence• The bisection method does not estimate $., the approximation of the

desired root $. It instead finds an interval smaller than a given tolerance that contains the root.

X*is the root

* → ta errox*

¥ tr L to I → stop

'⑤←t:i÷÷÷÷:¥÷÷.at

the

1¥ lr-Itc-Ige-adienea.nu.

Example:Consider the nonlinear equation

0 $ = 0.5$) − 2

and solving f x = 0 using the Bisection Method. For each of the initial intervals below, how many iterations are required to ensure the root is accurate within 267?

A) [−10,−1.8]

B) [−3,−2.1]

C) [−4, 1.9]

in general : ties lol

1bjgIcto12k2lb-alTJeogzfbIoaT@fkslogzf8z.Ig)-7.3flat flb)

ffa).f(b) so →ok ! ( 8 iterations )

Muffat.fCbI2o→hot0k!k > logzf5.fi/- 6.56f-(a) fasho → on ! G- iterations )

Bisection method

Algorithm:1.Take two points, ! and ", on each side of the root such that #(!) and #(") have opposite signs.

2.Calculate the midpoint & = !"#$

3. Evaluate #(&) and use & to replace either ! or ", keeping the signs of the endpoints opposite.

.i..

'' Ii iii.

Bisection Method - summary

q The function must be continuous with a root in the interval L, M

q Requires only one function evaluations for each iteration!o The first iteration requires two function evaluations.

q Given the initial internal [L, M], the length of the interval after #iterations is 869)!

q Has linear convergence

=

e-

Newton’s method• Recall we want to solve ! " = 0 for !:ℛ → ℛ

• The Taylor expansion:

! "! + ℎ ≈ ! "! + !′ "! ℎ

gives a linear approximation for the nonlinear function ! near "!.

linear approximationof fCx

)

EE-- -

= Ich)

f- (xnth) = O

Ich)⇒ → fan) t f'Au) h = O

Neawetgoritnm : fh=-s¥;I→Newtons×. = random tininess )#→ Newton updatef-Go), f'Go) → h → ftp.ti-Xkth

Newton’s method

:"

Find x# s .t . f#too

ft (Stx)

- line atxn

fed-- - - -- - - - -•- slope.IS#-O---ftxn)Jg.tangent

Xin - Xktl

° . .

t'O / Yeti g

xf'Gr)

×* *FfHH=×k-f¥ITy

ExampleConsider solving the nonlinear equation

5 = 2.0 /" + "#

What is the result of applying one iteration of Newton’s method for solving nonlinear equations with initial starting guess "$ = 0, i.e. what is "%?

A) −2B) 0.75C) −1.5D) 1.5E) 3.0

X,= ?

Xo = O

↳→

ffxf-2ettx-5.gl/kti=Xrthh=-fCXn)DftXI=2e+zxf'Cxn)

Xo → ffto)- I -5=-3

six, - ah=¥÷fIh -4.5

x,= Xo th - Ot 1.5 -71×1=1.57

Newton’s Method - summary

q Must be started with initial guess close enough to root (convergence is only local). Otherwise it may not converge at all.

q Requires function and first derivative evaluation at each iteration (think about two function evaluations)

q Typically has quadratic convergence

lim.→0||>.34||||>.||)

= +, 0 < + < ∞

q What can we do when the derivative evaluation is too costly (or difficult to evaluate)?

-

-

-

④re

Secant methodAlso derived from Taylor expansion, but instead of using 0′ $. , it approximates the tangent with the secant line:

$.34 = $. − 0 $. /0′ $.

If ⇒ approximation for f'(x)

→ Xkt, = Xk- f(X#§ df(xn)

Tzpointss ! 1×0,57

*÷⇒;/:::÷÷÷¥far)--- - -- -9-t, XkH=Xk-f§¥÷#¥• t !

y* Xk Xk-l

re

Secant Method - summary

q Still local convergence

q Requires only one function evaluation per iteration (only the first iteration requires two function evaluations)

q Needs two starting guesses

q Has slower convergence than Newton’s Method – superlinearconvergence

lim.→0||>.34||||>.||5

= +, 1 < 5 < 2

f'→ df

1D methods for root finding:Method Update Convergence Cost

Bisection Check signs of & ' and & (

)! =|( − '|2!

Linear (/ = 1 and c = 0.5) One function evaluation per iteration, no need to compute derivatives

Secant 6!"# = 6! + ℎ

ℎ = −& 6! /&′ 6!

Superlinear / = 1.618 ,local convergence properties, convergence depends on the initial guess

One function evaluation per iteration (two evaluations for the initial guesses only), no need to compute derivatives

Newton 6!"# = 6! + ℎ

ℎ = −& 6! />&'

>&' = & 6! − & 6!$#6! − 6!$#

Quadratic / = 2 , local convergence properties,convergence depends on the initial guess

Two function evaluations per iteration, requires first order derivatives

Nonlinear system of equations

https://www.youtube.com/watch?v=NRgNDlVtmz0 (Robotic arm 1)https://www.youtube.com/watch?v=9DqRkLQ5Sv8 (Robotic arm 2)https://www.youtube.com/watch?v=DZ_ocmY8xEI (Blender)

Robotic arms

Inverse Kinematicsn

Given :

a

i.ee#----*fflQl--oIf- , ( Oi ,02,8) = O

:*: :: Him.is. iterative=

Nonlinear system of equationsGoal: Solve P Q = R for P:ℛ; → ℛ;

= =

Elk) -- e

*⇒ if:c:: ÷:sew

ink . x. x. . . . . .

-It? If? -- 4→ fi -42-24×2-¥4 --of-z =-2X, -3×2+5--0

Newton’s methodApproximate the nonlinear function P Q by a linear function using Taylor expansion:

( ND)

Z

1-(Its) - I t.IE) g- - Ef 'T vectorvector vector-

vector

i:::÷÷÷÷÷i÷÷÷÷:÷÷÷÷÷¥÷÷

f-Ets) EEE) t EE) E÷O O

ICI t EG) E - on→ solve for E

|IEIE=-f Linear system-

of equation

Io - initial vector-

IKH - Intf,I

Newton’s methodAlgorithm:

Convergence: • Typically has quadratic convergence• Drawback: Still only locally convergent

Cost:• Main cost associated with computing the Jacobian matrix and solving

the Newton step.

IE) -- e

Io : initial guessfor i = I , R , - - -ri - evaluate Jkr) -

an.÷÷÷÷÷¥¥÷÷.:S. Is:÷÷:÷÷÷of equation-

④O r

-#

ExampleConsider solving the nonlinear system of equations

2 = 2) + +4 = +$ + 4)$

What is the result of applying one iteration of Newton’s method with the following initial guess?

-% = 10

µ

E-

-0¥'t. :DE- rats . its:]

:¥¥¥**i::F÷¥÷÷.↳=L'oItfz.FI?oIs] es.

Newton’s method!0 = #$#%#&' ()*++

,-. / = 1,2, …

Evaluate 4 = 5 !1

Evaluate 6 !1

Factorization of Jacobian (for example 78 = 5)

Solve using factorized J (for example 78 91 = −6 !1

Update !123 = !1+ 91

we

-

Newton’s method - summaryq Typically quadratic convergence (local convergence)

q Computing the Jacobian matrix requires the equivalent of S) function evaluations for a dense problem (where every function of P Q depends on every component of Q).

q Computation of the Jacobian may be cheaper if the matrix is sparse.

q The cost of calculating the step T is U S< for a dense Jacobian matrix (Factorization + Solve)

q If the same Jacobian matrix V Q. is reused for several consecutive iterations, the convergence rate will suffer accordingly (trade-off between cost per iteration and number of iterations needed for convergence)

-

o

E

Inverse Kinematics X, y , p → 9 , Oz , 03^

a =Nifty ✓

a.bgivezga.a.no¥i¥ : Eat:im ri.←*#

Tc'= oftb - Iab cosOz →fi-E-a-bt2abcos.bz#b2--atE-2accosa,→¥=b?aIt2accos(Oi-af=J

4/80-0 , -0e) tozt p+904940-da ) =360T

-69¥If?If, I fffs=-9-0ztOstpTI