Certified and fast computation of supremum norms of approximation

Certi�ed and fast computation of supremumnorms of approximation errors

Sylvain Chevillard Mioara Joldes Christoph Lauter

May 26, 2009

1 / 32

Outline

Motivation

Correctly rounded elementary functionsSupremum norm of error functionsPrevious approaches and di�culties

Our approach

Automatic di�erentiation and Taylor modelsIsolation of roots of polynomialsEnclosure of roots of functions

Results & Conclusion

2 / 32

Motivation

�It makes me nervous to �y on airplanes, since I know they are

designed using �oating-point arithmetic.�

A. Householder

Software systems: scienti�c computing, �nancial, embeddedsystems.

Need to compute sin, cos, exp in �nite precision,�oating-point environment.

Most Mathematical Libraries do not provide correctly roundedfunctions.

IEEE-754 standard revision (June 2008) recommends correctrounding.

Arenaire team develops the Correctly Rounded Libm(CRLibm)1.

1http://lipforge.ens-lyon.fr/www/crlibm/

3 / 32

Motivation



A. Householder







3 / 32

Motivation



A. Householder







3 / 32

Motivation



A. Householder






1http://lipforge.ens-lyon.fr/www/crlibm/3 / 32

Correctly rounded functions

4 / 32


4 / 32


Table Maker's Dilemma

Increase working precision

Worst cases search - V. Lefevre and J-M. Muller

5 / 32

Supremum Norms of Error Functions

ε(x) = f(x)− p(x), x ∈ [a, b] or

ε(x) = p(x)f(x) − 1, x ∈ [a, b]

De�ne ‖ε‖∞ = supx∈[a, b]{|ε(x)|}

Compute a certi�ed bound for the supremum norm of an errorfunction

�Quick and dirty� supremum norms - another class ofalgorithms

6 / 32


ε(x) = f(x)− p(x), x ∈ [a, b] or

ε(x) = p(x)f(x) − 1, x ∈ [a, b]




6 / 32


ε(x) = f(x)− p(x), x ∈ [a, b] or

ε(x) = p(x)f(x) − 1, x ∈ [a, b]




6 / 32


ε(x) = f(x)− p(x), x ∈ [a, b] or

ε(x) = p(x)f(x) − 1, x ∈ [a, b]




6 / 32


Error ε(x) = f(x)− p(x) or ε(x) = p(x)f(x) − 1, x ∈ [a, b]

De�ne ‖ε‖∞ = supx∈[a, b]{|ε(x)|}Purpose: Compute a certi�ed bound for the supremum normof an error function

Given p and f �nd a narrow interval r such that ‖ε‖∞ ∈ r.

Need for a fast and certi�ed algorithm:

Correctly rounded elementary functions

For computing the minimum error between a function andthousands of polynomials with �oating-point coe�cients

7 / 32

Supremum norm of error functions

Example:

f(x) = ex, x ∈ [0, 1]p(x) =

∑5i=0 cix

i s.t. ‖f − p‖∞ is as small as possible (Remezalgorithm)ε(x) = f(x)− p(x)

How to obtain a certi�ed and tight bound?

8 / 32


Example:

f(x) = ex, x ∈ [0, 1]p(x) =

∑5i=0 cix


How to obtain a certi�ed and tight bound?

8 / 32


Example:

f(x) = ex, x ∈ [0, 1]p(x) =

∑5i=0 cix


How to obtain a certi�ed and tight bound?8 / 32

Our Problem - Prone to High Dependency Phenomenon

Example:

f(x) = ex, x ∈ [0, 1]p(x) =

∑5i=0 cix


Using IA, ε(x) ∈ [−0.4, 0.4], but ‖ε(x)‖∞ ' 1.1295e− 6:9 / 32


Example:

f(x) = ex, x ∈ [0, 1]p(x) =

∑5i=0 cix


Using IA, ε(x) ∈ [−0.4, 0.4], but ‖ε(x)‖∞ ' 1.1295e− 6:9 / 32


In this case, over [0, 1] we need 107 intervals!

10 / 32

Previous Approaches

Floating-point techniques - not �safe�

Existent Interval Arithmetic methods - Not su�cient

Global optimization software (eg. Globsol) - not tailored forour speci�c problem

Chevillard and Lauter's technique: interval arithmetic, tightbounding of the zeros of the derivative of the error function,removes false singularities (like x = 0 for sin(x)/x). Highcomputation time for deg(p) > 10.Techniques based on a high order Taylor expansion of the errorfunction and a su�ciently close bounding of the remainder

Certi�ed polynomial approximations of analytic functions.

11 / 32

Our Approach

How to obtain a certi�ed and tight bound for‖ε‖∞ = ‖f − p‖∞ AND ‖ε‖∞ = ‖p/f − 1‖∞, over giveninterval [a, b]?

Use multiprecision IA: MPFI 2

Use a higher degree approximation polynomial T for thefunction f

Compute (T,∆) s.t f(x)− T (x) ∈ ∆,∀x ∈ [a, b].Break the dependency problem in two:

‖f − p‖∞ ≤ ‖f − T‖∞︸︷︷︸

bounding a remainder

+ ‖T − p‖∞︸︷︷︸

tightly bounding a polynomial

1 Automatic Di�erentiation/ Taylor Models

2 Tightly bounding the polynomial di�erence - Roots isolationand re�nement techniques

2

http://gforge.inria.fr/projects/mp�/

12 / 32

Our Approach

How to obtain a certi�ed and tight bound for‖ε‖∞ = ‖f − p‖∞ AND ‖ε‖∞ = ‖p/f − 1‖∞, over giveninterval [a, b]?Use multiprecision IA: MPFI 2



‖f − p‖∞ ≤ ‖f − T‖∞︸︷︷︸


+ ‖T − p‖∞︸︷︷︸




2http://gforge.inria.fr/projects/mp�/12 / 32

Our Approach



Compute (T,∆) s.t f(x)− T (x) ∈ ∆,∀x ∈ [a, b].

Break the dependency problem in two:

‖f − p‖∞ ≤ ‖f − T‖∞︸︷︷︸


+ ‖T − p‖∞︸︷︷︸





Our Approach




‖f − p‖∞ ≤ ‖f − T‖∞︸︷︷︸


+ ‖T − p‖∞︸︷︷︸





Our Approach




‖f − p‖∞ ≤ ‖f − T‖∞︸︷︷︸bounding a remainder

+ ‖T − p‖∞︸︷︷︸





Our Approach





+ ‖T − p‖∞︸︷︷︸tightly bounding a polynomial




(1) Computing (T, ∆) - Automatic di�erentiation (AD)

Example:

f(x) = exp(x) over [0, 1]p(x) =

∑5i=0 cix

i

Introduce a higher degree polynomial T : Use AD

T (x) =8∑

i=0

f (i)(1/2)i! (x− 1/2)i =

8∑i=0

exp(1/2)i! (x− 1/2)i

Compute an enclosure of the remainder: Use AD

∆9(x, ξ) =f (9)(ξ)

9!︸︷︷︸∈exp([0, 1])

9!

× (x− 1/2)9︸︷︷︸≤(1/2)(−9)

|∆9(x, ξ)| ∈ 1.4630578142[1; 2]× e− 8

13 / 32


Example:

f(x) = exp(x) over [0, 1]p(x) =

∑5i=0 cix

i


T (x) =8∑

i=0

f (i)(1/2)i! (x− 1/2)i =

8∑i=0

exp(1/2)i! (x− 1/2)i


∆9(x, ξ) =f (9)(ξ)

9!︸︷︷︸∈exp([0, 1])

9!

× (x− 1/2)9︸︷︷︸≤(1/2)(−9)

|∆9(x, ξ)| ∈ 1.4630578142[1; 2]× e− 8

13 / 32


Example:

f(x) = exp(x) over [0, 1]p(x) =

∑5i=0 cix

i


T (x) =8∑

i=0

f (i)(1/2)i! (x− 1/2)i =

8∑i=0

exp(1/2)i! (x− 1/2)i


∆9(x, ξ) =f (9)(ξ)

9!︸︷︷︸∈exp([0, 1])

9!

× (x− 1/2)9︸︷︷︸≤(1/2)(−9)

|∆9(x, ξ)| ∈ 1.4630578142[1; 2]× e− 813 / 32


Main idea:


+ ‖T − p‖∞︸︷︷︸bounding a polynomial

Achieved so far:

‖exp−p‖∞ ≤ ‖exp−T‖∞︸︷︷︸∈ 1.4630578142[1;2]×e−8

+ ‖T − p‖∞︸︷︷︸bounding a polynomial

14 / 32

(2)Bounding the polynomial di�erence

Purpose: Tightly bound ‖T − p‖∞ over the interval [a, b]

Example:

p(x) =∑5

i=0 cixi

T (x) =8∑

i=0

f (i)(1/2)i! (x− 1/2)i

Tightly bound theroots of the derivative

Evaluate usinginterval arithmetic

15 / 32



Example:

p(x) =∑5

i=0 cixi

T (x) =8∑

i=0

f (i)(1/2)i! (x− 1/2)i



15 / 32



Example:

p(x) =∑5

i=0 cixi

T (x) =8∑

i=0

f (i)(1/2)i! (x− 1/2)i



15 / 32



Example:

p(x) =∑5

i=0 cixi

T (x) =8∑

i=0

f (i)(1/2)i! (x− 1/2)i



15 / 32

(3)Isolation and re�nement of roots of polynomials

Techniques based on counting the number of roots inside aninterval considered

Sturm TheoremDescartes' Rule of Signs

Use a bisection strategy for isolating the roots

Use dichotomy or Newton iteration process

16 / 32






16 / 32






16 / 32






16 / 32


r0 ∈ 0.0068[5; 6], r1 ∈ 0.2544[3; 4],r2 ∈ 0.5059[4; 5], r3 ∈ 0.7544[7; 8],r4 ∈ 0.9345[8; 9]

‖T − p‖∞ ∈ 1.11486943[1; 2]× e− 6

17 / 32


r0 ∈ 0.0068[5; 6], r1 ∈ 0.2544[3; 4],r2 ∈ 0.5059[4; 5], r3 ∈ 0.7544[7; 8],r4 ∈ 0.9345[8; 9]‖T − p‖∞ ∈ 1.11486943[1; 2]× e− 6

17 / 32

Our Approach - Summary

1 Purpose: fast and safely compute the supremum norm‖f − p‖∞ over an interval [a, b]

2 Introduce a higher degree approximation polynomial:‖f − p‖∞ ≤ ‖f − T‖∞ + ‖T − p‖∞ (AD/TM)

3 Bound the remainder (AD/TM)

4 Bound the polynomial di�erence (polynomial roots isolation)

5 Add the two bounds to obtain a tight and safe bound of theapproximation error

Our example:

‖f − p‖∞ ≤ ‖f − T‖∞︸︷︷︸∈ 1.4630578142[1;2]×e−8

+ ‖T − p‖∞︸︷︷︸∈ 1.11486943[1;2]×e−6

‖f − p‖∞ ∈ 1.1295[0; 1]× e− 6

IA: ‖f − p‖∞ ≤ 0.4, sampling: ‖f − p‖∞ ' 1.1295e− 6.

18 / 32







Our example:

‖f − p‖∞ ≤ ‖f − T‖∞︸︷︷︸∈ 1.4630578142[1;2]×e−8

+ ‖T − p‖∞︸︷︷︸∈ 1.11486943[1;2]×e−6

‖f − p‖∞ ∈ 1.1295[0; 1]× e− 6


18 / 32







Our example:

‖f − p‖∞ ≤ ‖f − T‖∞︸︷︷︸∈ 1.4630578142[1;2]×e−8

+ ‖T − p‖∞︸︷︷︸∈ 1.11486943[1;2]×e−6

‖f − p‖∞ ∈ 1.1295[0; 1]× e− 6


18 / 32







Our example:

‖f − p‖∞ ≤ ‖f − T‖∞︸︷︷︸∈ 1.4630578142[1;2]×e−8

+ ‖T − p‖∞︸︷︷︸∈ 1.11486943[1;2]×e−6

‖f − p‖∞ ∈ 1.1295[0; 1]× e− 6


18 / 32







Our example:

‖f − p‖∞ ≤ ‖f − T‖∞︸︷︷︸∈ 1.4630578142[1;2]×e−8

+ ‖T − p‖∞︸︷︷︸∈ 1.11486943[1;2]×e−6

‖f − p‖∞ ∈ 1.1295[0; 1]× e− 6


18 / 32







Our example:

‖f − p‖∞ ≤ ‖f − T‖∞︸︷︷︸∈ 1.4630578142[1;2]×e−8

+ ‖T − p‖∞︸︷︷︸∈ 1.11486943[1;2]×e−6

‖f − p‖∞ ∈ 1.1295[0; 1]× e− 6


18 / 32







Our example:

‖f − p‖∞ ≤ ‖f − T‖∞︸︷︷︸∈ 1.4630578142[1;2]×e−8

+ ‖T − p‖∞︸︷︷︸∈ 1.11486943[1;2]×e−6

‖f − p‖∞ ∈ 1.1295[0; 1]× e− 6

IA: ‖f − p‖∞ ≤ 0.4, sampling: ‖f − p‖∞ ' 1.1295e− 6.18 / 32

Our Approach - Relative Error

Compute: ‖ε‖∞ = ‖p/f − 1‖∞ over I

Brutal application of the same principle does not work! WHY?

19 / 32



Brutal application of the same principle does not work! WHY?

19 / 32

(1) Computing (T, ∆) - Taylor Models at a Glance

Couple of form (T,∆), s.t. f − T ∈ ∆.

Propagation of error bounds combined with AD

Start with trivial Taylor Model: (1, [0, 0]).Easily de�ne arithmetic operations: +,−, ∗. Eg.Multiplication:(T1,∆1) ∗ (T2,∆2) =((T1T2)0..n,∆1B(T2) + ∆2B(T1) +B((T1T2)n..2n))Note: The bound B(T ) computed is propagated in theremainder. It in�uences the quality of the remainder!

Usually the bounds on the occuring polynomials are rough!

Tight bounds imply a computational e�ort at each operation.

20 / 32

Relative Error Issues

Example:

f(x) = cos(x) over [−1, 1]p(x) =

∑5i=0 cix

i

ε(x) = p(x)/f(x)− 1

Compute the Taylor Model for ε(x), order 12.

TM for f(x), remainder: [−1.351322e− 10; 1.351322e− 10]TM for 1

f(x) , remainder: [−3.1634304e− 2; 0.258216]

Remainder bounds are unsatisfactory in our case.

21 / 32

Relative Error Issues

Example:

f(x) = cos(x) over [−1, 1]p(x) =

∑5i=0 cix

i

ε(x) = p(x)/f(x)− 1

Compute the Taylor Model for ε(x), order 12.

TM for f(x), remainder: [−1.351322e− 10; 1.351322e− 10]TM for 1

f(x) , remainder: [−3.1634304e− 2; 0.258216]

Remainder bounds are unsatisfactory in our case.

21 / 32



Brutal application of the same principle does not work:

Tight bounds for the polynomials interfering in Taylor Modelshave to be computedSome compositions (inverse function, logarithm) generateintervals that are too wide in our case

Idea: Tightly enclose all the extrema of ε: they are among thezeros of the �rst derivative.

ε′ = (p′f − pf ′)/f2. Consider τ = p′f − pf ′.Solve τ(x) = 0

HOW?

Find small intervals that enclose each root of τ .

Evaluate ε on these intervals.

22 / 32







HOW?



22 / 32






ε′ = (p′f − pf ′)/f2. Consider τ = p′f − pf ′.

Solve τ(x) = 0

HOW?



22 / 32







HOW?



22 / 32







HOW?



22 / 32







HOW?



22 / 32






ε′ = (p′f − pf ′)/f2. Consider τ = p′f − pf ′.Solve τ(x) = 0 HOW?



22 / 32

Finding enclosures of roots of a function

First idea: interval Newton Method BUT Dependencyphenomenon present in τ = p′f − pf ′ also!

Better idea: Compute a Taylor Model (T, [−θ, θ]) s.t.τ(x)− T (x) ∈ [−θ, θ],∀x ∈ Iτ(x) = 0 =⇒ T (x) ∈ [−θ, θ]Compute a list Lu of intervals where T − θ ≤ 0 and Ll whereT + θ ≥ 0

23 / 32


First idea: interval Newton Method BUT Dependencyphenomenon present in τ = p′f − pf ′ also!Better idea: Compute a Taylor Model (T, [−θ, θ]) s.t.τ(x)− T (x) ∈ [−θ, θ],∀x ∈ I

τ(x) = 0 =⇒ T (x) ∈ [−θ, θ]Compute a list Lu of intervals where T − θ ≤ 0 and Ll whereT + θ ≥ 0

23 / 32


First idea: interval Newton Method BUT Dependencyphenomenon present in τ = p′f − pf ′ also!Better idea: Compute a Taylor Model (T, [−θ, θ]) s.t.τ(x)− T (x) ∈ [−θ, θ],∀x ∈ Iτ(x) = 0 =⇒ T (x) ∈ [−θ, θ]

Compute a list Lu of intervals where T − θ ≤ 0 and Ll whereT + θ ≥ 0

23 / 32




23 / 32




23 / 32


First idea: interval Newton Method BUT Dependencyphenomenon present in τ = p′f − pf ′ also!Better idea: Compute a Taylor Model (T, [−θ, θ]) s.t.τ(x)− T (x) ∈ [−θ, θ],∀x ∈ Iτ(x) = 0 =⇒ T (x) ∈ [−θ, θ]Compute a list Lu of intervals where T − θ ≤ 0 and Ll whereT + θ ≥ 0

23 / 32


First idea: interval Newton Method BUT Dependencyphenomenon present in τ = p′f − pf ′ also!Better idea: Compute a Taylor Model (T, [−θ, θ]) s.t.τ(x)− T (x) ∈ [−θ, θ],∀x ∈ Iτ(x) = 0 =⇒ T (x) ∈ [−θ, θ]Compute a list Lu of intervals where T − θ ≤ 0 and Ll whereT + θ ≥ 0

23 / 32


Compute a list of intervals where T − θ ≤ 0, T is a polynomial

Compute enclosures of the roots of T − θCompute sign changes

Find suitable intervals

24 / 32



Compute enclosures of the roots of T − θ

Compute sign changes


24 / 32





24 / 32





24 / 32

Our example - Relative error

Example:

f(x) = cos(x) over [−1, 1], p(x) =∑5

i=0 cixi,

ε(x) = p(x)/f(x)− 1

25 / 32


Example:

f(x) = cos(x) over [−1, 1], p(x) =∑5

i=0 cixi,

ε(x) = p(x)/f(x)− 1

Compute Taylor Model for τ = p′f − pf ′, order 12.

Remainder: [−1.67352e− 7; 1.67352e− 7]

25 / 32


r0 ∈ −0.886[9; 8], r1 ∈ −0.536[6; 4],r2 ∈ [−0.000001; 0.000001], r3 ∈ 0.536[4; 6],r4 ∈ 0.886[8; 9]

25 / 32


r0 ∈ −0.886[9; 8], r1 ∈ −0.536[6; 4],r2 ∈ [−0.000001; 0.000001], r3 ∈ 0.536[4; 6],r4 ∈ 0.886[8; 9]‖ε‖∞ ∈ 5.59[3/4]× e− 5.

25 / 32


Purpose: fast and safely compute the supremum norm‖f − p‖∞ and ‖p/f − 1‖∞ over an interval [a, b]Enclose all the zeros of the �rst derivative of the error

Evaluate the approximation error on these small intervals only,using IA

Use a Taylor Model based approach to overcome thedependency

Enclose the zeros of a function using our algorithm

26 / 32

Our example - Relative error - How small should theremainder be?

τ(x)− T (x) ∈ [−θ, θ],∀x ∈ Iτ(x) = 0 =⇒ T (x) ∈ [−θ, θ]

27 / 32


τ(x)− T (x) ∈ [−θ, θ],∀x ∈ Iτ(x) = 0 =⇒ T (x) ∈ [−θ, θ]

Degree: 1227 / 32


τ(x)− T (x) ∈ [−θ, θ],∀x ∈ Iτ(x) = 0 =⇒ T (x) ∈ [−θ, θ]

27 / 32


τ(x)− T (x) ∈ [−θ, θ],∀x ∈ Iτ(x) = 0 =⇒ T (x) ∈ [−θ, θ]

Degree: 1427 / 32


τ(x)− T (x) ∈ [−θ, θ],∀x ∈ Iτ(x) = 0 =⇒ T (x) ∈ [−θ, θ]

Degree: 827 / 32


Heuristics

Start with a value close to the approx. max value of τ

In our example: θi = 6.44e− 4

θ degree T ‖ε‖∞θi 8 [5.59e− 5; 1.84]θi/2 9 [5.59e− 5; 1e− 3]θi/10 10 [5e− 5; 8e− 5]θi/102 11 0.5[5; 7]e− 4θi/103 12 0.559[3; 4]e− 4θi/104 13 0.55935[2; 4]e− 4θi/105 14 0.5593528[2; 4]e− 4

28 / 32

Results

Experiments were made on an Intel Pentium D 3.00GHz with a2GB RAM.

f [a, b] dp1 m2 acc3 time4

exp(x)− 1 [−0.25, 0.25] 5 r 37.6 412log2(1 + x) [−2−9, 2−9] 7 r 83.3 2, 186

cos(x) [−0.5, 0.25] 15 r 19.5 2, 235exp(x) [−0.125, 0.125] 25 r 42.3 7, 753sin(x) [−0.5, 0.5] 9 a 21.5 520

exp(cos(x)2 + 1) [1, 2] 15 r 25.5 10, 984tan(x) [0.25, 0.5] 10 r 26.0 1, 072x2.5 [1, 2] 7 r 15.5 1, 362

1Degree of p2Error mode considered: a=absolute, r=relative3Accuracy4Timings in ms

29 / 32

Conclusion

Safe and fast algorithm for bounding the supremum norm ofthe error functions

Combination and reusal of various techniques (TM, polynomialroots isolation, interval arith)

Absolute and Relative errors handled

Faster and more accurate than other current approaches

Future works:

Formal proof (AD, isolation of roots, multiple precision intervalarithmetic are needed in the proof checker)Generalization of the algorithm for multivariate functions.

30 / 32

Questions

Thank you for your attention!

Questions?

31 / 32

Results

Experiments were made on an Intel Pentium D 3.00GHz with a2GB RAM.

f [a, b] dp1 m2 dT

3 acc4 time5

exp(x)− 1 [−0.25, 0.25] 5 r 11 37.6 412log2(1 + x) [−2−9, 2−9] 7 r 23 83.3 2, 186

cos(x) [−0.5, 0.25] 15 r 28 19.5 2, 235exp(x) [−0.125, 0.125] 25 r 41 42.3 7, 753sin(x) [−0.5, 0.5] 9 a 14 21.5 520

exp(cos(x)2 + 1) [1, 2] 15 r 60 25.5 10, 984tan(x) [0.25, 0.5] 10 r 21 26.0 1, 072x2.5 [1, 2] 7 r 26 15.5 1, 362

1Degree of p2Error mode considered: a=absolute, r=relative3Degree of T4Accuracy5Timings in ms

32 / 32

Documents

Certified and fast computation of supremum norms of approximation