Radial Basis Functions:

Developments and applications toplanetary scale flows

Natasha Flyer NCAR, IMAGeInstitute for Mathematics Applied to the Geosciences

in collaboration with, and presented by

Bengt Fornberg University of Colorado Boulder, Department of Applied Mathematics

Slide 1 of 27

Presentation outline:

Slides Topic

3 Finite Difference (FD) approximations

4 - 5 Pseudospectral (PS) approximations in 1-D and 2-D

6 - 8 RBF introduction

9 - 10 Flat RBFs - why are they interesting?

11 - 12 Numerically stable algorithms: Contour-Padé and RBF -QR

13 - 14 PS vs. RBF derivative approximations

15 - 17 RBF interpolation and PDE solution on a sphere

18 - 19 Vortex roll-up test case

20 - 22 Shallow water equations on a sphere

23 - 26 RBFs for computing mantle convection

27 Conclusions

Slide 2 of 27

Finite difference methods: Limits of increasing o rders exist

Approximate derivatives by local difference formulas.

Examples of some explicit FD formulas on an equispa ced grid:

[1]ØuØx = [ − 1

2 0 12 ] u

h + O(h2)= [ 1

12 − 23 0 2

3 − 112 ] u

h + O(h4)= [ − 1

603

20 − 34 0 3

4 − 320

160 ] u

h + O(h6)o o o o o o o

[¢ − 13

12 − 1 0 1 − 1

213 ¢] u

h PS limit

Slide 3 of 27

Pseudospectral (PS) methods can be seen as limits of increasing order FD methods

Periodic problem - Fourier-PS Method:

Use equispaced grid: We get exactly the same derivative approximations at nodes if we:

i. Extend periodically, and then apply limiting FD method,ii. Find interpolating trig polynomial by FFT, take its analytic derivative. O(N log N) op. for N points

Non-Periodic Problem - Chebyshev-PS Method:

Runge Phenomenon Does not arise if the nodes are suitably clustered near the boundaries.Example: f(x) = 1/(1+16x2)

Equispaced nodes Chebyshev-spaced nodes

i. Create global FD formulas separately for each node point, O(N 2) operations for N pointsii. Get Chebyshev polynomial expansion (by FFT), take its analytic derivative; O(N log N) operations.

Slide 4 of 27

How do PS methods work in more than 1-D ?

2-D:

They generalize to Tensor product type grids

BUT

NO set of basis functions can guarantee a nonsingular system in the case of scattered nodes

Scattered nodes Interpolant s(x) = �k=0

N

ck Tk(x)System that determines the expansion coefficients:

T0(x0) T1(x

0) £ TN(x

0)

T0(x1) T1(x

1) £ TN(x

1)

§ § • §

T0(xN

) T1(xN) £ TN(x

N)

c0

c1

§

cN

=

f0

f1

§

fN

Move two nodes so they exchange locations:

Two rows become interchanged, determinant changes sign

fl determinant is zero somewhere along the way.

Slide 5 of 27

RBF idea, In pictures:

1970 Invention of RBFs (for application in cartography)

Some other key dates:

1940 Unconditional non-singularity for many types of radial functions1984 Unconditional non-singularity for multiquadrics ( )�(r) = 1 + (� r )2

1990 First application to numerical solutions of PDEs2002 Flat RBF limit exists - generalizes all 'classical' pseudospectral methods 2004 First numerically stable algorithm in flat basis function limit2007 First application of RBFs to large-scale geophysical test problems2009 A new physical phenomenon (a certain mantle convection instability) initially discovered

in an RBF calculation

Slide 6 of 27

RBF idea, In formulas:

Given scattered data (xk , fk), k = 1, 2, ... , N, in d-D, the coefficients in the interpolant�k

s(x) = �k=1

N

�k �(|| x − xk||)

are found by collocation: , k = 1, 2, ... , N :s(xk) = fk

�(||x1

− x1||) �(||x

1− x

2||) £ �(||x1 − x

N||)

�(||x2

− x1||) �(||x

2− x

2||) £ �(||x

2− x

N||)

§ § •§

�(||xN

− x1||) �(||x

N− x

2||)£ �(||x

N− x

N||)

�1

§

§

�N

=

f1

§

§

fN

Two key theorems:

- For most radial functions , this system can never be singular�(r)

- The interpolation is spectrally accurate for smooth radial functions

Slide 7 of 27

Many types of RBF s:

Piecewise smooth φ(r) Infinitely smooth φ(r)

Cubics TP splines Multiquadric Gaussian Inverse quadratic

r3 r2 log r 1 + (� r)2 e−(� r)2 11 + (� r)2

Interpolation guaranteed nonsingular forall commonly used radial functions.

- Piecewise smooth RBFs:- algebraic accuracy (cf. cubic splines)- minimize 'wiggles'- basis function can have compact support

- Infinitely smooth RBFs:- spectral accuracy (if no Runge phenomenon)

Slide 8 of 27

Numerical conditioning, and the flat RBF limit (ε →ε →ε →ε → 0)

Classical basis functions are usually RBFs are translates of onehighly oscillatory single function - here �(r) = e−(� r)2

In case of 41 scattered nodes in 1-D: cond(A) = , det(A) = .O(�−80) O(�1640)

2-D: cond(A) = , det(A) = .O(�−16) O(�416)

Exact formulas available for any number of nodes in any number of dimensions(Fornberg and Zuev, 2007)

Extreme ill-conditioning typical as ε → 0.

Slide 9 of 27

Why are flat (or near-flat) RBFs interesting ?

- Intriguing error trends as ε → 0'Toy-problem' example: 41 node MQ interpolation of f(x1,x2) = 59

67 + (x1 + 17 )2 + (x2 − 1

11 )

- RBF interpolant in 1-D reduces to Lagrange's interpolation polynomial (Driscoll and Fornberg, 2002)

- The ε → 0 limit reduces to 'classical' PS methods if used on tensor type grids.

- The RBF approach generalize PS methods in many ways:- Guaranteed nonsingular also for scattered nodes on irregular geometries- Allow spectral accuracy to be combined with mesh refinement- Best accuracy often obtained for non-zero ε.

Solving followed by evaluating A� = f s(x, �) =�k=1N�k �(||x − x

k||)

is merely an unstable algorithm for a stable problem

Slide 10 of 27

Numerical computations for small values of εεεε (near-flat RBFs)

It is possible to create algorithms that completely bypass ill-conditioning all the wayinto ε→ 0 limit, while using only standard precision arithmetic:

Concept: Find a computational path from f to s(x,ε) that does not go via theill-conditioned λ.

- Contour-Padé algorithm First algorithm of its kind; established that the concept is possible

Limited to relatively small N-values (Fornberg and Wright, 2004)

Improved versions under development (Fornberg, Wright, and Trefethen).

- RBF-QR method Initially developed for nodes scattered over the surface of a sphereNo limit on N; cost about five times that of RBF-Direct

Original version for nodes on sphere (Fornberg and Piret, 2007).Version for 2-D developed (Fornberg, Larsson, and Flyer, 2009).Version for 3-D under development (Fornberg and Larsson).

Probably many more completely stable algorithms to come

Slide 11 of 27

The concept for the RBF-QR method Recognize that the existence of an ill-conditioned basis does not imply that the spanned space is bad.

Ex. 1: 3-D space Ex. 2: Polynomials of degree ≤≤≤≤ 100

Bad Basis Good Basis

xn , n = 0, 1,¢, 100 Tn(x) , n = 0, 1,¢, 100Bad Basis Good Basis

Ex. 3: Space spanned by RBFs in their flat limit� d 0- The spanned space turns out to be excellent

for computational work - just the basis that isbad.

- Is there any Good Basis in exactly the same space?

- RBF-QR finds such a basis through someanalytical expansions, leading to numerical steps that all remain completely stable evenin the flat basis function limit.

Bad Basis

Slide 12 of 27

-1-0.5

00.5

10

2

4

10

20

50

100

-1

0

1

n

x

y

-1-0.5

0

0.5

10

2

4

10

20

50

100

0

0.5

1

n

x

y

-1-0.5

0

0.51

0

2

4

10

20

50

100

0

0.5

1

n

x

y

PS vs. RBF derivative approximations (Fornberg, Flyer, Russell, 2008)

There is something strange about how FD Where should pick up its data from?Ø

Øxand PS methods approximate Answer: Very heuristic analysis suggests1

2Ø

Øx + ØØy

something reminiscent of the

function x8 0F1(3,− 14 (x2 + y2))

Even on a lattice, RBFs typically pick up

information for also from beside theØ

Øxx-axis.

Slide 13 of 27

→x

↑ y

The main strength of RBFs is their geometric flexib ility. However, even in a pure 2-Dperiodic geometry, they can approximate just as wel l or even better than Fourier-PS

Example: Interpolation errors in a 2-D periodic test case

Cartesian : Fourier - PS

Other three cases : RBFs on different node layouts

Spectral convergence in all cases. Slide 14 of 27

Interpolation on a sphere - RBF-Direct vs. RBF-QR (Fornberg and Piret, 2007)

Test function: 1849 minimal energy nodes Errors as function of εεεε

f(x) = e−7(x+ 12 )2−8(y+ 1

2 )2−9(z− 12

)2

RBF-Direct: with the obtained by solving .s(x) = �k=1

N

�k �(|| x − xk||) �k A� = f

cond(A) = O(ε -84)

To lower the critical ε by a factor of 100 by means of high precision arithmetic requiresthe numerical precision to be raised from standard 16 digits to close to 1000 digits.

RBF-QR: With the new basis functions in exactly the same approximation space,cond(A') = O(1).

Slide 15 of 27

Solving PDEs on a sphere

Governing PDE:

⇒ØuØt + (cos � − tan � sin� sin �) Øu

Ø�− cos� sin � Øu

�= 0

RBF collocation of PDE in spherical coordinates eliminatesall coordinate-based singularities

Spectrally accurate numerical methods: DF: Double Fourier series

RBF: Radial Basis Functions Below: 1849 minimum energy nodes

SE: Spectral ElementsImplemented by means of cubed sphere

SPH: Spherical HarmonicsCollocation with orthogonal set of trig-like basis functions, which lead to entirely uniformresolution over surface of sphere

Slide 16 of 27

Convective flow over a sphere - Comparison between methods (Flyer and Wright, 2007) (10- 3)

Snapshot of 4096 node RBF calculation after 12 days Error (parts per thousand)(1 full revolution) of a cosine bell

yes> 10006 minutes7,7760.005Spectral elementsSEno> 10090 seconds32,7680.005Double FourierDFno> 50090 seconds32,7680.005Spherical harmonicsSPHyes< 401/2 hour4,0960.006Radial basis functionsRBF

Local refine-ment feasible

Code length(lines)

Time step(RK4)

Number of nodepoints/

free parameters

l2 errorMethod

(Fornberg and Piret, 2008): Time stepping thousands of full revolutions and using RBF-QR - still nosignificant error growth or trailing dispersive waves.

Slide 17 of 27

Moving Vortex Roll-Up on A Sphere: Local Node Refin ement (Flyer and Lehto, 2009)

Initial condition Solution after 12 daysLinear convectionwith a vortex-likeflow field

Numerical implementation Minimal energy (ME) nodes Refined nodes

IMQ RBFs: �(r) = 11 + �2r2

N = 3136 nodes (1849 shown in figures to the right)

Method of lines (MOL) time stepping with standardRunge-Kutta, 4th order

Slide 18 of 27

Numerical RBF solution at 12 days Error at 12 days (10- 4)

2 ⋅ 10 -31-35 o - 0.625 o- Finite Volume (3 levels. lat-long)FV8 ⋅ 10 -520-3,136Radial basis functionsRBF

With local refinement7 ⋅ 10 -36-9,600Discontinuous GalerkinDG2 ⋅ 10 -330-38,400Finite Volume (cubed sphere)FV2 ⋅ 10 -3100.625 o165,888Finite Volume (lat-long grid)FV4 ⋅ 10 -3606.4 o3,136Radial basis functionsRBF

Without local refinementl2 errorMinutesTypical angularN (total)ErrorTime stepResolutionMethod

Slide 19 of 27

Nonlinear Unsteady Shallow Water Equations (Flyer and Wright, 2009)

Forcing terms added to the shallow water equations to generate a flow that mimics a shortwave trough embedded in a westerly jet.

Initial Velocity Field Initial geopotential height field(equivalent to pressure field)

Slide 20 of 27

Errors after wave trough traveled one revolution

Total physical run time 5 days

Increase in error with time for different N nodes, N = 3136Numerical time step �t = 10 minutes,Error in max norm, white < 10−5

Slide 21 of 27

Comparison with Commonly Used Methods and Benchmark s

Accuracy obtained with different methods, and numeri cal time step needed for stability

4.0 ⋅ 10 -545 seconds24,5766.5 ⋅ 10 -390 seconds6,144Spectral Element2.0 ⋅ 10 -33 minutes8,192 (1,849)Spherical Harmonic4.0 ⋅ 10 -490 seconds32,7688.2 ⋅ 10 -43 minutes8,1923.9 ⋅ 10 -16 minutes2,048Double Fourier1.0 ⋅ 10 -86 minutes5,0412.5 ⋅ 10 -7 8 minutes4,0968.8 ⋅ 10 -615 minutes3,1363.5 ⋅ 10 -324 minutes1,8494.8 ⋅ 10 -140 minutes784RBF

Relative l2 errorTime stepNMethod

Computational times for the RBF method , in Matlab on 2.66 GHz PC

24 minutes 0.605,041 6 minutes0.414,096 2 minutes0.253,13633 seconds0.111,849

5 seconds 0.03784Total RuntimeRuntime per time step (s)N

Slide 22 of 27

Thermal Convection in a 3-D Spherical Shell (Flyer and Wright, 2009)

Equations : (Ra = Rayleigh number; related to ratio of heat convection to heat conduction)

= $u = 0<u −=p + RaTe

r= 0

ØTØt + u $ =T = <T

Node Layout for hybrid RBF-Chebyshev discretization :

Slide 23 of 27

Ra = 7,000 test case

Initial condition N =1849 nodes on each spherical shellY40 + 5

7Y44

M = 31 shells, Perturbation temperature shownBlue - down flow, Yellow - up flow, Red - core

Comparisons against main previous results from the literature

0.2157831.08233.6096 57,319RBF-ChebyshevRBF-CH0.2157831.08213.6096 552,960Spherical harmonics -FDSH-FD0.2159431.02263.5983 663,552Finite volumeFV0.2159732.63083.4945 12,582,912Finite differencesFD0.2176 31.09 3.6254 393,216Finite elementsFE

T<VRMS >NuouterNo of nodesMethod

Slide 24 of 27

Example of computed solution for Ra = 500,000

Isosurfaces of perturbedtemperature:

Calculation on PC system;

Regime not reached withany alternative numericalmethod.

At somewhat lower Ra numbers, a similar RBF calculation revealed anunexpected physical instability, afterwards confirmed on the Japanese EarthSimulator.

Slide 25 of 27

What is next in modeling geophysical flows with RBF s?

Some things currently on the radar:

1) Dynamically adaptive node refinement

2) RBF-FD, implemented on scattered nodes (create FD formulas are exact forRBFs rather than for polynomials)

3) Carry out RBF-FD analysis, and devise hyperviscosity-type local smoothingoperators.

4) Implement RBF codes on parallel Graphical Processing Units (GPUs), and lateron other parallel architectures.

Slide 26 of 27

Conclusions

Established:

- RBFs can be seen as a generalization of PS methods to arbitrarily shaped domains.

- RBFs combine spectral accuracy with flexible opportunities for local node refinement.

- RBFs can offer excellent accuracy also over very long integration times.

- The near-flat basis function regime (ε small) has been found to be of particular interest,and genuinely stable numerical algorithms have been developed.

- For certain classes of convective flow problems, RBF-based solutions are more accurateand more cost-effective than any other numerical method.

Current research issues:

- Compare RBFs against alternative methods in more major applications.

- Explore further the combination of spectral accuracy with local node refinement.

- Find RBF algorithms that combine high speed with numerical stability (for small ε).

- Develop further the concept of RBF-FD (RBF-generated FD formulas for scattered nodecases ).

- Develop effective implementations first on GPUs, and later on massively parallel(peta-scale) computer hardware.

Slide 27 of 27