Characteristic-Based Flux Splitting for Implicit-Explicit Time Integration of Low-Mach Number Flows

Mathematics & Computer Science Argonne National Laboratory

Debojyoti Ghosh Emil M. Constantinescu

13th U. S. National Congress on Computational Mechanics (USNCCM13) San Diego, CA, July 26 – 30, 2015

Motivation & Objectives

2  

Numerical simulation of atmospheric flows Governing equations: 2D Euler equations with gravitational forces (conservation of mass, momentum and energy)

Other (more popular) forms of the governing equations o  Exner pressure, velocity,

p o t e n t i a l t e m p e r a t u re : COAMPS (US Navy), NMM (NCEP), MM5 (NCAR/PSU) .

o  Mass, momentum, potential temperature: WRF (NCAR), NUMA (NPS).

Time scales: entropy (u) << acoustic (u ± a)

Time integration o  Explicit time-integration à time step size restricted by acoustic waves;

but acoustic waves do not significantly impact any atmospheric phenomenon.

o  Implicit time-integration à Unconditionally stable; but requires solutions to non-linear system or linearized approximation.

Ø  Implicit-Explicit (IMEX) time-integration à Integrate “fast” waves implicitly, “slow” waves explicitly.

Ø  Characteristic-based partitioning of the hyperbolic flux (Acoustic waves integrated implicitly, entropy waves integrated explicitly)

Giraldo, Restelli, Laeuter, 2010: P e r t u r b a t i o n - b a s e d I M E X splitting of the hyperbolic flux (first-order perturbations implicit, higher-order perturbations explicit)

Selective preconditioning of acoustic modes •  Implicit Continuous Eulerian

( ICE) t echn ique (Har low, Amsden, 1971)

•  Preconditioning applied to stiff modes (Reynolds, Samtaney, Woodward, 2010)

@

@t

2

664

⇢

⇢u

⇢v

e

3

775+@

@x

2

664

⇢u

⇢u

2 + p

⇢uv

(e+ p)u

3

775+@

@y

2

664

⇢v

⇢uv

⇢v

2 + p

(e+ p)v

3

775 =

2

664

0⇢g · i⇢g · j

⇢ug · i+ ⇢vg · j

3

775

Spatial Discretization

Conservative finite-difference discretization of a hyperbolic conservation law

3

dujdt

+1Δx

f (x j+1/2, t)− f (x j−1/2, t)#$ %&= 0u t+ f (u)x = 0; f '(u)∈ℜ

j j+1 0 N j-1 j-1/2 j+1/2

Cell centers Cell interfaces

Δx

Weighted Essentially Non-Oscillatory (WENO) Schemes §  Weights depend on the local smoothness of the

solution §  Optimal weights in smooth regions allow (2r-1)th

order accuracy §  Near-ze ro we igh t s fo r s t enc i l s w i th

discontinuities à non-oscillatory behavior §  Compact-Reconstruction WENO (CRWENO)

à Higher spectral resolution and lower absolute errors for same order of convergence

CRWENO5 (Compact finite difference scheme)

WENO5 Smoothness

indicator

Characteristic-based Flux Splitting (1)

4  

Separation of acoustic and entropy modes in the flux for implicit-explicit time integration

1D Euler equations Semi-discrete ODE in time Spatial discretization

Discretization operator (e.g.:WENO5, CRWENO5)

Flux Jacobian

-3

-2

-1

0

1

2

3

-3 -2.5 -2 -1.5 -1 -0.5 0

Imaginary

Real

CRWENO5

-200

-150

-100

-50

0

50

100

150

200

-250 -200 -150 -100 -50 0 50

Imag

inary

Real

u+a u-a

u

E i g e n v a l u e s o f t h e CRWENO5 discretization

Eigenvalues of the r i g h t - h a n d - s i d e operator (u=0.1, a=1.0, dx=0.0125)

Example: Periodic density sine wave on a unit domain discretized by N=80 points.

Eigenvalues of the right-hand-side of the ODE are the eigenvalues of the discretization operator times the characteristic speeds of the physical system

Time step size limit for linear stability

Characteristic-based Flux Splitting (2)

5  

Splitting of the flux Jacobian based on its eigenvalues

“Slow” flux

“Fast” Flux

⇤F =

2

40

u+ au� a

3

5

-200

-150

-100

-50

0

50

100

150

200

-250 -200 -150 -100 -50 0 50

Imaginary

Real

F(u) FF(u) FS(u)

Example: Periodic density sine wave on a unit domain discretized by N=80 points (CRWENO5).

Small difference between the eigenvalues of the complete operator and the split operator. (Not an error)

IMEX Time Integration with Characteristic-based Flux Splitting (1)

6  

Apply Implicit-Explicit Runge-Kutta (PETSc - TSARKIMEX) time-integrators

Stage values (s stages)

Step completion

Non-linear system of equations

Solution-dependent weights for the WENO5/CRWENO5 scheme

Linearized Formulation

Redefine the splitting as

Note: Introduces no error in the governing equation.

-200

-150

-100

-50

0

50

100

150

200

-250 -200 -150 -100 -50 0 50

Imaginary

Real

F(u) FF(u) FS(u)

At the beginning of a time step:-

Is FF a good approximation at each stage?

7  

Linearization of the WENO/CRWENO discretization: Within a stage, the non-linear coefficients are kept fixed.

IMEX Time Integration with Characteristic-based Flux Splitting (2)

Linear system of equations for implicit stages:

ARKIMEX 2c •  2nd order accurate •  3 stage (1 explicit, 2 implicit) •  L-Stable implicit part •  Large real stability of explicit part

ARKIMEX 2e •  2nd order accurate •  3 stage (1 explicit, 2 implicit) •  L-Stable implicit part

ARKIMEX 3 •  3rd order accurate •  4 stage (1 explicit, 3 implicit) •  L-Stable implicit part

ARKIMEX 4 •  4th order accurate •  5 stage (1 explicit, 4 implicit) •  L-Stable implicit part

ARK Methods (PETSc)

Preconditioning (Preliminary attempts)

First order upwind discretization Periodic boundaries ignored

•  Jacobian-free approach à Linear Jacobian defined as a function describing its action on a vector (MatShell)

•  Preconditioning matrix à Stored as a sparse matrix (MatAIJ)

Block n-diagonal matrices •  Block tri-diagonal (1D) •  Block penta-diagonal (2D) •  Block septa-diagonal (3D)

Example: 1D Density Wave Advection

8  

Initial solution ⇢ = ⇢1 + ⇢ sin (2⇡x) , u = u1, p = p1; 0 x 1

-100

-50

0

50

100

-100 -80 -60 -40 -20 0

Imaginary

Real

F(u) FF(u) FS(u)

-800

-600

-400

-200

0

200

400

600

800

-1000-900-800-700-600-500-400-300-200-100 0 100

Imaginary

Real

F(u) FF(u) FS(u)

Explicit Implicit

C R W E N O 5 , 320 grid points

-800

-600

-400

-200

0

200

400

600

800

-1000 -800 -600 -400 -200 0

Imaginary

Real

F(u) FF(u) FS(u)

-100

-50

0

50

100

-100 -80 -60 -40 -20 0

Imaginary

Real

F(u) FF(u) FS(u)

Explicit Implicit

10−4 10−3 10−2 10−110−13

10−12

10−11

10−10

10−9

10−8

10−7

10−6

10−5

dt

Erro

r

arkimex(2c )arkimex(2e )arkimex(3 )rk (2a )rk (3 )

≈10x Explicit limit

Semi-implicit limit

10−4 10−3 10−2 10−1 10010−13

10−12

10−11

10−10

10−9

10−8

10−7

10−6

10−5

dt

Erro

r

arkimex(2c )arkimex(2e )arkimex(3 )rk (2a )rk (3 )

≈100x

Explicit limit

Semi-implicit limit S e m i - i m p l i c i t

time step size limit determined b y t h e f l o w velocity

Eigenvalues

Example: 1D Density Wave Advection (Computational Cost)

9  

Number of function calls

Wall time

1e-11

1e-10

1e-09

1e-08

1e-07

1e-06

1e-05

1e-04

1e+03 1e+04 1e+05

Error (L2)

Number of RHS function calls

ARK2c - No preconditionerARK2c - Block Jacobi

ARK2c - ILU(0)RK2a

≈ 5x

1e-11

1e-10

1e-09

1e-08

1e-07

1e-06

1e-05

1e-04

1e-01 1e+00 1e+01

Error (L2)

Wall time (seconds)

ARK2c - No preconditionerARK2c - Block Jacobi

ARK2c - ILU(0)RK2a

≈ 4x

1e-13

1e-12

1e-11

1e-10

1e-09

1e-08

1e-07

1e-06

1e-05

1e-04

1e+03 1e+04 1e+05 1e+06

Error (L2)

Number of RHS function calls

ARK2c - No preconditionerARK2c - Block Jacobi

ARK2c - ILU(0)RK2a

≈ 60x

1e-13

1e-12

1e-11

1e-10

1e-09

1e-08

1e-07

1e-06

1e-05

1e-04

1e-01 1e+00 1e+01 1e+02

Error (L2)

Wall time (seconds)

ARK2c - No preconditionerARK2c - Block Jacobi

ARK2c - ILU(0)RK2a

≈ 45x

Example: Low Mach Isentropic Vortex Convection

10  

Freestream flow

Vortex (Strength b = 0.5)

Eigenvalues of the right-hand-side operators

-15

-10

-5

0

5

10

15

-20-18-16-14-12-10 -8 -6 -4 -2 0 2

ImaginaryReal

F(u) FF(u) FS(u)

-1.5

-1

-0.5

0

0.5

1

1.5

-1.4 -1.2 -1 -0.8 -0.6 -0.4 -0.2 0

Imaginary

Real

F(u) FF(u) FS(u)Grid:322 points, CRWENO5  

10−3 10−2 10−1 100 10110−14

10−12

10−10

10−8

10−6

10−4

10−2

dt

Erro

r

arkimex(2e )arkimex(2c )arkimex(3 )arkimex(4 )rk (2a )rk (3 )rk (4 )

≈10x

Semi-implicit limit

Explicit limit

−3 −2.5 −2 −1.5 −1 −0.5 0 0.5−2.5

−2

−1.5

−1

−0.5

0

0.5

1

1.5

2

2.5

Re(λ) ∆ t

Im(λ

) ∆ t

ARK2e (IMEX)ARK2e (Expl)

−4 −3.5 −3 −2.5 −2 −1.5 −1 −0.5 0 0.5−3

−2

−1

0

1

2

3

Re(λ) ∆ t

Im(λ

) ∆ t

ARK3 (IMEX)ARK3 (Expl)

§  O p t i m a l o r d e r s o f convergence observed for all methods

§  Time step size limited by the “slow” eigenvalues.

Example: Vortex Convection (Computational Cost)

11  

1e-09

1e-08

1e-07

1e-06

1e-05

1e-04

1e-03

1e-02

1e+03 1e+04 1e+05

Error (L2)

Number of RHS function calls

ARK2c - No preconditionerARK2c - Block Jacobi

ARK2c - ILU(0)ARK2c - LU

RK2a

1e-09

1e-08

1e-07

1e-06

1e-05

1e-04

1e-03

1e-02

1e+01 1e+02 1e+03 1e+04

Error (L2)

Wall time (seconds)

ARK2c - No preconditionerARK2c - Block Jacobi

ARK2c - ILU(0)ARK2c - LU

RK2a

1e-11

1e-10

1e-09

1e-08

1e-07

1e-06

1e-05

1e-04

1e-03

1e-02

1e+03 1e+04 1e+05 1e+06

Error (L2)

Number of RHS function calls

ARK3 - No preconditionerARK3 - Block Jacobi

ARK3 - ILU(0)ARK3 - LU

RK3

1e-11

1e-10

1e-09

1e-08

1e-07

1e-06

1e-05

1e-04

1e-03

1e-02

1e+01 1e+02 1e+03 1e+04

Error (L2)

Wall time (seconds)

ARK3 - No preconditionerARK3 - Block Jacobi

ARK3 - ILU(0)ARK3 - LU

RK3

ARK 2c ARK 3

Number of function calls

Wall time

Example: Inertia – Gravity Wave

12  

−  Periodic channel – 300 km x 10 km −  No-flux boundary conditions at top and bottom

boundaries −  Mean horizontal velocity of 20 m/s in a uniformly

stratified atmosphere (M∞≈ 0.06) −  Initial solution – Potential temperature

perturbation

x

y

0 0.5 1 1.5 2 2.5 3x 105

0

1000

2000

3000

4000

5000

6000

7000

8000

9000

10000

−1.5

−1

−0.5

0

0.5

1

1.5

2

2.5x 10−3

Potential temperature perturbations at 3000 seconds (Solution obtained with WENO5 and ARKIMEX 2e, 1200x50 grid points)

Cross-sect ional potential temperature perturbations at 3000 seconds (y = 5 km) at various CFL numbers (0.2 – 13.6)

Eigenvalues of the right-hand-side operators

Grid: 300x10 points, CRWENO5  

CFL Wall time (s) Function counts

8.5 6149 24800

13.6 4118 17457

17.0 3492 14820

20.4 2934 12895

RK4 CFL ~ 1.0 Wall time: 5400 s Function counts: 24000

Conclusions

13  

Characteristic-based flux splitting (Work in progress): §  Partitioning of flux separates the acoustic and entropy modes à Allows larger

time step sizes (determined by flow velocity, not speed of sound). §  Comparison to alternatives

–  Vs. explicit time integration: Larger time steps à More efficient algorithm

–  Vs. implicit time integration: Semi-implicit solves a linear system without any approximations to the overall governing equations (as opposed to: solve non-linear system of equations or linearize governing equations in a time step).

To do: §  Improve efficiency of the linear solve

–  Better preconditioning of the linear system

§  Extend to 3D flow problems

Acknowledgements U.S. Department of Energy, Office of Science, Advanced Scientific Computing Research

14

Thank you!