A Parallel Simulator for Streamer Discharges in Three ... · A Parallel Simulator for Streamer Discharges in Three Dimensions Bo LIN Department of Mathematics National University

A Parallel Simulator for Streamer Discharges inThree Dimensions

Bo LIN

Department of MathematicsNational University of Singapore

Joint works withWeizhu BAO(NUS), Zhenning CAI(NUS) and Chijie ZHUANG(THU)

Workshop 2: Modeling and Simulation for Quantum Condensation,Fluids and Information

IMS, 21 November, 2019

Outline

Background

Model

Three-dimensional parallel simulator in N2

Parallel simulator in air

Conclusion

Background

Streamer:

I Cold plasma

I Initial stages of sparks and lightning

I High altitude sprite discharges above thunderclouds

Experimental photos are taken from Chen, She, et al. ”Nanosecond repetitivelypulsed discharges in N2-O2 mixtures: inception cloud and streamer emergence.”Journal of Physics D: Applied Physics 48.17 (2015): 175201.

Background 1 / 46

Lightning

URL: https://youtu.be/8ny3N-smAUc

Background 2 / 46

High altitude discharges

Consecutive 1ms. Photos are taken from Moudry, Dana, et al. ”Imaging of elves,halos and sprite initiation at 1ms time resolution.” Journal of Atmospheric andSolar-Terrestrial Physics 65.5 (2003): 509-518.

Background 3 / 46

Applications

I Dust precipitators

I Polymer selection

I Production of ozone

I · · ·

Background 4 / 46

Dust precipitation

Photo is taken from Mizuno, A. ”Electrostatic precipitation.” IEEE Transactionson Dielectrics and Electrical Insulation 7.5 (2000): 615-624.

Background 5 / 46

Dust precipitation

Photo is taken from Wikipedia.Background 6 / 46

Polymer selection

Photos is taken from Borcia, G., C. A. Anderson, and N. M. D. Brown.”Dielectric barrier discharge for surface treatment: application to selected polymers infilm and fibre form.” Plasma Sources Science and Technology 12.3 (2003): 335.

Background 7 / 46

Polymer selection

0, 0.2, 1.5, 1.5 s on cylinder. Photos are taken from Borcia, G., C. A. Anderson,and N. M. D. Brown. ”Dielectric barrier discharge for surface treatment: applicationto selected polymers in film and fibre form.” Plasma Sources Science and Technology12.3 (2003): 335.

Background 8 / 46

Prevention

I Corona discharge in power transmission

I Partial discharge in transformerI · · ·

Background 9 / 46

Corona discharge

Electric field (potential gradient) around a conductor is high enough.

Photo is taken fromhttps://electricalbaba.com/corona-discharge-factors-affecting-corona-2/.

Background 10 / 46

Summary

I Cold plasma, initial stage and building block

I Frequently-seen in life

I Applications

I Prevention

Background 11 / 46

Outline

Background

Model



Conclusion

Fluid model

Under some assumptions, the following fluid model1 can be used todescribe the streamer discharges between two parallel planes in threedimensions:

∂ne∂t−∇ · (µe ~Ene)−∇ · (De∇ne) = Si + Sph,

∂np∂t

+∇ · (µp ~Enp) = Si + Sph,

−∆φ = eε0

(np − ne), ~E = −∇φ.

where Si = α(| ~E|)µe| ~E|ne is the effective impact ionization, and Sphdenotes the photoionization.

I Cubic domain Ω. 0 < t < T .I For potential φ, Dirichlet boundaries + Neumann boundaries.I For np and ne, Neumann boundary conditions.

1Ward(1965, PR), Kline(1974, JAP), Yoshida and Tagashira(1976, JPD),...Model 12 / 46

Photoionization

The nonlocal photoionization Sph for N2-O2 mixture2.

Collisions with electrons → Radiating states of N2 (b′Π, b′′Σ+u , C

′′

4 Σ+u )

→ Ground states X ′Σ+g → Photoionization of O2

Sph(x)

=

∫Ω

pqp+ pq

ξω

αSi(y)

exp(−χminpO2 |y− x|)− exp(−χmaxpO2 |y− x|)4π|y− x|3 ln(χmax/χmin)

dy.

ω: ionizing photons created by an electron over 1 cm path.χ: absorption cross section.

2Zhelezniak, M. B., A. Kh Mnatsakanian, and Sergei Vasil’evich Sizykh.”Photoionization of nitrogen and oxygen mixtures by radiation from a gas discharge.”High Temperature Science 20 (1982): 423-428.

Model 13 / 46

Numerical difficulties

I Multiscale structure. Thin layer at the streamer head (micrometermagnitude), comparing with the size of the problem (centimeter).

I Large computation in 3D.I Small time step.I Large linear system for electric potential (Poisson equation).I Nonlocal solution for photoionization.

Model 14 / 46

Outline

Background

Model



Conclusion

Pure N2

In pure N2, no photoionization. Exclude Sph.

∂ne∂t−∇ · (µe ~Ene)−∇ · (De∇ne) = Si,

∂np∂t

+∇ · (µp ~Enp) = Si,

−∆φ = eε0

(np − ne), ~E = −∇φ,

where Si = α(| ~E|)µe| ~E|ne.

Three-dimensional parallel simulator in N2 15 / 46

Previous temporal discretization

Before showing our scheme, let us review the first-order explicit scheme:

nn+1e − nneτn

−∇ · (µe ~Ennne )−∇ · (De∇nne ) = α(| ~En|)µe| ~En|nne ,

nn+1p − nnpτn

+∇ · (µp ~Ennnp ) = α(| ~En|)µe| ~En|nne ,

−∆φn = eε0

(nnp − nne ), ~En = −∇φn.

Time steps τn suffer from dielectric relaxation time constraint:

τn < τdiel :=ε0

e(µenne + µpnnp )≈ 10−12s 3.

3Typically, final time T has magnitude of nanosecond 10−9sThree-dimensional parallel simulator in N2 16 / 46

Semi-implicit scheme

The following semi-implicit scheme is designed to remove the dielectricrelaxation constraint4 in time step:

nn+1e − nneτn

−∇ · (µe ~En+1nne )−∇ · (De∇nne ) = α(| ~En+1|)µe| ~En+1|nn+1e ,

nn+1p − nnpτn

+∇ · (µp ~En+1nnp ) = α(| ~En+1|)µe| ~En+1|nn+1e ,

−∆φn+1 = eε0

(nn+1p − nn+1e ), ~En+1 = −∇φn+1.

Poisson equation can be solved in an explicit way.

−∇ ·((ε0

e+ τn

(µpn

np + µen

ne

))∇φn+1

)= nnp − nne − τn∇ · (De∇nne ).

4The scheme is proved asymptotic preserving in Villa, Andrea, et al. ”Anasymptotic preserving scheme for the streamer simulation.” Journal of ComputationalPhysics 242 (2013): 86-102.


Plots of two schemes

:tn+1

:tn,nnp n

ne ,ϕn E⃗

n

,nn+1p n

n+1e

First order explicit scheme

:tn+1

:tn,nnp n

ne

,ϕn+1 E⃗ n+1

,nn+1p n

n+1e

First order semi-implicit scheme

For the semi-implicit one, ∆tn < ε0e(µenne +µpnnp ).


Second order explicit scheme

Second order:

I Heun’s method

I Solve all equations twice

Questions: Second-order with

I More stability, like thesemi-implicit kind

I Fast resolution

:tn+1

:tn ,nnp n

ne

Heun's method

,n∗

p n∗

e

,n∗∗

p n∗∗

e

,nn+1p n

n+1e

Average


Our second order semi-implicit scheme

Inspired by midpoint method, we use two steps (predictor-corrector) toconstruct a second-order scheme. The first step (predictor) is

nn+1/2e − nneτn/2

−∇ · (µe ~En+1/2nne )−∇ · (De∇nne ) = α(| ~En+1/2|)µe| ~En+1/2|nne ,

nn+1/2p − nnpτn/2

+∇ · (µp ~En+1/2nnp ) = α(| ~En+1/2|)µe| ~En+1/2|nne ,

−∆φn+1/2 = eε0

(nn+1/2p − nn+1/2e ), ~En+1/2 = −∇φn+1/2.

The second step (corrector) isnn+1e − n

ne

τn−∇ · (µe ~En+1/2nn+1/2e )−∇ · (De∇n

n+1/2e ) = α(|~E

n+1/2|)µe|~En+1/2|nn+1/2e ,

nn+1p − nnp

τn+∇ · (µp ~En+1/2nn+1/2p ) = α(|~E

n+1/2|)µe|~En+1/2|nn+1/2e .

Solve Poisson equation only once!


Our second order semi-implicit scheme

:tn+1

:tn ,nnp n

ne

,ϕn+1/2 E⃗ n+1/2

,nn+1p n

n+1e

Our second-order semi-implicit scheme

,nn+1/2p n

n+1/2e

(1)

(2)

(2)

(3)(3)

(3)

I Second order with solving one variable elliptic equation per time stepI More stable than explicit schemes


Spatial discretization

I Finite volume method in space.

I For drift-diffusion-reaction equations: Second order MUSCL withKoren limiter.

I For elliptic problem, central difference. Fast solver?


Fast solver for the elliptic equation

Combine geometric multigrid with Krylov subspace method.

I Geometric multigrid preconditionerI Flexible Generalized Minimal Residual (FGMRES) method


Why parallel computing?

Our simulator is implemented in MPI library.

A snapshot5, where the authors use first order explicit scheme.

We want to simulate in 1cm3 domain, with more than 10003 grid points.

5Shi, Feng, Ningyu Liu, and Joseph R. Dwyer. ”Three-Dimensional Modeling ofTwo Interacting Streamers.” Journal of Geophysical Research: Atmospheres 122.19(2017): 10-169.


Numerical experiments

I Second-order convergenceI Stability test w.r.t. dielectric relaxation timeI Scalability (Parallel efficiency)I Performance of fast elliptic solverI Application

The High performance computing facilities we use is called Tianhe2-JK, which islocated at Beijing Computational Science Research Center (CSRC) in China.


Second order in time

Table: Error of our second order semi-implicit scheme in 1D testing problem.

∆t 0.005 0.005/2 0.005/22 0.005/23 0.005/24 0.005/25

‖ne − (ne)ref‖ 3.1659e-4 8.3831e-5 2.1649e-5 5.5091e-6 1.3869e-6 3.4440e-7order – 1.9171 1.9532 1.9744 1.9900 2.0097‖np − (np)ref‖ 2.5302e-4 6.3421e-5 1.6027e-5 4.0407e-6 1.0129e-6 2.5099e-7order – 1.9962 1.9844 1.9879 1.9962 2.0127


Stability test w.r.t. dielectric relaxation time

Table: Stability of different temporal discretizations on a 1D testing problem.

Temporal scheme τ = 0.5τdiel τ = τdiel τ = 3τdiel τ = 10τdiel τ = 50τdiel2nd order semi-implicit stable stable stable stable stable1st order semi-implicit stable stable stable stable stable

2nd order explicit stable stable unstable unstable unstable1st order explicit stable stable unstable unstable unstable


Scalability of our codes

100

101

102

number of node

100

101

102

rela

tiv

e s

pe

ed

up

ideal

our simulator

101

102

number of node

100

101

rela

tiv

e s

pe

ed

up

ideal

our simulator

Fig. Scalability of our second-order semi-implicit scheme over two meshes512× 512× 640, 1024× 1024× 1280. 6

6The codes are run over different number of nodes, with all 20 cores used in eachnode. Domain is set as 1cm3, where τ proportional to mesh size.


Performance of fast elliptic solver

0 200 400 600 800 1000 1200

time step

0

1

2

3

4

5

6

7

nu

mb

er

of

ite

rati

on

0 200 400 600 800 1000 1200

time step

0

1

2

3

4

5

6

7

nu

mb

er

of

ite

rati

on

Fig. Iteration number of multigrid preconditioned FGMRES solver in second-ordersemi-implicit scheme over two meshes 512× 512× 640, 1024× 1024× 1280. 7Relative tolerance 10−8.

7Same setting as the previous testing, except we fix 32 nodes (640 cores).Three-dimensional parallel simulator in N2 29 / 46

Performance of fast elliptic solver

Table: Calculation time for different algebraic elliptic solvers 8 by second ordersemi-implicit scheme. (50 steps)

mesh size: 512× 512× 640method FGMRES GMRES CG FCG CGS BICGSTABmean time [s] 75.401 91.058 91.696 91.913 103.19 103.31ratio (on FGMRES) 1 1.2076 1.2161 1.2190 1.3685 1.3701

mesh size: 1024× 1024× 1280method FGMRES GMRES CG FCG CGS BICGSTABmean time [s] 511.38 654.50 655.65 659.29 741.57 740.00ratio (on FGMRES) 1 1.2799 1.2821 1.2892 1.4501 1.4471

8Saad, Yousef. Iterative methods for sparse linear systems. Vol. 82. siam, 2003.Chapter 6–9.


Application: Interaction of two streamers

I Domain [0, 1]× [0, 1]× [0, 1] cm3

I Mesh size 2048× 2048× 2560, more than 10 billion cellsI Number of nodes for computing: 64 (each node has 20 processors)

I Simulation time: Less than two days


Interaction of two streamers

Fig. Electron density which greater than 1013 cm−3 at 2.5, 3.5 ns.


Outline

Background

Model



Conclusion

Photoionization in air

Now we should consider Sph9.

Sph(x)

=

∫Ω

pqp+ pq

ξω

αSi(y)

exp(−χminpO2 |y− x|)− exp(−χmaxpO2 |y− x|)4π|y− x|3 ln(χmax/χmin)

dy

=

∫Ω

ψ(y)ξf(|y− x|)4π|y− x|2

dy,

where ψ(y) = pqp+pqωαSi(y), and

f(|y− x|) = exp(−χminpO2 |y− x|)− exp(−χmaxpO2 |y− x|)|y− x| ln(χmax/χmin)

.

Kernel function depends on |y− x|, which is denoted as r in the followingslides.

9Zhelezniak, M. B., A. Kh Mnatsakanian, and Sergei Vasil’evich Sizykh.”Photoionization of nitrogen and oxygen mixtures by radiation from a gas discharge.”High Temperature Science 20 (1982):423-428.

Parallel simulator in air 33 / 46

Two approximation methods

1. Helmholtz approximation

2. Radiative transfer equation, SP3 approximation


Helmholtz approximation

Key idea: kernel approximation10.

ξf(r) = ξexp(−χminpO2r)− exp(−χmaxpO2r)

r ln(χmax/χmin)

≈ rN∑j=1

Aj exp(−λjr).

Then,

Sph(x) =N∑j=1

Aj

∫Ω

ψ(y)exp(−λjr)

4πrdy︸︷︷︸

Sph,j

=

N∑j=1

AjSph,j .

10Luque, Alejandro, et al. ”Photoionization in negative streamers: Fastcomputations and two propagation modes.” Applied physics letters 90.8 (2007):081501.


Helmholtz approximation

Assume Sph(x)→ 0 on the boundary,

(−∆ + (λj)2)Sph,j(x) = ψ(x).

I Elliptic solver is applicable.I Integral → PDE.


Radiative transfer equation

Consider a steady-state radiative transfer equation without scattering,

~Ω · ∇Iν(x, ~Ω) =nu(x)φu(ν)

4πcτu− κνIν(x, ~Ω),

where ~Ω ∈ S2, 1/τu is the Einstein coefficient for spontaneous transition.The isotropic solution is

ϕν(x) =

∫S2Iνd~Ω =

∫Ω

exp(−κνr)nu(y)φu(ν)

4πcτur2dy.

By multi-group method11, the photoionization is approximated as:

Sph =

N∑i=1

ξκ̄icϕi.

One further approximation will be applied: SPN method.11Lewis, Elmer Eugene, and Warren F. Miller. ”Computational methods of neutron

transport.” (1984).Parallel simulator in air 37 / 46

Radiative transfer equation: multi-group method

Compare the results from multi-group method and the originalphotoionization, they are identical if12

exp(−χminpO2r)− exp(−χmaxpO2r)r ln(χmax/χmin)

= f(r) =

N∑i=1

φu,iκ̄i exp(−κ̄ir),

where κ̄i = λipO2 in the air.

12Bourdon, Anne, et al. ”Efficient models for photoionization produced bynon-thermal gas discharges in air based on radiative transfer and the Helmholtzequations.” Plasma Sources Science and Technology 16.3 (2007): 656.


Radiative transfer equation: SPN method13

~Ω · ∇Ii(x, ~Ω) = Su − κ̄iIi(x, ~Ω),

can be rewritten as (1 +

1

κ̄i~Ω · ∇

)Ii = Su.

For optically thick equation,

Ii =

(1 +

1

κ̄i~Ω · ∇

)−1Su

=

(1− 1

κ̄i~Ω · ∇+ 1

κ̄2i(~Ω · ∇)2 + 1

κ̄3i(~Ω · ∇)3 + · · ·

)Su.

Then take integration both side for ~Ω ∈ S2.

13Larsen, Edward W., et al. ”Simplified PN approximations to the equations ofradiative heat transfer and applications.” Journal of Computational Physics 183.2(2002): 652-675.


Summary of multi-group SP3

−∇ · µ21

κ̄iΨi + κ̄iΨi = κ̄i4πSu,

−∇ · µ22

κ̄iΨ̂i + κ̄iΨ̂i = κ̄i4πSu.

Boundary:

α1Ψi +1

κ̄i∇Ψi · n = −β2Ψ̂i + η1Su,

α2Ψ̂i +1

κ̄i∇Ψ̂i · n = −β1Ψi + η2Su.

One group solution:

ϕi =γ2Ψi − γ1Ψ̂iγ2 − γ1

.

Photoionization:

Sph =

N∑i=1

ξκ̄icϕi.


Numerical experiments

I Alternating number of SP3I Performance of fast elliptic solver on SP3I Application

I Role of the photoionization


Alternating number of SP3

0 200 400 600Time step

1

2

3

4

5

6

Altern

ating ite

ration n

um

ber

First group

Second group

Third group

Fig. Iteration numbers of the multigrid preconditioned FGMRES solver at each timestep for photoionization. ”first group”, ”second group” and ”third group” refer to ϕ1,ϕ2, ϕ3. One “alternating iteration” means to solve the equation of Ψ and then switchto solve the equation of Ψ̂ once.


Interaction between two streamers

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

z (cm)

0.4

0.4

z (cm)

0.9

0.3

0.5

0.7

0.4

0.5y (cm)

y (cm)0.4

0.6 0.40.5

x (cm)

x (cm)0.6

0.2

0.3

0.4

0.5

0.6

0.7

0.8

z (cm)

0.9y (cm)

0.40.5x (cm)

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

z (cm)

0.4

0.4

z (cm)

0.9

0.3

0.5

0.7

0.4

0.5y (cm)

y (cm)0.4

0.6 0.40.5

x (cm)

x (cm)0.6

0.2

0.3

0.4

0.5

0.6

0.7

0.8

z (cm)

0.9y (cm)

0.40.5x (cm)

Fig. Three contours of the electron density. Left: 0.5 ns, right: 1.5 ns. Blue color:1× 1013 cm−3; red color: 5× 1013 cm−3; green color: 9× 1013 cm−3.


Role of the photoionization

0 0.2 0.4 0.6 0.8 1z (cm)

-150

-100

-50

0E

z (

kV

/ c

m)

With photoionization

Without photoionization

Fig. Electric field Ez on x = y = 0.25 cm, from 0.5 ns to 1.5 ns.


Outline

Background

Model



Conclusion

Conclusion

Summary:

I New second-order semi-implicit scheme• Solving elliptic equation once at each time step• More stable than explicit schemes

I Fast elliptic solver• Multigrid preconditioned FGMRES• Fast convergence

I MPI parallelism• Good parallel efficiency

I Photoionization• Two approximations: Kernel expansion, multi-group SP3• Applications of elliptic solver

Future work:

I Adaptive mesh

I Curve boundary

Conclusion 45 / 46

Conclusion

Summary:I New second-order semi-implicit scheme

• Solving elliptic equation once at each time step• More stable than explicit schemes

I Fast elliptic solver• Multigrid preconditioned FGMRES• Fast convergence

I MPI parallelism• Good parallel efficiency

I Photoionization• Two approximations: Kernel expansion, multi-group SP3• Applications of elliptic solver

Future work:

I Adaptive meshI Curve boundary

Thank you for your attention!Conclusion 46 / 46

BackgroundModelThree-dimensional parallel simulator in N2Numerical methodsMesh and fully discretizationVariable coefficient elliptic solverNumerical experiments

Parallel simulator in airConclusion

fd@rm@0:

Documents

A Parallel Simulator for Streamer Discharges in Three ... · A Parallel Simulator for Streamer Discharges in Three Dimensions Bo LIN Department of Mathematics National University