The Heavy Ion Fusion Virtual National Laboratory
Vay 06/08/04
Application of Adaptive Mesh Refinement to PIC simulations in inertial fusion
J.-L. Vay, P. Colella, J.W. Kwan, P. McCorquodale, D. SerafiniLawrence Berkeley National Laboratory
A. Friedman, D.P. Grote, G. WestenskowLawrence Livermore National Laboratory
J.-C. Adam, A. HéronCPHT, Ecole Polytechnique, France
I. HaberUniversity of Maryland
15th International Symposium on Heavy Ion Inertial Fusion Princeton, New Jersey
June 7-11, 2004
The Heavy Ion Fusion Virtual National Laboratory
Vay 06/08/04
Outline
• Motivations for coupling PIC with AMR
• Issues
• Examples of electrostatic and electromagnetic PIC-AMR
• Joint project at LBNL to develop AMR library for PIC
• Conclusion
The Heavy Ion Fusion Virtual National Laboratory
Vay 06/08/04
challenging because length scales span a wide range: m to km(s)
Goal: end-to-end modeling of a Heavy Ion Fusion driver
The Heavy Ion Fusion Virtual National Laboratory
Vay 06/08/04
The Adaptive-Mesh-Refinement (AMR) method
• addresses the issue of wide range of space scales
• well established method in fluid calculations
AMR concentrates the resolution around the edge which contains the most interesting scientific features.
3D AMR simulation of an explosion (microseconds after ignition)
The Heavy Ion Fusion Virtual National Laboratory
Vay 06/08/04
Electrostatic PIC+AMR: issues
The Heavy Ion Fusion Virtual National Laboratory
Vay 06/08/04
Mesh Refinement in Particle-In-Cell: Issues
• Asymmetry of grid implies asymmetry
of field solution for one particle spurious self-force
• Some implementations may violate Gauss’ Law Total charge may not be conserved exactly
• EM: shortest wavelength resolved on fine grid not resolved on coarse grid reflect at interface with factor>1
May cause instability by multiple reflections
As shown in the following slides, the choice of algorithm matters.
The Heavy Ion Fusion Virtual National Laboratory
Vay 06/08/04
Electrostatic: possible implementations
• Given a hierarchy of grids, there exists several ways to solve Poisson
• Two considered:
1. ‘1-pass’• solve on coarse grid • interpolate solution on fine grid boundary • solve on fine grid different values on collocated nodes
2. ‘back-and-forth’ • interleave coarse and fine grid relaxations • collocated nodes values reconciliation same values on collocated nodes
Patch grid
“Mother” grid
The Heavy Ion Fusion Virtual National Laboratory
Vay 06/08/04
Illustration of the spurious self-force effect
• 2-grid set with metallic boundary;
Patch grid
“Mother” grid
Metallic boundary
Test particle
v
one particle attracted by its image
Spurious “image”
as if there was a spurious image
0 100 200 300 400 500
25
26
X
reference case linear - 1p linear - bf quad. - 1p quad. - bf
X
T
0
10
20
30
zoom
MR introduces spurious force, particle trapped in patch
The Heavy Ion Fusion Virtual National Laboratory
Vay 06/08/04
Self-force amplitude map and mitigation
y
Line
arQ
uadr
atic
1-pass
x
multipass Log(E)
• Magnitude of self force decreases rapidly with distance from edge
• with the 1-pass method, the self-force effect can be mitigated by defining a transition region surrounding the patch in which deposit charge and solve but get field from underlying coarse patch
patch
main grid
transition region
The Heavy Ion Fusion Virtual National Laboratory
Vay 06/08/04
Electromagnetics: usual scheme
Rc
Rf
G Rc: coarse resolutionRf: fine resolution
P
• the solution is computed as usual in the main grid and in the patch
• interpolation is performed at the interface
• unfortunately, most schemes relying on interpolations have instability issues at short wavelengths (the ones that may be generated in the patch but cannot propagate in the main grid)
The Heavy Ion Fusion Virtual National Laboratory
Vay 06/08/04
Illustration of instability in 1-D EM tests
10 1001E-7
1E-6
1E-5
1E-4
1E-3
0.01
0.1
1
10 'jump' (n=3) finite volume (n=3) 'directional' (n=3)
R
2c/xfine grid (xcoarse grid=nxfine grid)10 100
1E-5
1E-4
1E-3
0.01
0.1
1
10 'jump' space-time (n=3) energy conserving (n=2) 'directional' (n=2)
R
2c/xfine grid (xcoarse grid=nxfine grid)
Space only Space+Time
x o x o x o x o x oj-5/2 j-2 j-3/2 j-1 j-1/2 j j+1/2 j+1 j+3/2 j+5/2
x1 x2n.x1
Boundary
grid 1 grid 2
o: E[+],E[-]
x: B[+],B[-]
o: E, x:B
The Heavy Ion Fusion Virtual National Laboratory
Vay 06/08/04
Rc
ABC
RcP1
RfP2
G
Outside patch:F = F(G)
Inside patch:F = F(G)+F(P2)-F(P1)
Electromagnetics: we proposed a method by “substitution”
Rc: coarse resolutionRf: fine resolution
The Heavy Ion Fusion Virtual National Laboratory
Vay 06/08/04
Electrostatic PIC+AMR examples
The Heavy Ion Fusion Virtual National Laboratory
Vay 06/08/04
Study of steady-state regime of HCX triode
The Heavy Ion Fusion Virtual National Laboratory
Vay 06/08/04
3D WARP simulation of High-Current Experiment (HCX)
Modeling of source is critical since it determines initial shape of beam
0.0 0.1 0.2 0.3 0.40.2
0.4
0.6
0.8
1.0 Low res. Medium res.
High res. Very High res.
4N
RM
S ( m
m.m
rad)
Z(m)
Run Grid size Nb particles
Low res. 56x640 ~1M
Medium res. 112x1280
~4M
High res. 224x2560
~16M
Very High res. 448x5120
~64M
WARP simulations show that a fairly high resolution is needed to reach convergence
The Heavy Ion Fusion Virtual National Laboratory
Vay 06/08/04
0.0 0.1 0.2 0.3 0.40.2
0.4
0.6
0.8
1.0 Low res. Medium res.
High res. Medium res. + MR
4N
RM
S ( m
m.m
rad)
Z(m)
Prototype MR implemented in WARPrz ( axisymmetric )
Low res. Medium res.
High res. Medium res.+MR
• Three runs with single uniform grid• One run at medium resolution + MR patch
Run Grid size Nb particles
Low res. 56x640 ~1M
Medium res. 112x1280
~4M
High res. 224x2560
~16M
Medium res. + MR 112x1280
~4M
In this case: speedup ~ 4
Low res. Medium res.
High res. Medium res.+MR
The Heavy Ion Fusion Virtual National Laboratory
Vay 06/08/04In this case: speedup ~ 4
0.0 0.1 0.2 0.3 0.40.2
0.4
0.6
0.8
1.0 Low res. Medium res.
High res. Medium res. + MR
4N
RM
S ( m
m.m
rad)
Z(m)
Prototype AMR implemented in WARPrz ( axisymmetric )
Low res. Medium res.
High res. Medium res.+AMR
• Better results obtained with a dynamic AMR mockup refining emitter area + beam edge
Run Grid size Nb particles
Low res. 56x640 ~1M
Medium res. 112x1280
~4M
High res. 224x2560
~16M
Medium res. + AMR 112x1280
~4M
In this case: speedup ~ 4
Low res. Medium res.
High res. Medium res.+AMR
The Heavy Ion Fusion Virtual National Laboratory
Vay 06/08/04
0.0 0.1 0.2 0.3 0.40.2
0.4
0.6
0.8
1.0 Low res. Medium res.
High res. Low res. + AMR
4N
RM
S ( m
m.m
rad)
Z(m)
Prototype AMR implemented in WARPrz (
• Higher speedup obtained with a “true” dynamic AMR implementation
Z (m)
R (
m)
In this case: speedup ~ 11.3
Run Grid size Nb particles
Low res. 56x640 ~1M
Medium res. 112x1280
~4M
High res. 224x2560
~16M
Low res. + AMR 56x640 ~1M
Low res.+AMR
(Ntransit=2)
Low res.
High res. Low res.+AMR
(Ntransit=0)
Medium res.
Z (m) Z (m)Refinement of gradients: emitting area,
beam edge and front.
zoom
Z (m)
R (
m)
The Heavy Ion Fusion Virtual National Laboratory
Vay 06/08/04
Time-dependent modeling of ion source rise-time
The Heavy Ion Fusion Virtual National Laboratory
Vay 06/08/04
3D WARP simulation of HCX shows beam head scrapping Rise-time = 800 ns
beam head particle loss < 0.1%
z (m)
z (m)
x (
m)
x (
m)
Rise-time = 400 nszero beam head particle loss
• Can we get even cleaner head with faster rise-time?
• Optimum?
The Heavy Ion Fusion Virtual National Laboratory
Vay 06/08/04
1D time-dependent modeling of ion diode: fast rise-timeEmitter Collector
V V=0d
virtual surface
di
Vi
tIQ
d
VχI
2i
2/3i
I (A
)
Time (s)
N = 160t = 1nsd = 0.4m
“L-T” waveform
Ns = 200
irregular patch in di
Time (s)
x0/x~10-5!
3
max
t4
3
t
V
V(t)
time
curr
ent
0.0 1.00.0
1.0
t/
V/V
max
Lampel-Tiefenback
AMR ratio = 16
irregular patch in di + AMR following front
Time (s)
Careful analysis shows that di too large by >104
=> irregular patch
Careful analysis shows that di too large by >104
=> irregular patch
Insufficient resolution of beam front=> AMR patch
Insufficient resolution of beam front=> AMR patch
Irregular MR patch covering emission area suppresses long wavelength oscillation
Adaptive MR patch following the beam head
suppresses front peak
Irregular MR patch covering emission area suppresses long wavelength oscillation
Adaptive MR patch following the beam head
suppresses front peak
The Heavy Ion Fusion Virtual National Laboratory
Vay 06/08/04
• Specialized 1-D patch implemented in 3-D injection routine (2-D array)
• Extension Lampel-Tiefenback technique to 3-D implemented in WARP predicts a voltage waveform which extracts a nearly flat current at emitter
• Without MR, WARP predicts overshoot
• Run with MR predicts very sharp risetime (not square due to erosion)
Application to three dimensions
T (s)
V (
kV
)
“Optimized” Voltage Current at Z=0.62m
X (
m)
Z (m)
STS500 experiment
The Heavy Ion Fusion Virtual National Laboratory
Vay 06/08/04
• Experimental voltage lowered so that risetime = particle transit time
• Mesh Refinement essential to recover experimental results
• Ratio of smaller mesh to main grid mesh ~ 1/1000
Z (m)
No MR With MRCurrent history (Z=0.62m) Current history (Z=0.62m)
Comparison with experiment
The Heavy Ion Fusion Virtual National Laboratory
Vay 06/08/04
Electromagnetic PIC+MR example
The Heavy Ion Fusion Virtual National Laboratory
Vay 06/08/04
Laser-plasma interaction in the context of fast ignition
• A laser impinges on a cylindrical target which density is far greater than the critical density.
• The center of the plasma is artificially cooled to simulate a cold high-density core.
• Patch boundary surrounds plasma. Laser launched outside the patch.
coreLaser beam
=1m,1020W.cm-2
(Posc/mec~8,83)
2=28/k0
10nc, 10keV
Patch
• Implemented new MR technique in EM PIC code Emi2d (E. Polytech.)
The Heavy Ion Fusion Virtual National Laboratory
Vay 06/08/04
Comparison single uniform high res. grid / low res. + patchw
ithou
t pa
tch
with
pat
ch
• same results except for small residual incident laser outside region of interest (well understood, possible cures)
• same results except for small residual incident laser outside region of interest (well understood, possible cures)
• no instability nor spurious wave reflection observed at patch border
• no instability nor spurious wave reflection observed at patch border
• can be used as is for various applications and we are also exploring improvements and variants
• can be used as is for various applications and we are also exploring improvements and variants
The Heavy Ion Fusion Virtual National Laboratory
Vay 06/08/04
AMR library for PIC
The Heavy Ion Fusion Virtual National Laboratory
Vay 06/08/04
• Researchers from AFRD (PIC) and ANAG (AMR-Phil Colella’s group) collaborate to provide a library of tools that will give AMR capability to existing PIC codes (on serial and parallel computers)
• The base is the existing ANAG’s AMR library Chombo
• The way it works
• WARP is test PIC code but library will be usable by any PIC code
Effort to develop AMR library for PIC at LBNL
The Heavy Ion Fusion Virtual National Laboratory
Vay 06/08/04
Example of WARP-Chombo injector field calculation
• Interactions with particles is being implemented
The Heavy Ion Fusion Virtual National Laboratory
Vay 06/08/04
Conclusion
• PIC and AMR are numerical techniques that have proven to be very valuable in various fields and their combination may lead to more powerful tools for beams and plasmas modeling in inertial fusion (and beyond).
• The implementation must be done with care (beware of potential spurious self-forces, violation of Gauss’ Law, reflection of smallest wavelengths).
• Prototypes of AMR methods were implemented in existing PIC codes and test runs demonstrated the effectiveness of the method in ES-PIC and a proof-of-principle of a new method was performed in EM-PIC.
• There is an ongoing effort at LBNL to build an AMR library which will ultimately provide AMR capabilities to existing PIC codes.
The Heavy Ion Fusion Virtual National Laboratory
Vay 06/08/04
Backup slides
The Heavy Ion Fusion Virtual National Laboratory
Vay 06/08/04
Time and length scales in driver and chamber span a wide range
Length scales:
-11-12 -10 -9 -8 -7 -6 -5 -4 -3 -2 -1 0
electroncyclotronin magnet pulse
electron driftout of magnet
beamresidence
pb
latticeperiod
betatrondepressedbetatron
pe
transitthru
fringefields
beamresidence
pulse log of timescalein seconds
In driver
In chamberpi
pb
• electron gyroradius in magnet ~10 m• D,beam ~ 1 mm• beam radius ~ cm• machine length ~ km's
Time scales:
The Heavy Ion Fusion Virtual National Laboratory
Vay 06/08/04
global error larger with BF than 1P BF: Gauss’ law not satisfied; error transmitted to coarse grid solution
y
Line
arQ
uadr
atic
1 pass
x
y
x
Back and forth
x
Back and forth
VV
S
d/dSdD
N// refrefGlobal error
The Heavy Ion Fusion Virtual National Laboratory
Vay 06/08/04
Electrostatic issues: summary
• Mesh Refinement introduces spurious self-force that has a repulsive effect on a macroparticle close to coarse-fine interface in fine grid, but:
- real simulations involve many macroparticles: dilution of the spurious force
- for some coarse-fine grid coupling, the magnitude of the spurious effect can be reduced by an order of magnitude by interpolating to and from collocated nodes in band in fine grid along coarse-fine interface
- we may also simply discard the fine grid solution in band and use coarse grid solution instead for force gathering (or ramp)
• some scheme may violate Gauss’ law and may introduce unphysical non-linearities into “mother” grid solution: hopefully there is also dilution of the effect in real simulations– we note that our tests were performed for a node-centered
implementation and our conclusion applies to this case only. For example, a cell-centered implementation does strictly enforce Gauss’ Law and results may differ.