35
Advances in simulation capability: A path towards modeling 10-100GeV plasma accelerator stages W.B.Mori University of California at Los Angeles Collaborators: Chengkun Huang, F.S.Tsung, V.K. Decyk, W.Lu, M.Zhou, J.Cooley, T.Antonsen Jr., A.Ghalam, T.Katsouleas, S.Deng, R. Fonseca, and L.O. Silva And E-164x collaboration http://exodus.physics.ucla.edu

Advances in simulation capability · 2014. 2. 11. · Assuming OSIRIS runs at a speed of 3µs per particle timestep (on SP3 processor), with 256 × 256 × 256 grids and 4 particles

  • Upload
    others

  • View
    1

  • Download
    0

Embed Size (px)

Citation preview

Page 1: Advances in simulation capability · 2014. 2. 11. · Assuming OSIRIS runs at a speed of 3µs per particle timestep (on SP3 processor), with 256 × 256 × 256 grids and 4 particles

1

Advances in simulation capability:A path towards modeling 10-100GeV plasma

accelerator stages

W.B.MoriUniversity of California at Los Angeles

Collaborators:Chengkun Huang, F.S.Tsung, V.K. Decyk, W.Lu, M.Zhou,J.Cooley, T.Antonsen Jr., A.Ghalam, T.Katsouleas, S.Deng, R. Fonseca, and L.O. SilvaAnd E-164x collaboration

http://exodus.physics.ucla.edu

Page 2: Advances in simulation capability · 2014. 2. 11. · Assuming OSIRIS runs at a speed of 3µs per particle timestep (on SP3 processor), with 256 × 256 × 256 grids and 4 particles

Concepts For Plasma Based Accelerators

! Plasma Wake Field Accelerator(PWFA)A high energy electron bunch

! Laser Wake Field Accelerator(LWFA, SMLWFA, PBWA)A single short-pulse of photons

! Drive beam ! Trailing beam

! Need to model:! Wake excitation: need to resolve ωp, λp

! Evolution of drive beam: evolves on β and/or xR >> σz scales! Evolution of trailing beam: need to run pump depletion distances

! To model all of these necessitates particle-in-cell models

Page 3: Advances in simulation capability · 2014. 2. 11. · Assuming OSIRIS runs at a speed of 3µs per particle timestep (on SP3 processor), with 256 × 256 × 256 grids and 4 particles

What Is a Fully Explicit Particle-in-cell Code?

• Maxwell’s equations for field solver

• Lorentz force updates particle’s position and momentum

Interpolate to particles

Particle positions

Lorentz Force

push

parti

cles weight to grid

z vi i,

ρn m n mj, ,,r

dpdt

E v B cr r r r

= + ×

r rE Bn m n m, ,,

∆t

Computational cycle (at each step in time)

Typical simulation parameters:~107-109 particles ~1-100 Gbytes~105 time steps ~102-105 cpu hours

Page 4: Advances in simulation capability · 2014. 2. 11. · Assuming OSIRIS runs at a speed of 3µs per particle timestep (on SP3 processor), with 256 × 256 × 256 grids and 4 particles

Computational challenges for modeling plasma-based acceleration

(1 GeV Stage): 5000 hours/ GeVBeam-driven wake* Fully Explicit

∆z ≤ .05 c/ωp

∆y, ∆x ≤ .05 c/ωp

∆t ≤ .02 c/ωp

# grids in z ≥350# grids in x, y ≥150

# steps ≥2 x 105

Nparticles ~.25-1. x 108 (3D)

Particles x steps ~.5 x 1013 (3D) - ≥ 5000 hrs

*Laser-driven GeV stage requires on the order of (ωo/ωp)2=1000 x longer, however, the the resolution can usually be relaxed (~200,000 hours).

Page 5: Advances in simulation capability · 2014. 2. 11. · Assuming OSIRIS runs at a speed of 3µs per particle timestep (on SP3 processor), with 256 × 256 × 256 grids and 4 particles

Plasma Accelerators and the Livingston Curve

Ε162/e+

C. Joshi and T. Katsouleas, Physics Today, June 2003

Page 6: Advances in simulation capability · 2014. 2. 11. · Assuming OSIRIS runs at a speed of 3µs per particle timestep (on SP3 processor), with 256 × 256 × 256 grids and 4 particles

Afterburners30 m

One goal is to build a virtual accelerator:

A 100 GeV-on-100 GeV e-e+ Collider

Based on Plasma Afterburners

3 km

Page 7: Advances in simulation capability · 2014. 2. 11. · Assuming OSIRIS runs at a speed of 3µs per particle timestep (on SP3 processor), with 256 × 256 × 256 grids and 4 particles

Motivation:

•Particle models are required but full PIC codes are too CPU intensive to model GeV LWFA stages and/or 100 GeV PWFA stages in three dimensions.•Real time feedback between simulation and experiment is not possible

•What is the solution?

•“Exploit” the physics: utilize the huge disparity in time scales.

•In 1997 Whittum (PWFA) and Antonsen and Mora (LWFA) independently wrote particle-in-cell codes that made the frozen field or quasi-static approximation.

•We have used some of these ideas to construct a fully three dimensional, fully parallelized, quasi-static PIC code that can model PWFA/LWFA plus beam loading: QUICKPIC

Page 8: Advances in simulation capability · 2014. 2. 11. · Assuming OSIRIS runs at a speed of 3µs per particle timestep (on SP3 processor), with 256 × 256 × 256 grids and 4 particles

QuickPIC:quasi-static PIC + fully parallelized+ fully 3D

• Quasi-static approximation: driver evolves on a much longer distances than wake wavelength· Frozen wakefield over time scale of the bunch length

·

· => β and/or xR >> σz (very good approximation!)

· Includes the best ideas from Antonsen and Mora with Whittum

BeamWake

Page 9: Advances in simulation capability · 2014. 2. 11. · Assuming OSIRIS runs at a speed of 3µs per particle timestep (on SP3 processor), with 256 × 256 × 256 grids and 4 particles

Basic equations for approximate QuickPIC

• Quasi-static or frozen field approximation converts Maxwell’s equations into electrostatic equations•Use ideas from Whittum 1997

Maxwell equations in Lorentz gauge Reduced Maxwell equations

( 1c 2

∂2

∂t2 − ∇ 2 )A = 4πc

j

(1c 2

∂2

∂t2 − ∇ 2 )φ = 4πρ

Quasi-static approx.−∇⊥

2 A =4πc

−∇⊥2 φ = 4πρ

φ, A = ϕ, A(z − ct )

Local--φ,Α at any z-slice depend only on ρ,j at that slice!

• j = jb + je ≈ jb = cρb ˆ z )ˆ( //zA=AForces :

plasma : Fe⊥ = −e∇ ⊥φ

beam : Fb⊥ = −e∇ ⊥Ψ

• Ψ = φ − A//

Ez(ξ) =∂ψ∂ξ

Page 10: Advances in simulation capability · 2014. 2. 11. · Assuming OSIRIS runs at a speed of 3µs per particle timestep (on SP3 processor), with 256 × 256 × 256 grids and 4 particles

Quasi-static PIC code

(n+2) ∆s

Field frozen

(n+1) ∆s

Field frozen

Beam evolves due tobetatron oscillation determined by:

kβ =k p

2γ b

Moving Window

…plasma response

mm+1 56

n∆ss

ξ 1234

Page 11: Advances in simulation capability · 2014. 2. 11. · Assuming OSIRIS runs at a speed of 3µs per particle timestep (on SP3 processor), with 256 × 256 × 256 grids and 4 particles

Parallelization of QuickPIC

zy

x

Node 3 Node 2 Node 1 Node 0

xy

Node 3

Node 2

Node 1

Node 0

Page 12: Advances in simulation capability · 2014. 2. 11. · Assuming OSIRIS runs at a speed of 3µs per particle timestep (on SP3 processor), with 256 × 256 × 256 grids and 4 particles

Full QuickPIC:PWFA or LWFA

• The axial motion of the plasma can be important! Need to include it.•Use ideas from Antonsen and Mora 1997

Reduced Maxwell equations

ρ = iq1−

// ivparticles

r j = iq

1−// i

vparticles∑ r v

γ − p// =1− qψ−∇⊥

2 A = 4πc

j

−∇⊥2 φ = 4πρ

γ = (1+ p2 + a2 /2)1/ 2

ψ = φ − A//

For beam electrons :∂Pb

∂s= −

qb

cE +

Vb

c× B

= −

qb

c∇ψ

For plasma electrons : ∂Pe⊥

∂ξ=

qe

c −Ve //

[E⊥ + (Ve

c× B)⊥ ]

χ =−q

γ − p//∑For laser beam 2iω∂τ − 2∂τξ − ∇⊥

2( )a = χ a

Page 13: Advances in simulation capability · 2014. 2. 11. · Assuming OSIRIS runs at a speed of 3µs per particle timestep (on SP3 processor), with 256 × 256 × 256 grids and 4 particles

Assuming OSIRIS runs at a speed of 3µs per particle timestep (on SP3 processor), with 256 × 256 × 256 grids and 4 particles per cell, beam particles gain 1GeV in 0.1m (acceleration gradient is 10Gev/m).

For 50Gev energy gain, 250,000 node-hours is needed for the OSIRIS simulation.

Potential savings forPWFA

Quasi-static PIC code, beam, plasma

evolution time scale separated

Full electromagnetic PIC code

Feature

5,234∆t< 0.05ωp-1~0.05c/ωpOSIRIS

∆t<0.05ωβ-1

=

Timestep limit

~0.05c/ωp

Grid size limit

67QuickPIC

Total time of simulation per GeV stage (node-hour)

Simulation Codes

0.05 × 2γω p−1

Page 14: Advances in simulation capability · 2014. 2. 11. · Assuming OSIRIS runs at a speed of 3µs per particle timestep (on SP3 processor), with 256 × 256 × 256 grids and 4 particles

Benchmark result of longitudinal wakefield between QuickPIC and 2D OSIRIS for botha) electron drive beam ( nb n p = 25.9 ) and b) positron drive beam ( nb np =15.2 )

Benchmark:QuickPIC=OSIRIS at <.01 the cost

a) b)

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

-8 -6 -4 -2 0 2 4 6 8

Focu

sing

Fie

ld E

r - B

θ (mcω

p/e)

r (c/ωp)

Blue --- OSIRIS(2D)Red --- QuickPIC

a) -0.4

-0.2

0

0.2

0.4

-6 -4 -2 0 2 4 6

Focu

sing

Fie

ld E

r - B

θ (mcω

p/e)

r (c/ωp)

Blue --- OSIRIS(2D)Red --- QuickPIC

b)

Longitudinal Wakefield

-5

-4

-3

-2

-1

0

1

2

3

-8 -6 -4 -2 0 2 4 6 8

Osiris_2DFull QuickPICBasic QuickPIC

Logi

tudi

nal W

akef

ield

(mcω

p/e)

Z (c/ ωp)

electron benchmark: sigma_r=7; sigma_z = 45; N= 1.8e10; n0=2e16

-0.3

-0.2

-0.1

0

0.1

0.2

0.3

-10 -5 0 5 10

Osiris_2dFull QuickPICBasic QuickPIC

Logi

tudi

nal W

akef

ield

(mcω

p/e)

Z (c/ ωp)

positron benchmark: sigma_r=25; sigma_z = 600; N= 1.8e10; n0=2e14

Focusing Field

Page 15: Advances in simulation capability · 2014. 2. 11. · Assuming OSIRIS runs at a speed of 3µs per particle timestep (on SP3 processor), with 256 × 256 × 256 grids and 4 particles

Full 3D Simulation parameters

Beam shape

Beam charge

Plasma Density

Longitudinal:

Drive beam : wedge shape, L=145µm=6.5c/ωp

Trailing beam : Gaussian profile, σz= 10µm = 0.45c/ωp

Transverse: Gaussian profile, σr= 15µm = 0.67c/ωp

Longitudinal:

Transverse: Gaussian profile, σr= 12µm = 0.9c/ωp

Ndrive=3E1010, Ntrailing=1E1010N=1.8E1010

n=5.66E16cm-3n=1.6E17cm-3

AfterburnerE164X

-1

0

1

2

3

4

5

0 50 100 150 200 250 300 350 400

beam profile

Bea

m p

rofil

e

Z (micron)

Page 16: Advances in simulation capability · 2014. 2. 11. · Assuming OSIRIS runs at a speed of 3µs per particle timestep (on SP3 processor), with 256 × 256 × 256 grids and 4 particles

E164-X

QuickTime™ and aMPEG-4 Video decompressor

are needed to see this picture.

QuickTime™ and aMPEG-4 Video decompressor

are needed to see this picture.

QuickTime™ and aMPEG-4 Video decompressor

are needed to see this picture.

Page 17: Advances in simulation capability · 2014. 2. 11. · Assuming OSIRIS runs at a speed of 3µs per particle timestep (on SP3 processor), with 256 × 256 × 256 grids and 4 particles

Even in Blowout regimebunch shaping can lead tohigher transformer ratios

Wedge shape, beam length = 18 c/ωp, nb/np= 3

Wedge shape, beam length = 6 c/ωp, nb/np= 8.4

Bi-Gaussian shape, σz= 1.2 c/ωp, nb/np= 26

Page 18: Advances in simulation capability · 2014. 2. 11. · Assuming OSIRIS runs at a speed of 3µs per particle timestep (on SP3 processor), with 256 × 256 × 256 grids and 4 particles

Ideal beam loading scenarioscan be found

Trailing beam spot size and charge

Ndrive = 3E1010, Ntrailing = 0.5E1010Ndrive = 3E1010, Ntrailing = 1E1010

Page 19: Advances in simulation capability · 2014. 2. 11. · Assuming OSIRIS runs at a speed of 3µs per particle timestep (on SP3 processor), with 256 × 256 × 256 grids and 4 particles

Afterburner simulationMinimal hosing!

1) Matched wedge shape drive beam with trailing bi-Gaussian beam.

2) Background plasma density.

QuickTime™ and aMPEG-4 Video decompressor

are needed to see this picture.

Page 20: Advances in simulation capability · 2014. 2. 11. · Assuming OSIRIS runs at a speed of 3µs per particle timestep (on SP3 processor), with 256 × 256 × 256 grids and 4 particles

50 Gev energy gain in 3 meters !

QuickTime™ and aMPEG-4 Video decompressor

are needed to see this picture.

Accelerating field24GeV/m at the load

Page 21: Advances in simulation capability · 2014. 2. 11. · Assuming OSIRIS runs at a speed of 3µs per particle timestep (on SP3 processor), with 256 × 256 × 256 grids and 4 particles

QuickPIC provides the properLWFA wakes

-0.3

-0.2

-0.1

0

0.1

0.2

0.3

-5 0 5 10 15 20 25 30

QuickPICOSIRIS (1D)Linear Theory

Wak

efie

ld (m

cωp/e

)

Z (c/ ω p)

Laser wakefield in a near-1D QuickPIC benchmark run Benchmarked against

Theory and OSIRIS

Page 22: Advances in simulation capability · 2014. 2. 11. · Assuming OSIRIS runs at a speed of 3µs per particle timestep (on SP3 processor), with 256 × 256 × 256 grids and 4 particles

Axial Electric Fieldξ = ct − z

x

Laser Pulse

1.8 nC electron bunch25 MeV injection energy

Reduced amplitude due to effects of beam loading

Page 23: Advances in simulation capability · 2014. 2. 11. · Assuming OSIRIS runs at a speed of 3µs per particle timestep (on SP3 processor), with 256 × 256 × 256 grids and 4 particles

23

Full-PIC is still needed!

1.When self-trapping is important2.For benchmarking3.For moderate energy beams

OSIRIS, VORPAL, TurboWAVE ….

Page 24: Advances in simulation capability · 2014. 2. 11. · Assuming OSIRIS runs at a speed of 3µs per particle timestep (on SP3 processor), with 256 × 256 × 256 grids and 4 particles

•Simulation Parameters–Laser:

• a0 = 3• W0=9.25 λ=7.4 µm• ωl/ωp = 22.5

–Particles• 1x2x2 particles/cell• 240 million total

–Channel length• L=.828cm• 300,000 timesteps

•The parameters are similar to those at LOA and LBNL

Full scale 3D LWFA simulation

3712 cells136 µm

256 cells136 µm

256 cells136 µm

State-of- the- art ultrashort laser pulse

λ0 = 800 nm, ∆t = 50 fs

I = 1.5x1019 W/cm-2, W =7.4 µm

Laser propagation

Plasma Backgroundne = 3x1018 cm-3 channel

Simulation ran for 300,000 time steps(~40 Rayleigh lengths)

Simulation ran for 300,000 time steps(~40 Rayleigh lengths)

Page 25: Advances in simulation capability · 2014. 2. 11. · Assuming OSIRIS runs at a speed of 3µs per particle timestep (on SP3 processor), with 256 × 256 × 256 grids and 4 particles

( ˆ k )

( ˆ e )

( ˆ b )

Evolution of laser leads to self-injection

Page 26: Advances in simulation capability · 2014. 2. 11. · Assuming OSIRIS runs at a speed of 3µs per particle timestep (on SP3 processor), with 256 × 256 × 256 grids and 4 particles

UCLA

Nanocoulombs of charge is accelerated

104

105

106

107

108

0 200 400 600 800 1000

fe(E) (0.5cm)

fe(E) (0.84cm)

E (MeV)

fe(E)

(e's/MeV)

Total # of Electrons = 4.14 x 10 9

Total # of Electrons = 3.24 x 10 9

Energy Distribution of Fast Electrons

εN[ ]i =πc

∆ui2 ∆xi

2 − ∆xi∆ui2[ ]1/ 2

First Bunch

• Total Charge: 0.519 nC

• Beam well-confined within the channel

Second Bunch

• Total Charge: .663nC

• Emittance: 18.81 µm (along b) 40.89 µm (along e)

Page 27: Advances in simulation capability · 2014. 2. 11. · Assuming OSIRIS runs at a speed of 3µs per particle timestep (on SP3 processor), with 256 × 256 × 256 grids and 4 particles

Wakes can be produced in field ionized produced plasma by either Laser or Particle beam drivers:

OSIRIS

PWFALWFA

Page 28: Advances in simulation capability · 2014. 2. 11. · Assuming OSIRIS runs at a speed of 3µs per particle timestep (on SP3 processor), with 256 × 256 × 256 grids and 4 particles

Summary and future work:

•quasi-static PIC (QuickPIC)•It works! Can reproduce full PIC results at .01 the cost.•50 GeV stages can be modeled in ~1000 node hours•E-164x can be modeled in ~500 node hours•Laser field solve is now working and simulations are beginning•Also being used to model the e-cloud problem for circular accelerators

•Full PIC is alive and well: OSIRIS, VORPAL, TurboWAVE …..•3D with ionization•GeV LWFA acceleration in a plasma channel

•Future work for QuickPIC:•Dynamic load balancing (a first pass should be completed in 1 month)•Ionization (work will begin over the summer)•Mesh refinement to handle tightly focused matched beams•A recipe for handling self-trapped particles

Page 29: Advances in simulation capability · 2014. 2. 11. · Assuming OSIRIS runs at a speed of 3µs per particle timestep (on SP3 processor), with 256 × 256 × 256 grids and 4 particles

29

QuickPIC is being applied to“mainstream” accelerator problems:

e-cloud

For high current circular accelerators the limiting issue is E-cloud.

The e-cloud is seeded by the beam halos or synchrotron radiation hitting the walls.

The proton/positron electron cloud mechanism is very similar to beam plasma interactions.

Page 30: Advances in simulation capability · 2014. 2. 11. · Assuming OSIRIS runs at a speed of 3µs per particle timestep (on SP3 processor), with 256 × 256 × 256 grids and 4 particles

(a) (b) (c)

-0.015 y(c/ωp) 0.015

-0.75

z(c/ωp)

0.75

D

0.2

Z(c/ωp)

-0.2 -0.004 Y(c/ωp) 0.004

(a) (b) (c)

3 snapshots of beam over CERN-SPS ring. a) At t0=0.4ms (18 turns). b) At t1=t0+TB/4c) At t2=t0+ TB/2, where TB is the nominal betatron period (0.9µs),The beam is initially off

centered 1mm from the axis of the pipe

3 snap shots of beam evolution over CERN-SPS a) At t=0 b) At t=136µs (6 turns) c) At t=0.8ms(35 turns). The beam is initially tilted

Initially Off-CenteredBeam

Initially Tilted Beam

Study of the Long Term Beam DynamicsStudy of the Long Term Beam Dynamics

These figures show that no matter how the beam is initially perturbed, the beam ends up having a similar long term dynamics

Page 31: Advances in simulation capability · 2014. 2. 11. · Assuming OSIRIS runs at a speed of 3µs per particle timestep (on SP3 processor), with 256 × 256 × 256 grids and 4 particles

HosingBeam centroid vs. propagation distance

0.0001

0.001

0.01

0.1

1

10

0 0.5 1 1.5 2 2.5

Linear theoryQuickPIC simulation

|xb| (

mm

) s (m)

E.S. Dodd PRL 88(12), 2002

Full QuickPIC shows less hosing than what theory and basic QuickPIC predict for afterburner.

Page 32: Advances in simulation capability · 2014. 2. 11. · Assuming OSIRIS runs at a speed of 3µs per particle timestep (on SP3 processor), with 256 × 256 × 256 grids and 4 particles

Other issue: Plasma lens

50GeV electron beam

10 cm wide plasma slab

compressed beam

Page 33: Advances in simulation capability · 2014. 2. 11. · Assuming OSIRIS runs at a speed of 3µs per particle timestep (on SP3 processor), with 256 × 256 × 256 grids and 4 particles

To meet our ever increasing computational needs we are building a 512 processor cluster: DAWSON

The “Dawson” cluster under construction at

UCLA: NSF MRI

Page 34: Advances in simulation capability · 2014. 2. 11. · Assuming OSIRIS runs at a speed of 3µs per particle timestep (on SP3 processor), with 256 × 256 × 256 grids and 4 particles

2-D plasma slabQuickPIC loop (based on UPIC):

Wake (3-D)Beam (3-D)

1. initialize beam

2. solve ∇⊥2 ϕ = ρ, ∇⊥

2ψ = ρe ⇒ Fp ,ψ

3. push plasma, store ψ4. step slab and repeat 2.5. use ψ to giant step beam

Page 35: Advances in simulation capability · 2014. 2. 11. · Assuming OSIRIS runs at a speed of 3µs per particle timestep (on SP3 processor), with 256 × 256 × 256 grids and 4 particles

Adding the laser field into quickPIC(A Maryland and UCLA collaboration)

Use ponderomotive guiding center approach of Antonsen and Mora

dr P

dt= q

r E +

r v ×

r B ( )−

1

4

q

γ∇ a

2 where γ = 1+ P2 +

a 2

2

A parallelized laser envelope solver has been added into the code by J. Cooley and T. Antonsen’s

χ = −qγ

∑2iω∂τ − 2∂τξ − ∇⊥2

a = χ a