Using/Extending LAMMPS for your InterfaceSimulations

Steve PlimptonSandia National Labs

CAMS Summer School ProgramSimulation of Complex Microstructure in Materials

May 2013 - U Florida

Outline

LAMMPS aims to be flexible

Three interface vignettes revisited

Interfacing LAMMPS with other codes

LAMMPS flexibility comes from styles

90% of 500K lines of code in LAMMPS is add-on classes

can optionally build with or without any of thempotentials: pair, bond, angle, ..., long-range K-spacefixes: integrators, boundary conditions, constraints, ...computes: temperature, per-atom coordination, rdf, ...input script commands: create atoms, run, temper, neb, ...output dump formats, geometric regions, etc

Virtual parent class defines the interface

User or developer writes a new child class:

provide required and optional methodsdrop 2 files in src directory and re-compileshould work with (and not break) anything else in LAMMPS

LAMMPS flexibility comes from styles

90% of 500K lines of code in LAMMPS is add-on classes

can optionally build with or without any of thempotentials: pair, bond, angle, ..., long-range K-spacefixes: integrators, boundary conditions, constraints, ...computes: temperature, per-atom coordination, rdf, ...input script commands: create atoms, run, temper, neb, ...output dump formats, geometric regions, etc

Virtual parent class defines the interface

User or developer writes a new child class:

provide required and optional methodsdrop 2 files in src directory and re-compileshould work with (and not break) anything else in LAMMPS

What options (styles) does LAMMPS already have?

http://lammps.sandia.gov and click on Commands

Compute styles (60) are diagnosticsglobal, per-atom, per-molecule quantities

Good to know about for 2 reasons:don’t need to re-invent the wheelstarting point for something you want to implement

What options (styles) does LAMMPS already have?

http://lammps.sandia.gov and click on Commands

Compute styles (60) are diagnosticsglobal, per-atom, per-molecule quantities

Good to know about for 2 reasons:don’t need to re-invent the wheelstarting point for something you want to implement

Pair styles (100) for pair & many-body interactions

Variety of fix styles (120)

Fixes allow tailoring of timestep

In hindsight, this is best feature of LAMMPS

Need control of “what” happens “when” within each timestep

Loop over timesteps:

fix initial NVE, NVT, NPT, rigid-body integration

communicate ghost atoms

fix neighbor insert particles

build neighbor list (once in a while)compute forcescommunicate ghost forces

fix force SHAKE, langevin drag, wall, spring, gravityfix final NVE, NVT, NPT, rigid-body integrationfix end volume rescaling, alchemy, diagnostics

output to screen and files

Fixes allow tailoring of timestep

In hindsight, this is best feature of LAMMPS

Need control of “what” happens “when” within each timestep

Loop over timesteps:fix initial NVE, NVT, NPT, rigid-body integrationcommunicate ghost atomsfix neighbor insert particlesbuild neighbor list (once in a while)compute forcescommunicate ghost forcesfix force SHAKE, langevin drag, wall, spring, gravityfix final NVE, NVT, NPT, rigid-body integrationfix end volume rescaling, alchemy, diagnosticsoutput to screen and files

Revisit the three interface vignettes

1 Grain boundary migration

interface = “surface” between 2 crystals, different orientations

2 Shock-wave initiation of explosions

2 interfaces = shock front and void/bulk

3 Nanoparticle self-assembly

2 interfaces = NP/solvent and liquid/vapor

Recurring theme: these simulations were enabled by features(styles) we added to LAMMPS

#1 - Grain boundary migration

Build a bi-crystal

Input script commands:

region lower box EDGE EDGE EDGE EDGE EDGE 20.0region upper box EDGE EDGE EDGE EDGE 20.0 EDGE

lattice fcc 4.04 origin 0 20 0 orient x -3 1 0 ...create atoms 1 region lower

lattice fcc 4.04 origin 0 20 0 orient x 3 1 0 ...create atoms 1 region upper

delete atoms overlap 0.5 all all

New fix orient/fcc command to impose driving force

2 files: src/fix orient fcc.cpp and fix orient fcc.h

Request full neighbor list, every timestep:

int irequest = neighbor->request((void *) this);neighbor->requests[irequest]->pair = 0;neighbor->requests[irequest]->fix = 1;neighbor->requests[irequest]->half = 0;neighbor->requests[irequest]->full = 1;

Post force() method for fix orient/fcc

double loop over atoms and neighbors:compute Rij and add to listsort list to find 12 nearest neighbors (fcc)

loop over atoms:compute contributions from 12 neighborsderivative of energy → forces on I and J atoms

communicate partial forces induced on ghost atoms

double loop over atoms and neighbors:compute full orientation force on each I atom

LAMMPS provides method to perform communication

∼250 lines of code

Post force() method for fix orient/fcc

double loop over atoms and neighbors:compute Rij and add to listsort list to find 12 nearest neighbors (fcc)

loop over atoms:compute contributions from 12 neighborsderivative of energy → forces on I and J atoms

communicate partial forces induced on ghost atoms

double loop over atoms and neighbors:compute full orientation force on each I atom

LAMMPS provides method to perform communication

∼250 lines of code

#2 - Shock-induced detonation of explosives

Post integrate() method for fix wall/reflect command

for (int m = 0; m < nwall; m++)coord = current wall position (fixed or variable)dim = wallwhich[m] / 2; side = wallwhich[m] % 2;

for (i = 0; i < nlocal; i++)if (side == 0)

if (x[i][dim] < coord)x[i][dim] = coord + (coord - x[i][dim]);v[i][dim] = -v[i][dim];

elseif (x[i][dim] > coord)x[i][dim] = coord - (x[i][dim] - coord);v[i][dim] = -v[i][dim];

Entire fix = ∼200 lines of code

Post integrate() method for fix wall/reflect command

for (int m = 0; m < nwall; m++)coord = current wall position (fixed or variable)dim = wallwhich[m] / 2; side = wallwhich[m] % 2;

for (i = 0; i < nlocal; i++)if (side == 0)

if (x[i][dim] < coord)x[i][dim] = coord + (coord - x[i][dim]);v[i][dim] = -v[i][dim];

elseif (x[i][dim] > coord)x[i][dim] = coord - (x[i][dim] - coord);v[i][dim] = -v[i][dim];

Entire fix = ∼200 lines of code

New fix reaxc/species command for molecule stats

Written by Ray Shan

Molecules in ReaxFF and a shock explosion are dynamic

not defined by permanent bonds, angles, etcdefined by instantaneous bond-order parameters

Useful to know numbers/locations/atoms of moleculesat any timestep, on-the-fly

Compute cluster/atom flags clusters based on cutoff

each atom starts as own clusterwalk outward, merging clusters with lower atom IDparallel communication when clusters overlap proc domains

Ray used this logic to merge based on bond-order criterion

Compile molecule stats and write details to file

New fix reaxc/species command for molecule stats

Written by Ray Shan

Molecules in ReaxFF and a shock explosion are dynamic

not defined by permanent bonds, angles, etcdefined by instantaneous bond-order parameters

Useful to know numbers/locations/atoms of moleculesat any timestep, on-the-fly

Compute cluster/atom flags clusters based on cutoff

each atom starts as own clusterwalk outward, merging clusters with lower atom IDparallel communication when clusters overlap proc domains

Ray used this logic to merge based on bond-order criterion

Compile molecule stats and write details to file

#3 - Nanoparticle self-assembly

Coarse-grained NP/NP and NP/LJ interactions

Integrated LJ potential over volume of nanoparticleEveraers & Ejtehadi, Phys Rev E, 67, 041710 (2003)Encoded as pair colloid.cpp in LAMMPS

Pairwise and analytic (albeit expensive), Hamaker constant,1/r2 dependence, similar formulation for ellipsoids

Coarse-grained NP/NP and NP/LJ interactions

Integrated LJ potential over volume of nanoparticleEveraers & Ejtehadi, Phys Rev E, 67, 041710 (2003)Encoded as pair colloid.cpp in LAMMPS

Pairwise and analytic (albeit expensive), Hamaker constant,1/r2 dependence, similar formulation for ellipsoids

Fix evaporate to remove solvent at controlled rate

Input script:fix 1 solvent evaporate

N M topbox 38277 {molecule yes}Pre exchange() method in fix evaporate

identify atoms in region volumepick random subset (consistent across procs)delete from systemalso remove molecules the deleted particles are in

∼200 lines of code (molecules add complexity)

Computational issue with dual particles sizes

P in ’t Veld, et al, Comp Comm Phys, 179, 320 (2008)Mixture system with 20/1 size ratio leads to 3 cutoffs

20σ for NP/NP, 14σ for NP/LJ, 3σ for LJ/LJ

Efficient neighbor finding is challenging

Easy to spend 95% of time building neighbor lists

Solution is small bins, stencils, and avoid computing Rij

Computational issue with dual particles sizes

P in ’t Veld, et al, Comp Comm Phys, 179, 320 (2008)Mixture system with 20/1 size ratio leads to 3 cutoffs

20σ for NP/NP, 14σ for NP/LJ, 3σ for LJ/LJ

Efficient neighbor finding is challenging

Easy to spend 95% of time building neighbor lists

Solution is small bins, stencils, and avoid computing Rij

Load-balancing for enhanced performance

Liquid/vapor interface → dramatic 1d density variation

Default processor domains are equal-sized 3d bricksunequal particles/proc → poor parallel scalability

Implemented load-balancing commands:static balance commanddynamic fix balance command

Much better performance on large number of procs

Algorithm is inadequate for general 3d density variation

Load-balancing for enhanced performance

Liquid/vapor interface → dramatic 1d density variation

Default processor domains are equal-sized 3d bricksunequal particles/proc → poor parallel scalability

Implemented load-balancing commands:static balance commanddynamic fix balance command

Much better performance on large number of procs

Algorithm is inadequate for general 3d density variation

Load-balancing for enhanced performance

Liquid/vapor interface → dramatic 1d density variation

Default processor domains are equal-sized 3d bricksunequal particles/proc → poor parallel scalability

Implemented load-balancing commands:static balance commanddynamic fix balance command

Much better performance on large number of procs

Algorithm is inadequate for general 3d density variation

Fix rigid and fix rigid/small commands

All-atom NP core and CG wedge are rigid bodies

Fix rigid and fix rigid/small (new) commandssum forces and torques on particles of each bodyintegrate body motion via quaternions

LAMMPS allows for rigid bodies with:atoms, spheroids, ellipsoids, lines, triangles

Fix rigid/small is 5-10x faster on large number of procsused for wedge → microtubule model

Fix rigid and fix rigid/small commands

All-atom NP core and CG wedge are rigid bodies

Fix rigid and fix rigid/small (new) commandssum forces and torques on particles of each bodyintegrate body motion via quaternions

LAMMPS allows for rigid bodies with:atoms, spheroids, ellipsoids, lines, triangles

Fix rigid/small is 5-10x faster on large number of procsused for wedge → microtubule model

Other LAMMPS features may help with complex models

Hybrid pair styles, disjoint or overlayedmetal/water interface → 3 potentialsCG nanoparticles with repulsive + lubrication + DLVO

Time averaging via fix ave/time, fix ave/atom, fix ave/spatial

Input scripts use simple, but powerful scripting languagevariables, loops, if tests, etcinvoke shell commands for pre- and post-processingset variables from shell when input script invoked

Use multiple processors in a single input scriptto run one (big) simulationto run many (small) simulations simultaneouslyto run a huge set of simulations

Dump image commandon-the-fly JPG files → instant moviescolor atoms/bonds by any computable property

centro-symmetry, CNA, coordination, Voronoi volume, etc

Other LAMMPS features may help with complex models

Hybrid pair styles, disjoint or overlayedmetal/water interface → 3 potentialsCG nanoparticles with repulsive + lubrication + DLVO

Time averaging via fix ave/time, fix ave/atom, fix ave/spatial

Input scripts use simple, but powerful scripting languagevariables, loops, if tests, etcinvoke shell commands for pre- and post-processingset variables from shell when input script invoked

Use multiple processors in a single input scriptto run one (big) simulationto run many (small) simulations simultaneouslyto run a huge set of simulations

Dump image commandon-the-fly JPG files → instant moviescolor atoms/bonds by any computable property

centro-symmetry, CNA, coordination, Voronoi volume, etc

Dump image examples

One more kind of (pseudo) interface

Multi-physics or multi-scale models often lead tonumeric or coupling interface between two methods or two codes

LAMMPS is really a library

Most people run it as stand-alone application, but ...

Another code can instantiate LAMMPS (one or more times)

LAMMPS is a C++ classC interface also providedenables LAMMPS to be called from C, Fortran, Python

One more kind of (pseudo) interface

Multi-physics or multi-scale models often lead tonumeric or coupling interface between two methods or two codes

LAMMPS is really a library

Most people run it as stand-alone application, but ...

Another code can instantiate LAMMPS (one or more times)

LAMMPS is a C++ classC interface also providedenables LAMMPS to be called from C, Fortran, Python

Examples of MD in multi-scale context

MD + DFT: dynamics with quantum forces

MD + FE: thermal/mechanical coupling to continuum

MD + OpenFoam Navier-Stokes: fluidized granular bed

MD + Navier-Stokes: flowing biomolecules

MD + off-lattice kinetic MC: defect dynamics in solids

MD + on-lattice kinetic MC: stress-driven grain growth

Materials modeling via on-lattice kMC

Variety of models for time evolution of processed materials

Length scale = atomic to mesoscale (fcc or cubic lattices)

Time scale = microseconds to days

Hamiltonian encodes physics of interest via local interactions

MD + kMC for stress-driven grain growth

SPPARKS (kMC) runs Potts model for grain growthHamiltonian includes stress termsend grain structure to LAMMPS

LAMMPS (MD) treats particles at grain boundary as largerrelaxes systemsend per-particle stress to SPPARKS

Library interface also enables Python to wrap LAMMPS

Write a Python script that calls LAMMPS or viz or plotting

Viz options: VMD, AtomEye, PyMol

Plotting options: GnuPlot, MatLab

170 lines of Python:

LAMMPS web site and tarball are resources

Web site:

Pictures & movies

Papers - titles, authors, abstracts - search in browser

Documentation - section 10 = modify/extend LAMMPS

Mail list - 37K postings, 12K threadsfast turn-around, searchable

Tutorials

Benchmark data

Tarball:

Source code, documentation

30 simple examples: input scripts and output

Post-processing tools and scripts

Library examples: examples/COUPLE (MD/kMC), Python

LAMMPS web site and tarball are resources

Web site:

Pictures & movies

Papers - titles, authors, abstracts - search in browser

Documentation - section 10 = modify/extend LAMMPS

Mail list - 37K postings, 12K threadsfast turn-around, searchable

Tutorials

Benchmark data

Tarball:

Source code, documentation

30 simple examples: input scripts and output

Post-processing tools and scripts

Library examples: examples/COUPLE (MD/kMC), Python

Per-atom, per-timestep cost for atomistic potentials

See lammps.sandia.gov/bench.html#potentials

Links and thanks

LAMMPS: http://lammps.sandia.gov

post a question: http://lammps.sandia.gov/mail.html

my email: sjplimp@sandia.gov

Thanks to LAMMPS co-developers at Sandia:

Paul CrozierStan MooreRay ShanAidan ThompsonChristian Trott