3. Efficient Many-Body Algorithms: Barnes-Hut and Fast ... · 3.2. Molecular Dynamics – the Physical Model Classical Molecular Dynamics Quantum mechanics! approximation classical

Simulation on the Molecular Level Physical Model Mathematical Model Long-Range Potentials

3. Efficient Many-Body Algorithms:Barnes-Hut and Fast Multipole

3. Efficient Many-Body Algorithms: Barnes-Hut and Fast Multipole

Perlen der Informatik I, Hans-Joachim Bungartz page 1 of 47

http://www5.in.tum.de/persons/bungartz.html


3.1. Simulation on the Molecular LevelHierarchy of Models

Different points of view for simulating human beings:

issue level of resolution model basis (e.g.!)global increase inpopulation

countries, regions population dynamics

local increase inpopulation

villages, individuals population dynamics

man circulations, organs system simulatorblood circulation pump/channels/valves network simulatorheart blood cells continuum mechanicscell macro molecules continuum mechanicsmacro molecules atoms molecular dynamicsatoms electrons or finer quantum mechanics





Scales – an Important Issue

• length scales in simulations:

– from 10−9m (atoms)– to 1023m (galaxy clusters)

• time scales in simulations:

– from 10−15s– to 1017s

• mass scales in simulations:

– from 10−24g (atoms)– to 1043g (galaxies)

• obviously impossible to take all scales into account in an explicitand simultaneous way

• first molecular dynamics simulations reported in 1957





Applications for Micro and Nano Simulations

Lab-on-a-chip, used in brewingtechnology (Siemens)

Flow through a nanotube (wherethe assumptions of continuum me-chanics are no longer valid)






Material science: hexagonal cry-stal grid of Bornitrid

Protein simulation: actin, importantcomponent of muscles (overlay ofmacromolecular model with elec-tron density obtained by X-ray cry-stallography (brown) and simulati-on (blue))






Protein simulation: human haemoglobin (light blue and purple: alphachains; red and green: beta chains; yellow, black, and dark blue:

docked stabilizers or potential docking positions for oxygen)





A Prominent Recent Example:

• Gordon-Bell-Prize 2005 (most important annual supercomputingaward)

• phenomenon studied: solidification processes in Tantalum andUranium

• method: 3D molecular dynamics, up to 524,000,000 atomssimulated

• machine: IBM Blue Gene/L, 131,072 processors (world’s #1 inNovember 2005)

• performance: more than 101 TeraFlops (almost 30% of the peakperformance)





3.2. Molecular Dynamics – the Physical Model

Classical Molecular Dynamics

• Quantum mechanicsapproximation−−−−−−−−→ classical Molecular Dynamics

• classical Molecular Dynamics is based on Newton’s equations ofmotion

• molecules are modelled as particles; simplest case: pointmasses

• there are interactions between molecules

• multibody potential functions describe the potential energy of thesystem; the velocities of the molecules (kinetic energy) are acomposition of

– the Brownian motion (high velocities, no macroscopicmovement),

– flow velocity (for fluids)





Fundamental Interactions• Classification of the fundamental

interactions:

– strong nuclear force– electromagnetic force– weak nuclear force– gravity

O

rk

ri

rj

• interaction→ potential energy

• total potential of N particles is the sum of multibody potentials:

– U :=∑

0<i<N U1(ri ) +∑

0<i<N∑

i<j<N U2(ri , rj )+∑

0<i<N∑

i<j<N∑

j<k<N U3(ri , rj , rk ) + . . .

– there are ( Nn ) = N!

n!(N−n)! ∈ O(Nn) n-body potentials Un,particulary N one-body and 1

2 N(N − 1) two-body potentials

• force ~F = −gradU





Lennard Jones Potential

e s

e,s

O

ri

rj

rij

• a widespread family of intermolecular two-body potentials

• Lennard Jones potential: ULJ(rij)

= αε

((σrij

)n−(σrij

)m)

with n > m and α = 1n−m

(nn

mm

) 1n−m

• continuous and differentiable (C∞), since rij > 0





• LJ 12-6 potential

ULJ(rij)

= 4ε((

σrij

)12−(σrij

)6)

– m = 6: van der Waals attraction (van der Waals potential)

– n = 12: Pauli repulsion (softsphere potential): heuristic– application: simulation of inert gases (e.g. Argon)

– force between 2 molecules:

Fij = −∂U(rij)∂rij

= 24εrij

(2(σrij

)12−(σrij

)6)

– very fast fade away⇒ short range (m = 6 > 3 = ddimension)





3.3. Molecular Dynamics – the MathematicalModel

System of ODE

• resulting force acting on a molecule: ~Fi =∑

j 6=i~Fij

• acceleration of a molecule (Newton’s 2nd law):

~ ir =~Fi

mi=

∑j 6=i~Fij

mi= −

∑j 6=i

∂U(~ri ,~rj )∂|rij |

mi(1)

• system of dN coupled ordinary differential equations of 2nd ordertransferable (as compared to Hamilton formalism) to 2dNcoupled ordinary differential equations of 1st order (N: number ofmolecules, d : dimension), e.g. independent variables q := r and pwith

~pi := mi~ri (2a)

~pi = ~Fi (2b)3. Efficient Many-Body Algorithms: Barnes-Hut and Fast Multipole




Boundary Conditions

• Initial Value Problem:position of the molecules and velocities have to be given;initial configuration e.g.:

– molecules in crystal lattice (body-/face-centered cell)– initial velocity

• random direction• absolute value dependent of the temperature

(normal distribution or uniform), e.g.32 NkBT = 1

2∑N

i=1 mv2i with vi := v0

⇒ v0 :=√

3kBTm resp. v∗

0 :=√

3T∗∆t∗

• time discretisation: t := t0 + i ·∆t → time integration procedure3. Efficient Many-Body Algorithms: Barnes-Hut and Fast Multipole




Short-Range Interactions

−1

−0.8

−0.6

−0.4

−0.2

0

0.2

0.4

0 1 2 3 4 5

U*

r*

dim. red. Lennard−Jones 12−6 Potential

−2.5

−2

−1.5

−1

−0.5

0

0.5

0 1 2 3 4 5

F*

r*

dim. red. Lennard−Jones 12−6 Force

1

2

3

4

5

Fij Force matrix/Interaction-graph

- F12 F13 F14 F15−F12 - F23 F24 F25−F13 −F23 - F34 F35−F14 −F24 −F34 - F45−F15 −F25 −F35 −F45 -

• fast decay of force contributions with increasing distancedense force matrix with O(n2), mostly very small, entries





Short-Range Interactions

−1

−0.8

−0.6

−0.4

−0.2

0

0.2

0.4

0 1 2 3 4 5

U*

r*

dim. red. finites Lennard−Jones 12−6 Potential (rc=2)

−2.5

−2

−1.5

−1

−0.5

0

0.5

0 1 2 3 4 5

F*

r*

dim. red. finite Lennard−Jones 12−6 Force (rc=2)

1

2

3

4

5

Fij Force matrix/Interaction-graph

- F12 F13 F14 0−F12 - 0 F24 F25−F13 0 - F34 0−F14 −F24 −F34 - F45

0 −F25 0 −F45 -

• cut-off radius leads to a reduction of the computational effortsparse force matrix with O(n) entries





NucleationNucleation process of supersaturated Argon

t∗ = 10000 t∗ = 125000 t∗ = 230000 t∗ = 360000

• nucleation process for an oversaturated Argon vapour at0.97 Mol/l and 80k

• the simulation program automatically detects clusters (droplets)





0

2

4

6

8

10

12

35030025020015010050

num

ber o

f clu

ster

s

timesteps (103)

Clusters in a supersaturated Argon vapor, 80 K, 0.97 Mol/l

f1(x) = 0.00731 x -2.36f2(x) = 0.00774 x -4.09f3(x) = 0.00749 x -4.698

cluster size > 20cluster size > 30cluster size > 40

f1(x)f2(x)f3(x)

• counting and grouping clusters of certain size ranges, a statisticcan be generated

• the growth of the clusters (slope) is known as nucleation rate andimportant for macroscopic simulations





3.4. Numerical Methods for Long-RangePotentials

Numerical Methods for Long-RangePotentials

• so far: focus on short-range potentials such as Lennard-Jones– resulting mutual interactions are restricted to particles in

some local neighbourhood– facilitates numerical treatment and algorithmic organization:

no quadratic complexity induced by an “each-with-each”behaviour

• now: tackle long-range potentials, too– examples: Coulomb or gravitation potential– interactions between remote particles must not be neglected– simple cut-off not possible– nevertheless need for approaches that avoid quadratic

complexity





• what is long-range?

– intuitively: potential function U(r) does not decrease rapidlywith increasing r

– formally (one possibility): for d > 2, potentials notdecreasing faster than r−d for increasing r (criterion:integrability over Rd )

• typical potentials in applications have both – a short-range part(to be dealt with according to the previous sections) and along-range part, represented as two additive components:

U(r) := Ushort + U long





Tree-Based Methods

• based on integral representation of the potential

Φ(x) =1

4πε0

∫%(y)

1‖y − x‖

dy

• hierarchical decompositions of the domain of simulation

• adaptive approximation of the particle distribution

• widespread scheme: octrees

• allow for separation of near-field and far-field influences

• log-linear or even linear complexity can be obtained





• high flexibility with respect to more general potentials (as neededfor special applications, such as biomolecular problems)

• advantageous especially for heterogeneous particle distributions(frequent in astrophysics, relevant also for molecular dynamics)

• examples:

– panel clustering– Barnes-Hut method– (fast) multipole methods





Series Expansion of the Potential

• general (integral) representation of the potential:

Φ(x) =

∫Ω

G(x , y)%(y)dy

(general kernel G, particle density %(y), and domain Ω)

• Taylor expansion of the kernel G (if sufficiently smooth apart fromthe singularity in x = y ) in y around y0:

G(x , y) =∑‖j‖1≤p

1j!

G0,j (x , y0)(y − y0)j + Rp(x , y)

(multi-index j = (j1, j2, j3), j! = j1!j2!j3!, Gk,j (x , y) mixed (k , j)-thderivative (k -th w.r.t. x , j-th w.r.t. y ), remainder Rp(x , y))





• leads to expansion (and approximation) of the potential:

Φ(x) ≈∑‖j‖1≤p

1j!

Mj (Ω, y0)G0,j (x , y0)

with the so-called moments

Mj (Ω, y0) :=

∫Ω

%(y)(y − y0)jdy





Subdivision of the Domain

• separation of near-field and far-field for given x :

Ω = Ωnear ∪ Ωfar , Ωnear ∩ Ωfar = ∅

• decomposition of the far-field into disjoint, convex subdomains:

Ωfar =⋃

i

Ωfari

• note: this decomposition depends on x , i.e. it is done for eachparticle position x (efficient derivation possible from onehierarchical tree structure)

• each Ωfari has an associated point y i

0

• how to choose the subdivision?

diam

‖x − y i0‖

:=supy∈Ωfar

i‖y − y i

0‖‖x − y i

0‖≤ θ

for some suitable constant 0 < θ < 13. Efficient Many-Body Algorithms: Barnes-Hut and Fast Multipole




• resulting approximation for Φ(x):

Φ(x) =

∫Ω

%(y)G(x , y)dy

=

∫Ωnear

%(y)G(x , y)dy +

∫Ωfar

%(y)G(x , y)dy

=

∫Ωnear

%(y)G(x , y)dy +∑

i

∫Ωfar

i

%(y)G(x , y)dy

≈∫

Ωnear

%(y)G(x , y)dy +∑

i

∑‖j‖1≤p

1j!

Mj (Ωfari , y i

0)G0,j (x , y i0)





Error Estimates

• error characteristics for one fix particle position x ∈ Ω:

– local relative approximation error for one Ωfari can be shown

to be of order O(θp+1)

– global relative approximation error (summation over wholefar-field) can be shown to be of order O(θp+1)

• this clarifies the role of θ:

– allows to control the global approximation error in x– geometric requirement to the far-field subdivision: the closer

Ωfari is located to x , the smaller it has to be to fulfil the

θ-condition

• hence: a typical ”level of detail”

– the closer, the higher resolved– cf. terrain representation in flight simulators





Tree Structures

• central question: how can we construct all these necessaryseparations of near-fields and far-fields and subdivisions offar-fields in an efficient way?

• idea: recursive decomposition of Ω (a square in 2D, a cube in 3D– without loss of generality) in cells of different size, terminatingthe subdivision process if a cell is either empty or contains justone particle

• concepts:– kd-tree: alternate subdivision in coordinate direction (x , y ,

and z), such that the separation produces two subdomainsthat roughly contain the same number of particles each

– quadtree (2D) or octree (3D): subdivision into fourcongruent subsquares or eight congruent subcubes,respectively

• the following algorithms (Barnes-Hut etc.) use the octreeapproach





kd-Trees – Example

3

4

5

7

611

9

8

10

12

13

1 2 3 4 5

6

7

9

8 10 1312

11

2

1





Quadtrees and Octrees – Examples





Recursive Computation of the Far-Field• starting point: create the octree corresponding to the set of

particles– each node of the octree represents a subdomain of Ω or one

cell– for each cell i , define some y i

0 (the centre point or the centreof gravity of all particles contained, e.g.) for doing the Taylorexpansion

– for each cell i , let the parameter diam just denote thediameter of the smallest surrounding sphere, e.g.

• objective: for each particle position x , use as few cells aspossible (i.e. as big cells as possible) for fulfilling the diam-θ rule

• hence: start from root node, checkdiam

‖x − y i0‖≤ θ ,

stop if fulfilled (no need for further subdivision) and proceed if notyet fulfilled





• note that for each x , we typically get a different subdivision

• but note also that all these subdivisions are just subtrees of ourconstructed octree





Recursive Computation of the Moments

• now: use this subdivision for the efficient calculation of the localmoments Mj (Ωfar

i , y i0)

• direct (numerical) integration or direct summation are not efficient• therefore: use hierarchical tree structure to calculate all moments

for all cells in one run and store them• crucial property for that:

Mj (Ω1 ∪ Ω2, y0) = Mj (Ω1, y0) + Mj (Ω2, y0) ,

if the point of expansion y0 is the same• in the (standard) case of different y1

0 and y20 , there are simple

conversion formulas:

Mj (Ω1, y0) =∑k≤j

(jk

)(y0 − y0)j−k Mk (Ω1, y0)

(k ≤ j component-wise, multiplicative binomial coefficients)





• this allows for a bottom-up calculation of the moments from theleaves to the root

• in the leaves:

– if no particle present: zero– if one particle of mass m there in x : m(x − y i

0)j





Using these Building Blocksstill to be done for a numerical routine:• how to construct the tree, starting from a given set of particles?

• how to store the tree?

• how to choose cells and expansion points?

• how to determine far-field and near-field?several algorithmic variants to be discussed in the following





The Barnes-Hut Method• the so-called Barnes-Hut method is the oldest, simplest, and

most widespread hierarchical tree-based approach, dates backto 1986

• original (and still main) target applications: astrophysics (highparticle numbers and a typically very heterogeneous densitydistribution)

• particle-particle interaction via gravitation potential (ormodifications):

U(rij ) = −GGravmimj

rij

• uses octrees: leaves of the tree represent empty cells or oneparticle, inner nodes represent clusters of particles or pseudoparticles

• main underlying idea: gravitation (or other force) induced bymany particles in one remote cell can be approximated by theinfluence induced by one pseudo particle of the accumulatedmass in the cell’s centre of gravity





• three main steps:

– construction of the octree: refinement, until just one or noparticle per cell (top-down recursion)

– calculation of pseudo particles: mass is just sum of the cell’sparticles’ masses, associated point is the mass-weightedaverage of the particles’ positions (bottom-up recursion)

– calculation of forces (N incomplete top-down traversals for Nparticles)





Calculation of the Forces

• remember: to be done for each particle

• assume a particle with position x ∈ Ω

– start in root node– proceed to son nodes, until the θ − rule: diam

r ≤ θis fulfilled, where r denotes the distance of thecorresponding pseudo particle to x

– then, add the resulting interaction influence to the overallresult

• implicit separation into near-field and far-field:

– near-field: all leaves reached during this traversal(single-particle influence)

– far-field: all inner nodes where the process stops, i.e. cellsrepresenting pseudo particles





• several variants concerning the determination of the parameterdiam

• Barnes-Hut method can be interpreted as a special case of theTaylor approximation discussed above for p = 0 (for thecalculation of the pseudo particles, we just sum up masses, i.e.zero-th moments)





Example of the Tree Construction and ForceCalculation

! ! ! !! ! ! !! ! ! !

" " " " " " " "" " " " " " " "" " " " " " " "

# # # # # # ## # # # # # ## # # # # # #

$ $ $ $ $$ $ $ $ $$ $ $ $ $

% % % %% % % %% % % %

& & && & && & &

' '' '' '

( (( (( (

) )) )) ) * *

* ** *

+ ++ ++ +

, ,, ,, ,

- -- -- -

. . . . . .. . . . . .. . . . . .

/ / / / / // / / / / // / / / / /

0 00 00 0

1 11 11 1

2223334 4 4 44 4 4 44 4 4 4

5 5 5 55 5 5 55 5 5 5

6 6 6 6 6 66 6 6 6 6 66 6 6 6 6 6

7 7 7 7 7 77 7 7 7 7 77 7 7 7 7 7

8 8 8 8 8 8 8 8 8 88 8 8 8 8 8 8 8 8 88 8 8 8 8 8 8 8 8 8

9 9 9 9 9 9 9 9 9 99 9 9 9 9 9 9 9 9 99 9 9 9 9 9 9 9 9 9

: : : : : : : :: : : : : : : :: : : : : : : :

; ; ; ; ; ; ; ;; ; ; ; ; ; ; ;; ; ; ; ; ; ; ;

< < < < << < < < << < < < <

= = = = == = = = == = = = =

> > >> > >> > >

? ? ?? ? ?? ? ?@ @ @ @ @ @

@ @ @ @ @ @@ @ @ @ @ @

A A A A A AA A A A A AA A A A A A

B BB BB B

C CC CC C

D DD DD D

E EE EE E

F F F F F FF F F F F FF F F F F F

G G G G G GG G G G G GG G G G G GH H H H H H H H H H H H H H H H H H H

H H H H H H H H H H H H H H H H H H HH H H H H H H H H H H H H H H H H H HH H H H H H H H H H H H H H H H H H H

I I I I I I I I I I I I I I I I I I II I I I I I I I I I I I I I I I I I II I I I I I I I I I I I I I I I I I II I I I I I I I I I I I I I I I I I I

x i

x i





Accuracy and Complexity

• accuracy:

– depends on control parameter θ– the smaller we choose θ, the larger gets the near-field and

the smaller, hence, gets the error resulting from thecorresponding far-field approximation

– however, slow O(θ) convergence due to p = 0

• complexity:

– increases for decreasing θ– for roughly homogeneous particle distributions:

• number of active cells bounded by C log N/θ3 for some constant Cand N particles

• overall cost of order O(θ−3N log N)

– for limit case θ → 0: method degenerates to the quadraticcomplexity of the original “each-with-each” approach withoutfar-field approximation





Some Remarks on the Implementation

• use a standard data structure for each (pseudo) particle• standard tree implementation with pointers:

– each node contains one (pseudo) particle at most– subdomain corresponding to a node/cell can be either stored

explicitly or calculated on-the-fly during a tree traversal– four (2D) or eight (3D) pointers from a node to its sons

• linearisation is possible, too (i.e. no need for pointers)• tree traversal typically by recursion

– pre-order (top-down)– post-order (bottom-up)

• tree construction:– successive insertion of particles– starting from the empty tree (i.e. the octree consisting of the

root node only)– refine, when necessary (i.e. two particles in one node)





Some Remarks on the Implementation (2)

• calculation of pseudo particles:– bottom-up (post-order) traversal– sum of masses, weighted average of positions as described

before• calculation of forces:

– outer loop: traversal visiting each leaf (to get the far-fieldapproximation for each particle)

– inner loop: top-down (pre-order) traversal until the θ-rule isfulfilled

– parameter diam on-the-fly• time integration: essentially as before, now as another tree

traversal• motion:

– either via constructing a new octree– or via modifying the existing octree (generally preferred)





The Fast-Multipole Method – Main Idea• idea of Barnes-Hut: replace particle-particle interactions by

interactions between particles and pseudo particles• effect is an improved complexity: O(N log N) instead of O(N2) for

almost homogeneous particle distributions, and still a significanteffect otherwise

• next step now:– avoid calculation of interactions with remote pseudo

particles on the “origin” side for each particle on the “effect”side individually

– instead, combine particles to pseudo particles on the “effect”side also and calculate interactions of pseudo particles withremote pseudo particles, only

– cells representing pseudo particles with their containedparticles are called clusters

– from such a clucter-cluster interaction, the particle-clusterinteractions can be derived for all of the cluster’s particles

– if done properly, this fast multipole method provides alinear O(N) complexity





Fundamentals

• starting point again: Taylor expansion of the kernel G(x , y), butnow in x and y around x l

0 ∈ Ωl and y i0 ∈ Ωi :

G(x , y) =∑‖k‖1≤p

∑‖j‖1≤p

1k !j!

(x−x l0)k (y−y i

0)jGk,j (x l0, y

i0) + Rp(x , y)

• leads to expansion (and approximation) of the potential (here:the part due to the subdomain Ωi ):

Φi (x) ≈∑‖k‖1≤p

1k !

(x − x l0)k

∑‖j‖1≤p

1j!

Gk,j (x l0, y

i0)Mj (Ωi , y i

0)

(moments defined as before)





• hence: interaction between Ωi and Ωl is calculated once and can,afterwards, be used for deriving the particle-cluster interactionsfor all x ∈ Ωl (using suitable transformation rules)

• this principle is now applied hierarchically or recursively:

– first, calculate interactions between large clusters– these are “inherited” to the son level, where now the

resulting interactions are determined– finally, get particle-X interactions in the leaves





Barnes-Hut vs. Fast Multipole – Comparison

x

x

y

yl0

i

i

0

0





Subdivision

• instead of a recursive subdivision of Ω, we have to do it for Ω× Ωnow

• same octree structure for origin and effect side• typically, same “midpoints”: x i

0 = y i0 ∈ Ωi (again, geometrical

midpoint or centre of gravity)• now, also a “double” θ-criterion:

‖x − x l0‖

‖x l0 − y i

0‖≤ θ and

‖y − y i0‖

‖x l0 − y i

0‖≤ θ ∀x ∈ Ωl , y ∈ Ωi

• hierarchy can be used for an efficient calculation of theinteractions (for details, we refer to the literature)

• accuracy: as before, an O(θp+1)-characteristics can be shown forthe relative error

• parallelisation: only slightly more complicated than forBarnes-Hut




Documents

3. Efficient Many-Body Algorithms: Barnes-Hut and Fast ... · 3.2. Molecular Dynamics – the Physical Model Classical Molecular Dynamics Quantum mechanics! approximation classical