Embed Size (px)
Citation preview
Particle Swarm Optimization:Application to Nanostructure Reconstruction from X-Ray Scattering Data
Abhinav Sarje
Computational Research DivisionLawrence Berkeley National Laboratory
December 09, 2015
Introduction Structure Reconstruction Particle Swarm Optimization End Notes
Small Angle X-ray Scattering at Synchrotrons
[email protected] Lawrence Berkeley National Laboratory
graphic: courtesy of A. Meyer, www.gisaxs.de
Introduction Structure Reconstruction Particle Swarm Optimization End Notes
Inverse Modeling for Reconstruction: The Optimization Problem
[email protected] Lawrence Berkeley National Laboratory
Start
Initial guess
forwardsimulation
compute error w.r.t.experimental data
tune modelparameters
End
Introduction Structure Reconstruction Particle Swarm Optimization End Notes
Inverse Modeling for Reconstruction
Various optimization algorithms exist. Some of the representative algorithms in our case:
• Stochastic: Reverse Monte-Carlo.
• Gradient based: LMVM (Limited-Memory Variable-Metric.)
• Derivative-free trust region-based: Pounders.
• Stochastic: Particle Swarm Optimization.
Objective functions: Forward simulation for computing scattered light intensities. E.g.
• Fast Fourier Transformation computations.
• Complex form factor and structure factor computations.
[email protected] Lawrence Berkeley National Laboratory
Introduction Structure Reconstruction Particle Swarm Optimization End Notes
The Objective Function: Forward Simulations
Given:1 a sample structure model, and
2 experimental configuration,simulate scattering patterns.
Scattered light intensity is computed at each point~q in the detector inverse space.
[email protected] Lawrence Berkeley National Laboratory
Introduction Structure Reconstruction Particle Swarm Optimization End Notes
The Objective Function: Forward Simulations
• Intensity is proportional to square of the sum of Form Factors (and Structure Factors) at~q:
I(~q) ∝
∣∣∣∣∣S∑
s=1
F(~q)
∣∣∣∣∣2
• Form Factor at~q is an integral over an object surface.
F(~q) = − i|q|2
∫S(~r)
ei~qr·~rqn(~r)d2~r
• Computed as summation over a discretization of the object surface:
F(~q) ≈ − i|q|2
t∑k=1
ei~q·~rk qn,kσk
[email protected] Lawrence Berkeley National Laboratory
Introduction Structure Reconstruction Particle Swarm Optimization End Notes
LMVM and POUNDerS: Two Parameter Case
A Single CylindricalNanoparticle X-Ray Scattering Pattern
16 18 20 22 24x1
16
18
20
22
24
x2
0
20
40
60
80
100
120
140
160
180
Objective Function Map
16 18 20 22 24x1
16
18
20
22
24
x2
15
30
45
60
75
90
105
LMVM Convergence Map
16 18 20 22 24x1
16
18
20
22
24
x2
3
6
9
12
15
18
21
24
27
POUNDerS Convergence Map
[email protected] Lawrence Berkeley National Laboratory
Introduction Structure Reconstruction Particle Swarm Optimization End Notes
LMVM and POUNDerS: Six Parameter Case
Pyramidal Nanoparticlesforming a Lattice
X-Ray Scattering Pattern36 37 38 39 40 41 42 43 44
x1
46
48
50
52
54
x3
6.0
7.5
9.0
10.5
12.0
13.5
15.0
16.5
18.0
Objective Function Map
LMVM does not converge
36 37 38 39 40 41 42 43 44x1
46
48
50
52
54
x3
12
14
16
18
20
22
24
26
28
POUNDerS Convergence Map
[email protected] Lawrence Berkeley National Laboratory
Introduction Structure Reconstruction Particle Swarm Optimization End Notes
Particle Swarm Optimization
• A class of stochastic methods.
• Multiple agents, “particle swarm”, evaluate for optimal positions within the parameter searchspace.
• Agents perform quasi “random-walks”.
• Agent “velocities” (displacements) are influenced by the histories of the traveled paths.
• No initial guesses of parameter values necessary.
[email protected] Lawrence Berkeley National Laboratory
Introduction Structure Reconstruction Particle Swarm Optimization End Notes
Particle Swarm Optimization
[email protected] Lawrence Berkeley National Laboratory
Introduction Structure Reconstruction Particle Swarm Optimization End Notes
Particle Swarm Optimization: Fitting X-Ray Scattering Data
2 4 6 8 10 12 14 16 18 20Number of agents
1
4
9
16
25
36
49
64
81
100
Para
mete
r sp
ace
volu
me
80
160
240
320
400
480
560
640
Funct
ion e
valu
ati
ons
Fitting 2 Parameters
2 4 6 8 10 12 14 16 18 20Number of agents
0.5
1.0
1.5
2.0
2.5
3.0
3.5
4.0
Para
mete
r sp
ace
volu
me
100
200
300
400
500
600
700
800
900
Funct
ion e
valu
ati
ons
Fitting 6 Parameters
[email protected] Lawrence Berkeley National Laboratory
Introduction Structure Reconstruction Particle Swarm Optimization End Notes
Particle Swarm Optimization: Runtime Performance
0 4 16 36 64 100 144Search Space Volume ([x1 ]x[x2 ])
0
5
10
15
20
25
30
Itera
tions
for
Converg
ence
15 agents20 agents
Convergence w.r.t. Search Space Volume
4 16 64 256Number of GPUs
102
103
104
Tim
e [
s]
sample 1
sample 2
Strong Scaling on Titan
[email protected] Lawrence Berkeley National Laboratory
Introduction Structure Reconstruction Particle Swarm Optimization End Notes
Particle Swarm Optimization: Agents vs. Generations
0 10 20 30 40 50Generation
10-12
10-11
10-10
10-9
10-8
10-7
10-6
10-5
10-4
10-3
10-2
10-1
100
Rela
tive E
rror
5101520253035404550556065707580859095100
0 10 20 30 40Generation
20
40
60
80
100
Num
. A
gents
10-11
10-10
10-9
10-8
10-7
10-6
10-5
10-4
10-3
10-2
10-1
Rela
tive E
rror
[email protected] Lawrence Berkeley National Laboratory
Introduction Structure Reconstruction Particle Swarm Optimization End Notes
Deeper into Particle Swarm Optimization
Selecting coefficient values ω, φ1, φ2:
0 1 2 3 4 5
φ1
0
1
2
3
4
5
φ2
10-5
10-4
10-3
10-2
10-1
100
ω = 0.00
0 1 2 3 4 5
φ1
0
1
2
3
4
5
φ2
10-5
10-4
10-3
10-2
10-1
100
ω = 0.25
0 1 2 3 4 5
φ1
0
1
2
3
4
5
φ2
10-5
10-4
10-3
10-2
10-1
100
ω = 0.50
0 1 2 3 4 5
φ1
0
1
2
3
4
5
φ2
10-3
10-2
10-1
100
ω = 0.75
0 1 2 3 4 5
φ1
0
1
2
3
4
5
φ2
100
ω = 1.00
[email protected] Lawrence Berkeley National Laboratory
Introduction Structure Reconstruction Particle Swarm Optimization End Notes
Particle Swarm Optimization: Variants
Numerous variations of the basic PSO algorithm exist. Each is a variant in either:
• Agent’s position displacement computation method.
• Connection topology among the agents.
• Combination of both.
[email protected] Lawrence Berkeley National Laboratory
Introduction Structure Reconstruction Particle Swarm Optimization End Notes
Particle Swarm Optimization: Basic
1 Each agent evaluates the objective function at its current position in the search space.
2 Each agent identifies its “local” best position yet.
3 With fully-connected topology, all agents collaborate to find the “global” best position yet.
4 Each agent computes its displacement and moves to the new position.
[email protected] Lawrence Berkeley National Laboratory
Introduction Structure Reconstruction Particle Swarm Optimization End Notes
Particle Swarm Optimization: Position Displacement
Inertia coefficient ω, as
• a constant value throughout,
• dampened value through constriction coefficient χ,
• a variable and/or tuned value.
Fully-informed particle swarm (FIPS).
• Each agent explicitly influences all other agents (all-to-all).
Fitness distance ratio (FDR).
• Replace Euclidean distance measure between agents’ position from local and global best with adistance ratio using the corresponding “fitness” values.
[email protected] Lawrence Berkeley National Laboratory
Introduction Structure Reconstruction Particle Swarm Optimization End Notes
Particle Swarm Optimization: Position DisplacementFIPS:
10-5
10-4
10-3
10-2
10-1
0 5 10
15
20
25
30
35
40
45
50
Rel
. Erro
r
Generation
49100144196240289342
10-2
10-1
100
0 5 10
15
20
25
30
35
40
45
50
Rel
. Erro
r
Generation
49100144196240
FDR:
10-12
10-10
10-8
10-6
10-4
10-2
100
0 5 10
15
20
25
30
35
40
45
50
Rel
. Erro
r
Generation
49100144196240289342
10-4
10-3
10-2
10-1
100
0 5 10
15
20
25
30
35
40
45
50
Rel
. Erro
r
Generation
49100144196240
[email protected] Lawrence Berkeley National Laboratory
Introduction Structure Reconstruction Particle Swarm Optimization End Notes
Particle Swarm Optimization: Topology
Position updates are influenced only by the “neighborhoods” defined by the topology used.
Lbest.
• Ring lattice topology – each agent has exactly two neighbors.
• Increases agent independence making them less susceptible by others’ local minima.
• Takes longer to converge, but in general provides better solutions.
• Also, avoids global communication.
von Neumann topology.
• Torus lattice topology – each agent has exactly four (2D) or six (3D) neighbors.
• Better balance between Lbest and fully-connected topologies.
[email protected] Lawrence Berkeley National Laboratory
Introduction Structure Reconstruction Particle Swarm Optimization End Notes
Particle Swarm Optimization: TopologyLbest:
10-1010-910-810-710-610-510-410-310-210-1100101
0 5 10
15
20
25
30
35
40
45
50
Rel
. Erro
r
Generation
49100144196240289342
10-3
10-2
10-1
100
0 5 10
15
20
25
30
35
40
45
50
Rel
. Erro
r
Generation
49100144196240
von Neumann:
10-12
10-10
10-8
10-6
10-4
10-2
100
0 5 10
15
20
25
30
35
40
45
50
Rel
. Erro
r
Generation
49100144196240289342
10-4
10-3
10-2
10-1
100
0 5 10
15
20
25
30
35
40
45
50
Rel
. Erro
r
Generation
49100144196240
[email protected] Lawrence Berkeley National Laboratory
Introduction Structure Reconstruction Particle Swarm Optimization End Notes
Particle Swarm Optimization: Look-Ahead
• Better exploit massive parallelism.
• Each agent identifies best of k possible positions from its current position.
• Uses more comprehensive knowledge of search space neighborhood to identify its path.
10-12
10-10
10-8
10-6
10-4
10-2
100
0 5 10
15
20
25
30
35
40
45
50
Rel
. Erro
r
Generation
49100144196240289342
[email protected] Lawrence Berkeley National Laboratory
Introduction Structure Reconstruction Particle Swarm Optimization End Notes
An Ongoing Work ...
We saw that
• Gradient based methods converge only for simple cases, and gradient-free trust region basedmethods are highly sensitive to the initial guess.
• PSO works without the need of an initial guess.
• PSO is robust, nearly always converging, but computationally expensive.
Near future:
• Evaluating other stochastic methods.
• Using machine learning for feature and structural classification to generate initial models to fit.
• Evaluating neuromorphic computing for classification.
[email protected] Lawrence Berkeley National Laboratory
Introduction Structure Reconstruction Particle Swarm Optimization End Notes
An Ongoing Work ...
We saw that
• Gradient based methods converge only for simple cases, and gradient-free trust region basedmethods are highly sensitive to the initial guess.
• PSO works without the need of an initial guess.
• PSO is robust, nearly always converging, but computationally expensive.
Near future:
• Evaluating other stochastic methods.
• Using machine learning for feature and structural classification to generate initial models to fit.
• Evaluating neuromorphic computing for classification.
[email protected] Lawrence Berkeley National Laboratory
Introduction Structure Reconstruction Particle Swarm Optimization End Notes
Further Information
1 HipGISAXS: a high-performance computing code for simulating grazing-incidence X-rayscattering data. Journal of Applied Crystallogrpahy, vol. 46, pp. 1781–1795, 2013.
2 Massively parallel X-ray scattering simulations. Supercomputing (SC), 2012.
3 Tuning HipGISAXS on multi- and many-core supercomputers. Performance Modeling,Benchmarking and Simulation of High Performance Computer Systems, 2013.
4 High performance inverse modeling with Reverse Monte Carlo simulations. InternationalConference on Parallel Processing, 2014.
[email protected] Lawrence Berkeley National Laboratory
Introduction Structure Reconstruction Particle Swarm Optimization End Notes
Acknowledgments
• Thanks to NVIDIA for donating several GPU cards to make this work possible.
• Supported by the Office of Science of the U.S. Department of Energy under Contract No.DE-AC02-05CH11231.
• Also supported by DoE Early Career Research grant awarded to Alexander Hexemer.
• Used resources of the Oak Ridge Leadership Computing Facility at the Oak Ridge NationalLaboratory, which is supported by the Office of Science of the U.S. Department of Energy underContract No. DE-AC05-00OR22725.
• Used resources of the National Energy Research Scientific Computing Center, which issupported by the Office of Science of the U.S. Department of Energy under Contract No.DE-AC02-05CH11231.
[email protected] Lawrence Berkeley National Laboratory
Introduction Structure Reconstruction Particle Swarm Optimization End Notes
Thank you!
Contact: [email protected]
[email protected] Lawrence Berkeley National Laboratory
