Embed Size (px)
Citation preview
Recent developments in MADS algorithms:ABAGUS and Squads
Dylan R. HarpVelimir V. Vesselinov
LA-UR-11-11957
2011 EES-16 Brownbag Series
Model analysis and decision support (MADS) for complexproblems
Complex problems:
Large number of model parameters
Nonlinear and hysteretic parameter correlations
Multiple maxima/minima
Flat response surface regions (portions of parameter spacewith low parameter sensitivity)
Long execution times
Require efficient and robust model analyses strategies
Model analysis and decision support (MADS) for complexmodels
Why do we care?
Model analysis
Calibration/parameter estimationUncertainty quantificationParameter sensitivities and correlationsPredictive analysisModel selectionModel averaging
Decision support
Robust and/or optimal decisions
Model analysis and decision support (MADS) for complexmodels
Why do we care?
Model analysis
Calibration/parameter estimationUncertainty quantificationParameter sensitivities and correlationsPredictive analysisModel selectionModel averaging
Decision support
Robust and/or optimal decisions
Model analysis and decision support (MADS) for complexmodels
Why do we care?
Model analysis
Calibration/parameter estimationUncertainty quantificationParameter sensitivities and correlationsPredictive analysisModel selectionModel averaging
Decision support
Robust and/or optimal decisions
Agent-Based Analysis of Global Uncertainty and Sensitivity
ABAGUS features:
“Agent-based” model analysis
Extends Particle Swarm Optimization(PSO) to uncertainty and sensitivityanalysis
Collects all model evaluation results inKD-Tree for efficient restart andhierarchical analysis
Response surface sculpting discouragesreinvestigation of ”collected” regionsof the parameter
Discretized parameter space
Automated discretization refinement
ABAGUS uses:
Identify acceptableparameter ranges
Sensitivity analysis
Identify parametercorrelations
Parameteruncertainty analysis
Predictive analysis
Decision support
Information forthese are containedin the results from asingle ABAGUS run
Harp, D.R. and V.V. Vesselinov (2011), An agent-based approach to global uncertainty
and sensitivity analysis, Computers & Geosciences, doi:10.1016/j.cageo.2011.06.025.
Agent-Based Analysis of Global Uncertainty and Sensitivity
ABAGUS features:
“Agent-based” model analysis
Extends Particle Swarm Optimization(PSO) to uncertainty and sensitivityanalysis
Collects all model evaluation results inKD-Tree for efficient restart andhierarchical analysis
Response surface sculpting discouragesreinvestigation of ”collected” regionsof the parameter
Discretized parameter space
Automated discretization refinement
ABAGUS uses:
Identify acceptableparameter ranges
Sensitivity analysis
Identify parametercorrelations
Parameteruncertainty analysis
Predictive analysis
Decision support
Information forthese are containedin the results from asingle ABAGUS run
Harp, D.R. and V.V. Vesselinov (2011), An agent-based approach to global uncertainty
and sensitivity analysis, Computers & Geosciences, doi:10.1016/j.cageo.2011.06.025.
Agent-Based Analysis of Global Uncertainty and Sensitivity
ABAGUS features:
“Agent-based” model analysis
Extends Particle Swarm Optimization(PSO) to uncertainty and sensitivityanalysis
Collects all model evaluation results inKD-Tree for efficient restart andhierarchical analysis
Response surface sculpting discouragesreinvestigation of ”collected” regionsof the parameter
Discretized parameter space
Automated discretization refinement
ABAGUS uses:
Identify acceptableparameter ranges
Sensitivity analysis
Identify parametercorrelations
Parameteruncertainty analysis
Predictive analysis
Decision support
Information forthese are containedin the results from asingle ABAGUS run
Harp, D.R. and V.V. Vesselinov (2011), An agent-based approach to global uncertainty
and sensitivity analysis, Computers & Geosciences, doi:10.1016/j.cageo.2011.06.025.
Agent-Based Analysis of Global Uncertainty and Sensitivity
ABAGUS features:
“Agent-based” model analysis
Extends Particle Swarm Optimization(PSO) to uncertainty and sensitivityanalysis
Collects all model evaluation results inKD-Tree for efficient restart andhierarchical analysis
Response surface sculpting discouragesreinvestigation of ”collected” regionsof the parameter
Discretized parameter space
Automated discretization refinement
ABAGUS uses:
Identify acceptableparameter ranges
Sensitivity analysis
Identify parametercorrelations
Parameteruncertainty analysis
Predictive analysis
Decision support
Information forthese are containedin the results from asingle ABAGUS run
Harp, D.R. and V.V. Vesselinov (2011), An agent-based approach to global uncertainty
and sensitivity analysis, Computers & Geosciences, doi:10.1016/j.cageo.2011.06.025.
Agent-Based Analysis of Global Uncertainty and Sensitivity
ABAGUS features:
“Agent-based” model analysis
Extends Particle Swarm Optimization(PSO) to uncertainty and sensitivityanalysis
Collects all model evaluation results inKD-Tree for efficient restart andhierarchical analysis
Response surface sculpting discouragesreinvestigation of ”collected” regionsof the parameter
Discretized parameter space
Automated discretization refinement
ABAGUS uses:
Identify acceptableparameter ranges
Sensitivity analysis
Identify parametercorrelations
Parameteruncertainty analysis
Predictive analysis
Decision support
Information forthese are containedin the results from asingle ABAGUS run
Harp, D.R. and V.V. Vesselinov (2011), An agent-based approach to global uncertainty
and sensitivity analysis, Computers & Geosciences, doi:10.1016/j.cageo.2011.06.025.
Agent-Based Analysis of Global Uncertainty and Sensitivity
ABAGUS features:
“Agent-based” model analysis
Extends Particle Swarm Optimization(PSO) to uncertainty and sensitivityanalysis
Collects all model evaluation results inKD-Tree for efficient restart andhierarchical analysis
Response surface sculpting discouragesreinvestigation of ”collected” regionsof the parameter
Discretized parameter space
Automated discretization refinement
ABAGUS uses:
Identify acceptableparameter ranges
Sensitivity analysis
Identify parametercorrelations
Parameteruncertainty analysis
Predictive analysis
Decision support
Information forthese are containedin the results from asingle ABAGUS run
Harp, D.R. and V.V. Vesselinov (2011), An agent-based approach to global uncertainty
and sensitivity analysis, Computers & Geosciences, doi:10.1016/j.cageo.2011.06.025.
Agent-Based Analysis of Global Uncertainty and Sensitivity
ABAGUS features:
“Agent-based” model analysis
Extends Particle Swarm Optimization(PSO) to uncertainty and sensitivityanalysis
Collects all model evaluation results inKD-Tree for efficient restart andhierarchical analysis
Response surface sculpting discouragesreinvestigation of ”collected” regionsof the parameter
Discretized parameter space
Automated discretization refinement
ABAGUS uses:
Identify acceptableparameter ranges
Sensitivity analysis
Identify parametercorrelations
Parameteruncertainty analysis
Predictive analysis
Decision support
Information forthese are containedin the results from asingle ABAGUS run
Harp, D.R. and V.V. Vesselinov (2011), An agent-based approach to global uncertainty
and sensitivity analysis, Computers & Geosciences, doi:10.1016/j.cageo.2011.06.025.
Monte Carlo vs ABAGUS: Estimation of probability ofsuccess/failure based
Example: parabola function
f (x1, x2) = x21 + x2
2
Goal: estimate area wheref (x1, x2) ≤ 160, (red circle)
f (x1, x2) ≤ 160 isapproximately 5% of domain
x uniformly distributed
Domain: x = [−50 : 50]
Monte Carlo vs ABAGUS: Estimation of probability ofsuccess/failure based
Example: parabola function
f (x1, x2) = x21 + x2
2
Goal: estimate area wheref (x1, x2) ≤ 160, (red circle)
f (x1, x2) ≤ 160 isapproximately 5% of domain
x uniformly distributed
Domain: x = [−50 : 50]
Monte Carlo estimation of probability of success/failure
Estimation of parameter space with f (x1, x2) ≤ 160
Probability of success/failure (i.e. domain fraction) estimated by fractionof random samples in “red circle”
Monte Carlo uses an Improved Distance Latin Hypercube Samplingmethod (encoded in MADS as well)
ABAGUS estimation of probability of success/failure
Before exploration
f (x1, x2) = 160 indicatedby red circle
Zoomed intox1, x2 = [−20 : 20]
After exploration
Response surface sculpted
“Acceptable” parametersets collected
ABAGUS estimation of probability of success/failure
Before exploration
f (x1, x2) = 160 indicatedby red circle
Zoomed intox1, x2 = [−20 : 20]
After exploration
Response surface sculpted
“Acceptable” parametersets collected
ABAGUS estimation of probability of success/failure
Estimation of parameter space with f (x1, x2) ≤ 160
ABAGUS results on more complicated response surfaces...
Griewank Rosenbrock
-100 -50 0 50 100
x
-100
-50
0
50
100
y
0
1
2
3
4
5
6
7
-100 -50 0 50 100
x
-100
-50
0
50
100
y
0
2e+09
4e+09
6e+09
8e+09
1e+10
1.2e+10
(a) (b)
-20 -15 -10 -5 0 5 10 15 20
x-20-15
-10-5
05
1015
20
y
00.5
11.5
22.5
z
0
0.5
1
1.5
2
2.5
-4 -2 0 2 4
x-4-2
02
4
y
0
25000
50000
75000
100000
z
0
25000
50000
75000
100000
(c) (d)
-20 -15 -10 -5 0 5 10 15 20
x
-20
-15
-10
-5
0
5
10
15
20
y
0
0.01
0.02
0.03
0.04
0.05
0.06
0.07
0.08
0.09
0.1
-4 -3 -2 -1 0 1 2 3 4 5
x
-1
0
1
2
3
4
5
6
7
8y
0
5
10
15
20
(e) (f)
ABAGUS as predictive analyzer
Identify “plausible”region based on 1st
criterion
Gradient contours of2nd criterion
Max/min values of
2nd criterion within1st criterion
ABAGUS as predictive analyzer
Identify “plausible”region based on 1st
criterion
Gradient contours of2nd criterion
Max/min values of
2nd criterion within1st criterion
ABAGUS as predictive analyzer
Identify “plausible”region based on 1st
criterion
Gradient contours of2nd criterion
Max/min values of
2nd criterion within1st criterion
ABAGUS application
Contaminant plume in aquifer...
w01
w02
w03
w04
w05
w06w07
w08
w09
w10
w11
w12
w13
d01
d02
d03
d04
0 500 1000 1500 2000 2500 3000 3500x [m]
500
1000
1500
2000
2500
3000
y [m
]
0.1
1
100
1000
c [ppb]
Source
Source location search domain
w wells (circles) - existing wells
d wells (stars) - proposal wells
Uncertain parameters: source location (xs , ys) dispervities(ax , ay , az)
ABAGUS application
Parameter histograms produced from ABAGUS:
ABAGUS application
Plausible source locations collected by ABAGUS:
Min OF at each source location plotted
ABAGUS application
Predictive analysis of concentrations at proposal wells:(a) d01 (b) d02
0
100
200
300
400
500
600
-3 -2 -1 0 1 2 3 4
Fre
quen
cy
log10(c [ppb])
0
100
200
300
400
500
600
-2 -1 0 1 2 3 4
Fre
quen
cy
log10(c [ppb])
(c) d03 (d) d04
0
100
200
300
400
500
600
-6 -5 -4 -3 -2 -1 0 1 2 3 4
Fre
quen
cy
log10(c [ppb])
0
100
200
300
400
500
600
-3 -2 -1 0 1 2 3
Fre
quen
cy
log10(c [ppb])
Adaptive Optimization: Squads
Squads
Global optimization with local optimization speedup
Global strategy: Adaptive Particle Swarm Optimization(APSO)
Local strategy: Levenberg-Marquardt (LM)
Adaptive rules balance strategies
Vesselinov, V.V. and D.R. Harp, Adaptive hybrid optimization strategy for
calibration and parameter estimation of physical model, Computers &
Geosciences, In Review.
Squads comparisons
Squads is compared to:
Levenberg-Marquardt (LM) - local strategy
Particle Swarm Optimization (PSO) Standard 2006 - globalstrategy
TRIBES Adaptive PSO - global strategy
hPSO (PSO + simplex) - alternative hybrid strategy
Comparison details:
2D, 5D, and 10D Rosenbrock and Griewank test functions
Domain: x = [−100 : 100]
20,000 allowable function evaluations for each optimizationrun
1000 runs per strategy for each test function
Success: all parameters within 0.1 of optimal parameters
Squads comparisons
Squads is compared to:
Levenberg-Marquardt (LM) - local strategy
Particle Swarm Optimization (PSO) Standard 2006 - globalstrategy
TRIBES Adaptive PSO - global strategy
hPSO (PSO + simplex) - alternative hybrid strategy
Comparison details:
2D, 5D, and 10D Rosenbrock and Griewank test functions
Domain: x = [−100 : 100]
20,000 allowable function evaluations for each optimizationrun
1000 runs per strategy for each test function
Success: all parameters within 0.1 of optimal parameters
Squads: Rosenbrock comparisons2D Rosenbrock
0 10000 20000
LM
PSO
TRIBES
hPSO
SQUADS
Function evaluations
360 runs
992 runs
982 runs
1000 runs
1000 runs
Boxes indicate 25th to 75th
percentile range for number ofevaluations needed to achievesuccess
Vertical lines in boxes indicatemedian value
“Whiskers” indicate max andmin values
Number of successful runs outof 1000 are indicated aboveboxes
2D Rosenbrock function
Global minimum: x = 1
Squads: Rosenbrock comparisons
Function evaluation boxplots 2D Rosenbrock function
Global minimum: x = 1
Squads: Griewank comparisons
Griewank Function:
Ideal for comparison ofhybrid methods
Becomes more difficult forglobal methods withincreased dimensionality
Becomes easier for localmethods with increaseddimensionality
Hybrid methods shouldhave a well balanced act
2D Griewank function
Global minimum: x = 0
Squads: Griewank comparisons
Function evaluation boxplots 2D Griewank function
Global minimum: x = 0
SQUADS application
Case A Case B Case C Case D
LM
-1
0
1
2
3
4
5
6
7
0 2000 4000 6000 8000 10000
log 1
0(O
F)
Function evaluations
-1
0
1
2
3
4
5
6
7
0 10 20 30 40 50 60 70 80 90 100
Frequency
-1
0
1
2
3
4
5
6
7
0 10 20 30 40 50 60 70 80 90 100
Frequency
-1
0
1
2
3
4
5
6
7
0 2000 4000 6000 8000 10000
log 1
0(O
F)
Function evaluations
-1
0
1
2
3
4
5
6
7
0 10 20 30 40 50 60 70 80 90 100
Frequency
-1
0
1
2
3
4
5
6
7
0 10 20 30 40 50 60 70 80 90 100
Frequency
-1
0
1
2
3
4
5
6
7
0 2000 4000 6000 8000 10000
log 1
0(O
F)
Function evaluations
-1
0
1
2
3
4
5
6
7
0 10 20 30 40 50 60 70 80 90 100
Frequency
-1
0
1
2
3
4
5
6
7
0 10 20 30 40 50 60 70 80 90 100
Frequency
-1
0
1
2
3
4
5
6
7
0 2000 4000 6000 8000 10000
log 1
0(O
F)
Function evaluations
-1
0
1
2
3
4
5
6
7
0 10 20 30 40 50 60 70 80 90 100
Frequency
-1
0
1
2
3
4
5
6
7
0 10 20 30 40 50 60 70 80 90 100
Frequency
PSO
-1
0
1
2
3
4
5
6
7
0 2000 4000 6000 8000 10000
log 1
0(O
F)
Function evaluations
-1
0
1
2
3
4
5
6
7
0 10 20 30 40 50 60 70 80 90 100
Frequency
-1
0
1
2
3
4
5
6
7
0 10 20 30 40 50 60 70 80 90 100
Frequency
-1
0
1
2
3
4
5
6
7
0 2000 4000 6000 8000 10000
log 1
0(O
F)
Function evaluations
-1
0
1
2
3
4
5
6
7
0 10 20 30 40 50 60 70 80 90 100
Frequency
-1
0
1
2
3
4
5
6
7
0 10 20 30 40 50 60 70 80 90 100
Frequency
-1
0
1
2
3
4
5
6
7
0 2000 4000 6000 8000 10000
log 1
0(O
F)
Function evaluations
-1
0
1
2
3
4
5
6
7
0 10 20 30 40 50 60 70 80 90 100
Frequency
-1
0
1
2
3
4
5
6
7
0 10 20 30 40 50 60 70 80 90 100
Frequency
-1
0
1
2
3
4
5
6
7
0 2000 4000 6000 8000 10000
log 1
0(O
F)
Function evaluations
-1
0
1
2
3
4
5
6
7
0 10 20 30 40 50 60 70 80 90 100
Frequency
-1
0
1
2
3
4
5
6
7
0 10 20 30 40 50 60 70 80 90 100
Frequency
TR
IBE
S
-1
0
1
2
3
4
5
6
7
0 2000 4000 6000 8000 10000
log 1
0(O
F)
Function evaluations
-1
0
1
2
3
4
5
6
7
0 10 20 30 40 50 60 70 80 90 100
Frequency
-1
0
1
2
3
4
5
6
7
0 10 20 30 40 50 60 70 80 90 100
Frequency
-1
0
1
2
3
4
5
6
7
0 2000 4000 6000 8000 10000
log 1
0(O
F)
Function evaluations
-1
0
1
2
3
4
5
6
7
0 10 20 30 40 50 60 70 80 90 100
Frequency
-1
0
1
2
3
4
5
6
7
0 10 20 30 40 50 60 70 80 90 100
Frequency
-1
0
1
2
3
4
5
6
7
0 2000 4000 6000 8000 10000
log 1
0(O
F)
Function evaluations
-1
0
1
2
3
4
5
6
7
0 10 20 30 40 50 60 70 80 90 100
Frequency
-1
0
1
2
3
4
5
6
7
0 10 20 30 40 50 60 70 80 90 100
Frequency
-1
0
1
2
3
4
5
6
7
0 2000 4000 6000 8000 10000
log 1
0(O
F)
Function evaluations
-1
0
1
2
3
4
5
6
7
0 10 20 30 40 50 60 70 80 90 100
Frequency
-1
0
1
2
3
4
5
6
7
0 10 20 30 40 50 60 70 80 90 100
Frequency
SQU
AD
S
-1
0
1
2
3
4
5
6
7
0 2000 4000 6000 8000 10000
log 1
0(O
F)
Function evaluations
-1
0
1
2
3
4
5
6
7
0 10 20 30 40 50 60 70 80 90 100
Frequency
-1
0
1
2
3
4
5
6
7
0 10 20 30 40 50 60 70 80 90 100
Frequency
-1
0
1
2
3
4
5
6
7
0 2000 4000 6000 8000 10000
log 1
0(O
F)
Function evaluations
-1
0
1
2
3
4
5
6
7
0 10 20 30 40 50 60 70 80 90 100
Frequency
-1
0
1
2
3
4
5
6
7
0 10 20 30 40 50 60 70 80 90 100
Frequency
-1
0
1
2
3
4
5
6
7
0 2000 4000 6000 8000 10000
log 1
0(O
F)
Function evaluations
-1
0
1
2
3
4
5
6
7
0 10 20 30 40 50 60 70 80 90 100
Frequency
-1
0
1
2
3
4
5
6
7
0 10 20 30 40 50 60 70 80 90 100
Frequency
-1
0
1
2
3
4
5
6
7
0 2000 4000 6000 8000 10000
log 1
0(O
F)
Function evaluations
-1
0
1
2
3
4
5
6
7
0 10 20 30 40 50 60 70 80 90 100
Frequency
-1
0
1
2
3
4
5
6
7
0 10 20 30 40 50 60 70 80 90 100
Frequency
Conclusions
ABAGUS presents efficient approach for model-baseduncertainty analyses
Squads provides an efficient and robust optimization strategy
