Multifidelity Optimization Via Pattern Search and Space Mapping Genetha Gray Computational Sciences...

Multifidelity Optimization Multifidelity Optimization Via Pattern Search and Via Pattern Search and

Space MappingSpace Mapping

Genetha GrayGenetha Gray

Computational Sciences & Mathematics ResearchComputational Sciences & Mathematics Research

Sandia National Labs, Livermore, CASandia National Labs, Livermore, CA

Sandia is a multiprogram laboratory operated by Sandia Corporation, a Lockheed Martin Company, for the United States Department of Energy’s

National Nuclear Security Administration under contract DE-AC04-94AL85000.

OutlineOutline

Multifidelity OptimizationMultifidelity Optimization APPSPACKAPPSPACK Space MappingSpace Mapping MFO schemeMFO scheme Descriptive ExampleDescriptive Example Groundwater Remediation ExampleGroundwater Remediation Example

Multifidelity Optimization (MFO)Multifidelity Optimization (MFO) The low fidelity model retains The low fidelity model retains

many of the properties of the many of the properties of the high fidelity model but is high fidelity model but is simplified in some waysimplified in some way

Decreased physical resolutionDecreased physical resolution Decreased FE mesh resolutionDecreased FE mesh resolution Simplified physicsSimplified physics

MFO optimizes an inexpensive, MFO optimizes an inexpensive, low fidelity model while making low fidelity model while making periodic corrections using the periodic corrections using the expensive, high fidelity model.expensive, high fidelity model.

Works well when low-fidelity Works well when low-fidelity trends match high-fidelity trends match high-fidelity trends.trends. Low Fidelity

30,000 DOFHigh Fidelity800,000 DOF

Finite ElementModels of the

Same Component

Asynchronous Parallel Pattern Search (APPS)

Direct method Direct method → no derivatives required→ no derivatives required Pattern of search directions drives search and Pattern of search directions drives search and

determines new trial points for evaluation determines new trial points for evaluation Objective function can be an entirely separate Objective function can be an entirely separate

program program Achieves parallelism by assigning function Achieves parallelism by assigning function

evaluations to different processors evaluations to different processors Freely available software under the GNU public Freely available software under the GNU public

license (APPSPACK)license (APPSPACK)

Synchronous Pattern SearchSynchronous Pattern Search

Inherently (or embarrassingly) parallel,but processor load should be considered.

Processor Load Balance Processor Load Balance ConsiderationsConsiderations

The number of trial points can varyThe number of trial points can vary at each at each iteration. iteration. Cached function valuesCached function values Search patterns changeSearch patterns change Constraints (infeasible trial points are not evaluated)Constraints (infeasible trial points are not evaluated)

Evaluation times can vary for each trial point.Evaluation times can vary for each trial point. Different processor characteristicsDifferent processor characteristics Effect of input on function Effect of input on function Function evaluation faultsFunction evaluation faults MFO: different function models with different MFO: different function models with different

evaluation times!!evaluation times!!

APPSPACK ExampleAPPSPACK ExampleWorkers

Waiting

APPSPACK ExampleAPPSPACK ExampleWorkers

Waiting

b

c

d

e

APPSPACK ExampleAPPSPACK ExampleWorkers

Waiting

c

d

APPSPACK ExampleAPPSPACK ExampleWorkers

Waiting

f

c

d

g

APPSPACK ExampleAPPSPACK ExampleWorkers

Waiting

c

d

APPSPACK ExampleAPPSPACK ExampleWorkers

Waiting

h

c

d

i

j,k

APPSPACK ExampleAPPSPACK ExampleWorkers

Waiting

i

APPSPACK ExampleAPPSPACK ExampleWorkers

Waiting

l

m

n

i

o

APPSPACK ExampleAPPSPACK ExampleWorkers

Waiting

i

APPSPACK ExampleAPPSPACK ExampleWorkers

Waiting

p

q

r

i

s

APPSPACK ExampleAPPSPACK ExampleWorkers

Waiting

sNote: Cannot Prune on Unsuccessful Iteration

APPSPACK ExampleAPPSPACK ExampleWorkers

Waiting

s

u

t

v

• Space mapping* is a technique that maps the design space of a low fidelity model to the design space of high fidelity model such that both models result in approximately the same response.

The parameters within xH need not match the parameters within xL

Space Mapping*: A Conduit Between the Space Mapping*: A Conduit Between the Low and High Fidelity Model Design SpacesLow and High Fidelity Model Design Spaces

xx – design – design variablesvariables

R - responseR - responsePP - mapping - mapping

xH

RH(xH)

high-fimodel

xL

RL(xL)

low-fimodel

*Developed by John Bandler, et. al.

xH

RL(P(xH))

mappedlow-fi model

P(xH)

xL=P(xH) RL(P(xH))RH(xH)such that

We’re using the mapping ( )H HP x x

?

OracleOracle An oracle predicts points at which a decrease in An oracle predicts points at which a decrease in

the objective function might be observed.the objective function might be observed. Analytically, an oracle can choose points by any Analytically, an oracle can choose points by any

finite process.finite process. Oracle points are used in addition to the points Oracle points are used in addition to the points

defined by the search pattern. defined by the search pattern. The MFO scheme employs an oracle framework The MFO scheme employs an oracle framework

to do a space mapping so that APPSPACK to do a space mapping so that APPSPACK convergence is not adversely affected. convergence is not adversely affected.

Future work may include investigating any Future work may include investigating any convergence improvement. convergence improvement.

The MFO Scheme: Combining The MFO Scheme: Combining APPSPACK and Space MappingAPPSPACK and Space Mapping

Outer LoopOuter Loop Inner LoopInner Loop

Low Fidelity ModelOptimizationxH

High Fidelity ModeOptimization

viaAPPSPACK

Space MappingVia NonlinearLeast Squares

Calculation

multiple

xH,f(xH

)

xH

trial

MFO AlgorithmMFO Algorithm1. Start the Outer Loop (APPSPACK)

Evaluate N high fidelity response points Produce xH, fH(xH) pairs

2. Start the Inner Loop Take data pairs from APPSPACK Run LS optimization

At each iteration, evaluate N low fidelity responses At conclusion, obtain , , for space map (xH) +

Optimize low fidelity model within space mapped high fidelity space. In other words, minimize fL((xH) + ) with respect to xH to obtain xH*.

3. Return xH* to APPSPACK to determine if a new best point has been found.

2 20 1 0 1( , ) (0.8 0.5) (0.5 0.83)Hf x x x x 2 2

0 1 0 1( , )Lf x x x x

View of High Fidelity Design Space View of Unmapped Low Fidelity Design Space

A Simple ExampleA Simple Example

MFO ResultsMFO Results

1

When the # response points is 8, there are two calls to inner loop.

2 20 1 0 1( , ) (0.8 0.5) (0.5 0.83)Hf x x x x

Approximate Inner Loop Call Locations within Hi-Fi Model

(-0.76,2.0)

(-0.8,-1.2)

1

2

•The numbered white boxes show approximately where the inner loop was called•The point in red brackets is where APPSPACK is before the inner loop call•The point in green was found by the inner loop

2

(-0.56,1.6)

(-0.61,1.25)

Groundwater RemediationGroundwater Remediation

Groundwater Remediation via OptimizationGroundwater Remediation via Optimization

Optimization techniques can aid the design process Optimization techniques can aid the design process to result in lower clean up costs.to result in lower clean up costs.

Use Hydraulic Capture (HC) models to alter the Use Hydraulic Capture (HC) models to alter the groundwater flow direction and control plume groundwater flow direction and control plume migrationmigration

1.1. Transport Based Concentration Control (TBCC)Transport Based Concentration Control (TBCC) Computationally expensiveComputationally expensive Well defined plume boundaryWell defined plume boundary

→ MFO high fidelity model

2.2. Flow Based Hydraulic Control (FBHC)Flow Based Hydraulic Control (FBHC) Orders of magnitude fasterOrders of magnitude faster Constraints require calibrationConstraints require calibration

→ MFO low fidelity model

OptimizationOptimization

Objective FunctionObjective Function J(u) = installation costs + operation costsJ(u) = installation costs + operation costs Evaluation requires results of a simulationEvaluation requires results of a simulation

Design VariablesDesign Variables Number of wellsNumber of wells Well pumping ratesWell pumping rates Well locationsWell locations

ConstraintsConstraints Well capacityWell capacity Net pumping rateNet pumping rate Don’t flood or dry out landDon’t flood or dry out land No useless wellsNo useless wells

ImplementationImplementation Derivatives are unavailableDerivatives are unavailable

SimulatorsSimulators MODFLOW: used for flow MODFLOW: used for flow

equation (USGS)equation (USGS) MT3D: used for transport MT3D: used for transport

equation (EPA)equation (EPA)

)(min uJDu

MFO Numerical TestMFO Numerical Test

Test the MFO method on the HC problem Test the MFO method on the HC problem included in the community problems set. included in the community problems set. (Mayer, Kelley, Miller)(Mayer, Kelley, Miller)

The FBHC formulation has been shown to The FBHC formulation has been shown to be sufficient for this simple domain. be sufficient for this simple domain. (Fowler, Kelley, Kees, Miller)(Fowler, Kelley, Kees, Miller)

Other approaches are needed for Other approaches are needed for heterogeneous more realistic domains. heterogeneous more realistic domains. (Ahlfeld, Page, Pinder)(Ahlfeld, Page, Pinder)

MFO ResultsMFO Results

MethodMethod CostCost % Decrease% Decrease # Fn Evals# Fn EvalsFBHCFBHC $24,176$24,176 69.2%69.2% 117 mf2k117 mf2k

0 mt3d0 mt3d

TBCCTBCC $20,362$20,362 74.1%74.1% 188 mf2k188 mf2k

160 mt3d160 mt3d

MFOMFO $22,428$22,428 71.5%71.5% 152 mf2k152 mf2k

86 mt3d86 mt3d

Initial cost: $78,586

MODFLOW (mf2k): ~2 seconds

mt3d: ~50 seconds

MFO ResultsMFO Results

AcknowledgementsAcknowledgements

MFO development team MFO development team (Sandia)(Sandia) Joe CastroJoe Castro (PI), Electrical & (PI), Electrical &

Microsystem Modeling, NMMicrosystem Modeling, NM Tony GiuntaTony Giunta, Validation & , Validation &

Uncertainty Quantification Uncertainty Quantification Processes, NMProcesses, NM

Patty HoughPatty Hough, CSMR, CA, CSMR, CA

Groundwater Groundwater ApplicationApplication Katie FowlerKatie Fowler, ,

Clarkson UniversityClarkson University

Questions??Questions??Genetha GrayGenetha Gray

gagray@sandia.govgagray@sandia.gov

SoftwareSoftware• APPSPACK: APPSPACK: software.sandia.gov/appspack/version4.0/index.html• DAKOTA: DAKOTA: http://endo.sandia.gov/DAKOTA