Learningprocessmodelsfrom( simulaons

This report was prepared as an account of work sponsored by an agency of the United States Government. Neither the United States Government nor any agency thereof, nor any of their employees, makes any warranty, express or implied, or assumes any legal liability or responsibility for the accuracy, completeness, or usefulness of any information, apparatus, product, or process disclosed, or represents that its use would not infringe privately owned rights. Reference herein to any specific commercial product, process, or service by trade name, trademark, manufacturer, or otherwise does not necessarily constitute or imply its endorsement, recommendation, or favoring by the United States Government or any agency thereof. The views and opinions of authors expressed herein do not necessarily state or reflect those of the United States Government or any agency thereof.

Learning process models from simula2ons

This report was prepared as an account of work sponsored by an agency of the United States Government. Neither the United States Government nor any agency thereof, nor any of their employees, makes any warranty, express or implied, or assumes any legal liability or responsibility for the accuracy, completeness, or usefulness of any information, apparatus, product, or process disclosed, or represents that its use would not infringe privately owned rights. Reference herein to any specific commercial product, process, or service by trade name, trademark, manufacturer, or otherwise does not necessarily constitute or imply its endorsement, recommendation, or favoring by the United States Government or any agency thereof. The views and opinions of authors expressed herein do not necessarily state or reflect those of the United States Government or any agency thereof.

Alison Cozad1, Nick Sahinidis12, and David Miller2

1 Department of Chemical Engineering, Carnegie Mellon University 2 Na2onal Energy and Technology Laboratory

Carnegie Mellon University

PROBLEM STATEMENT

2

! Challenges: ‒  Lack of an algebraic model ‒  Computa2onally costly simula2ons ‒  OKen noisy func2on evalua2ons ‒  Scarcity of fully robust simula2ons

Process simulation


SIMULATION-‐BASED METHODS

3

! Es2mated gradient based ‒  Finite element, perturba2on analysis,

etc. ! Deriva2ve-‐free op2miza2on (DFO)

‒  Local/global ‒  Stochas2c/determinis2c

! What is modeled? ‒  Objec2ve, objec2ve + constraints,

disaggregated system ! Type of model

‒  Linear/nonlinear ‒  Simple/Complex ‒  Algebraic/black-‐box

! Op2mizer ‒  Deriva2ve/deriva2ve-‐free

Simulator Optimizer Simulator Modeler Optimizer

Indirect methods

Direct methods


RECENT WORK IN CHEMICAL ENG

4

Simulator Modeler Optimizer

Indirect methods

Full process

Disaggregated

Kriging Neural nets Other

§  Michalopoulos, et. Al., 2001

§  Palmer and Realff, 2002

§  Huang, et. al., 2006 §  Davis and

Ierapetriton, 2012

§  Caballero and Grossmann, 2008

§  Palmer and Realff, 2002

§  Henao and Maravelias, 2011


PROCESS DISAGGREGATION

Surrogate models of blocks

Disaggregated blocks of process unit(s)

Simula2on

f1(x) f3(x) f2(x)

Algebraic constraints Mass balances Design specs

Nonlinear program Algebraic model for optimization


! Build a model of output variables z as a func2on of input variables x over a specified interval

MODELING PROBLEM STATEMENT

Independent variables: Operating conditions, inlet flow

properties, unit geometry

Dependent variables: Efficiency, outlet flow conditions,

conversions, heat flow, etc.

6

Process simulation


! Model ques2ons: ‒  What is the func2onal form of the model? ‒  How complex of a model is needed? ‒  Will this be tractable in an algebraic op2miza2on framework?

! Sampling ques2ons: ‒  How many sample points are needed to define an accurate model? ‒  Where should these points be sampled?

! Desired model traits: ü  Accurate ü  Tractable in algebraic op2miza2on: Simple func2onal forms ü  Generated from a minimal data set

MODELING PROBLEM STATEMENT

7


ALGORITHMIC FLOWSHEET

true

Stop

Update training data set

Start

false

Initial sampling

Build surrogate model

Adaptive sampling

Model converged?

8



true

Stop


Start

false

Initial sampling


Adaptive sampling

Model converged?

9


! Goal: To generate an ini2al set of input variables to evenly sample the problem space

! La2n hypercube design of experiments -‐ Space-‐filling design

DESIGN OF EXPERIMENTS

10


! AKer running the design of experiments, we will evaluate the black-‐box func2on to determine each zi

INITIAL SAMPLING

Ini2al training set

11

Process simulation



true

Stop


Start

false

Initial sampling


Adaptive sampling

Model converged?

12


! Goal: Iden2fy the func2onal form and complexity of the surrogate models

! Func2onal form: ‒  General func2onal form is unknown: Our method will iden2fy

models with combina2ons of simple basis func2ons

MODEL IDENTIFICATION

13


OVERFITTING AND TRUE ERROR

14

! Empirical error: ‒  Error between the model and the sampled data points

! True error: ‒  Error between the model and the true func2on

Complexity

Erro

r

Ideal Model

Overfitting Underfitting


SURROGATE MODEL

15

! Surrogate model can have the form

! Low-‐complexity desired surrogate form

! is chosen to ‒  Reduce overfidng ‒  Achieve surrogate simplicity for a tractable final op2miza2on model


BEST SUBSET METHOD

16

! Generalized best subset problem:

! Goodness of fit: ‒  Corrected Akaike Informa2on Criterion (AICc)

•  Gives an es2mate of the difference between a model and the true func2on

Accuracy + Complexity


FINAL BEST SUBSET MODEL

17

! This model is solved for increasing values of T un2l the AICc worsens



true

Stop


Start

false

Initial sampling


Adaptive sampling

Model converged?

18


! Goal: Search the problem space for areas of model inconsistency or model mismatch

! More succinctly, we are trying to find points that maximizes the model error with respect to the independent variables

‒  Op2mized using a black-‐box or deriva2ve-‐free solver (SNOBFIT) [Huyer and Neumaier, 08]

ADAPTIVE SAMPLING

Surrogate model

19


ADAPTIVE SAMPLING ! Goal: Search the problem space for areas of model

inconsistency or model mismatch


Black-box function Data points Surrogate model





Surrogate model

Black-box function Data points True

minimum

Current surrogate optimum





Black-box function Data points Surrogate model

New sample point after interrogating

the surrogate

Model error





Black-box function Data points New surrogate model


ERROR MAXIMIZATION SAMPLING

24

! Informa2on gained using error maximiza2on sampling: 1.   New data point loca2ons that will be used to befer train the next

itera2on’s surrogate model

2.   Conserva2ve es2mate of the true model error •  Defines a stopping criterion •  Es2mates the final model error


COMPUTATIONAL TESTING

25

! Surrogate genera2on methods have been implemented into a package:

ALAMO (Automated Learning of Algebraic Models for Op2miza2on)

! Modeling methods compared ‒  MIP – Proposed methodology ‒  EBS – Exhaus2ve best subset method

•  Note: due to high CPU 2mes this was only tested on smaller problems ‒  LASSO – The lasso regulariza2on ‒  OLR – Ordinary least-‐squares regression

! Sampling methods compared ‒  DFO – Proposed error maximiza2on technique ‒  SLH – Single la2n hypercube (no feedback)


DESCRIPTION – TEST SET A

26

! Two and three input black-‐box func2ons randomly chosen basis func2ons available to the algorithms with varying complexity from 2 to 10 terms

! Basis func2ons allowed:


RESULTS – TEST SET A

27

Model accuracy Function evaluations

45 test problems, repeated 5 2mes, tested against 1000 independent data points


MODEL COMPLEXITY – TEST SET A

28


DESCRIPTION – TEST SET B

29

! Two input black-‐box func2ons with basis func2ons unavailable to the algorithms with


RESULTS – TEST SET B

30

Model accuracy Function evaluations

12 test problems, repeated 5 2mes, tested against 1000 independent data points


TEST CASE: CUMENE PRODUCTION

Generate Models:

Over the Range:

Benzene Propylene

Cumene

31

Cumene producFon simulaFon is form the Aspen Plus® Library


! Maximum error found at each itera2on may increase ‒  Due to the deriva2ve-‐free solver is given more informa2on at each itera2on

GENERATING THE SURROGATES

0%

2%

4%

6%

8%

10%

12%

14%

1 2 3 4 5

Max

imum

real

tive e

rror

Iterations

xCTreactFrec

yes

true false

Initial data set: 3 points Final data set: 23 points


PROCESS OPTIMIZATION

Benzene Propylene

Cumene

33


CARBON CAPTURE OPTIMIZATION

34

! Problem statement: Capture 90% of CO2 from a 350MW power plant’s post combus2on flue gas with minimal increase in the cost of electricity

! Design considera2ons: ‒  Capture technology

•  Bubbling fluidized bed, moving bed, fast fluidized bed, transport bed, etc.

‒  Number of reactors ‒  Reactor configura2on and geometry ‒  Opera2ng condi2ons

350 MW Coal fired power

plant

CO2 rich flue gas

CO2 poor flue gas

Ads

orbe

r

Reg

ener

ator


SUPERSTRUCTURE OPTIMIZATION

35

Flue gas from power plant

a1

a2

a3

a4

d1

d2

d3

d4

Solid sorbent stream

Cleaned gas

Other capture trains

Cooling water Steam Work

Solid sorbent CO2 Capture

Carnegie Mellon University 36

! Model outputs (13 total) ! Geometry required (2) ! Opera2ng condi2on required (1) ! Gas mole frac2ons (2) ! Solid composi2ons (2) ! Flow rates (2) ! Outlet temperatures (3) ! Design constraint (1)

BUBBLING FLUIDIZED BED

! Model inputs (14 total) ‒  Geometry (3) ‒  Opera2ng condi2ons (4) ‒  Gas mole frac2ons (2) ‒  Solid composi2ons (2) ‒  Flow rates (4)

Bubbling fluidized bed adsorber diagram Outlet gas Solid feed

CO2 rich gas CO2 rich solid outlet

Cooling water


ADAPTIVE SAMPLING

yes

true false

0%

4%

8%

12%

16%

1 3 5 7 9

Est

imat

ed re

lativ

e err

or

IterationsFGas_out Gas_In_P THX_out Tgas_out Tsorb_outdt gamma_out lp vtr xH2O_ads_outxHCO3_ads_out zCO2_gas_out zH2O_gas_out

Progression of mean error through the algorithm

Initial data set: 137 pts

Final data set: 261


EXAMPLE MODELS Solid feed

CO2 rich gas

Cooling water

38



39


a1

a2

a3

a4

d1

d2

d3

d4


Cleaned gas






40


a1

a2

a3

a4

d1

d2

d3

d4


Cleaned gas




a1

a2

a3

a4

Add the set of surrogate models

generated for each adsorber



a1


PRELIMINARY RESULTS

41

a2

d1


Cleaned gas


Regen. gas



! The algorithm we developed is able to model black-‐box func2ons for use in op2miza2on such that the models are ü  Accurate ü  Tractable in an op2miza2on framework (low-‐complexity models) ü  Generated from a minimal number of func2on evalua2ons

! Surrogate models can then be incorporated within a op2miza2on framework flexible objec2ve func2ons and addi2onal constraints

CONCLUSIONS

Automated Learning of Algebraic Models for Optimization ALAMO

42


MODEL REDUCTION TECHNIQUES

43

! Qualita2ve tradeoffs of model reduc2on methods

Backward elimina2on [Oosterhof, 63] Forward selec2on [Hamaker, 62]

Stepwise regression [Efroymson, 60]

Regularized regression techniques •  Penalize the least squares objec3ve using the magnitude of the regressors

Best subset methods •  Enumerate all possible subsets


BEST SUBSET METHOD

44

! Surrogate subset model:

! Mixed-‐integer surrogate subset model:

! Generalized best subset problem mixed-‐integer formula2on:


! Further reformula2on ‒  Replace bilinear terms with big-‐M constraints

‒  Decouple objec2ve into two problems

‒  Inner minimiza2on objec2ve reformula2on

MIXED-‐INTEGER PROBLEM

45

b) basis and coefficient selection

a) model sizing


PROBLEM SIMPLIFICATIONS

46

‒  Inner problem •  Sta2onarity condi2on used to solve for con2nuous variables

•  Linear objec2ve used to solved for integer variables

-‐5

-‐4

-‐3

-‐2

-‐1

00 2 4 6

AICc

Number of basis functions in modelT

Solution

! Simplifica2ons: ‒  Outer problem

•  The outer problem is parameterized by T and a local minima is found

Documents

Learningprocessmodelsfrom( simulaons