Use of Latent Variables in the Parameter Estimation Process

MODEGAT 2009-09-14

Chalmers University of Technology

Use of Latent Variables in the Parameter Estimation Process

Jonas Sjöblom

Energy and Environment


MODEGAT 2009-09-14


NOX Reduction catalysis

BART

Ea

BA Aekr

Introduction

~mm ~µm ~nm

MODEGAT 2009-09-14


Use of Latent Variables (LV)

• What is LV? How does it work?

• How can it be applied in the parameter estimation process?– 3 case studies

• Why is it good?

Outline

MODEGAT 2009-09-14


Latent Variable modelling

• Reduces a data matrix (using projections) to new, few and independent components (Latent Variables).

• Latent Variable (LV) Model: – P: loadings (linear combination of original

variables)– T: scores (projections on the subspace

defined by P)– # components: # linear independent

directions• Different types of Latent Variable (LV)

models:– Principal Components Analysis (PCA)– Partial Least Squares (PLS)

'ˆ TPXX

x1=dY/d1

x2=dY/d2

x3=dY/d3

p1

p2

What is LV modelling?

MODEGAT 2009-09-14


Parameter Estimation Process

How can LV models be applied?

Define model and model assumptions

Define ”experimental space”

Fit parameters

Satisfactory results?Yes!

Evaluate the Design by LV-model (experimental rank)

No!No!

Evaluate the Design (perform experiments)

Choice of experiments to perform

Use of LV modelsUse of LV models

1.

2.

3.

1&2

MODEGAT 2009-09-14


Application 1: LV models during the fitting process

• NOX Storage and Reduction (NSR) Mechanism– 62 parameters

• Poor experimental design

• Jacobian f/ used in gradient search– ill-conditioned– Local minima

Objective: to improve parameter fitting by analysing parameter correlations and make parameters more orthogonal

Ref: Sjoblom et al, Comput. Chem. Eng. 31 (2007) 307-317

MODEGAT 2009-09-14


Parameter assessment

• Jacobian f/– Evaluated for ALL adjustable parameters (not only fitted

ones)• Latent Variable (LV) method:

– Partial Least Squares (PLS) using the Jacobian as "X" and f (Residual: simulated-observed gas phase concentrations) as “Y”

• Outcomes:1. Correlation structure !2. Number of independent directions (# parameters to fit) !3. Which parameters to choose ! (method 1)4. Parameter fit in LV space (method 2)

How can LV models be used? -appl.1

MODEGAT 2009-09-14


LV example: "loading" plotXY

k02_

k03_k04_

k05_

k06_

k07_

k08_

k09_

k10_

k11_

k12_

k13_

k14_

k15_k16_

k17_k18_

k19_

k20_

k21_

k22_

k23_

k24_

k25_

k26_k27_

k28_

k29_

k30_

k31_

k32_k33_

k34_

k35_

Ea02

Ea03 Ea04Ea05Ea06

Ea07

Ea09Ea10

Ea11

Ea14Ea15Ea16Ea17Ea18

Ea19Ea20 Ea22

Ea23

Ea24Ea25

Ea26Ea27Ea28

Ea29

Ea30

Ea31

Ea32

Ea33

Ea34

Ea35

Eth0Eth3

s-NO

sNO2

sNOx

sCO2

w*c

[2]

-0,30

-0,20

-0,10

0,00

0,10

0,20

0,30

-0,20 -0,10 0,00 0,10 0,20 0,30

w*c[1]

k01_


MODEGAT 2009-09-14


Results

Fitting results are comparable, but the fitting is more efficient (faster) due to fewer and more independent parameters, adopted for the data set at hand

Method I (9 selected parameters)

Method II(fitting of 9 scores)

Method “brute force”(all 62 parameters)

90 90 500


MODEGAT 2009-09-14


Application 2: Model-based DoE for precise parameter estimation

• "Simple" but realistic system: – NO-oxidation on Pt– Model from Olsson et.al. (1999)– Using simulated data (noise

added) as experiments

• Objective: – How to find the experiments that

enable precise estimation of the kinetic parameters

*

2*2

*

)(2)(2

)(2)()(

)()(2

)()(

7

8

5

6

3

4

1

2

g

r

rads

ads

r

radsg

ads

r

rg

ads

r

rg

NONO

NOONO

OO

NONO

Ref: Sjoblom et al, Comput . Chem. Eng 32 (2008) 3121-3129

MODEGAT 2009-09-14


Experiment assessment• Jacobian f/

– Evaluated for ALL "possible" experiments (3 iterations)

• Latent Variable (LV) method:– Principal Component Analysis

(PCA) of J (unfolded 3 way matrix)

– D-optimal design to select experiments

• Outcomes:– Correlation structure !– Number of independent

directions (# parameters to fit) !– Which experiments to choose !

How can LV models be applied? -appl.2


Define experimental space

D-optimalChoice of experiments to performusing X or T from LV-model

fit, analyze



No!No!

Evaluate the Design by LV- model


MODEGAT 2009-09-14


Results• Overcomes dimensional reduction of the Fischer

information matrix:

by use of PCA (LV model of unfolded 3-way matrix)• Almost perfect fit was obtained but parameter values

were different (J not full rank)• Using X (as is) or an LV approximation of X performs

equally well– but becomes more efficient since it requires less

experiments

• The LV model gives additional information of the dimensionality of selected experiments before they are performed.

sr

m

r

m

srs JJM '

1 1


Define “experimental space”

D-optimal Choice of experiments to performusing X or T from LV-model

fit, analyze



"novel" use of LV-models

No!No!

Evaluate the Design by LV-model



MODEGAT 2009-09-14


Application 3: Extended Sensitivity Analysis for targeted Model Improvements

• H2 assisted HC-SCR over Ag-Al2O3

– Detailed model (23 reactions, heat balance)

– Acceptable fit, but still significant Lack-of-Fit

• Objectives:– Verify (falsify) model

assumptions– Get indications on how to

improve model fit

Refs: Creaser et al. Appl.Catal.B 90 (2009) 18-28, Sjöblom PhD Thesis (2009) Chalmers

Thesis available at: http://publications.lib.chalmers.se/records/fulltext/92706.pdf

NO2NO3

CH2

NO2O O

O

C8H18

CO2, H

N2

NO, H2

MODEGAT 2009-09-14


Experimental• Sensitivity analysis of 62 model parameters (not

only fitted ones, not only kinetic parameters)– 46 kinetic parameters– 10 mass and heat transport parameters– 6 other parameters

• Scaled local sensitivities

– Unfold 3-way matrix to size n x pk, where n=26025 time points, p=62 parameters and k=5 responses

• Univariate analysis as well as LV modelling


),,()(*3

),,( tconf

t locallevellevel

adj βysy

β

β

y

y

ββyS

MODEGAT 2009-09-14


LV-model and univariate measures


• PCA model– Scores plot

– Loadings plot

– 25 components

• Univariate table data– Confidence intervals

– Sensitivity average, std, max

– Correlations -0,10

-0,05

-0,00

0,05

0,10

-0,10 -0,08 -0,06 -0,04 -0,02 0,00 0,02 0,04 0,06 0,08 0,10

p[2]

p[1]

GasSensitivity-thesis3.M1 (PCA-X), all paramsp[Comp. 1]/p[Comp. 2]Colored according to Var ID (Primary)

R2X[1] = 0,281539 R2X[2] = 0,209535

CO2CO_NO2NO_T__

SIMCA-P 11.5 - 2009-09-08 09:40:37-0,10

-0,05

-0,00

0,05

0,10

-0,10 -0,08 -0,06 -0,04 -0,02 0,00 0,02 0,04 0,06 0,08 0,10

p[2]

p[1]

GasSensitivity-thesis3.M1 (PCA-X), all paramsp[Comp. 1]/p[Comp. 2]Colored according to Var ID (Primary)

R2X[1] = 0,281539 R2X[2] = 0,209535

CO2CO_NO2NO_T__

SIMCA-P 11.5 - 2009-09-08 09:40:37

-20

-10

0

10

20

-50 -40 -30 -20 -10 0 10 20

t[2]

t[1]

GasSensitivity-thesis3.M1 (PCA-X), all paramst[Comp. 1]/t[Comp. 2]Colored according to Obs ID (Primary)

R2X[1] = 0,281539 R2X[2] = 0,209535 Ellipse: Hotelling T2 (0,95)

exp1exp2exp3exp4exp5

SIMCA-P 11.5 - 2009-09-08 09:53:58-20

-10

0

10

20

-50 -40 -30 -20 -10 0 10 20t[2

]

t[1]

GasSensitivity-thesis3.M1 (PCA-X), all paramst[Comp. 1]/t[Comp. 2]Colored according to Obs ID (Primary)

R2X[1] = 0,281539 R2X[2] = 0,209535 Ellipse: Hotelling T2 (0,95)

exp1exp2exp3exp4exp5

SIMCA-P 11.5 - 2009-09-08 09:53:58

MODEGAT 2009-09-14


Sensitivity Analysis results (examples)

• Mass transfer model needs attention– Include diffusivities in fitting?– Include internal mass transport?– Targeted transients?

• Heat transfer model needs attention– Improve/extend temperature measurements?– Consider additional sensors (HC, H2)?– Modify heat transfer model?– Targeted experiments?

(For more details, see poster)


MODEGAT 2009-09-14


• Ability to master different parts of the process– The model (assumptions)– The available data (experiments)– The parameter values (which to fit)

• Ability to “change focus” in the process as the fit develops

Why are LV models good?

Factors for successful parameter estimation

Model assumptions

”experimental space”

Fit parameters

Happy?Yes!No!No!

Evaluate the Design

Choice of experiments

New PhD project: “Improved methods for parameter estimation”Advertisement out now! Application dead line 20th sept 2009http://www.chalmers.se/chem/EN/news/vacancies/positions/phd-student-position-in8778

MODEGAT 2009-09-14


LV Components:Few, New & linearly Independent

• Few: Improved efficiency• Linear: Non-linear systems, LV models provide

more robust linearisations• Independent: Orthogonal sensitivities fulfils

statistical requirements

Why are LV models good?

MODEGAT 2009-09-14


Conclusions

• The LV concept is a viable way in the Parameter estimation process

• Widely applicable – during fitting, DoE, evaluation

• Proven more efficient (due to fewer dimensions)– Superior? Yet to be “proven”...

MODEGAT 2009-09-14


End

AcknowledgementsThe Swedish Research council for financial

supportThe Competence Centre for Catalysis (KCK)

for good collaborationDerek Creaser & Bengt Andersson for fruitful

supervision

Thank you for your attention!

Documents

Use of Latent Variables in the Parameter Estimation Process