Robust Bayesian linear regression for Tokamak plasma boundary … · 2017-06-15 · Robust Bayesian...

Robust Bayesian linear regression for Tokamakplasma boundary estimation

Vít Škvára1,2, Jakub Urban1, Václav Šmídl2,3

1Institute of Plasma Physics, Prague, Czech Republic2Institute of Information Theory and Automation, Prague, Czech Republic

3Regional Innovation Center for Electrical Engineering, Pilsen, Czech Republic

2nd IAEA Technical Meeting on Fusion Data Processing, Validation andAnalysis / May 30, 2017

V. Škvára: Bayesian linear regression for boundary estimation / 2nd IAEA TM FDPVA / May 19, 2017 1/16

Outline

1 Toroidal harmonics equilibrium model

2 Bayesian formulation of the regression problem

3 Results evaluation

4 Conclusion

[1] Faugeras, B. et al. 2D interpolation and extrapolation of discrete magneticmeasurements with toroidal harmonics for equilibrium reconstruction in aTokamak. Plasma Physics and Controlled Fusion. 2014, 56.

Magnetic flux in a tokamak

0.3 0.4 0.5 0.6 0.7 0.8

−0.4

−0.3

−0.2

−0.1

chamber wall

magnetic probe

We seek a form of magnetic flux ψ, that satisfies partial differential equation

−∆∗ψ = 0 in domain D,ψ = ψ0 on ∂D,

where D is a domain without plasma or other current source.

1 There is a known solution, given by decomposition into toroidalharmonics series:

ψ(r , z) =∞∑

unF (n, r , z)

where F (.) are known functions (containing the toroidal harmonics),un ∈ R are unknown coefficients.

2 Knowledge of measurements ψ(r , z) + selection of a finite number ofelements of the decomposition⇒ inverse problem of coefficientsdetermination.

3 With vector u, we reconstruct ψ(r , z)⇒ boundary is identified as lastclosed flux surface.

ψ(r , z) =∞∑

unF (n, r , z)

ψ(r , z) =∞∑

unF (n, r , z)

Problem formulation

The previous transforms into a linear model

Y = Xu + e,

whereY are magnetic flux measurements from coils,X is regressor matrix given by F (n, r , z),u is a vector of unknown coefficients,e is noise.

Optimal choice of the degree of decomposition (length of u) is unknown butthe result is extremely sensitive to it.

Proposed solutionConstruction of a hierarchical probability model that

automatically selects the proper decomposition modelis robust to outliers in measurements.

Problem formulation

The previous transforms into a linear model

Y = Xu + e,

whereY are magnetic flux measurements from coils,X is regressor matrix given by F (n, r , z),u is a vector of unknown coefficients,e is noise.

Optimal choice of the degree of decomposition (length of u) is unknown butthe result is extremely sensitive to it.

Proposed solutionConstruction of a hierarchical probability model that

automatically selects the proper decomposition modelis robust to outliers in measurements.

Least squares approach

The modelY = Xu + e,e ∼ N (e|0, β−1I)

leads to the likelihood function

p(Y |u, β) = N (Y |Xu, β−1I).

An estimate of u is obtained by maximizing the likelihood which results in theclosed-form solution

u = (X T X )−1X T Y .

How to extend this?

p(Y |u, β) = N (Y |Xu, β−1I).

u = (X T X )−1X T Y .

How to extend this?

p(Y |u, β) = N (Y |Xu, β−1I).

u = (X T X )−1X T Y .

How to extend this?

Hierarchical model 1 - LS-ARD

Now begins the construction of a hierarchical probability model. So far wehave

p(Y |u, β) = N (Y |Xu, β−1I).

Now, we add a prior distribution on the vector u:

p(u|α) = N (u|0,diag(α−1)).

Also, we add a gamma prior on both precisions parameters:

p(β) = G(β|a0,b0),

p(αi ) = G(αi |c0,d0).

Now, u, β, α are unknown. How to estimate them?

p(Y |u, β) = N (Y |Xu, β−1I).

p(u|α) = N (u|0,diag(α−1)).

p(β) = G(β|a0,b0),

p(αi ) = G(αi |c0,d0).

p(Y |u, β) = N (Y |Xu, β−1I).

p(u|α) = N (u|0,diag(α−1)).

p(β) = G(β|a0,b0),

p(αi ) = G(αi |c0,d0).

Variational Bayes

We must estimate the posterior distribution p(u, β, α|Y ) = distribution ofparameters given the data Y . For that, we use the variational Bayesframework. We approximate the true posterior with a mixture of marginals

p(u, β, α|Y ) ≈ p(u, β, α|Y ) = p(u|Y )p(β|Y )p(α|Y ).

What does this result into?

We obtain an iterative algorithm that computes hyperparameters (means andvariances) of the posterior distributions.

In effect, the iterative algorithm solves the regularized least squares problem

(Yi − Xiu)2 +∑

αju2j

with adaptive αj .

Variational Bayes

We must estimate the posterior distribution p(u, β, α|Y ) = distribution ofparameters given the data Y . For that, we use the variational Bayesframework. We approximate the true posterior with a mixture of marginals

p(u, β, α|Y ) ≈ p(u, β, α|Y ) = p(u|Y )p(β|Y )p(α|Y ).

What does this result into?

We obtain an iterative algorithm that computes hyperparameters (means andvariances) of the posterior distributions.

In effect, the iterative algorithm solves the regularized least squares problem

(Yi − Xiu)2 +∑

αju2j

with adaptive αj .

Sparsity of the model

Equations of the posterior distribution moments are solved iteratively.By estimating α - the precision (inverse of variance) of u ⇒ less informativeelements are forced to zero.

1 30 60

−0.04

−0.03

−0.02

−0.01

1 30 60

Figure : Convergence of elements of ui and their respective precisions αi .

Hierarchical model 2-RLS-ARD

To achieve robustness, we assume that the noise has the Student’s tdistribution

p(Y |u, ω, z) = N(

Y |Xu, (ω diag(z))−1),

p(ω) = G(ω|ρ0, κ0),

p(zi |d) = G(z|d/2,d/2),

p(d) = G(d |ψ0, φ0),

which follows from∫N (e|0, (ω diag(z))−1)G(z|d/2,d/2)dz = S(e|0, ω,d).

Again, the iterative algorithm solves the problem

√zi (Yi − Xiu)2 +

αju2j

with adaptive zi , αj .

Hierarchical model 2-RLS-ARD

To achieve robustness, we assume that the noise has the Student’s tdistribution

p(Y |u, ω, z) = N(

Y |Xu, (ω diag(z))−1),

p(ω) = G(ω|ρ0, κ0),

p(zi |d) = G(z|d/2,d/2),

p(d) = G(d |ψ0, φ0),

which follows from∫N (e|0, (ω diag(z))−1)G(z|d/2,d/2)dz = S(e|0, ω,d).

Again, the iterative algorithm solves the problem

√zi (Yi − Xiu)2 +

αju2j

with adaptive zi , αj .

Outlier detection

Estimate of d represents normality of the measurements, whilemeasurements with zi < 1 are regarded as outliers.

1.00 1.05 1.10 1.15 1.20 1.25

Figure : Time evolution of d and zi during a discharge.

Validation

COMPASS shot number 9339.Ground truth data simulated with the EFIT++ equilibrium reconstructioncode.The second model was updated to distinguish between flux loops andmagnetic probes.

−0.4

−0.3

−0.2

−0.1

t = 1.02 s t = 1.13 s

limiter

EFIT++

OLS Student2 measurement point

t = 1.21 s

0.3 0.4 0.5 0.6 0.7 0.8

−0.4

−0.3

−0.2

−0.1

real m

0.3 0.4 0.5 0.6 0.7 0.8

z [m]0.3 0.4 0.5 0.6 0.7 0.8

Figure : Boundary shapes computed with EFIT++ (ground truth), original model (OLS)and Student’s t noise models. Enlarged circles depict "outlying" measurements.

1.00 1.05 1.10 1.15 1.20 1.250.00

simulated data

1.00 1.05 1.10 1.15 1.20 1.250.00

0.35measured data

Student1

Student2

1.00 1.05 1.10 1.15 1.20 1.25

Figure : Fit MSE (upper row) and Hausdorff distance DH for different estimationmethods during a discharge.

EFIT++

It solves the Grad-Shafranov equation

jp(ψ, r) = rp′ +1µ0r

(ff ′)(ψ)

for unknown functions p′, ff ′.These are approximated with a decomposition into polynomial, spline orChebyshev basis functions.With a proper model, we may extend the base and achieve robustness.

Conclusion

We have presented a Bayesian approach to linear regression.Universal algorithm with self-tuning regularization and robustnessparameters.By modeling prior knowledge, we improved the results of boundaryidentification.Outlook - implementation of the method into EFIT++ code.

Robust Bayesian linear regression for Tokamak plasma boundary … · 2017-06-15 · Robust Bayesian...

Documents

Bayesian Heteroskedasticity-Robust Regression …econ.ucsb.edu/~startz/Bayesian Heteroskedasticity-Robust...Bayesian Heteroskedasticity-Robust Regression Richard Startz* revised February

Multivariate Bayesian Logistic Regression for …of clinical safety data called multivariate Bayesian logistic regression (MBLR). Parallel logistic regression models are ﬁt to a

Bayesian Linear Regression - unibas.chinformatik.unibas.ch/fileadmin/Lectures/HS2016/pattern-recognition/... · Figs: Bishop PRML, 2006 ... • Bayesian linear regression is a nice

Bayesian Semiparametric Regression for Median Residual Life

Bayesian Learning, Regression-based learning. Overview Bayesian Learning Full MAP learning Maximum Likelihood Learning Learning Bayesian Networks

Bayesian Multivariate Isotonic Regression Splines: Applications to

Bayesian Regression Analysis

Differentially Private Bayesian Linear Regression Garrett

Bayesian Regularisation in Geoadditive Expectile Regression...Bayesian Regularisation in Geoadditive Expectile Regression Elisabeth Waldmann, Fabian Sobotka, Thomas Kneib April 20,

Bayesian Methods in Multilevel Regression

Penalized Regression, Standard Errors, and Bayesian Lassos

Bayesian Linear Regression Modelling for Sperm Quality

Bayesian Quantile Mixture Regressionthanos/BNP2019_talk.pdfPrologos Quantile Mixture Regression Data Application Epilogos Bayesian Quantile Mixture Regression Athanasios Kottas Department

Lecture 4: Types of errors. Bayesian regression models ... · Lecture 4: Types of errors. Bayesian regression models. Logistic regression A Bayesian interpretation of regularization

Bayesian regression

Bayesian Linear Regression - Vision Labsvision.psych.umn.edu/users/schrater/schrater_lab/courses/PattRecog... · Bayesian Linear Regression • Given target values, ... – Style

Bayesian Quantile Regression for Ordinal Models

Ridge regression and Bayesian linear regression Kenneth D. Harris 6/5/15

Bayesian Regression: Basis Functions, MLE & Regularized Least … · 2019-02-06 · Bayesian Regression: Basis Functions, MLE & Regularized Least Squares, Multiple Outputs, Inference

Gaussian Process and Prediction. (C) 2001 SNU CSE Artificial Intelligence Lab (SCAI)2 Outline Gaussian Process and Bayesian Regression Bayesian regression