Upload
adonai
View
26
Download
0
Tags:
Embed Size (px)
DESCRIPTION
Higher-order Confidence Intervals for Stochastic Programming using Bootstrapping. Cosmin G. Petra Joint work with Mihai Anitescu Mathematics and Computer Science Division Argonne National Laboratory [email protected] INFORMS ANNUAL MEETING 2012. Outline. Confidence intervals Motivation - PowerPoint PPT Presentation
Citation preview
Higher-order Confidence Intervals for Stochastic Programming using Bootstrapping
Cosmin G. Petra
Joint work with Mihai Anitescu
Mathematics and Computer Science DivisionArgonne National Laboratory
INFORMS ANNUAL MEETING 2012
2
Outline
Confidence intervals
Motivation– statistical inference for the stochastic optimization of power grid
Our statistical estimator for the optimal value
Bootstrapping
Second-order bootstrapped confidence intervals
Numerical example
3
Confidence intervals (CIs) for a statistic
* ,( [ ])LP U
** Want an interval [L,U] where resides with high probability
Need the knowledge of the probability distribution Example: Confidence intervals for the mean of Gaussian (normal) random
variable
Normal distribution, also called Gaussian or "bell curve“ distribution. Image source: Wikipedia.
4
Approximating CIs
* [ ( )]f X E 1
1ˆ ( )N
ii
f X f XN
In many cases the distribution function is not known.
Such intervals are approximated based on the central limit theorem (CLT)
Normal approximation for equal-tailed 95% CI
Notation
*
1/2
ˆ(0,1), as (by CLT)
ˆN
N
N
1/2 1/2/2 /2
ˆ ˆˆ ˆ, ][ ,N NU N z NL z
1( ) is the normal quantilez
x) is t( ) ( ( he 0,1 normal cdf)x P N
5
Optimal value in stochastic programming
Monotonically shrinking negative bias: Consistency
Arbitrary slow convergence
Non-normal bias
* min ( ) : ( , )f x F x Ex X 1
1min ( ) , ) : (N
N N ii
f x F xN
x X
Sample average approximation (SAA)Stochastic programming (SP) problem
Properties*
1NN E E*
N
* 1/2( )N O N E1/2 *( ) inf ( )N x S
Y xN
D
Stochastic unit commitment with wind power
Wind Forecast – WRF(Weather Research and Forecasting) Model– Real-time grid-nested 24h simulation – 30 samples require 1h on 500 CPUs (Jazz@Argonne)
6
1min COST
s.t. , ,
, ,
ramping constr., min. up/down constr.
wind
wind
p u dsjk jk jk
s j ks
sjk kj
windsjk
j
wik ksj
ndsk
jjk
j
c c cN
p D s k
p D R s k
p
p
S N T
N
N
N
N
S T
S T
Slide courtesy of V. Zavala & E. Constantinescu
Wind farmThermal generator
7
The specific of stochastic optimization of energy systems
SAA
discrete continuous
Sampling
Statistical inference
uncertainty
is expensive
Only a small number of samples are available.
8
Standard methodology for stochastic programming – Linderoth, Shapiro, Wright (2004)
Lower bound CI CI for based on M batches of N samples
Upper bound CI CI for (obtained similarly)
Needs a relatively large number of samples (2MN)
First-order correct and therefore unreliable for small number of samples
1
1 ( , )N
ii
F xN
1/2 1/2/
1/
12 2
1 1,M M
m
m m m mN N N N
m
M z MM M
z
N
Correctness of a CI – order k if* /2[ , ])( )( k
N NL U O NP
9
Our approach for SP with low-size samples
1. Novel estimator
2. Bootstrapping
Converges one order faster than– Excepting for a set whose measure converges exponentially to 0.
N
Allows the construction of reliable CIs in the low-size samples situation. Bootstrap CIs are second-order correct
M. Anitescu, C. Petra: “Higher-Order Confidence Intervals for Stochastic Programming using Bootstrapping”, submitted to Math. Prog.
10
The estimator
12* ( , ) ( , ) ) ( , )1( )
2 0(
( 0) 0N
TN N N N N T N N
N
L x L x L xF x
J xJ x
E E EE[
L is the Lagrangian of SP and J is the Jacobian of the constraints is the solution of the SAA problem – obtained using N samples Intended for nonlinear recourse terms
Theorem 1: (Anitescu & P.) Under some regularity and smoothness conditions
Proof: based on the theory of large deviations.
CIs constructed for are based on a second batch of N samples.
A total of 2N sample needed when using bootstrapping
Nx
2 ( )3/2 * *1 3| ) ( ) ( (( )| )ra cr N xP p r NN e òò ò ò
*
11
Bootstrapping – a textbook example
1. 1930 population = 1920 population X mean of the ratios
2. needs the distribution of the ratios - not enough samples -> Bootstrapping– Sample the existing samples (with replacement)– For each sample compute the mean– Bootstrapping distribution is obtained– Build CIs based on the bootstrapping distribution
Histogram for the ratio of 1930 and 1920 populationsfor N=49 US cities
“Bootstrapped” distribution clearly not a GaussianBootstrap CIs outperform normal CIs.
US population known in 1920. 1930 population of 49 cities knownWant 1. estimation of the 1930 population2. CIs for the estimation
Solution
12
The methodology of bootstrapping
BCa (bias corrected and accelerated) confidence intervals– second-order correct– the method of choice when an accurate estimate of the variance is not available
13
What does bootstrapping do?
Edgeworth expansions for cdfs
Bootstrapping accounts also for the second term in the expansion
The quantiles are also second-order correct (Cornish-Fisher inverse expansions)
(Some) Bootstrapped CIs are second-order correct
1/2 ( 1)/21( ) ( ) (( ) ( ) ( ) ( )) k
kx N p x x p x x O NH x
1/2 1 1/21 1 1
ˆ ˆ ˆ( ) ( ) ( ) ( ), with( )) (pH x N p x x O N p Ox p N
Reference: Peter Hall, “The Bootstrap and Edgeworth Expansion”, 1994.
14
Bootstrapping the estimator
* 1,
ˆ ) ( )( .aBCaP J O N
Theorem 2: (Anitescu & P.) Let be a second order bootstrapping confidence interval for . Then for any
,ˆBCaJ * 0,a
15
Numerical order of correctness
Correctness order 0.32
Correctness order 0.82
Correctness order 1.14
Correctness order 2.11
Observed order of correctness*
*
4 4
1
1min ( ) : ( ) min ( ) : ( )
~ (0,
1
)
NN
u ii
f x E x u f x x uN
u U
16
Coverage for small number of samples
*
1 2
1 2
2 21 2 1 2 1 2 , 1 2 1 2
2 21 2 1 2 , 1 2 1 2
1 1 1 2
2 2 1 2
5 5min ( , ) : 7.4 2.4 ( , ; , )2 2
1( , ; , ) min ( ) 2 22
s.t. 20 (2 6 )10 (3 3 )
u u
y y
f x x x x x x E Q x x u u
Q x x u u y y y y
y u x xy u x x
1
2
~ ( 10,10)~ ( 5, 5)
u Uu U
Concluding remarks and future work
Proposed and analyzed a novel statistical estimator for the optimal solution of nonlinear stochastic optimization
Almost second order correct confidence intervals using bootstrapping
Theoretical properties confirmed by numerical testing
Some assumptions are rather strict and can/should be relaxed
Parallelization of the CI computations for large problems needed
17
Thank you for your attention!
Questions?
18
19
Bootstrapping - theory
Edgeworth expansions for pdfs1/2 ( 1)/2
1( ) ( ) (( ) ( ) ( ) ( )) kkx N p x x p x x O NH x
1/2 ( 1)/211 1( ) ( ) ( )k
kz N p z p z O Ny
1/21 1ˆ ( )k kp p O N
Bootstrapping also accounts for the second term of in the expansion.
Cornish-Fisher expansion for quantiles (inverting Edgeworth expansion)
Bootstrapped quantiles possess similar expansion
But
(Some) Bootstrap CIs are second-order correct (Hall’s book is really detailed on this)
1/2 ( 1)/211 1ˆ ˆ ˆ( ) ( ) ( ),k
k py z N p z p z O N