Upload
hang
View
73
Download
0
Tags:
Embed Size (px)
DESCRIPTION
Bootstrap Confidence Intervals for Reservoir Model Selection Techniques. Céline Scheidt and Jef Caers. SCRF Affiliate Meeting– April 30, 2009. Model Selection Techniques. Uncertainty in reservoir modeling is represented through a possibly large set of reservoir models - PowerPoint PPT Presentation
Citation preview
Bootstrap Confidence Intervals for Reservoir
Model Selection Techniques
Céline Scheidt and Jef Caers
SCRF Affiliate Meeting– April 30, 2009
Uncertainty in reservoir modeling is represented through a possibly large set of reservoir models◦ Generated by varying several input parameters
High CPU demand for flow simulations requires the use of model selection techniques ◦ Evaluate uncertainty on a subset of models
Model selection techniques select a subset of representative realizations which should preserve the statistics of the entire set of realizations◦ Eg.: Ranking, Distance-Kernel Method (DKM)
Model Selection Techniques
2SCRF Affiliate Meeting – 04/30/09
If we select N realizations, perform flow simulation, and quantify uncertainty:◦ How do we know if the results are accurate?◦ Can we be confident with the results?◦ Should we do more simulations?
We use of bootstrap methodology to evaluate the accuracy of the uncertainty quantification ◦ Applicable to standard ranking or new distance-kernel
method (DKM)
Goal
3SCRF Affiliate Meeting – 04/30/09
Distance Kernel Method (DKM)
Distance Matrix D1 2 3 4
1 d11 d12 d13 d14
2 d21 d22 d23 d24
3 d31 d32 d33 d34
4 d41 d42 d43 d44
Model 1 Model 2
Model 3 Model 4
d12
d13 d24
d34
d32
d14
-0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8-0.6
-0.4
-0.2
0
0.2
0.4
0.6
0.8
F
2D projection of Feature Space
-5 -4 -3 -2 -1 0 1 2 3 4
x 104
-3000
-2500
-2000
-1500
-1000
-500
0
500
1000
1500
2000
All realizationsSelected realizations
2D projection of Metric Space
Apply Clustering in F
P10,P50,P90 model selection
-5 -4 -3 -2 -1 0 1 2 3 4
x 104
-3000
-2500
-2000
-1500
-1000
-500
0
500
1000
1500
2000
M
2D projection of Metric Space
MDS
Kernels j
j-1
Pre-image
4SCRF Affiliate Meeting – 04/30/09
SCRF, 2008SPE Journal, 2009
Generate a proxy response for each L realizations (ranking measure)◦ Should be strongly correlated to the actual response
Select N realizations for flow simulations◦ Traditionally, N=3◦ Realizations equally spaced according to the ranking measure
Estimation of the distribution of the response using the N simulations◦ Compute P10, P50 and P90 statistics
Traditional Ranking Techinque
5SCRF Affiliate Meeting – 04/30/09
Review: Parametric Bootstrap – Simple Example
)ˆ,ˆ(ˆ NF
],...,[ **1
* bL
bb xxX
)ˆ̂,ˆ̂(ˆ̂ *** bbb
],...,[ 1 LxxX
2 3 4 5 6 7 80
50
100
150
200
250
300
Bootstrap Mean
Freq
uenc
y
0 5 10 15 20 250
50
100
150
200
250
300
Bootstrap Variance
Freq
uenc
y
SCRF Affiliate Meeting – 04/30/09
),( NF
)ˆ,ˆ(ˆ B bootstrap estimates of the mean and variance
6
?
1st estimate
2nd estimate
b = 1,..,B
b*ˆ̂b*ˆ̂
: Proxy response (ranking measure) Eg. Streamline simulations
: True response Eg. Eclipse simulations
: Selected realizations by model selection
: estimate of P10, P50 and P90 values From ranking or DKM & flow simulation (1st estimate)
: bootstrap estimate of P10, P50 and P90 values
From ranking or DKM & parametric distribution (2nd estimate)No additional flow simulations
Notations
7
Lyy ,,1
Nxx ,,1
**1 ,, Nxx
***905010
ˆ,ˆ,ˆ PPP xxx
bP
bP
bP xxx ***
905010
ˆ̂,ˆ̂,ˆ̂
SCRF Affiliate Meeting – 04/30/09
Proposed Bootstrap Methodology
Model selection
+ flow simulation
**1 ,, Nxx
**1 ,, Nyy
bN
b xx **1 ,,
bN
b yy **1 ,,
Proxy Values
Lyy ,,1
)()(1 ,, b
Lb yy
)()(1 ,, b
Lb xx b
PbP
bP xxx ***
905010
ˆ̂,ˆ̂,ˆ̂
***905010
ˆ,ˆ,ˆ PPP xxx
Application to model selection technique
8
Model selection+ response evaluation
Parametric BootstrapEstimation of distribution
Generation of B samples from NF̂
b = 1,…,B
NF̂
SCRF Affiliate Meeting – 04/30/094 4.5 5 5.5 6 6.5
0
50
100
150
200
250
Bootstrap P50Fr
eque
ncy
Distribution of the target and proxy responses:
Proposed bootstrap technique applied for several correlation scenarios between target and proxy responses◦ Scenarios for:
rxy = 1, 0.9, 0.8, 0.7, 0.6,0.5
Illustration: Bivariate Gaussian Distribution
),(~),( biNYX
r = 0.9
9
yyxy
xyxx
rr
SCRF Affiliate Meeting – 04/30/09
L = 100, r = 0.9 Selection of 15 realizations using DKM Number of bootstrap samples: B = 1000
Histograms of the Estimated Quantiles
1 2 3 4 50
50
100
150
200
250
300
Bootstrap P10
Freq
uenc
y
4 4.5 5 5.5 6 6.50
50
100
150
200
250
Bootstrap P50
Freq
uenc
y
5 6 7 8 9 100
50
100
150
200
250
300
Bootstrap P90
Freq
uenc
y
10
Bootstrap estimated P90Bootstrap estimated P50Bootstrap estimated P10
SCRF Affiliate Meeting – 04/30/09
Estimated P10 Estimated P50 Estimated P90
For each of the B samples, a dimensionless error is defined to evaluate the accuracy of the estimated quantiles:
Definition of the Error on the Bootstrap Estimated Quantiles
11
-
-
-
90
9090
50
5050
10
1010
ˆ
ˆˆ̂
ˆ
ˆˆ̂
ˆ
ˆˆ̂
31
***
*
P
PbP
P
PbP
P
PbPb
x
xx
x
xx
x
xxerror
Error on bootstrap estimated quantiles:
5 10 15 200
0.05
0.1
0.15
0.2
0.25
0.3
r = 1.0
# of function evaluation
Err
or o
n qu
antil
e es
timat
ion
Bivariate Gaussian distribution Confidence Intervals for different correlations scenarios
5 10 15 200
0.05
0.1
0.15
0.2
0.25
0.3
r = 0.9
# of function evaluation
Err
or o
n qu
antil
e es
timat
ion
5 10 15 200
0.05
0.1
0.15
0.2
0.25
0.3
r = 0.7
# of function evaluation
Err
or o
n qu
antil
e es
timat
ion
02
46
810
0 2 4 6 8 10
# of function evaluation
legend
Quantiles estim
ation
KK
MR
ankingR
andom
0 2 4 6 8 10
0
2
4
6
8
10
# of function evaluation
lege
nd
Quantiles estimation
KKMRankingRandom
12
5 10 15 200
0.05
0.1
0.15
0.2
0.25
0.3
r = 0.8
# of function evaluation
Err
or o
n qu
antil
e es
timat
ion
5 10 15 200
0.05
0.1
0.15
0.2
0.25
0.3
r = 0.6
# of function evaluation
Err
or o
n qu
antil
e es
timat
ion
5 10 15 200
0.05
0.1
0.15
0.2
0.25
0.3
r = 0.5
# of function evaluationE
rror
on
quan
tile
estim
atio
n
r = 1.0 r = 0.9 r = 0.8
r = 0.5r = 0.6r = 0.7
SCRF Affiliate Meeting – 04/30/09
WCA is a deepwater turbidite offshore reservoir located in a slope valley
Dimensions of the reservoir model◦ 78 x 59 x 116 gridblocks◦ 100,000 active gridblocks
28 wells◦ 20 production wells (red)◦ 8 injection wells (blue)
West Coast African Reservoir (WCA) Courtesy of Chevron
1 mile0.5 m
ile
800 feet13
SCRF Affiliate Meeting – 04/30/09
4 depositional facies◦ Facies 1: Shale (55% of the reservoir)◦ Facies 2: Poor quality sand #1 (debris flows or levees)◦ Facies 3: Poor quality sand #2 (debris flows or levees)◦ Facies 4: Good quality channels (28 %)
West Coast African Reservoir
Porosity for each facies determined by SGS conditioned to well data
Vshale for each facies modeled by SGS correlated to porosity
Permeability calculated analytically from Vshale
14SCRF Affiliate Meeting – 04/30/09
Uncertainty exists for:◦ Depositional environment
Modeled using 12 training images (TI) & snesim◦ Facies proportions
Modeled with 3 different probability cubes Probability cubes come from seismic
2 realizations were generated for each combination of TI and facies probability cube◦ 72 possible realizations of the WCA reservoir
Uncertainty in Reservoir Description
15SCRF Affiliate Meeting – 04/30/09
True response X:◦ Cumulative oil production after 1200 days of production
(evaluated by full flow simulation)
Proxy response Y:◦ Cumulative oil production after 1215 days of production
(evaluated by fast streamline simulation)
Correlation coefficient: r(X,Y) = 0.92
Definition of the Responses
16SCRF Affiliate Meeting – 04/30/09
Parametric bootstrap requires an assumption of the bivariate distribution function ( )◦ Not known a priori in real case (contrary to previous
example)
Use of a smoothing technique to obtain the distribution of the N selected bivariate samples
Definition of Distribution Function
NF̂
NF̂
17SCRF Affiliate Meeting – 04/30/09 True Response (Eclipse)
Pro
xy R
espo
nse
(Stre
amlin
es)
5.5 6 6.5 7 7.5 8x 10
4
4.5
5
5.5
6
x 104
True and proxy responses on the N Selected points
( Niyx ii ,,1,, **
NF̂
True Response (Chears)
Pro
xy R
espo
nse
(Stre
amlin
es)
5.5 6 6.5 7 7.5 8x 10
4
4.5
5
5.5
6
x 104
True Response (Eclipse)
Pro
xy R
espo
nse
(Stre
amlin
es)
5.5 6 6.5 7 7.5 8x 10
4
4.5
5
5.5
6
x 104
Sampling to generate new bivariate bootstrap datasets
NF̂
Lyy ,,1
Proxy measure (Streamline)
Flow simulations (Chears)on N selected realizations
)()(1 ,, b
Lb yy
)()(1 ,, b
Lb xx b
Nb xx **
1 ,,bN
b yy **1 ,,
1st Model Selection
to select N real.
**1 ,, Nxx
**1 ,, Nyy
2nd Model Selection
to select N real.bP
bP
bP xxx ***
905010
ˆ̂,ˆ̂,ˆ̂
Generation of Bootstrap Samples
SCRF Affiliate Meeting – 04/30/09
**1 ,, Nxx
Bivariate response
True Response (Eclipse)
Pro
xy R
espo
nse
(Stre
amlin
es)
5.5 6 6.5 7 7.5 8x 10
4
4.5
5
5.5
6
x 104
Smoothing on N selected realizations
NF̂
B times
True Response (Chears)
Pro
xy R
espo
nse
(Stre
amlin
es)
5.5 6 6.5 7 7.5 8x 10
4
4.5
5
5.5
6
x 104
True Response (Chears)
Pro
xy R
espo
nse
(Stre
amlin
es)
5.5 6 6.5 7 7.5 8x 10
4
4.5
5
5.5
6
x 104
Distance (for DKM only)◦ Difference in proxy response for every pair of
realizations
Comparison between 3 model selection methods:◦ DKM, ranking and random selection
Selection of N realizations: N = 3,5,8,10,15,20◦ The set of selected realizations are different for each N
Number of new bootstrap data sets generated: B = 1000
Application of Bootstrap to WCA
19SCRF Affiliate Meeting – 04/30/09
Bootstrap Estimated P10, P50 and P90 Quantiles
5 10 15 206.6
6.8
7
7.2
7.4
7.6
7.8
8
8.2x 10
4 Quantiles estimation
# of function evaluation
P90
02
46
810
0 2 4 6 8 10
# of function evaluation
legend
Quantiles estim
ation
KK
MR
ankingR
andom0 2 4 6 8 10
0
2
4
6
8
10
# of function evaluation
lege
nd
Quantiles estimation
KKMRankingRandom
5 10 15 206
6.2
6.4
6.6
6.8
7
7.2
7.4
7.6x 10
4 Quantiles estimation
# of function evaluation
P50
5 10 15 205.5
6
6.5
7
7.5x 10
4 Quantiles estimation
# of function evaluation
P10
20SCRF Affiliate Meeting – 04/30/09
SCRF Affiliate Meeting – 04/30/09
Error on Bootstrap Quantiles Estimations
0 0.02 0.04 0.06 0.08 0.1 0.120
50
100
150
200
250
300
Response Value
Freq
uenc
yN = 5
DKMRankingRandom
0 0.02 0.04 0.06 0.08 0.1 0.120
50
100
150
200
250
300
Response Value
Freq
uenc
y
N = 10
DKMRankingRandom
0 0.02 0.04 0.06 0.08 0.1 0.120
50
100
150
200
250
300
Response Value
Freq
uenc
y
N = 20
DKMRankingRandom
0 0.02 0.04 0.06 0.08 0.1 0.120
50
100
150
200
250
300
Response Value
Freq
uenc
y
N = 15
DKMRankingRandom
21
5 simulations 10 simulations
15 simulations 20 simulations
Error on Bootstrap Quantiles Estimations
5 10 15 200
0.02
0.04
0.06
0.08
0.1Quantiles estimation
# of function evaluation
Err
or o
n qu
antil
e es
timat
ion
02
46
810
0 2 4 6 8 10
# of function evaluation
legend
Quantiles estim
ation
KK
MR
ankingR
andom
0 2 4 6 8 10
0
2
4
6
8
10
# of function evaluation
lege
nd
Quantiles estimation
KKMRankingRandom
22
N = 8 or 10 simulations should be sufficient to obtain an accurate uncertainty quantification Previous work (SCRF 2008) showed that with 7 simulations, uncertainty
quantification on cumulative oil production was very accurate
SCRF Affiliate Meeting – 04/30/09
Comparison of the Results to the “Truth”
23
N = 3 N = 83 simulations 8 simulations
Conclusion We have established a workflow to construct
confidence intervals for quantile estimations
Workflow uses any model selection technique and parametric bootstrap procedure
DKM provides more robust results and outperforms ranking
The magnitude of the confidence intervals can show if more simulations are required for a better uncertainty quantification◦ Does not suggest how many more, only if sufficiently accurate
24SCRF Affiliate Meeting – 04/30/09