Renewable Energy Research Laboratory University of Massachusetts Prediction Uncertainties in Measure- Correlate-Predict Analyses Anthony L. Rogers, Ph.D

Renewable Energy Research Laboratory

University of Massachusetts

Prediction Uncertainties in Measure-Correlate-Predict Analyses

Anthony L. Rogers, Ph.D.

March 1, 2006



Measure-Correlate-Predict (MCP)• Provides estimate of mean wind speed and

wind speed and direction distributions – Uses a short-term data set and a long-term

reference site data set

• How can we estimate prediction uncertainties?– Review of measured uncertainties– Evaluation of jackknife estimate of variance– Discussion of issues



Measure-Correlate-Predict (MCP)• Apply relationship between concurrent target and

reference site data to long-term reference site data. 25

20

15

10

5

0Win

d S

pe

ed

, m/s

40x103 3020100

Time

Reference Site Data

25

20

15

10

5

0Win

d S

pe

ed

, m/s

40x103 3020100

Time

Target Site Data

Mean of X = 6.5X =

Y =

Y = aX+b Predicted Mean of Y = 5.2



Measure-Correlate-Predict (MCP)• Relationship may be a function of wind speed,

direction, time, temperature, …• (SpeedT, DirT)=f(SpeedR, DirR , Time, TempR)

• “Variance” Method used here– Slope = ratio of standard

deviations of x and y data– Line goes through the mean

of x and y

– Provides unbiased estimates

• Correlations done in 8 direction sectors

xyx

yx

x

yy

ˆ



Determining Prediction Uncertainties• Assemble multiple pairs of long-term

concurrent data sets– e.g. US176-US127

97,357 hourly averages

• Determine MCP estimates for multiple independent concurrent subsets– e.g. 21 MCP estimates for 4000 hr segments– Estimate long-term mean, Weibull parameters

• Evaluate how estimates vary

25

20

15

10

5

0

Win

d S

pe

ed

, m/s

80x103 6040200

Data Point Index

US176 US127



Data Sets Used for Analysis• Six inland pairs

– Oregon, Iowa, Indiana

• Six offshore pairs– N. Atlantic, Hawaii

• 4 to 16 years of data

Site LocationDistance

kmYears of good data

Inland

1 Kennewick - Goodnoe* Oregon 112 11.52

2 Red Oak/Cedar* Iowa 219 4.53

3 Estherville/Forest City* Iowa 100 4.23

4 Inwood/Sibley* Iowa 66 4.33

5 Radcliffe/Sutherland* Iowa 186 3.15

6 US176x1 /US127x07* Indiana 9 10.00

Offshore

7 44005/44007* New England 87 10.23

8 Buoy 44013/44008* New England 231 13.49

9 Buoy BUZM3/IOSN3* New England 178 14.33

10 Buoy MDRM1/MISM1* New England 62 16.71

11 Buoy 51001/51003* Hawaii 497 13.49

12 Buoy 51002/51004* Hawaii 566 13.75

* Reference site



Measured Mean Wind SpeedUncertainties

• Normalized standard deviation of mean:– Uncertainty decreases as

concurrent data length increases

– Beyond ~8000 hrs little improvement

– Value depends on site

• Normalized standard deviation of Weibull shape factor:

– Value very site dependant

0.12

0.08

0.04

0.00

8000600040002000

Inland

0.20

0.15

0.10

0.05

0.00

8000600040002000

Offshore

0.20

0.15

0.10

0.05

0.00

8000600040002000

Inland0.20

0.15

0.10

0.05

0.00

8000600040002000

Offshore



Estimating Uncertainty

• In practice– Only one set of concurrent data

– Characteristics of concurrent data may not represent long-term behavior

– Confidence interval may not fall out of the analysis

• Are there methods to determine the confidence one can have in the MCP results?– Linear regression statistics

– Jackknife estimate of variance

– Estimates from correlation coefficients



Estimating Uncertainty from Linear Regression

• Linear regression estimate ≠ measured!– Linear regression assumes data are not serially correlated

– But wind data ARE serially correlated

• Linear regression estimate = measured value when data are randomly jumbled, removing serial correlation

0.8

0.6

0.4

0.2

0.0S

tand

rd D

evia

tion,

m/s

8000600040002000Length of Concurrent Data, hrs

Standard Deviation of MCP Estimatesof Long Term Wind Speed

Measured - original data Measured - randomized data Linear regression - original data Linear regression - randomized data



Jackknife Estimate of Variance• Applicable to any MCP algorithm • Typically works when other methods not available

1) Find long-term predicted value, , using all of concurrent data

2) Find n long term predicted values, , using concurrent data sets that each have a different 1/nth of the data file missing

3) Number of subsets, n, fixed at value that minimizes RMS error over all data sets

4) The estimated uncertainty is:

• Jackknife subsets need to be independent

n

iiy yy

n

n 22ˆ ˆˆ

1

y

iy



Jackknife Results – Mean Wind SpeedInland Offshore

Blue = measured, Red = Estimated

0.25

0.20

0.15

0.10

0.05

0.00

Std

. D

ev.,

m/s

8000600040002000Number of hours in concurrent data

RS

0.30

0.20

0.10

0.00

Std

. D

ev.,

m/s


EF

0.15

0.10

0.05

0.00

Std

. D

ev.,

m/s


IS

0.5

0.4

0.3

0.2

0.1

0.0

Std

. D

ev.,

m/s


ROC

1.6

1.2

0.8

0.4

0.0

Std

. D

ev.,

m/s


KG

0.5

0.4

0.3

0.2

0.1

0.0

Std

. D

ev.,

m/s


US

0.5

0.4

0.3

0.2

0.1

0.0

Std

. D

ev.,

m/s


570.25

0.20

0.15

0.10

0.05

0.00

Std

. D

ev.,

m/s


MDR

0.6

0.4

0.2

0.0

Std

. D

ev.,

m/s


B440.4

0.3

0.2

0.1

0.0

Std

. D

ev.,

m/s


BUZ

0.30

0.20

0.10

0.00

Std

. D

ev.,

m/s


240.6

0.4

0.2

0.0

Std

. D

ev.,

m/s


13



Jackknife Results – Mean Wind Speed• Ratio of measured to estimated standard deviation

• Jackknife estimate of uncertainty of mean typically somewhat underestimates correct value

5

4

3

2

1

0

Rat

io,

-

8000600040002000Numbers of hours in concurrent data

Offshore

5

4

3

2

1

0

Rat

io,

-


Inland



Jackknife Results – Weibull Shape FactorInland Offshore

Blue = measured, Red = Estimated

0.16

0.12

0.08

0.04

0.00

Std

. D

ev.


570.16

0.12

0.08

0.04

0.00

Std

. D

ev.


B44

0.15

0.10

0.05

0.00

Std

. D

ev.


BUZ

0.6

0.4

0.2

0.0

Std

. D

ev.


13

0.4

0.3

0.2

0.1

0.0

Std

. D

ev.


24

0.12

0.08

0.04

0.00

Std

. D

ev.


MDR

0.10

0.08

0.06

0.04

0.02

0.00

Std

. D

ev.


EF

0.12

0.08

0.04

0.00

Std

. D

ev.


IS

0.15

0.10

0.05

0.00S

td.

Dev

.


RS

0.15

0.10

0.05

0.00

Std

. D

ev.


ROC

0.12

0.08

0.04

0.00

Std

. D

ev.


KG

0.20

0.15

0.10

0.05

0.00

Std

. D

ev.


US



Jackknife Results – Weibull Shape Factor

• Ratio of measured to estimated standard deviation:

• Jackknife estimate of uncertainty of Weibull shape factor provides reasonable estimates

4

3

2

1

0

Rat

io,

-


Inland

4

3

2

1

0

Rat

io,

-


Offshore



Limitations of EstimatingUncertainty from Short Data Sets

• Uncertainty within concurrent data set may not be same as uncertainty at longer time intervals

7

6

5

4

3

2

1

Mea

n W

ind

Spe

ed,

m/s

12108642Time, years

MCP Predictions and Jackknife UncertantiesUS176 - US127

1000 data points 9000 data points

Uncertainty within 1000 pt segments << variability of 1000 pt MCP predictions

Uncertainty within 9000 pt segments ~ variability of 9000 pt MCP predictions

Better estimates at one year0.6

0.5

0.4

0.3

0.2

0.1

0.0

Std

. D

ev.,

m/s


Measured and Estimated UncertaintyUS176x14 - US127x07 Data

Measured Estimated



Possible Jackknife Modifications• Inclusion of seasonal model

– e.g Monthly correlations• If no correlation for month,

use overall correlation

– Little improvement in ratios

• Empirical correction factors– e.g Scale estimate of standard deviation

of mean wind speed by 1.6– Ratios show great improvement– Does empirical factor apply to all sites?

2.5

2.0

1.5

1.0

0.5

0.0

Ra

tio

8000600040002000Concurrent Data Length, hr

Inland

Correlations using: Concurrent data length Months, where possible

2.5

2.0

1.5

1.0

0.5

0.0

Ra

tio


Offshore

Correlations using: Concurrent data length Months, where possible

4

3

2

1

0R

atio

, -


Offshore

4

3

2

1

0

Rat

io,

-


Inland

4

3

2

1

0

Rat

io


Inland

4

3

2

1

0

Rat

io


Offshore



Alternative Approaches• Correlation coefficients

– Uncertainty weakly correlated with correlation coefficients

– No improvement over jackknife at these sites

40x10-3

30

20

10

0

Nor

mal

ized

Std

. D

ev.

of M

ean

0.90.80.70.60.5Correlation coefficient between data sets

1000 pts fit_1000 pts 9000 pts fit_9000 pts



Conclusions• Jackknife should correctly estimate uncertainty based

on concurrent data– Much better than using linear regression results– Better than using fit to correlation coefficients

• Empirical correction may be used to account for variability at time scales greater than concurrent data length

• Variability at time scales greater than concurrent data length still a problem

• Jackknife estimate can be used with any MCP algorithm

Documents

Renewable Energy Research Laboratory University of Massachusetts Prediction Uncertainties in Measure- Correlate-Predict Analyses Anthony L. Rogers, Ph.D