Retrospective Evaluation of the Performance of Experimental Long-Lead Columbia River Streamflow Forecasts

0

100000

200000

300000

400000

500000

600000

700000

J J A S O N D J F M A M J J A S

Climate Forecast and Estimated Initial Soil

Moisture

ForecastEnsemble

Lead time = 8 months

ENSO

PDO Run Initialized Hydrologic Model

Ensemble StreamflowForecast

Select Temperature and Precipitation Data from Historic Record Associated with

Forecast Climate Category

ClimateForecast

Schematic for Streamflow Forecasting Methodology Using VIC Hydrologic Model and Resampled Observed Data

Elevation (m)

The Dalles

Forecasts

150000

200000

250000

300000

350000

400000

450000

1900

1910

1920

1930

1940

1950

1960

1970

1980

1990

2000

Ap

r-S

ept F

low

(cfs

)

Effects of the Pacific Decadal Oscillation (warm and cool epochs) and ENSO (red El Niño, blue La Niña, green ENSO Neutral) on Columbia River Summer Streamflows at The Dalles, OR.

Cool CoolWarm Warm

Interpreting Ensemble Streamflow ForecastsThe figures above show a time line for the long range streamflow forecasts, and a climatology based on simulated streamflows from 1950-2000. To produce a forecast, a climate forecast and initial soil conditions are estimated on October 1 of each water year. Then meteorological sequences are resampled from the historic record from1950-2000 according to the climate forecast, and the initialized hydrologic model is used to simulate the natural streamflow for each of these resampled driving sequences. The forecasts are informed both by initial conditions and the climate forecasts. The primary objective of the forecast is to predict April-September average runoff, an important quantity in the Columbia’s management scheme. The lead time from the forecast to peak flows in mid summer is about eight months in this case study. Longer lead times of up to 12 months are also possible with very similar results.

Overview of Streamflow Forecasting MethodsLong-lead forecasts of the Pacific Decadal Oscillation (PDO) and ENSO, global climate phenomena which exert strong controls on Pacific Northwest winter climate, are used to produce streamflow forecasts with 8-month lead times using techniques shown in the schematic below. A long retrospective streamflow forecast was made for water years1951-2000 using the VIC hydrologic model. A perfect categorical forecast of winter average ENSO state (warm, neutral, cool) is assumed, and heuristic methods of predicting relatively uncertain decadal changes in the PDO are used. Simulated soil moisture storage on October 1 is quantified using the hydrologic model driven by observed meteorological data. The climate forecasts contain misclassifications of the PDO for some years , based on these methods ( Hamlet and Lettenmaier, 1999 and 2002). 1998, 1999, 2000 are classified as cool PDO based on the high flow event in 1997, although this classification is uncertain.

Schematic and Timeline for Streamflow Forecasts

Dept. of Civil Engineering, UW, Box 352700, Seattle, WA 98195hamleaf@u.washington.edu http://www.ce.washington.edu/~hamleaf/hamlet/alan_f_hamlet.html

Long-Range Streamflow Forecasting

JISAO Climate Impacts Group and the Department of Civil and Environmental Engineering, University of Washington

Key to Forecast GraphicsYear shown on each plot is water year (Oct-September). Y axis is monthly mean streamflow in cms.

Gray lines are the ensemble forecast members, light black lines are the highest and lowest values from the climatology, the red lines are simulated natural streamflows (from VIC), blue lines are estimated natural flows from observations.

Colored circles in the upper left corner of each plot show the winter ENSO state for each forecast period. Red = warm ENSO, Green = ENSO neutral, Blue = cool ENSO.

Forecast Performance

Years in which the PDO state is misidentified are 1974, 1977, 1996(?) and 1997(?).

Using April-September average streamflow as the forecast variable, the following performance is achieved:

Years Where Obs are Below Lowest Ensemble Member

Years Where Obs are Above Highest Ensemble Member

1955 19821973 19961977 1997197919851994

Reliability 88% 94%

Overall reliability, defined as the probability of the observation being contained within the forecast envelope, is 82%

Skill ScoreQuantitative performance of the ensemble forecasts relative to the 50-year simulated climatology is assessed using a Brier-based skill score defined as follows:

Skill = 1 - [ 1/N1 * (obs-forecast)2 / 1/N2 * (obs-climatology)2 ]

where the summations in the second term are over all forecast/observation pairs in each case. This performance metric rewards accuracy and punishes higher variability in the forecast ensemble. Using this metric, a perfect forecast would have a skill of 1.0, whereas forecast performance equivalent to using climatological ensemble would have a skill of 0.

Skill of the forecast scheme assessed over all years is 0.078, an improvement of 8% over climatology.

Conclusions

For this fifty year retrospective assessment, the forecasts are shown to be 82% reliable, and improve on ensemble forecasts based on climatology by about 8% using a Brier-based skill metric.

The forecasting technique is least effective at capturing unusual droughts that are infrequent or unobserved in the data set used for resampling. In the period from 1950 to 2000, for example, there were no droughts with the same severity as 1973 or 1977 in the same composite PDO/ENSO category, and as a result the observation is outside of the forecast envelope in these years. Increasing the length of the record of observations would improve this aspect of the forecasts, but problems with limited sample size are an inherent weakness in forecasting methods of this type.

Str

eam

flow

(cm

s)

Alan F. Hamlet Dennis P. Lettenmaier

VIC Simulated Climatology