Do Higher Wind Power Penetration Levels P o s e a C h a l l e n g e t o E l e c t r i c P o w e r R e l i a b i l i t y ? : Evidence from the ERCOT Power

Grid in Texas

Kevin F. Forbes The Center for the Study of Energy and Environmental Stewardship

Department of Business and Economics The Catholic University of America

Washington, DC Forbes@CUA.edu

Marco Stampini

African Development Bank

Ernest M. Zampelli The Center for the Study of Energy and Environmental Stewardship

Department of Business and Economics The Catholic University of America

Center for Research in Regulated Industries 29th Annual Eastern Conference

Skytop, Pennsylvania, May 19-21, 2010

Abstract

Avoiding the high societal costs of blackouts requires that the amount of power generation in a balancing authority area match exactly, on a near-instantaneous basis, the system load, net of losses and interchange with other balancing authority areas. This paper presents econometric evidence that wind power increases this challenge in the ERCOT power grid in Texas. Specifically, the paper presents evidence that changes in wind energy production in ERCOT from one 15 minute market interval to the next are significantly offset by deployments of regulation power by the system operator. Evidence is also presented that the wind forecast errors in ERCOT are biased, very large, and have consequences for ERCOT’s deployments of balancing power to relieve market shortages. Additionally, the forecast errors are found to have implications for the deployments of reserve power and are shown to be statistically related to the hour of the day and the forecasted level of wind energy. Finally, the paper offers evidence that the prices in ERCOT’s day-ahead market for ancillary services reflect the anticipated adverse impact of the forecast errors on power grid operations.

1. Introduction There exists broad political support for increasing substantially the share of electricity generation from wind energy. However, the output of wind generation plants is inherently variable with upward dispatch by the system operator not being possible. Some also contend that wind energy production is difficult to forecast accurately. It therefore seems prudent to consider seriously the potential consequences of increased wind energy penetration on the reliability of the power grid since avoidance of the high societal costs of blackouts requires that the amount of power generation in a balancing authority area match exactly, on a near-instantaneous basis, the system load, net of losses and interchange with other balancing authority areas. This paper focuses on the ERCOT power grid in Texas where wind power accounted for approximately six percent of generation in 2009. An example of the challenge that wind energy poses for power grid reliability is ERCOT’s implementation of Step Two of its Emergency Electric Curtailment Plan on 26 February 2008 in response to a drop in system frequency. The paper assesses the effect of higher wind energy penetration levels on system reliability using econometric analyses of the impacts of wind power’s variability and relative unpredictability on the deployment of ancillary services, balancing power, and reserve power. Two sample periods are considered. The first sample period spans the time period from 1 October 2008 through 30 November 2009. The results indicate that changes in wind energy production from one fifteen minute market period to the next are significantly offset by changes in the dispatch of regulation power. The second sample is the period 13 June through 30 November 2009. For this sample period we make use of ERCOT’s wind forecast data to assess the impacts of the wind forecast errors on the deployment of balancing and reserve power.

2. ERCOT The Electric Reliability Council of Texas, Inc. (ERCOT) is responsible for ensuring the reliability of approximately 85 percent of the state's electric load and 75 percent of the geographic land area in Texas (Figure 1). ERCOT manages the flow of electric power to approximately 21 million Texas customers and oversees the operation of approximately 73,000 megawatts of generation and 40,000 miles of transmission lines (ERCOT, 2008). The vast proportion of electricity (90-95 %) in ERCOT is traded via bilateral contracts. Prices in these agreements are considered confidential and thus are not known by ERCOT. However, the quantity of electricity agreed to is reported to ERCOT through a day-ahead scheduling process. While ERCOT does not concern itself with the contract terms of these base electricity supplies, it is charged with managing transmission congestion as well as ensuring that the overall market is balanced in terms of supply and demand. ERCOT is an almost ideal power grid in which to study the impact of wind penetration on power grid operations. Analysis of the effects of wind penetration in say Denmark or Germany, both of which have high wind power penetration rates, yields an incomplete conclusion given that there is evidence that a portion of the effects are dispersed to other electricity control areas in the form of inadvertent electricity flows (Forbes, Stampini, and Zampelli, 2009). This is highly unlikely in the case of ERCOT because it is only interconnected with other control areas through five high-voltage direct-current (DC) interconnections that have aggregate transmission limits of about 1,000 MW. Moreover, these DC tie lines are far less susceptible than AC interconnections to inadvertent electricity flows. As a result, market disturbances in ERCOT are largely confined to ERCOT. The primary ancillary services in the ERCOT are:

• Regulation Services – are deployed in virtually every market period to help maintain the instantaneous balance between load and generation resources. Upward regulation is deployed to help resolve shortages while downward regulation is deployed to resolve excess generation. • Responsive Reserve Service – are generation resources held in reserve to address the unexpected loss of generation resources and/or unexpected large changes in load. These resources are deployed less than two percent of the time. • Non-Spinning Reserve Service (NSRS) – are resources that can come on line with short notice. These resources are deployed less than two percent of the time.

The capacity prices of ancillary services are determined in a day-ahead market that clears hourly. This market reports capacity prices for the following categories of ancillary services: regulation service up, regulation service down, responsive reserve service, and non-spinning reserve service.

ERCOT also operates a balancing market that allows it to acquire additional resources to correct imbalances between generation and load. Upward balancing energy services are generating resources that are deployed to help resolve shortages while downward balancing energy services are deployed to resolve excess generation. The prices in this market are determined in near-real-time. Figure 2 reports on the fuel mix in ERCOT over the first eleven months of 2009. Figure 3 depicts the pattern of generation and load in ERCOT over the period 1 -7 September 2009. Figures 4 and 5 depicts the deployment of ancillary services and balancing power over the period 1 -7 September 2009. Figures 6 and 7 are histograms of the regulation and balancing deployments. Observe in Figure 7 that downward deployments of balancing power are more likely than upward deployments.

3. Wind Energy Variability and Predictability in ERCOT

Figure 8 is a histogram of the quarter-hour changes in wind energy production in ERCOT over the period 1 October 2008 through 30 November 2009. Almost all observations are within the interval of +/- 500 MW. Over the sample period, the standard deviation of the changes in the quarter hour load was about 4.5 times the standard deviation in the changes in quarter-hour wind production. One possible interpretation of this lower relative variability is that changes in wind energy production have relatively minor impacts on power grid operations. In the next section, this hypothesis is subjected to rigorous testing.

Before analyzing ERCOT’s wind forecasting errors, it is useful to report on the wind forecast errors by other power grids. Unfortunately, some researchers have not reported forecast errors in the most meaningful way. For example, Cali, et. al. (2006) presented evidence that the root-mean-squared-errors (RMSE) of the day-ahead wind energy forecasts at an unidentified German TSO declined from approximately 10 percent of installed wind capacity in 2001 to approximately six percent of installed wind energy capacity in 2006. This finding has been cited by the EWEA (2007) as evidence that wind power is a reliable source of electricity supply. We are unimpressed by this statistic. In our opinion, an unweighted statistic or one weighted by the actual (or forecasted) level of wind energy production would be more relevant. Fortunately, the recent improvement in the transparency of the German power grid allows the calculation of forecasting errors for wind and load for the three largest transmission systems in Germany (Table 1). The principal finding is that the errors in wind forecasting in Germany, a country with vast experience regarding issues of wind integration, are very large relative to the errors in forecasting load. Specifically, in Amprion (formerly RWE), the RMSE for wind forecasting over the period 1 January – 15 December 2009 was 37.9 percent, more than three times the corresponding value for load forecasting.

Table 1

Wind and Load Forecast Errors Summary Statistics for the Amprion, Transpower, and Vattenfall Power Grids in Germany, 1 January – 15 December 2009

Wind Load Root Mean

Squared Percent Error

Mean Absolute Percent Error

Percent Root Mean Squared Error

Mean Absolute Percent Error

Amprion (formerly known as RWE)

37.9 26.0 9.0 3.4

TransPower (formerly known as E.ON Netz)

35.9 24.8 NA NA

Vattenfall 36.3 25.3 13.5 10.2

In ERCOT, a short term wind power forecast (STWPF) is generated each hour and represents the forecast of the most likely hourly values of wind power production in MW for the next 48 hours. The STWPF is derived with the intent that there is a 50 percent probability that actual wind generation in each hour will exceed the STWPF. An 80% “probability of exceedence” hourly forecast is also calculated for each of the following 48 hours and is known as the Wind Generation Resource Power Potential (WGRPP). These forecasts are posted each hour on ERCOT’s website (http://www.ercot.com/gridinfo/generation/). The forecasts posted at 5:14 AM on 27 December 2009 are depicted in Figure 9.

Table 2 reports on the percent of observations that exceeded WGRPP and STWPF by forecast hour over the period 13 June through 30 November. The table indicates that the forecasts are systematically inaccurate. For example, only 56 percent of the wind production levels exceeded the day-ahead hour 12 WGRPP while 31.4 percent of the wind production levels exceeded the day-ahead hour 12 STWPF. Observe that these values do not improve as the forecast period gets closer to real-time.

Table 2 The Percent of Observations that Exceed WGRPP and STWPF

by Forecast Hour, 13 June – 30 November 2009

Percent of the Observations in which Wind Energy Production Exceeded the Forecasted Level of WGRPP

Percent of the Observations in which Wind Energy Production Exceeded the Forecasted Level of STWPF

Day-Ahead Hour 9 55.5 30.7Day-Ahead Hour 10 56.2 31.6Day-Ahead Hour 11 56.3 31.8Day-Ahead Hour 12 56.0 31.4Day-Ahead Hour 13 56.0 30.7Day-Ahead Hour 14 56.9 31.0Day-Ahead Hour 15 58.4 33.4Day-Ahead Hour 16 59.0 32.9Day-Ahead Hour 17 58.8 32.8Day-Ahead Hour 18 57.7 32.1Day-Ahead Hour 19 56.0 31.0Day-Ahead Hour 20 54.8 30.2Day-Ahead Hour 21 47.8 24.1Day-Ahead Hour 22 47.6 23.8Day-Ahead Hour 23 47.2 23.3Five Hours Ahead 52.8 27.8Four Hours Ahead 52.4 29.2Three Hours Ahead 51.9 27.1Two Hours Ahead 52.5 27.3One Hour Ahead 51.9 26.0 Table 3 reports summary statistics of the wind forecast errors by forecast hour for the period 13 June through 30 November 2009. The errors are sizeable. For example, the RMSE of the day-ahead hour 12 forecast is 65.6 percent while the MAPE is 49.5 percent. The errors do not decline as the forecast period approaches real-time. For example, the RMSE and MAPE for the hour ahead forecast are 69.1 and 51.2 percent, respectively. Figures 10 through 12 illustrate the differences between actual and forecasted wind energy production for various segments of the sample. Figure 13 shows a histogram of the hour-ahead forecast errors for the entire sample. Figure 14 depicts the root mean squared forecast error for each day relative to the day-ahead forecasted wind production (both are based on the hour 12 forecast). This figure suggests that the forecast errors are highly clustered by day.    

Table 3 Wind Forecast Errors Summary Statistics

for ERCOT by Forecast Hour, 13 June 30 November 2009

Root Mean Forecast Error Relative as a Percent of the Mean level of Wind Energy

Mean Absolute Percentage Error

Mean Error (MW) (actual minus forecasted wind)

Day-Ahead Hour 9 65.5 49.7 -582Day-Ahead Hour 10 65.1 49.3 -563Day-Ahead Hour 11 65.0 49.2 -558Day-Ahead Hour 12 65.6 49.5 -572Day-Ahead Hour 13 65.5 49.3 -584Day-Ahead Hour 14 65.2 49.0 -572Day-Ahead Hour 15 67.8 50.3 -564Day-Ahead Hour 16 65.9 48.9 -552Day-Ahead Hour 17 65.9 49.0 -556Day-Ahead Hour 18 66.4 49.5 -580Day-Ahead Hour 19 67.2 49.7 -636Day-Ahead Hour 20 67.4 50.0 -659Day-Ahead Hour 21 70.3 53.6 -803Day-Ahead Hour 22 69.8 53.2 -801Day-Ahead Hour 23 70.1 53.4 -812Five Hours Ahead 68.3 51.4 -697Four Hours Ahead 71.5 54.4 -713Three Hours Ahead 69.9 52.4 -745Two Hours Ahead 68.7 51.3 -732One Hour –Ahead 69.1 51.2 -761

4. An Empirical Analysis of the Variability of Wind Energy Production and the Deployment of Regulation Services

Both wind energy and load are variable over the course of the day. Some researchers contend that the long experience of system operators in managing the variability in load gives them the requisite expertise to manage the variability in wind energy production. As stated by Milligan et. al. (2009):

“Although wind is a variable resource, grid operators have experience with managing variability that comes from handling the variability of load.”

It has also been suggested that it is the net variability of load and wind that is important and thus the analysis should focus on so-called “net load”, i.e., load minus wind energy production. In our view, focusing on “net load” makes sense only if the marginal impact of a change in wind energy production on power grid operations equals the marginal impact of a change in load.

To test the equivalence of the marginal impacts, we obtained load and wind data by 15 minute intervals from ERCOT for the period 1 October 2008 through 30 November 2009. We also obtained data on regulation deployments at five minute intervals. The regulation deployment data were aggregated to the 15 minute level and the following regression equation was estimated:

∆ ∑ ∆ ∆ (1)

where:

∆ is the change in net deployed regulation services from the previous 15 minute market period where net deployed regulation is defined at upward regulation minus downward regulation, This variable is measured in MW.

∆ is the change in wind energy production from the previous 15 minute market period. This variable is measured in MW.

∆ is the change in load from the previous 15 minute market period. This variable is measured in MW.

The results are reported in Table 4. The coefficients on ∆WIND and ∆LOAD are highly statistically significant. If the power system is indifferent between load and wind variability, then coefficients on ∆WIND and ∆LOAD should have opposite signs but be equal in absolute value. The reported coefficients indicate that this is not the case. Specifically, the results indicate that a one MW increase in wind power leads to a decrease in net regulation by about 0.52 MW while a one MW increase in load increases net regulation by about 0.02 MW.

Table 4

Estimation Results for Equation 1

Variable Estimated Coefficient

t- Statistic P-Value

C  18.59 3.27 0.001Hour1  ‐59.05 ‐6.93 0Hour2 ‐55.83 ‐6.51 0Hour3 ‐21.47 ‐2.61 0.009Hour4 ‐10.05 ‐1.25 0.21Hour5 ‐50.69 ‐6.5 0Hour6 ‐17.61 ‐2.3 0.021Hour7 ‐2.05 ‐0.27 0.784Hour8 ‐35.45 ‐4.84 0Hour9 ‐1.30 ‐0.17 0.866Hour10 ‐20.26 ‐2.44 0.015Hour11 ‐30.82 ‐3.86 0Hour12 19.25 2.38 0.017Hour13 ‐7.39 ‐0.88 0.379Hour14 ‐21.00 ‐2.69 0.007Hour15 56.41 7.07 0Hour16 38.61 4.69 0Hour17 ‐24.14 ‐3.36 0.001Hour18 ‐27.73 ‐3.91 0Hour19 ‐39.44 ‐5.33 0Hour20 ‐57.48 ‐6.82 0Hour21 ‐7.31 ‐0.76 0.444Hour22 ‐56.70 ‐6.33 0Hour23 ‐13.24 ‐1.46 0.143

∆   ‐0.52 ‐41.35 0∆   0.02 5.17 0

R Squared 0.0701Number of

Observations 37,666Reported t-statistics p-values are robust to arbitrary heteroskedasticity and autocorrelation.

5. An Empirical Analysis of Wind Energy Forecasting Errors and the Deployment of Upward Balancing Energy

Section 3 of this paper offered evidence that the wind forecasts in ERCOT are systematically upwardly biased. It would seem therefore that the ability of the system operator to compensate for the forecast errors through the upward deployment of balancing power would be among the principal challenges induced by higher wind power penetration levels. In this section, we estimate a model of upward deployments in ERCOT’s balancing market. A histogram of the upward deployments over the period 13 June through 30 November 2009 is depicted in Figure 15. Observe that a nontrivial number of the deployments are zero rendering ordinary least squares inappropriate as the estimation method. For this reason, we employ a standard Tobit model.

The set of explanatory variables includes scheduled load, a measure of actual load relative to scheduled load, two measures of the unexpected level of wind production, the expected level of wind production ( STWPF), and a measure of perceived forecast uncertainty (STWPF - WGRPP). Binary variables for each hour of the day are also included. The estimating equation is:

, , , , , ,

, ,

where

UpBESi* is the latent level of deployed upward balancing energy services in period i

UpBESi is the observed level of deployed upward balancing energy services in period i. It

equals zero if UpBESi* ≤ 0 and equals UpBESi

* if UpBESi* > 0.

SchLoad is the level of scheduled load in period i.

is the square of the ratio of actual load to scheduled load in period i. This particular formulation was chosen based on a preliminary analysis that employed the fractional polynomial approach advanced by Royston and Altman (1994).

NegativeError is the absolute value of actual wind minus forecasted wind when actual wind is less than forecasted and is zero otherwise.

NonNegativeError is the absolute value of actual wind minus forecasted wind when actual wind is greater than forecasted and is zero otherwise.

WindUncertainty is the level of perceived uncertainty in the wind forecast. It is proxied by (STWPFi,k - WGRPPi,k)

It is not known which forecast hour is the most relevant in determining UpBES. On one hand, the hour-ahead forecast has obvious appeal. On the other hand, ramping constraints on conventional generating units probably make the wind forecasts from some other hour more relevant in operating the grid. To assess which forecast hour is most relevant, equation (2) was estimated 20 times with a common sample of 12,451 observations using the STWPF and WGRPP values for the 20 different hours. Specifically, the equation was estimated for the following day-ahead hours: 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, and 23. It was also estimated using the five hour-ahead, four hour-ahead, three hour-ahead, two hour-ahead and the one hour-ahead values of STWPF and WGRPP. Inspection of Figure 16 reveals that the Pseudo R2 is maximized for the day-ahead hour 15 values of STWPF and WGRPP.

The estimation results for equation (2) corresponding to the day-ahead hour 15 values of STWPF and WGRPP are presented in Table 5. Observe that the coefficients on ScheduledLoad and ActSchLoad2 are both positive and highly statistically significant indicating that the upward deployment of balancing power is more likely and of larger magnitudes when scheduled load is high and actual load is high relative to scheduled. The coefficient on NegativeError is positive and highly significant indicating that actual wind production being less than forecasted contributes to the upward deployment of balancing power. In contrast, the coefficient on NonNegativeError is negative and highly significant indicating that actual wind production being greater than forecasted contributes to reductions in the upward deployment of balancing power. The coefficient on STWPF, is positive and highly statistically significant indicating that the upward deployment of balancing power is more likely and larger when the forecasted wind levels are high, ceteris paribus. The coefficient on the variable WindUncertainty is negative and highly statistically significant indicating that upward deployments of balancing power are less likely and smaller when the forecast is more uncertain. One possible explanation for this finding is that the system operator is prudent and “overschedules” conventional generation when the wind forecast indicates a high degree of uncertainty.

Table 5 Estimation Results for Equation 2

Variable Estimated

Coefficient t- Statistic P-Value

C  ‐5458.41 ‐74.07 0 Hour1  18.37 0.42 0.674 Hour2 80.40 1.82 0.068 Hour3 163.83 3.67 0 Hour4 181.05 4.07 0 Hour5 120.99 2.75 0.006 Hour6 990.90 21.63 0 Hour7 967.22 21.36 0 Hour8 768.76 17.14 0 Hour9 544.86 12.33 0 Hour10 399.02 8.90 0 Hour11 418.98 9.26 0 Hour12 484.54 10.66 0 Hour13 712.90 15.64 0 Hour14 1003.74 21.94 0 Hour15 1188.64 25.87 0 Hour16 1283.02 28.04 0 Hour17 1082.50 23.71 0 Hour18 784.30 17.13 0 Hour19 570.17 12.53 0 Hour20 548.46 12.06 0 Hour21 623.86 13.65 0 Hour22 214.66 4.98 0 Hour23 81.01 1.89 0.059 SchLoad  0.02 18.53 0 

ActSchLoad  7636.05 85.60 0 NegativeError 0.15 13.99 0 NonNegativeError ‐0.12 ‐7.58 0 STWPF 0.04 5.54 0 WindUncertainty ‐0.97 ‐16.03 0 

Sigma 663.85    655.1845 Number of

Observations 12,451  Pseudo R Squared 0.0797  

6. Wind Energy and Reserve Deployments Reserve deployments are clear instances in which system reliability is challenged. There are two types of reserve deployments in ERCOT: responsive reserves and non-spinning reserves. These deployments are relatively rare events. Over the sample period 13 June through 30 November 2009, reserves were deployed less than two percent of the time. In this section, we examine whether wind forecast errors have contributed to the deployments. The dependent variable in our analysis is binary. It has a value of one if either responsive or non-spinning reserves are deployed. It is zero otherwise. Because the dependent variable is binary, we make use of the standard probit model. Additionally, we make use of the binominal complementary log-log model, which is an appropriate estimation technique when attempting to estimate the marginal impacts of factors on the probability of a rare binary event. Except for the hourly binary variables, the explanatory variables include those presented in Table 5. The binary variables are excluded from the model because of singularity issues. Two other variables are added. The first is the change in the value of STWPF between day-ahead hour 15 and one hour before real-time (∆Forecast). The second is the change in the measure of perceived uncertainty between day-ahead hour 15 and one hour ahead of real-time (∆Uncertainty). Because the analysis only makes use of the hour–ahead and day-ahead hour 15 forecasts, there are significantly fewer missing values than when the results for Table 5 were generated and thus the sample is substantially larger than in the previous section. The results are presented in Table 6. In both models, the coefficients on SchLoad and are positive and highly statistically significant and, as expected, the coefficients on NonnegativeError are negative and statistically significant. In both models, the coefficients on NegativeError are positive as expected but significant only at the 10 percent level in the probit model. In the complementary log-log model, the coefficient is statistically significant at the one percent level. In both models, the coefficients on WindUncertainty are negative and highly statistically significant. Finally, in both models, the coefficients on ∆Forecast and ∆Uncertainty are statistically insignificant. While some might find this result troubling, it is just as well given that the hour-ahead forecast is inferior to the day-ahead hour 15 forecast as measured by the forecast errors reported in Table 3.

Table 6

Estimation Results for the Deployment of Reserve Power in ERCOT Using the Probit and Binomial Complementary Log Log Model Specifications

Probit Complementary Log Log

Variable Estimated Coefficient

t- Statistic Estimated Coefficient

t- Statistic

Constant ‐3.983  ‐14.55 ‐9.42164  ‐13.9SchLoad  3.29E‐05  8.09 8.82E‐05  8.69

ActSchLoad  9.38E‐01  2.82 2.682513  3.64NegativeError 1.10E‐04  1.93 0.000401  2.6NonNegativeError ‐3.66E‐04  ‐3.85 ‐0.00089  ‐3.85STWPF 7.70E‐07  0.02 ‐5.3E‐05  ‐0.57WindUncertainty ‐9.03E‐04  ‐3.44 ‐0.00213  ‐3.66∆Forecast ‐1.14E‐05  ‐0.26 2.72E‐05  0.24∆Uncertainty ‐2.59E‐04  ‐0.88 ‐0.00102  ‐1.51log likelihood ‐1073.391  ‐1063.4482   Number of Observations 15079  15079 Number of Deployments 230  230 

7. Can the Forecasts be Improved? In this section of the paper, we analyze the day-ahead hour 15 wind forecast errors. to assess the feasibility of improving the forecasts. The starting point is the hypothesis that the forecast errors are not statistically independent of the values of STWPF and WGRPP. We also hypothesize that the errors are not independent of the hour of the day. These hypotheses are based in part on our observation that the errors are highly correlated across forecast hours as well as the statistics presented in Tables 2 and 3 that strongly suggest STWPF is a biased predictor of wind energy production. We also suspect that the wind forecast errors could be reduced if the system operator revised its forecasts based on the prices reported in its day-ahead capacity markets for ancillary services. This hypothesis follows from Bachelier [1900] who was the first to recognize what has come to be known as the efficient market hypothesis. Following from this hypothesis, the day-ahead prices of ancillary services should reflect expected operating conditions to the extent that the market is efficient. Thus, if ERCOT’s day-ahead market for ancillary services is efficient, the market prices should be useful in forecasting operating conditions on a day-ahead basis. In testing this hypothesis, we follow Forbes and Zampelli (2009) who have provided evidence that the coefficient of variation and the skewness in day-ahead electricity prices can help explain load forecasting errors. This approach is consistent with the clustering of the forecast errors by day as revealed by Figure 14. To test these hypotheses, the following regression equation was estimated:

15 DaStwpfHR15 DaWgrppHR15

β CVDRS β SKEWNESSDRS β CVURS β SKEWNESSURS (3) + β CVRRS β SKEWNESSRRS β CVNSRS β SKEWNESSNSRS where DaErrorHR15 is the hour 15 day-ahead wind forecast error DaStwpfHR15 is the hour 15 day-ahead value of STWPF DaWgrppHR15 is the hour 15 day-ahead value of WGRPP

is the daily measure of the coefficient of variation in the hourly day-ahead prices for downward regulation services; SKEWNESSDRS is the daily measure of the skewness in the hourly day-ahead prices for downward regulation services; CVURS is the daily measure of the coefficient of variation in the hourly day-ahead prices for upward regulation services; SKEWNESSURS is the daily measure of the skewness in the hourly day-ahead prices for upward regulation services; CVRRS is the daily measure of the coefficient of variation in the hourly day-ahead prices for regulation reserve services; SKEWNESSRRS is the daily measure of the skewness in the hourly day-ahead prices for regulation reserve services; CVNSRS is the daily measure of the coefficient of variation in the hourly day-ahead prices for non-spinning reserve services; SKEWNESSNSRS is the daily measure of the skewness in the hourly day-ahead prices for non-spinning reserve services;

In many markets, the prices exhibit a unit root which precludes conventional regression analysis. Inspection of Table 7 reveals that the null hypothesis of a unit root can be rejected at the one percent level. Accordingly, the regression results corresponding to (3) are not expected to be plagued by unit root issues. The regression results are reported in Table 8. Observe that over 50 percent of the variation in the errors is accounted for by the estimated equation. Nine of the binary variables representing each hour are statistically significant at either the five or one percent level. Reflecting the bias in the forecasts noted in Table 3, the coefficients on DaStwpfHR15 and DaWgrppHR15 are both highly statistically significant. The coefficients on the various coefficients of variation and skewness are also highly statistically significant. One puzzling result is that the coefficients on responsive reserve service have the opposite signs from the coefficients on non-spinning reserve service. It not clear why this is the case and suggests that further research is needed.

Table 7 Dickey-Fuller Tests for Unit Root

Variable Test Statistic 1 Percent Critical Value of

the Statistic DaErrorHR15 -12.089 -3.430

StwpfHR15 -7.786 -3.430 WgrppHR15 -7.870 -3.430

CVDRS -7.081 -3.430 SKEWNESSDRS -6.528 -3.430

CVURS -9.480 -3.430 SKEWNESSURS -6.589 -3.430

CVRRS -4.440 -3.430 SKEWNESSRRS -6.742 -3.430

CVNSRS -6.268 -3.430 SKEWNESSNSRS -6.030 -3.430

Table 8 Estimation Results for Equation 3

Variable Estimated Coefficient t- Statistic P-Value C  -236.13 -1.1 0.271

Hour1  -71.19 -1.46 0.145Hour2 -118.27 -1.6 0.109Hour3 -124.54 -1.43 0.152Hour4 -132.14 -1.54 0.123Hour5 -134.65 -1.56 0.118Hour6 -134.11 -1.56 0.119Hour7 -196.87 -2.26 0.024Hour8 -270.17 -3.16 0.002Hour9 -186.80 -2.12 0.034Hour10 -21.84 -0.24 0.808Hour11 12.78 0.14 0.887Hour12 14.70 0.16 0.871Hour13 74.79 0.82 0.415Hour14 155.41 1.67 0.095Hour15 175.06 1.87 0.061Hour16 217.70 2.3 0.021Hour17 234.75 2.48 0.013Hour18 311.14 3.27 0.001Hour19 354.64 3.7 0Hour20 305.58 3.19 0.001Hour21 235.95 2.56 0.011Hour22 92.85 1.2 0.231Hour23 -66.64 -1.06 0.289

DaStwpfHR15 0.43 2.63 0.008DaWgrppHR15 -0.93 -5.43 0

CVDRS 550.03 3.23 0.001SKEWNESSDRS -142.19 -4.27 0

CVURS 685.79 5.65 0SKEWNESSURS -302.10 -5.72 0

CVRRS -427.09 -3.27 0.001SKEWNESSRRS 300.99 5.26 0

CVNSRS 376.63 3.81 0SKEWNESSNSRS -267.85 -4.86 0

Number of Observations 15079

R Squared 0.5628The reported results are Heteroskedasticity and autocorrelation-consistent statistics

8. Conclusion This paper has presented evidence that changes in wind energy production in ERCOT from one 15 minute market interval to the next are significantly offset by the deployment of regulation power by the system operator. The paper has also presented evidence that the wind forecast errors in ERCOT are very large as measured by both the root mean squared percentage error and the mean absolute percentage error. The errors were found to have consequences for ERCOT’s balancing market and the deployment of reserve power. The paper has also presented evidence that the forecast errors are statistically related with the hour of the day and the forecasted level of wind energy. There is also evidence that the prices in ERCOT’s day-ahead markets for ancillary service reflect the anticipated adverse impact of the forecast errors on power grid operations.

Acknowledgments We thank Michael A. Forbes of MIT for writing the software that performed the approximately 4,000 downloads of the hourly wind forecast data and the programs used to process the data. We also thank the staff of ERCOT for providing us with access to their data series and for answering our many questions. Any errors are the full responsibility of the authors.

Figure 1. The ERCOT Power Grid

Source : ERCOT

Figure 2. The Generation Fuel Mix in ERCOT, 1 January – 30 November 2009

Note: this figure only represents the primary fuel for those generators capable of using multiple fuels. Source: ERCOT

Coal36.2%

Gas42.7%

Hydro0.2%

Nuclear13.4%

Other1.3%

Wind6.1%

Figure 3. ERCOT Load and Generation by Fuel Type, 1 – 7 September 2009

Note: this figure does not depict electricity generation from hydro and miscellaneous sources

0

2000

4000

6000

8000

10000

12000

14000

16000M

Wh

Wind Gas Turbines Gas CC Coal Nuclear load Load Minus Wind

Figure 4. The Deployment of Ancillary Services in ERCOT, 1 – 7 September 2009

0

200

400

600

800

1000

1200

MW

Downward Regulation Upward Regulation Responsive Reserves NSRS

Figure 5. The Deployment of Balancing Energy in ERCOT, 1 – 7 September 2009

-6000

-4000

-2000

0

2000

4000

6000

MW

DOWNWARD BES UPWARD BES NET BES

Figure 6. A Histogram of Net Regulation Deployments in ERCOT, 1 October 2008 – 30 November 2009.

02

46

8P

erce

nt

-2000 -1000 0 1000 2000mw

Figure 7. A Histogram of Net Balancing Deployments in ERCOT, 13 June 2009 – 30 November 2009.

02

46

8P

erce

nt

-10000 -5000 0 5000 10000mw

Figure 8. A Histogram of the Changes in Wind Energy Production in ERCOT across 15 Minute Market Periods, 1 October 2008 – 30 November 2009

010

2030

Per

cent

-1500 -1000 -500 0 500 1000delta_wind_mw

Figure 9. The Wind Energy Forecasts that ERCOT posted at 5:14 AM on 27 December 2009

0

500

1000

1500

2000

2500

6 8 10 12 14 16 18 20 22 0 2 4 6 8 10 12 14 16 18 20 22 0 2 4Hour

MW

STWPF WGRPP

Figure 10. Actual and Day-Ahead Forecasted Levels of Wind Energy in ERCOT, 1- 7 September 2009

0

1000

2000

3000

4000

5000

6000

MW

Actual Wind Energy Hour 12 Day-Ahead Forecasted Hour 23 Day-Ahead Forecasted

Figure 11. Actual and Day-Ahead Forecasted Levels of Wind Energy in ERCOT, 1- 30 October 2009

0

1000

2000

3000

4000

5000

6000

7000

8000

MW

Actual Wind Energy Hour 12 Day-Ahead Forecasted

Figure 12. Actual and Hour-Ahead Forecasted Levels of Wind Energy in ERCOT, 1- 31 July 2009

Note: This figure is only illustrative. Fifty six of the 2,976 forecast values represented in this figure were missing. Their values are represented using the other hour-ahead forecast data.

0

1000

2000

3000

4000

5000

6000

7000

8000

MW

Figure 13. A Histogram of Hour-Ahead Wind Energy Forecasting Errors in ERCOT, 13 June – 30 November 2009

02

46

810

Per

cent

-6000 -4000 -2000 0 2000 4000mw

Figure 14. The Daily Forecast Errors

0

0.2

0.4

0.6

0.8

1

1.2

1.4

6/13/2

009

6/20/2

009

6/27/2

009

7/4/20

09

7/11/2

009

7/18/2

009

7/25/2

009

8/1/20

09

8/8/20

09

8/15/2

009

8/22/2

009

8/29/2

009

9/5/20

09

9/12/2

009

9/19/2

009

9/26/2

009

10/3/

2009

10/10

/2009

10/17

/2009

10/24

/2009

10/31

/2009

11/7/

2009

11/14

/2009

11/21

/2009

11/28

/2009D

aily

RM

SE R

elat

ive

to th

e D

aily

For

ecas

ted

Win

d Pr

oduc

tion

Figure 15. A Histogram of the Upward Deployments of Balancing Power in ERCOT, 13 June – 30 November 2009

010

2030

Per

cent

0 2000 4000 6000 8000up_bes_mw

Figure 16. Pseudo R2 for Equation 2 by Forecast Hour

0.077

0.0775

0.078

0.0785

0.079

0.0795

0.08

Day-A

head

Hou

r 9

Day-A

head

Hou

r 10

Day-A

head

Hou

r 11

Day-A

head

Hou

r 12

Day-A

head

Hou

r 13

Day-A

head

Hou

r 14

Day-A

head

Hou

r 15

Day-A

head

Hou

r 16

Day-A

head

Hou

r 17

Day-A

head

Hou

r 18

Day-A

head

Hou

r 19

Day-A

head

Hou

r 20

Day-A

head

Hou

r 21

Day-A

head

Hou

r 22

Day-A

head

Hou

r 23

Five H

ours

Ahead

Four H

ours

Ahead

Three H

ours

Ahead

Two Hou

rs Ahe

ad

One H

our A

head

References

Allison, P. D., 1999, Logistic Regression Using SAS®: Theory and Application, Cary, NC: SAS Institute Inc. Bachelier, Louis (1900) trans. James Boness. Theory of Speculation, in Cootner (ed) (1964) The Random Character of Stock Market Prices. MIT Press. pp. 17-78. DeCarolis, J. and D. Keith (2002), The Real Cost of Wind Energy, Science, wysiwyg://272/http://www.sciencemag.org/cgi/content/full/294/5544/1000. Cali, Ü., B. Lange, R. Jursa, and K. Biermann, (2006), Short-term prediction of distributed generation – Recent advances and future Challenges, Elftes Kasseler Symposium Energie-Systemtechnik November 2006. Available on the Internet at http://www.iset.uni-kassel.de/public/kss2006/KSES_2006.pdf Energy Information Administration, (2009) Electric Power Annual. Available on the internet at http://www.eia.doe.gov/cneaf/electricity/epa/epa_sum.html E.ON Netz (2005), Wind Report 2005. Available on the internet at http://www.windaction.org/documents/461 ERCOT (2008), ERCOT Quick Facts, Available on the internet at http://www.ercot.com/content/news/presentations/2008/ERCOT_Quick_Facts_May_2008.pdf EWEA, (2008) Pure Power: Wind Energy Scenarios up to 2030. Available on the internet at http://www.ewea.org/index.php?id=11 EWEA, (2007), Debunking The Myths. Available on the Internet at http://www.ewea.org/fileadmin/ewea_documents/documents/publications/wind_benefits/Windpower_is_unreliable.pdf Fabbri, A, T. Gomez San Romain, J. Rivier Abbad, and V.H. Mendez Quezada (2005), “Assessment of the Cost Associated With Wind Generation Prediction Errors in a Liberalized Market,” IEEE Transactions on Power Systems, 20(3), 1440-1446. Forbes, K., M. Stampini, and E. Zampelli (2009) Wind Power Variability, Wind Power Forecasting Errors, and the Cost of Balancing Power: Evidence from the E.ON Transmission System in Central Germany, unpublished manuscript.

Forbes, K., and E. Zampelli (2009) Do Day-Ahead Electricity Prices Contain “Hidden Secrets”?: Evidence from Six Power Grids, unpublished manuscript.

Lively, M. B., (2009), Renewable Electric Power—Too Much of a Good Thing: Looking At ERCOT, USAEE Dialogue, Vol 17 No. 2, August 2009

Hirst, E. (2001), Interactions of wind farms with bulk-power operations and markets, http://www.EHirst.com/PDF/WindIntegration.pdf. Newey, W. K. and K. D. West (1987), “A Simple Positive Semi-Definite Heteroskedasticity and Autocorrelation Consistent Covariance Matrix,” Econometrica 55, 703-708. Royston P, Altman DG (1994) Regression using fractional polynomials of continuous covariates: parsimonious parametric modelling (with Discussion). Appl Stat 43: 429–467 Samuelson, Paul (1965), “Proof that Properly Anticipated Prices Fluctuate Randomly,” Industrial Management Review, 6, 41-49.