Embed Size (px)
Citation preview
Discussion of PJM Forecasting ModelTim McCliveOPSI Annual MeetingOctober 12, 2015
Navigant Overview
» Navigant’s core business areas› Management Consulting, Economics, Financial Advisory, Disputes &
Investigations› Publicly traded since 1996 (NYSE: NCI), 35 offices in N.A., Europe,
Asia» Navigant’s global energy practice
› Clients: 50 largest electric and gas utilities, 20 largest independent power generators, 20 largest gas distribution and pipeline companies, Federal/State governments, new entrants, investors
› Personnel: 450+ consultants, average 15 years experience, 60% with advanced degree, 51% with engineering degree
Page 2
Overview of Navigant’s Engagement with P3
» Provide an independent review of› PJM’s proposed changes of its load forecasting models› the structure, data, and estimation techniques of the models
» Work with PJM LAS to investigate and recommend steps to improve accuracy and stability of the forecasts
» Preliminary findings› Near term – improvements to a key energy efficiency variable› Longer term – potential theoretical and empirical modifications to
the models
Page 3
Econometrics – not as hard as rocket science, but close
» “Econometrics is a special type of economic analysis in which the general theoretical approach is combined with empirical measurement of economic phenomena.” Leontief, 1948
» Econometrics has two elements, and both are important:› Economics – e.g., demand is higher with a stronger economy, lower
prices, hotter summers, or colder winters (or lower when the opposite happens), and demand is lower when more efficient technology is used to meet the same service needs
› Mathematics – the estimated models for demand should meet specified statistical conditions with an objective of producing unbiased estimates with minimum variance and error
Page 4
Issue: SEER Forecasts Underestimate Peak Demand
» PJM used available SEER data, and Navigant offered a way to augment that with publicly available data to improve accuracy
» Cooling demand forecast – conditioned by projected efficiency improvements for different equipment classes. › For residential AC and heat pumps, PJM uses efficiency indexes based
on seasonal energy efficiency ratio (SEER)› But SEER is calculated using full-load and part-load test conditions, while
the energy efficiency ratio (EER) measures equipment running at full load› Correlations show that large improvements in SEER correlate with only
moderate improvements in EER» Forecasts based on SEER may overestimate efficiency gains over
time, and underestimate peak electrical demandPage 5
Illustration of SEER-EER Relationship
» There is no direct relationship between SEER and EER» AC, HP manufacturers can increase SEER in ways that do
not increase EER (advanced motors, compressors, controls) Page 6
Points represent SEER and EER ratings of individual AC systems in the AHRI database of certified products. Note metrics’ ranges:SEER: 13.0 to 26.0, 100% above lowestEER: 9.0 to 16.5, 83% above lowest
The Effect on PJM’s Total Cooling Index
» Graphical comparison of load-weighted average Total Cooling Index (SEER-based vs. EER-based).
Page 7
Sensitivity of Forecast to Projected Index Values
» PJM’s May 2015 forecast update demonstrated that the forecast is highly sensitive to the efficiency index. › Efficiency growth that is one % point faster than projected results in
peak demand forecast falling by 5,000 – 7,000 MW by 2020» Navigant tested the sensitivity of the forecast to a change
from a SEER-based to an EER-based projected index and calculated a material impact on the forecast.› Efficiency-related parameters in the forecast equations will likely
change when the historical SEER-based index is replaced› Nevertheless, the transition from SEER to EER-based index would
still materially affect the peak demand forecast.
Page 8
On Forecast Accuracy
Page 9
» From Nov ‘14 LAS presentation
» The over-forecast in 2010-14 prompted the current reviews
» Forecasts are not consistently high nor low over the long-term
» PJM’s work in 2015 is a good start to resolve issues
Future Work and Recommendations
» PJM Staff and LAS members work continuously and proactively to monitor forecast model performance and improve the accuracy of the models
» Navigant’s review has identified areas for consideration and future study and will raise these with PJM and the LAS, e.g.› Simplify interactive structure of the explanatory variables to separate
the discernible impacts of economic, technology and weather variables› Consider a two-part process – use current models to control for the
short-term variances (“normalization”) and develop new model for the long term effects of economic and technology adoption variables
› Various “in the weeds” ideas around technical methods to address data pooling, multicollinearity, and missing-variables issues
Page 10
Navigant Contacts
Tim McClive, Director, Washington [email protected]
(202) 973-4555
Ken Seiden, Director, Boulder, [email protected]
(303) 728-2479
Peter Steele-Mosey, Managing Consultant, Toronto, [email protected]
(416) 956-5050
J. Decker Ringo, Managing Consultant, Burlington [email protected]
(781) 270-8410Page 11
