Tales from the Murky Depths

Dubious Tales from Deep Dark Waters Past(Being a very selective history of hydro management and NZEM origins.. with some observations on various kinds of bias) E.G.Read with contributions from S. Starkey

Presented to EPOC Winter WorkshopAuckland, New Zealand5 September 2013CaveatThis presentation should not be quoted, or relied upon

It has been largely prepared from memorywhich is inevitably selective and biased

(and not aided by the fact that any files I still have are currently in post-quake limbo-land)Some Overview Papers J.G. Culy, E.G. Read, and B. Wright: "Structure and Regulation of the New Zealand Electricity Sector", in R Gilbert and E Kahn (eds.) International Comparison of Electricity Regulation, Cambridge University Press, 1996, p. 312-365.

E.G. Read, J.G. Culy, and S.J. Gale: "Operations Research in Energy Planning for a Small Country", European Journal of Operational Research, vol. 56, 1992, p. 237-248.

E.G. Read: "OR Modelling for a Deregulated Electricity Sector", International Transactions in Operations Research, vol. 3, no. 2, 1996, p. 129-138.

E.G. Read: "Electricity Sector Reform in New Zealand: Lessons from the Last Decade Pacific Asia Journal of Energy Vol 7, No 2, 1997, p. 175-191

Episode 1: Guidelines Meremere:The Basic Rule Curve/ Forbidden Zone/Guideline Basically a critical probability calculation (as in the classic newsagent problem) Backwards projection of minimum storage to survive Design Dry Year (e.g. 1:20 if marginal cost of running out is 20 times marginal cost of fuel) Marsden:- Multiple GuidelinesGuideline Diagram (From Read and Boshier 1989)

Episode II: VALWAT/STAGEExperimented with SDP involving a variety of forward simulation strategies (Programmed by Chris Lusk c.1979, reported by Read and Boshier, 1989)Lead to abandonment of:Deterministic LP model (Boshier and Lermit, 1977)Deterministic decomposition model (Read, 1983)Produced STAGEDeveloped 1980ish, reported by Boshier et al (1983)Based on Swedish model by Stage and Larsson :Uses Marginalistic SDP - Setting MWVt = marginal value of release in t = E{MWVt+1 }. Subject to bounds But with about 6 months forward simulation - Setting MWVt =E{ MWVt+26 , or spill/shortage value if storage bound reached sooner}Trying to capture effects of correlation - Spreading trajectories over a wider range - So typically raising expected MWVs (see later)Optimal (SDP/STAGE) guidelines

Optimal deterministic guidelines assuming average inflows

Optimal guidelines from averaging deterministic optima for historical inflows

Optimal guidelines assuming nave future management Biases?The optimum is actually rather flat:The two extreme solutions performed badly, but While the SDP But was preferred for its conservatism It actually produced similar simulated performance to the average of deterministic policies

Why do these biases occur, though? The System Marginal Cost Curveis basically convex

Optimal management will try to control variability of marginal cost as best it can Bias arises due to sub-optimal/unrealistic control of variabilityTrue expected marginal cost with optimal (well controlled) degree of variation Under-estimated expected marginal cost from expected situation /perfect foresightOver-estimated expected marginal cost due to sub-optimal (poorly controlled) degree of variation Episode III: RESOP/PRISM>SPECTRA2-D Constructive Dual Dynamic Programmingso named retrospectively by Read and Hindsberger(2010)Conceived by Read (1984ish)Programmed by Culy, Davies, and many more over the yearsReported by Read et al(1987) RESOP reservoir optimisation module:Originally developed within a long term planning model (PRISM) Soon used for operational planning and much more PRISM was later re-developed as SPECTRABasic Concept: Guideline augmentation by inserting flatscorresponding to varying utilisation level of one thermal station

Uncertainty Adjustment to produce new expected MWV curve

Another source of bias: decision period lengthNotice that, a longer decision period means:The flats get wider in proportion The uncertainty adjustment interpolates over a wider rangeBut the MWV curve is still basically convex, so:The Expected MWV curve may be expected to riseAnd also lose its detailed structureRelease decisions become more moderate (since we could not stick with more extreme release levels for any length of time)This was very evident in early RESOP experimentsA more recent example(using the model of Dye et al, 2012)And another source of bias: Interpolation True expected marginal cost Over-estimated expected marginal cost due to nave linear interpolation on a (relatively coarse) grid Yet another source of bias: assuming independence

Cumulative pdf of independent in flowsEffect of correlationCounteracting biases in SPECTRAIndependent inflows:Too optimistic (lowering MWV)Linear interpolation on a fairly coarse gridToo pessimistic/cautious (raising MWV)Originally a coarse enough grid was used to give a cautious biasBut tuning was possible using a finer gridAnd it was tuned to achieve the 1:20 criterion exactlyIn 1992, which was worse than a 1:20 eventSo we should have run out of water And nearly did After the 1992 crisisOperational reliability standard raised:From 1:20 DDY with 7% load reductionTo 1:60 DDY without load reduction (The operational optimum being quite flat due to excess supply due to overbuilding in NZED/MoE period) Heuristic added to account for correlation in optimisation phaseby:Inflow pdf spreading '= (t) / t where (t) is the s.d of t week cumulative flow distributionSo assuming independent inflows in optimisation gives effective cumulative inflow s.d of:(t) = t * / t = (t)

Heuristic added to account for correlation simulation/operation phaseby:Episode IV: DecentralisationThe 1992 crisis provided some impetus for change, by denting ECNZs reputation,Quite unfairly IMHO

But some kind of competitive market was always on the agendaAnd ECNZs dominance was obviously a much more significant issueImpact of decentralisation?Less precise coordinationLess detailed informationLess sophisticated modellingGreater risk aversionLess pooling available to meet contract commitmentsMore innovation????????Greater diversity of perspective/techniqueLower overall riskBenefits of Diversity?fullfullemptyemptyManager 1Manager 2Diverse strategies reduce probability of extremes Both emptyBoth fullA Comment on Bias in Performance EvaluationPerformance evaluation seems much more difficult than most OR papers recogniseClassic paradigm:I make assumptions/collected data/built a model I show that my policy is better than yours given my assumptions/data/model I get my work published, and you get no say in itBut reality is very different not least in that it is neither known nor agreed What About Reality?Even if reality was as simple as an (agreed) inflow distribution:We will not observe that reality, only randomised outcomes Policies will tend to be judged by what happened, not what might have happenedObservers differ widely in their assessment of:Underlying driving forcesCurrent data values ForecastsWhat should drive decision-making Their personal situation and stake The interpretation of historical eventsA Rational Approach?Given this variety What seems optimal to one analyst may seem totally irrational to another.And vice versa!!We should be (a)ware of (the extent to which we are) privileging our own perspectives when comparing our own recommendations with those of othersThat leaves us with a much more difficult task:Assessing internal consistency with stated assumptionsAssessing robustness of outcomes in terms of a range of criteria, across a range of possible assumptionsOrWe could just entrust decision-making to those who have most at stake And hope for robust outcomesKnowing that these will always seem sub-optimal From (almost) everyones perspective Reservoir Management Outcomes?We should expect storage coordination to look wrong, from any individual perspectiveBut Tipping and Read (2010) Tuned a model to show that, in aggregate, hydro was operating as if using a plausible looking MWV curve Subsequently tested to find storage policyAbout as cautious as that under MoE Somewhat more cautious than that under ECNZAs should be expected? BUTNone of this was really what motivated the NZEM reforms

Advantages were seen in increased decision-making diversityBut maybe only enough to offset loss in coordination efficiency The real issue was creating a competitive market as a means of controlling cost and pricesRather than relying on Government At the timeHistory and common sense both seemed to suggest that it was politically impossible for a Government (of any stripe):To make unbiased growth forecasts for the economy (and hence for electricity)To back away from plans and promises that turned out to be unwiseTo set fair, honest and realistic prices for industry, commerce, or domestic consumers To resist biasing technology choicesThe fear was that: Gains in operational efficiency at ECNZ would be lost by gradual reversion to public sector norms, but particularly Forecasting, planning and pricing would be re-politicisedWith potentially severe consequences for allocative and investment efficiencyWhich are where most of the sectors costs are incurred Concluding Perspective Harker (2013) claims that the true energy component of domestic power bills has risen in recent yearsAlmost to the level reached under the MoE in 1982

But.1982 was the only year, in the MoE period, in which After rising by 124% in 12 monthsPrices (briefly) reached LRMC levelsAs calculated using cost projections from SCM and MWDMost of which turned out to be significant under-estimates30 years later, with cheap gas and hydro both gone:The New Zealand electricity sector is still (apparently) producing at those pricesAnd actually paying its own way!

If Dr Harker is correct I am personally astounded

How did such a marvellous thing ever come about?

Maybe Max was right? More ReferencesJ.F. Boshier and R. J Lermit: A Network Flow Formulation for Optimum Reservoir Management of the New Zealand Power Generating System , NZOR vol5 #2., 1977, p85-10085J.F. Boshier, G.B. Manning, and E.G. Read: "Scheduling Releases from New Zealand's Hydro Reservoirs" Transactions of the Institute of Professional Engineers in New Zealand, vol. 10, no. 2/EMCh, July 1983, p.33-41.S. Dye, E.G. Read, R.A Read, S.R. Starkey Easy Implementations of Generalised Stochastic CDDP Models for Market Simulation Studies Proceedings 4th IEEE/Cigr International Workshop on Hydro Scheduling in Competitive Markets. Bergen, Norway, 2012B. Harker Chairman's Address, TrustPower AGM, July 2013 https://www.nzx.com/companies/TPW/announcements/239025E.G. Read: "Reservoir Release Scheduling for New Zealand Electricity - A Non-Linear Decomposition Algorithm", New Zealand Operational Research, vol. 11, no. 2, July 1983, p.125-142.E.G. Read, J.G. Culy, T.S. Halliburton, and N.L. Winter: "A Simulation Model for Long-term Planning of the New Zealand Power System", in G.K. Rand (ed.) Operational Research 1987, North Holland, p.493-507.E.G. Read: "A Dual Approach to Stochastic Dynamic Programming for Reservoir Release Scheduling", in A.O. Esogbue (ed.) Dynamic Programming for Optimal Water Resources System Management, Prentice Hall NY, 1989, p.361-372.E.G. Read and J.F. Boshier: "Biases in Stochastic Reservoir Scheduling Models", in A.O. Esogbue (ed.) Dynamic Programming for Optimal Water Resources System Management, Prentice Hall NY, 1989, p.386-398.E. G. Read and M. Hindsberger Constructive Dual DP for Reservoir Optimisation in S. Rebennback, P.M. Pardalos, M.V.F. Pereira and N.A. Iliadis (eds) Handbook on Power Systems Optimisation Springer, 2010, Vol I p3-32 J. Tipping and E. G. Read Hybrid bottom-up/top-down modelling of prices in hydro-dominated power markets in S. Rebennback, P.M. Pardalos, M.V.F. Pereira and N.A. Iliadis (eds) Handbook on Power Systems Optimisation Springer, 2010, Vol II, p213-238