How predictable is technological progress?

J. Doyne Farmer1,2,3 and Franois Lafond1,4,5

1Institute for New Economic Thinking at the Oxford Martin School, University of

Oxford, Oxford OX2 6ED, U.K.2Mathematical Institute, University of Oxford, Oxford OX1 3LP, U. K.

3Santa-Fe Institute, Santa Fe, NM 87501, U.S.A4London Institute for Mathematical Sciences, London W1K 2XF, U.K.

5United Nations University - MERIT, 6211TC Maastricht, The Netherlands

February 18, 2015

Abstract

Recently it has become clear that many tech-nologies follow a generalized version of Mooreslaw, i.e. costs tend to drop exponentially, at dif-ferent rates that depend on the technology. Herewe formulate Moores law as a time series modeland apply it to historical data on 53 technolo-gies. Under the simple assumption of a corre-lated geometric random walk we derive a closedform expression approximating the distributionof forecast errors as a function of time. Based onhind-casting experiments we show that it is pos-sible to collapse the forecast errors for many dif-

Acknowledgements: We would like to acknowl-edge Diana Greenwald and Aimee Bailey for theirhelp in gathering and selecting data, as well asGlen Otero for help acquiring data on genomics,Chris Goodall for bringing us up to date on de-velopments in solar PV, and Christopher LlewellynSmith for comments. This project was supportedby the European Commission project FP7-ICT-2013-611272 (GROWTHCOM) and by the U.S. Dept. ofSolar Energy Technologies Office under grant DE-EE0006133. Contacts: doyne.farmer@inet.ox.ac.uk;francois.lafond@oxfordmartin.ox.ac.uk

ferent technologies at many time horizons ontothe same universal distribution. As a practicaldemonstration we make distributional forecastsat different time horizons for solar photovoltaicmodules, and show how our method can be usedto estimate the probability that a given tech-nology will outperform another technology at agiven point in the future.

Keywords: forecasting, technologicalprogress, Moores law, solar energy.

JEL codes: C53, O30, Q47.

1 Introduction

Technological progress is widely acknowledgedas the main driver of economic growth, and thusany method for improved technological forecast-ing is potentially very useful. Given that tech-nological progress depends on innovation, whichis generally thought of as something new andunanticipated, forecasting it might seem to bean oxymoron. In fact there are several postu-lated laws for technological improvement, such

1

as Moores law and Wrights law, that have beenused to make predictions about technology costand performance. But how well do these meth-ods work?

Predictions are useful because they allow us toplan, but to form good plans it is necessary toknow probabilities of possible outcomes. Pointforecasts are of limited value unless they are veryaccurate, and when uncertainties are large theycan even be dangerous if they are taken too seri-ously. At the very least one needs error bars, orbetter yet, a distributional forecast, estimatingthe likelihood of different future outcomes. Al-though there are now a few papers testing tech-nological forecasts1, there is as yet no methodthat gives distributional forecasts. In this paperwe remedy this situation by deriving the distri-butional errors for a simple forecasting methodand testing our predictions on empirical data ontechnology costs.

To motivate the problem that we address,consider three technologies related to electricitygeneration: coal mining, nuclear power and pho-tovoltaic modules. Figure 1 compares their long-term historical prices. Over the last 150 yearsthe inflation-adjusted price of coal has fluctu-ated by a factor of three or so, but shows nolong term trend, and indeed from the historicaltime series one cannot reject the null hypothe-sis of a random walk with no drift2 (McNerneyet al. 2011). The first commercial nuclear powerplant was opened in 1956. The cost of electricitygenerated by nuclear power is highly variable,but has generally increased by a factor of twoor three during the period shown here. In con-

1See e.g. Alchian (1963), Alberth (2008). Nagy et al.(2013) test the relative accuracy of different methodsof forecasting statistically but do not produce and testa distributional estimate of forecast reliability for anyparticular method.

2To drive home the point that fossil fuels show nolong term trend of dropping in cost, coal now costs aboutwhat it did in 1890, and a similar statement applies tooil and gas.

trast, since their first practical use as a powersupply for the Vanguard I satellite in 1958, so-lar photovoltaic modules have dropped in priceby a factor of about 2,330 between 1956 and2013, and since 1980 have decreased in cost atan average rate of about 10% per year3.

In giving this example we are not trying tomake a head-to-head comparison of the full sys-tem costs for generating electricity. Rather, weare comparing three different technologies, coalmining, nuclear power and photovoltaic man-ufacture. Generating electricity with coal re-quires plant construction (whose historical costhas dropped considerably since the first plantscame online at the beginning of the 20th cen-tury). Generating electricity via solar photo-voltaics has balance of system costs that havenot dropped as fast as that of modules in recentyears. Our point here is that different technolo-gies can decrease in cost at very different rates.

Predicting the rate of technological improve-ment is obviously very useful for planning andinvestment. But how consistent are such trends?In response to a forecast that the trends abovewill continue, a skeptic would rightfully respond,How do we know that the historical trend willcontinue? Isnt it possible that things will re-verse, and over the next 20 years coal will drop inprice dramatically and solar will go back up?".

3The factor of 2,330 is based on the fact that a onewatt PV cell cost $286 in 1956 (Perlin 1999), which us-ing the US GDP deflator corresponds to $1,910 in 2013dollars, vs. $0.82 for a kilowatt of capacity in 2013.

2

1900 1950 2000 2050

1e

021e

+00

1e+

02

pric

e in

$/k

Wh

0.17$/kWh= 2013 price in $/Wp

DOE SunShot Target = coal elec. price

Coal fuel price for electricity generation

nuclear

solar energy

0.1

0.5

5.0

50.0

pric

e in

$/W

p

photovoltaic module pricesU.S. nuclear electrity pricesU.K. Hinkley Point price in 2023

All prices in 2011 U.S. dollars

Figure 1: A comparison of long-term price trendsfor coal, nuclear power and solar photovoltaic mod-ules. Prices for coal and nuclear power are levelizedcosts in the US in dollars per kilowatt hour (scaleon the left) whereas solar modules are in dollars perwatt-peak, i.e. the cost for the capacity to gener-ate a watt of electricity in full sunlight (scale onthe right). For coal we use units of the cost ofthe coal that would need to be burned in a mod-ern US plant if it were necessary to buy the coal atits inflation-adjusted price at different points in thepast. Nuclear prices are levelized electricity costs forUS nuclear plants in the year in which they becameoperational (from Cooper (2009)). The alignmentof the left and right vertical axes is purely sugges-tive; based on recent estimates of levelized costs, wetook $0.177/kWh = $0.82/Wp in 2013 (2013$). Thenumber $0.177/kWh is a global value produced asa projection for 2013 by the International EnergyAgency (Table 4 in International Energy Agency(2014)). We note that it is compatible with esti-mated values (Table 1 in Baker et al. (2013), Fig.4 in International Energy Agency (2014)). The redcross is the agreed price for the planned UK Nuclearpower plant at Hinkley Point which is scheduled tocome online in 2023 (0.0925 $0.14). The dashedline corresponds to an earlier target of $0.5/kWh setby the DOE (the U.S. Department of Energy).

By studying the history of many technologies

our paper provides a quantitative answer to thisquestion. We put ourselves in the past, pre-tend we dont know the future, and use sim-ple methods to forecast the costs of 53 dif-ferent technologies. Actually going throughthe exercise of making out-of-sample forecastsrather than simply performing in-sample testsand computing error bars has the essential ad-vantage that it allows us to say precisely howwell forecasts would have performed. Out-of-sample testing is particularly important whenmodels are mis-specified, which one expects fora complicated phenomenon such as technologi-cal improvement.

We show how one can combine the experi-ence from all these different technologies to say,Based on experience with many other technolo-gies, the trend in photovoltaic modules is un-likely to reverse". Indeed we can assign a prob-ability to different price levels at different pointsin the future, as is done later in Figure 10. Ofcourse technological costs occasionally experi-ence structural breaks where trends change there are several clear examples in our histor-ical data. The point is that, while such struc-tural breaks happen, they are not so commonas to over-ride our ability to forecast. And ofcourse, every technology has its own story, itsown specific set of causes and effects that ex-plain why costs went up or down in any givenyear. Nonetheless, as we demonstrate here, thelong term trends tend to be consistent, and canbe captured via historical time series methodswith no direct information about the underly-ing technology-specific stories.

In this paper we use a very simple approachto forecasting, which was originally motivatedby Moores Law. As everyone knows, Intels ex-CEO, Gordon Moore, famously predicted thatthe number of transistors on integrated circuitswould double every two years, i.e. at an annualrate of about 40% (Moore 1965). Because mak-ing transistors smaller also brings along a variety

3

of other benefits, such as increased speed, de-creased power consumption, and cheaper manu-facture costs per unit of computation, with ap-propriate alterations in the exponent, it quicklybecame clear that Moores law applies morebroadly. For example, when combined with thepredicted increase in transistor density, the scal-ing of transistor speed as a function of size givesa doubling of computational performance every18 months.

Moores law stimulated others to look at re-lated data more carefully, and they discoveredthat exponential improvement is a reasonableapproximation for other types of computer hard-ware as well, such as hard drives. Since theperformance of hard drives depends on physicalfactors that are unrelated to transistor densitythis is an independent fact (though of course thefact that mass storage is essential for compu-tation causes a tight coupling between the twotechnologies). Lienhard, Koh and Magee, andothers4 have examined data for other productsand postulated that exponential improvement isa much more general phenomenon that appliesto many different technologies. An importantdifference that in part explains why exponentialscaling was discovered later for other technolo-gies is that the rates of improvement vary widelyand computer hardware is an outlier in terms ofrate.

Although Moores law is traditionally appliedas a regression model, we reformulate it here asa geometric random walk with drift. This allows

4Examples include Lienhard (2006), Koh & Magee(2006, 2008), Bailey et al. (2012), Benson & Magee(2014a,b), Nagy et al. (2013). Studies of improvementin computers over long spans of time indicate super-exponential improvement (Nordhaus 2007, Nagy et al.2011), suggesting that Moores law may only be an ap-proximation reasonably valid over spans of time of 50years or less. See also e.g. Funk (2013) for an expla-nation of Moores law based on geometric scaling, andFunk & Magee (2014) for empirical evidence regardingfast improvement prior to large production increase.

us to use standard results from the time seriesforecasting literature5. The technology time se-ries in our sample are typically rather short, of-ten only 15 or 20 points long, so to test hypothe-ses it is essential to pool the data. Because thegeometric random walk is so simple it is possibleto derive formulas for the errors in closed form.This makes it possible to estimate the errors asa function of both sample size and forecastinghorizon, and to combine data from many dif-ferent technologies into a single analysis. Thisallows us to get highly statistically significantresults.We find that even though the random walk

with drift is a better model than the generally-used linear regression on a time trend, it doesnot fully capture the temporal structure of tech-nological progress. We present evidence suggest-ing that a key difference is that the data are pos-itively autocorrelated. As an alternative we de-velop the hypothesis that technological progressfollows a random walk with drift and autocorre-lated noise, which we capture via an IntegratedMoving Average process of order (1,1), hereafterreferred to as an IMA(1,1) model. Under theassumption of sufficiently large autocorrelationthis method produces a good fit to the empiri-cally observed forecasting errors. We apply ourmethod to forecast the likely distribution of theprice of photovoltaic solar modules and show itcan be used to estimate the probability that theprice will undercut a competing technology at a

5Several methods have been defined to obtain pre-diction intervals, i.e. error bars for the forecasts (Chat-field 1993). The classical Box-Jenkins methodology forARIMA processes uses a theoretical formula for the vari-ance of the process, but does not account for uncertaintydue to parameter estimates. Another approach is to usethe empirical forecast errors to estimate the distributionof forecast errors. In this case, one can use either thein-sample errors (the residuals, as in e.g. Taylor & Bunn(1999)), or the out-of-sample forecast errors (Williams& Goodman 1971, Lee & Scholtes 2014). Several studieshave found that using residuals leads to prediction inter-vals which are too tight (Makridakis & Winkler 1989).

4

given date in the future.We want to stress that we do not mean to

claim that the generalizations of Moores lawexplored here provide the best method for fore-casting technological progress. There is a largeliterature on experience curves6, studying the re-lationship between cost and cumulative produc-tion originally suggested by Wright (1936), andmany authors have proposed alternatives andgeneralizations7. Nagy et al. (2013) tested thesealternatives using a data set that is very close toours and found that Moores and Wrights lawswere roughly tied for first place in terms of theirforecasting performance. An important caveatis that Nagy et al.s study was based on regres-sions, and as we argue here, time series methodsare superior, both for forecasting and for statis-tical testing. It is our intuition that methodsusing auxiliary data such as production, patentactivity, or R&D are likely to be superior8.

The key assumption made here is that alltechnologies follow the same random process,even if the parameters of the random processare technology specific. This allows us to de-velop distributional forecasts in a highly parsi-monious manner and efficiently test them out ofsample. We develop our results for Moores lawbecause it is the simplest method to analyze, butthis does not mean that we necessarily believeit is the best method of prediction. We restrictourselves to forecasting unit cost in this paper,for the simple reason that we have data for itand it is comparable across different technolo-gies. The work presented here provides a sim-ple benchmark against which to compare fore-casts of future technological performance based

6Arrow (1962), Alchian (1963), Argote & Epple(1990), Dutton & Thomas (1984), Thompson (2012).

7See Goddard (1982), Sinclair et al. (2000), Jamasb(2007), Nordhaus (2009).

8See for example Benson & Magee (2014b) for an ex-ample of how patent data can be used to explain varia-tion in rates of improvement among different technolo-gies.

on other methods.

The general approach of basing technologicalforecasts on historical data, which we pursuehere, stands in sharp contrast to the most widelyused method, which is based on expert opinion.The use of expert opinions is clearly valuable,and we do not suggest that it should be sup-planted, but it has several serious drawbacks.Expert opinions are subjective and can be biasedfor a variety of reasons, including common in-formation, herding, or vested interest (Albright2002, National Research Council 2009). Fore-casts for the costs of nuclear power in the US, forexample, were for several decades consistentlylow by roughly a factor of three (Cooper 2009).A second problem is that it is very hard to as-sess the accuracy of expert forecasts. In contrastthe method we develop here is objective and thequality of the forecasts is known. Nonethelesswe believe that both methods are valuable andthat they should be used side-by-side.

The remainder of the paper develops as fol-lows: In Section 2 we derive the error distribu-tion for forecasts based on the geometric randomwalk as a function of time horizon and othervariables and show how the data for differenttechnologies and time horizons should be col-lapsed. We also introduce the Integrated Mov-ing Average Model and derive similar (approx-imate) formulas. In Section 3 we describe ourdata set and present an empirical relationshipbetween the variance of the noise and the im-provement rate for different technologies. InSection 4 we describe our method of testing themodels against the data and present the results.We then apply our method to give a distribu-tional forecast for solar module prices in Sec-tion 5 and show how this can be used to forecastthe likelihood that one technology will overtakeanother. Finally we give some concluding re-marks in Section 6. A variety of technical resultsare given in the appendices.

5

2 Models

2.1 Geometric random walk

The generalized version of Moores law we studyhere is a postulated relationship which in its de-terministic form is

pt = p0et,

where pt is either the unit cost or the unit priceof a technology at time t; we will hereafter referto it as the cost. p0 is the initial cost and is theexponential rate of change. (If the technology isimproving then < 0.) In order to fit this todata one has to allow for the possibility of errorsand make an assumption about their structure.Typically the literature has treated Moores lawusing linear regression, minimizing least squareserrors to fit a model of the form

yt = y0 + t+ et, (1)

where yt = log(pt). From the point of view ofthe regression, y0 is the intercept, is the slopeand et is independent and identically distributed(IID) noise.We instead cast Moores law as a geometric

random walk with drift by writing it in the form

yt = yt1 + + nt, (2)

where as before is the drift and nt is an IIDnoise process. Letting the noise go to zero recov-ers the deterministic version of Moores law ineither case. When the noise is nonzero, however,the models behave quite differently. For the re-gression model the shocks are purely transitory,i.e. they do not accumulate. In contrast, if y0is the cost at time t = 0, Eq. 2 can be iteratedand written in the form

yt = y0 + t+t

i=1

ni. (3)

This is equivalent to Eq. 1 except for the lastterm. While in the regression model of Eq. 1 the

value of yt depends only on the current noise andthe slope , in the random walk model (Eq. 2) itdepends on the sum of previous shocks. Hence,shocks in the random walk model accumulate,and the forecasting errors grow with time hori-zon as one would expect, even if the parametersof the model are perfectly estimated.Another important difference is that because

Eq. 2 is a time series model the residual as-sociated with the most recently observed datapoint is by definition zero. For the regressionmodel, in contrast, the most recent point mayhave a large error; this is problematic since as aresult the error in the forecast for time horizon = 1 can be large. (Indeed the model gener-ally doesnt even agree with the data for t = 0,where pt is known).For time series models a key question is

whether the process is stationary, i.e. whetherthe process has a unit root. Most of our timeseries are much too short for unit root tests tobe effective (Blough 1992). Nonetheless, we findthat our time series forecasts are consistent withthe hypothesis of a unit root and that they per-form better than several alternatives.

2.2 Prediction of forecast errors

We assume that all technologies follow the samerandom process except for technology-specificparameters. Rewriting Eq. 2 slightly, it be-comes

yit = yi,(t1) + i + nit, (4)

where the index i indicates technology i. Forconvenience we assume that noise nit is IID nor-mal, i.e. nit N (0, K2i ). This means that tech-nology i is characterized by the drift i and thestandard deviation of the noise Ki. We will typ-ically not include the indices for the technologyunless we want to emphasize the dependence ontechnology.We now derive the expected error distribution

for Eq. 2 as a function of the time horizon .

6

Eq. 2 implies that

yt+ yt = +t+

i=t+1

ni (5)

The prediction steps ahead is

yt+ = yt + , (6)

where is the estimated . The forecast erroris defined as

E = yt+ yt+ . (7)

Putting Eqs. 5 and 6 into 7 gives

E = ( ) +t+

i=t+1

ni, (8)

which separates the error into two parts. Thefirst term is the error due to the fact that themean is an estimated parameter, and the sec-ond term represents the error due to the factthat unpredictable random shocks accumulate(Sampson 1991). Assuming that the noise incre-ments are i.i.d normal, and that the estimationof the parameters is based on m data points, inappendix B.1 we derive the scaling of the errorswith m, and K, where K2 is the estimatedvariance based on a trailing sample of m datapoints. To study how the errors grow as a func-tion of , because we want to aggregate forecasterrors for technologies with different volatilities,we use the normalized mean squared forecast er-ror (), which is

() E[

( EK

)2]

=m 1m 3

(

+ 2

m

)

. (9)

This formula makes intuitive sense. The termthat grows proportional to is the diffusiveterm, i.e. the growth of errors due to the noise.This term is present even in the limit m ,where the estimation is perfect. The term that is

proportional to 2/m is due to estimation error.The prefactor is due to the fact that we use theestimated variance, not the true one, implyingthat the distribution is not normal9 but ratheris Student t:

=1A

( EK

)

t(m 1) (10)

with

A = + 2/m. (11)

The linear term corresponds to diffusion due tonoise and the quadratic term to the propagationof errors due to misestimation of . Equation (9)is universal, in the sense that the right hand sidedepends neither on i nor Ki, hence it does notdepend on the technology. As a result, we canpool all the technologies to analyze the errors.Moreover, the right hand side of Eq. 10 doesnot even depend on , so we can pool differentforecast horizons together as well.

2.3 Integrated Moving Average

model

Although the random walk model above doessurprisingly well, and has the important advan-tage that all the formulas above are simple andintuitive, as we will see when we discuss ourempirical results in the next section, there isgood evidence that there are positive autocor-relations in the data. In order to incorporatethis structure we extend the results above for anARIMA(0,1,1) (autoregressive integrated mov-ing average) model. The zero indicates that wedo not use the autoregressive part, so we will

9The prefactor is different from one only when m issmall. Sampson (1991) derived the same formula butwithout the prefactor since he worked with the true vari-ance. Sampson (1991) also showed that the square termdue to error in the estimation of the drift exists for theregression on a time tend model, and for more generalnoise processes. See also Clements & Hendry (2001).

7

abbreviate this as an IMA(1,1) model in whatfollows. The IMA(1,1) model is of the form

yt yt1 = + vt + vt1, (12)

with the noise vt N (0, 2). When 6= 0 thetime series are autocorrelated; > 0 impliespositive autocorrelation.

We chose this model mainly because perform-ing Box-Jenkins analysis on the series for indi-vidual technologies tend to suggest it more oftenthan other ARIMA models (but less often thanthe pure random walk with drift10). Moreover,our data are often time-aggregated, that is, ouryearly observations are averages of the observedcosts over the year. It has been shown that if thetrue process is a random walk with drift thenaggregation can lead to substantial autocorre-lation (Working 1960). In any case, while ev-ery technology certainly follows an idiosyncraticpattern and may have a complex autocorrela-tion structure and specific measurement errors,using the IMA(1,1) as a universal model allowsus to parsimoniously understand the empiricalforecast errors and generate robust predictionintervals.

A key quantity for pooling the data is the vari-ance, which by analogy with the previous modelwe call K for this model as well. It is easy toshow that

K2 var(ytyt1) = var(vt+vt1) = (1+2)2,

see e.g. Box & Jenkins (1970). If we make theforecasts as before (Eq. 6), the distribution offorecast errors is (Appendix B.2)

E N (0, 2A), (13)10Bear in mind that our individual time series are very

short, which forces the selection of a very parsimoniousmodel, and likely explains the choice of the random walkmodel when selection is based on individual series in iso-lation.

with

A = 2 +(

1 +2(m 1)

m+ 2

)(

+ 2

m

)

(14)Note that we recover Eq. 11 when = 0. Inthe case where the variance is estimated, it ispossible to derive an approximate formula forthe growth and distribution of the forecast errorsby assuming that K and E are independent. Theexpected mean squared normalized error is

() E[

( EK

)2]

=m 1m 3

A

1 + 2, (15)

and the distribution of rescaled normalized fore-cast errors is

=1

A/(1 + 2)

( EK

)

t(m 1). (16)

Because the assumption that the numeratorand denominator are independent is true onlyin the limit m , these formulas are approx-imations. We compare these to more exact re-sults obtained through simulations in AppendixB.2 see in particular Figure 13. For m > 30the approximation is excellent, but there are dis-crepancies for small values of m.

3 Data

3.1 Data collection

The bulk of our data on technology costs comesfrom the Santa Fe Institutes Performance CurveDataBase11, which was originally developed byBela Nagy and collaborators; we augment itwith a few other datasets. These data were col-lected via literature search, with the principalcriterion for selection being availability. Figure2 plots the time series for each data set. Thesharp cutoff for the chemical data, for example,

11pcdb.santafe.edu

8

reflects the fact that it comes from a book pub-lished by the Boston Consulting Group in 1972.Table 1 gives a summary of the properties of thedata and more description of the sources can befound in appendix A.

1940 1960 1980 2000

1e

081e

05

1e

021e

+01

1e+

04

log(

cost

)

ChemicalHardwareConsumer GoodsEnergyFoodGenomics

Figure 2: Log cost vs. time for each technology inour dataset. This shows the 53 technologies out ofthe original set of 66 that have a significant rate ofcost improvement (DNA sequencing is divided by1000 to fit on the plot). The Figure makes the mot-ley character of our dataset clear, e.g. the technolo-gies span widely different periods. More details canbe found in Table 1 and Appendix A.

A ubiquitous problem in forecasting techno-logical progress is finding invariant units. Afavorable example is electricity. The cost ofgenerating electricity can be measured in dol-lars per kW/h, making it possible to sensiblycompare competing technologies and measuretheir progress through time. Even in this fa-vorable example, however, making electricitycleaner and safer has a cost, which has affectedhistorical prices for technologies such as coal andnuclear power in recent years, and means thattheir costs are difficult to compare to clean andsafe but intermittent sources of power such assolar energy. To take an unfavorable example,

our dataset contains appliances such as televi-sion sets, that have dramatically increased inquality through time12.One should therefore regard our results here

as a lower bound on what is possible, in the sensethat performing the analysis with better data inwhich all technologies had invariant units wouldvery likely improve the quality of the forecasts.We would love to be able to make appropriatenormalizations, but the work involved is pro-hibitive; if we dropped all questionable exampleswe would end with little remaining data. Mostof the data are costs, but in a few cases they areprices; again, this adds noise but if we were ableto be consistent that should only improve ourresults. We have done various tests removingdata and the basic results are not sensitive towhat is included and what is omitted (see Fig.15 in the appendix).We have removed some technologies that are

too similar to each other from the PerformanceCurve Database. For instance, when we havetwo datasets for the same technology, we keeponly one of them. Our choice was based on dataquality and length of the time series. This se-lection left us with 66 technologies, belongingto different sectors that we label as chemistry,genomics, energy, hardware, consumer durablesand food.

3.2 Data selection and descriptive

statistics

In this paper we are interested in technologiesthat are improving, so we restrict our analysisto those technologies whose rate of improvementis statistically significant based on the availablesample. We used a simple one-sided t-test onthe (log) series and removed all technologies forwhich the p-value indicates that we cant reject

12Gordon (1990) provides quality change adjustmentsfor a number of durable goods. These methods (typi-cally, hedonic regressions) require additional data.

9

the null that i = 0 at a 10% confidence level.Note that our results actually get better if wedo not exclude the non-improving technologies;this is particularly true for the geometric ran-dom walk model. We have removed the non-improving technologies because including tech-nologies that that do not improve swamps ourresults for improving technologies and makes thesituation unduly favorable for the random walkmodel.

Fre

quen

cy

0.0 0.2 0.4 0.6 0.8

02

46

810

12

~

Fre

quen

cy

0.0 0.2 0.4 0.6 0.8

05

1015

2025

K~

Fre

quen

cy

20 40 60 80

05

1015

20

T

Fre

quen

cy

1.0 0.5 0.0 0.5 1.0

02

46

8

~

Figure 3: Histogram for the estimated parametersfor each technology i based on the full sample (seealso Table 1). i is the annual logarithmic rate ofdecrease in cost, Ki is the standard deviation of thenoise, Ti is the number of available years of dataand is the autocorrelation.

Table 1 reports the p-values for the one sidedt-test and the bottom of the table shows thetechnologies that are excluded as a result. Table1 also shows the estimated drift rate i and theestimated standard deviation Ki based on thefull sample for each technology i. (Throughoutthe paper we use a hat to denote estimates per-formed within an estimation window of size mand a tilde to denote the estimates made usingthe full sample). Histograms of i, Ki, samplesize Ti and are given in Figure 3.

0 10 20 30 40 50

010

2030

4050

# of

tech

nolo

gies

0 10 20 30 40 50

010

020

030

040

050

060

070

0

# of

fore

cast

s

# technologies# of forecasts

Figure 4: Technologies, forecasts and time horizon.Because we use a hindcasting procedure the numberof possible forecasts that can be made with a givendata window m decreases as the time horizon ofthe forecast increases. For m = 5 we show the num-ber of possible forecasts as well as the number oftechnologies for which at least one forecast is pos-sible at that time horizon. The horizontal line at = 20 indicates our (somewhat arbitrary) choice ofa maximum time horizon. There are a total of 8212forecast errors and 6391 with 20.

Since our dataset includes technologies of dif-ferent length (see Table 1 and Figure 4), andbecause we are doing exhaustive hindcasting, asdescribed in the next section13, the number ofpossible forecasts is highest for = 1 and de-creases for longer horizons. Figure 4 shows thenumber of possible forecasts and the number oftechnologies as a function of the forecast hori-zon . We somewhat arbitrarily impose an up-per bound of max = 20, but find this makesvery little difference in the results (see Appendix

13The number of forecast errors produced by a tech-nology of length Ti is [Ti (m + 1)][Ti m]/2 whichis O(T 2

i). Hence the total number of forecast errors con-

tributed by a technology is disproportionately dependenton its length. However, we have checked that aggregat-ing the forecast errors so that each technology has anequal weight does not qualitatively change the results.

10

Technology Industry T p value K Transistor Hardware 38 -0.50 0.00 0.24 0.19

Geothermal.Electricity Energy 26 -0.05 0.00 0.02 0.15Milk..US. Food 79 -0.02 0.00 0.02 0.04DRAM Hardware 37 -0.45 0.00 0.38 0.14

Hard.Disk.Drive Hardware 20 -0.58 0.00 0.32 -0.15Automotive..US. Consumer Goods 21 -0.08 0.00 0.05 1.00

Low.Density.Polyethylene Chemical 17 -0.10 0.00 0.06 0.46Polyvinylchloride Chemical 23 -0.07 0.00 0.06 0.32Ethanolamine Chemical 18 -0.06 0.00 0.04 0.36

Concentrating.Solar Energy 26 -0.07 0.00 0.07 0.91AcrylicFiber Chemical 13 -0.10 0.00 0.06 0.02

Styrene Chemical 15 -0.07 0.00 0.05 0.74Titanium.Sponge Chemical 19 -0.10 0.00 0.10 0.61VinylChloride Chemical 11 -0.08 0.00 0.05 -0.22Photovoltaics Energy 34 -0.10 0.00 0.15 0.05

PolyethyleneHD Chemical 15 -0.09 0.00 0.08 0.12VinylAcetate Chemical 13 -0.08 0.00 0.06 0.33Cyclohexane Chemical 17 -0.05 0.00 0.05 0.38BisphenolA Chemical 14 -0.06 0.00 0.05 -0.03

Monochrome.Television Consumer Goods 22 -0.07 0.00 0.08 0.02PolyethyleneLD Chemical 15 -0.08 0.00 0.08 0.88Laser.Diode Hardware 13 -0.36 0.00 0.29 0.37

PolyesterFiber Chemical 13 -0.12 0.00 0.10 -0.16Caprolactam Chemical 11 -0.10 0.00 0.08 0.40

IsopropylAlcohol Chemical 9 -0.04 0.00 0.02 -0.24Polystyrene Chemical 26 -0.06 0.00 0.09 -0.04

Polypropylene Chemical 10 -0.10 0.00 0.07 0.26Pentaerythritol Chemical 21 -0.05 0.00 0.07 0.30

Ethylene Chemical 13 -0.06 0.00 0.06 -0.26Wind.Turbine..Denmark. Energy 20 -0.04 0.00 0.05 0.75

Paraxylene Chemical 12 -0.10 0.00 0.09 -1.00DNA.Sequencing Genomics 13 -0.84 0.00 0.83 0.26NeopreneRubber Chemical 13 -0.02 0.00 0.02 0.83Formaldehyde Chemical 11 -0.07 0.00 0.06 0.36SodiumChlorate Chemical 15 -0.03 0.00 0.04 0.85

Phenol Chemical 14 -0.08 0.00 0.09 -1.00Acrylonitrile Chemical 14 -0.08 0.01 0.11 1.00Beer..Japan. Food 18 -0.03 0.01 0.05 -1.00

Primary.Magnesium Chemical 40 -0.04 0.01 0.09 0.24Ammonia Chemical 13 -0.07 0.02 0.10 1.00Aniline Chemical 12 -0.07 0.02 0.10 0.75Benzene Chemical 17 -0.05 0.02 0.09 -0.10Sodium Chemical 16 -0.01 0.02 0.02 0.42Methanol Chemical 16 -0.08 0.02 0.14 0.29

MaleicAnhydride Chemical 14 -0.07 0.03 0.11 0.73Urea Chemical 12 -0.06 0.03 0.09 0.04

Electric.Range Consumer Goods 22 -0.02 0.03 0.04 -0.14PhthalicAnhydride Chemical 18 -0.08 0.03 0.15 0.31

CarbonBlack Chemical 9 -0.01 0.03 0.02 -1.00Titanium.Dioxide Chemical 9 -0.04 0.04 0.05 -0.41Primary.Aluminum Chemical 40 -0.02 0.06 0.08 0.39

Sorbitol Chemical 8 -0.03 0.06 0.05 -1.00Aluminum Chemical 17 -0.02 0.09 0.04 0.73

Free.Standing.Gas.Range Consumer Goods 22 -0.01 0.10 0.04 -0.30CarbonDisulfide Chemical 10 -0.03 0.12 0.06 -0.04Ethanol..Brazil. Energy 25 -0.05 0.13 0.22 -0.62

Refined.Cane.Sugar Food 34 -0.01 0.23 0.06 -1.00CCGT.Power Energy 10 -0.04 0.25 0.15 -1.00

HydrofluoricAcid Chemical 11 -0.01 0.25 0.04 0.13SodiumHydrosulfite Chemical 9 -0.01 0.29 0.07 -1.00

Corn..US. Food 34 -0.02 0.30 0.17 -1.00Onshore.Gas.Pipeline Energy 14 -0.02 0.31 0.14 0.62

Motor.Gasoline Energy 23 -0.00 0.47 0.05 0.43Magnesium Chemical 19 -0.00 0.47 0.04 0.58Crude.Oil Energy 23 0.01 0.66 0.07 0.63

Nuclear.Electricity Energy 20 0.13 0.99 0.22 -0.13

Table 1: Descriptive statistics and parameter estimates (using the full sample) for all available technologies,ordered by p-value of a one-sided t-test for . The improvement of the last 13 technologies is not statisticallysignificant and so they are dropped from further analysis.

11

C.3).

3.3 Relation between drift and

volatility

Figure 5 shows a scatter plot of the estimatedstandard deviation Ki for technology i vs. theestimated improvement rate i. This showsvery clearly that on average the uncertainty Kigets bigger as the improvement rate i in-creases. There is no reason that we are awareof to expect this a priori. One possible expla-nation is that for technological investment thereis a trade-off between risk and returns. Anotherpossibility is that faster improvement amplifiesfluctuations.

0.02 0.05 0.10 0.20 0.50

0.02

0.05

0.10

0.20

0.50

~

K~

ChemicalEnergyHardwareConsumer GoodsFood

loglog fitlinear fit

Figure 5: Scatter plot of and K for technologieswith a significant improvement rate. The linear fit(which is curved when represented in log scale) isK = 0.020.76, with R2 = 0.87 and a p-value 0.The log-log fit is K = 0.51()0.72, with R2 = 0.73and p-value 0. Technologies with a faster rate ofimprovement also have higher uncertainty in theirimprovement.

4 Empirical results

4.1 Statistical validation proce-

dure

We use hindcasting for statistical validation, i.e.for each technology we pretend to be at a givendate in the past and make forecasts for datesin the future relative to the chosen date14. Wehave chosen this procedure for several reasons.First, it directly tests the predictive power ofthe model rather than its goodness of fit to thedata, and so is resistant to overfitting. Second,it mimics the same procedure that one wouldfollow in making real predictions, and third, itmakes efficient use of the data available for test-ing.We fit the model at each time step to the m

most recent changes in cost (i.e. the most recentm+1 years of data). We use the same value ofmfor all technologies and for all forecasts. Becausemost of the time series in our dataset are quiteshort, and because we are more concerned herewith testing the procedure we have developedthan with making optimal forecasts, unless oth-erwise noted we choose m = 5. This is admit-tedly very small, but it has the advantage thatit allows us to make a large number of forecasts.We will return later to discuss the question ofwhich value of m makes the best forecasts.We perform hindcasting exhaustively in the

sense that we make as many forecasts as possiblegiven the choice of m. For convenience assumethat the cost data yt = log pt for technology iexists in years t = 1, 2, . . . , Ti. We then makeforecasts for each feasible year and each feasibletime horizon, i.e. we make forecasts yt0+ (t0)rooted in years t0 = (m + 1, . . . , Ti 1) withforecast horizon = (1, . . . , Ti t0).In each year t0 for which forecasts are made

the drift t0 is estimated as the sample mean of

14This method is also sometimes called backtestingand is a form of cross-validation.

12

the first differences

t0 =1

m

t01

j=t0m

(yj+1 yj) =yt0 yt0m

m, (17)

where the last equality follows from the fact thatthe sum is telescopic, and implies that only twopoints are needed to estimate the drift. Thevolatility is estimated using the unbiased esti-mator15

K2t0 =1

m 1t01

j=t0m

[(yj+1 yj) t0 ]2. (18)

4.2 Comparison of models to data

This procedure gives us a variable number offorecasts for each technology i and time horizon , rooted at all feasible times t0. We record theforecasting errors Et0, = yt+ (t0) yt+ (t0) andthe associated values of Kt0 for all t0 and all where we can make forecasts. We then test themodels by computing the mean squared normal-ized forecast error,

() = Et0

[

(Et0,Kt0

)2]

,

averaging over all technologies and for all fore-casts with < 20, as shown by the black dots inFigure 6. The dashed line compares this to thepredicted normalized error under the geometricrandom walk hypothesis.The random walk model does a good job of

predicting the scaling of the forecast errors as afunction of , but the predicted errors are toosmall by roughly a factor of two. The fact thatit underestimates the errors is not surprising

15This is different from the maximum likelihood esti-mator, which does not make use of Bessels correction(i.e. dividing by (m 1) instead of m). Our choice isdriven by the fact that in practice we use a very smallm, making the bias of the maximum likelihood estimatorrather large.

1 2 5 10 20

510

2050

100

200

500

Forecast horizon

()

real data

IMA( = 0.63, mean)

m 1

m 3

+

2

m

IMA( = 0.25, mean)

IMA( = 0.25, 95 % CI)

Figure 6: Growth of the mean squared normalizedforecast error () for forecasts on real data, com-pared to predictions. The empirical value of the nor-malised error () is shown by black dots. Thedashed line is the prediction of the random walkwith drift according to equation (9). The solid lineis based on the expected errors for the IMA(1,1)model using a value of = m = 0.63 that best fitsthe empirically observed errors. The dotted line isbased on = w = 0.25, which is a sample meanweighted by the specific number of forecasts ofeach technology. The shaded error corresponds tothe 95% error quantile using w. See appendix Dfor details on the estimation of .

the random walk model is an extreme assump-tion. The correct prediction of the scaling of theerrors as a function of the forecast horizon is notan obvious result; alternative hypotheses, suchas long-memory, can produce errors that grow intime faster than the random walk. Given thatlong-memory is a natural result of nonstation-arity, which is commonly associated with tech-nological change, our prior was that it was ahighly plausible alternative16. These procedures

16A process has long-memory if the autocorrelationfunction of its increments is not integrable. Under thelong-memory hypothesis one expects the diffusion termof the normalized squared errors to scale as () 2H ,

13

involve some data snooping, i.e. they use knowl-edge about the future to set , but since this isonly one parameter estimated from a sample ofmore than 6, 000 forecasts the resulting estimateshould be reasonably reliable. See the more de-tailed discussion in Appendix D.

To understand why the errors in the model arelarger than those of the random walk we inves-tigated two alternative hypotheses, heavy tailsand autocorrelated innovations. We found lit-tle evidence for heavy tails (appendix C.4) butmuch stronger evidence for positive autocorrela-tions. Testing for autocorrelation is difficult dueto the fact that the series are short, and the sam-ple autocorrelations of the individual samplesare highly variable and in many cases not verystatistically significant (see Figure 3). Nonethe-less, on average the individual series have posi-tive autocorrelations, suggesting that this is atleast a reasonable starting point.

We thus selected the IMA(1,1) model dis-cussed in Section 2.3 as an alternative to thesimple random walk with drift. This modelrequires an additional parameter , which de-scribes the strength of the autocorrelation. Be-cause we are using a small number m of pastdata points to fit the model, a dynamic sam-ple estimate based on a moving average is notstatistically stable.

To counter this problem we use a global valueof , i.e. we use the same value for all technolo-gies and all points in time. We use two differ-ent methods for doing this and compare them inwhat follows. The first method takes advantageof the fact that the magnitude of the forecasterrors is an increasing function of (we assume > 0) and chooses m to match the empirically

where H is the Hurst exponent. In the absence of long-memory H = 1/2, but for long-memory 1/2 < H < 1.Long-memory can arise from many causes, includingnonstationarity. It is easy to construct plausible pro-cesses with the parameter varying where the meansquared errors grow faster than 2.

observed forecast errors.

10 5 0 5 10

0.00

20.

005

0.02

00.

050

0.20

00.

500

cum

ulat

ive

dist

ribut

ion

120

Figure 7: Cumulative distribution of empiricalrescaled normalized forecast errors at different fore-cast horizons . The forecast errors for each tech-nology i are collapsed using Eq. 16 with =w = 0.25. This is done for each forecast horizon = 1, 2, . . . , 20 as indicated in the legend. Thegreen thick curve is the theoretical prediction. Thepositive and negative errors are plotted separately.For the positive errors we compute the number oferrors greater than a given value X and divide bythe total number of errors to estimate the cumula-tive probability and plot in semi-log scale. For thenegative errors we do the same except that we takethe absolute value of the error and plot against X.

The second method takes a weighted averagew calculated as follows: We exclude all tech-nologies for which the estimate of reveals spec-ification or estimation issues ( 1 or 1).Then at each horizon we compute a weightedaverage, with the weights proportional to thenumber of forecasts made with that technology.Finally we average the first 20 estimates. SeeAppendix D.

14

15 10 5 0 5 10 15

2e

041e

03

5e

032e

02

1e

015e

01

cum

ulat

ive

dist

ribut

ion

Student t(m1)IMA(1,1)RWD

Figure 8: Cumulative distribution of empiricalrescaled normalized forecast errors with all pooledtogether for both the autocorrelation IMA(1,1)model and the random walk. The empirical distri-bution when the normalization is based on a randomwalk (Eq. 10) is denoted by a dashed line and whenbased on an IMA(1,1) model (Eq. 16) is denotedby a solid line. See the caption of Figure 7 for adescription of how the cumulative distributions arecomputed and plotted.

Note that although we derived an asymptoticformula for the errors of the IMA(1,1) processequation (15), withm = 5 there is too little datafor this to be accurate. Instead we simulate theIMA(1,1) process to create an artificial data setthat mimics our real data17.

We then test to see whether we correctly pre-dict the distribution of forecast errors. Figure

17More specifically, for each technology we generateTi pseudo cost data points using Eq. 12 with = i,K = Ki and =

w = 0.25. We then estimate the pa-rameters just as we did for the real data and compute themean squared normalized error in forecasting the artifi-cial data as a function of . The curves corresponding tothe IMA models in Figure 6 are the result of doing this1000 times and taking the average. The error bars arealso computed using the simulations (95% of the simu-lated datasets ( = 0.25) have a mean square normalizedforecast error within the grey area)

7 shows the distribution of rescaled forecast er-rors at several different values of using theIMA(1,1) rescaling factor A (eq. 14) and com-pares it to the predicted distribution. Fig. 8shows the distribution with all values of pooledtogether and compares the empirical results forthe random walk with drift to the empirical re-sults with the IMA(1,1). The predicted distribu-tion is fairly close to the theoretical prediction,and as expected the IMA(1,1) is better than therandom walk with drift.

Given the complicated overlapping structureof the exhaustive hindcasting approach that weuse here, the only feasible approach to statis-tically testing our results is simulation18. Togive a feeling for the expected size of the fluc-tuations, in Appendix E we generate an ensem-ble of results under the null hypothesis of theIMA(1,1) model, generating an artificial data setthat mimics the real data, as described above.This gives a feeling for the expected fluctua-tions in Figure 8 and makes it clear that therandom walk can confidently be rejected. Theestimate based on the IMA(1,1) model with = w = 0.25 is also rejected, with no morethan 1% of the simulations generating deviationsas large as the real data (this depends on the wayin which deviations from the distribution aremeasured). Unsurprisingly the IMA(1,1) modelwith = m = 0.63 is not rejected (see Ap-pendix E for details).

4.3 Dependence on m

So far we have used a small value of m fortesting purposes; this allows us to generate alarge number of forecasts and thereby test to

18The fact that we use a rolling window approach im-plies overlaps in both the historical sample used to esti-mate parameters at each time t0 and overlapping inter-vals in the future for horizons with > 1. This impliessubstantial correlations in the empirical forecast errors,which complicates statistical testing.

15

see whether we are correctly predicting the dis-tributional forecasting accuracy of our method.We now address the question of the optimal

value of m. In a stationary world, in which themodels are well-specified and there is a constanttrue value of , one should always use the largestpossible value of m. In a non-stationary world,however, it can be advantageous to use a smallervalue of m, or alternatively a weighted averagethat decays as it goes into the past. We experi-mented with increasing m as shown in Figure 9and as described in more detail in Appendix C.1,and we find that the errors drop as m increasesroughly as one would expect if the process werestationary. Note that to check forecast errors forhigh m we have used technologies for which atleast m+ 2 years were available.

1 2 5 10 20

25

1020

5010

050

0

Forecast horizon

()

m

481216

EmpiricalIMA( = 0.25)

Figure 9: Mean squared normalized forecast error as a function of the forecast horizon for differentsizes of the learning window m.

5 Application to solar PV

modules

In this section we provide a distributional fore-cast for the price of solar photovoltaic modules.We then show how this can be used to make a

comparison to a hypothetical competing tech-nology in order to estimate the probability thatone technology will be cheaper than another ata given time horizon, and finally, we use solarenergy as an example to illustrate how extrap-olation can be useful to forecast production aswell as cost.

5.1 A distributional forecast for

solar energy

PV

mod

ule

pric

e in

201

3 $/

Wp

1980 1986 1992 1998 2004 2010 2016 2022 2028

0.02

0.05

0.12

0.33

0.9

2.46

6.69

1 1.5 2

Figure 10: Forecast for the cost of photovoltaicsmodules, in 2013 $/ Wp. The parameters used forthe point forecast and the error bars are taken fromTable 1, that is = 0.10, K = 0.15. For the errorbars we used = 0.63 and m = 33.

Figure 10 shows the predicted distribution oflikely prices for solar photovoltaic module fortime horizons up to 2030. To make this fore-cast, we use all available years (m = 33), andthe normal approximation Eq.13 with = m.The prediction says that it is likely that solarPV modules will continue to drop in cost at theroughly 10% rate that they have in the past.Nonetheless there is a small probability (about5%) that the price in 2030 will be higher than it

16

is now19.

5.2 Estimating the probability

that one technology will be

cheaper than another

Consider comparing the price of photovoltaicmodules yS with the price of an alternative tech-nology yC . We assume that both technologiesfollow an IMA(1,1) process with = m, andas previously we make forecasts using the sim-ple random walk with drift predictions. To keepthings simple we assume that the estimated vari-ance is the true variance, so that the forecasterror is normally distributed (Eq. 13). Assumethe estimated parameters for the two technolo-gies are S and KS for solar modules and Cand KC for the competing technology. We wantto compute the probability that steps aheadyS < yC. Under these assumptions

yS(t+ ) N (yS(t) + S, K2SA/(1 + 2S)),

where A() is defined in Eq. 14 with S =

m,and m = 33. Technology C is similarly defined,but for the sake of argument we assume thattechnology C has historically on average had aconstant cost, i.e. C = 0. We also assumethat the estimation period is the same, and thatC = S =

m. The probability that yS < yCis the probability that the random variable Z =yS yC is positive. Since yS and yC are normal,assuming they are independent their differenceis normal

Z N(

Z , 2Z

)

,

where Z = (yS(t) yC(t)) + (S C) and2Z = (A

/(1+ 2m ))(K2S + K

2C). The probability

that yS < yC is the integral for the positive part,

19This forecast is consistent with the one made severalyears ago by Nagy et al. (2013) using data only until2009.

which is expressed in terms of the error function

Pr(yS < yC) =

0

fZ(z)dz

=1

2

[

1 + Erf

(

Z2 Z

)]

.

(19)

0.0

0.2

0.4

0.6

0.8

Forecast horizon

Pro

b(y S

15.

C Robustness checks

C.1 Size of the learning window

4 6 8 10 12 14 16

23

45

m

()

= 1

4 6 8 10 12 14 16

510

1520

m

()

= 2

4 6 8 10 12 14 16

2040

6080

100

m

()

= 5

4 6 8 10 12 14 1650

150

250

350

m

()

= 15

Figure 14: Empirical mean squared normalized fore-cast errors as a function of the size of learning win-dow, for different forecast horizons. The dots arethe empirical errors, and the plain lines are thoseexpected if the true model was an IMA(1,1) with = w = 0.25.

The results are robust to a change of the sizeof learning window m. It is not possible to gobelow m = 4, as when m = 3 the Student distri-bution has m 1 = 2 degrees of freedom, hencean infinite variance. Note that to make fore-casts using a large m, only the datasets whichare long enough can be included. The results fora few values of m are shown in Figure 9. Figure

22

14 shows that the normalized mean square fore-cast error consistently decreases as the learningwindow increases.

C.2 Data selection

We have checked how the results change whenabout half of the technologies are randomly se-lected and removed from the dataset. The shapeof the normalized mean square forecast errorgrowth does not change and is shown in Figure15. The procedure is based on 10000 randomtrials selecting half the technologies.

1 2 5 10 20

510

2050

100

200

500

Forecast horizon

()

All technologiesHalf technologies: MeanHalf technologies: 95% C.I.

Figure 15: Robustness to dataset selection. Meansquared normalized forecast errors as a function of , when using only half of the technologies (26 out53), chosen at random. The 95% confidence in-tervals, shown as dashed lines, are for the meansquared normalized forecast errors when we ran-domly select 26 technologies.

C.3 Increasing max

1 2 5 10 20 50

510

2050

200

500

2000

Forecast horizon

()

Figure 16: Robustness to increasing max. Main re-sults (i.e as in Figures 6 and 8) using max = 73.We use again = 0.25

In the main text we have shown the results for aforecast horizon up to max = 20. Moreover, wehave used only the forecast errors up to max toconstruct the empirical distribution of forecasterrors in Figure 8 and to estimate in appendixD. Figure 16 shows that if we use all the fore-cast errors up to the maximum with = 73 theresults do not change significantly.

C.4 Heavy tail innovations

To check the effect of non-normal noise incre-ments on (), we simulated random walks withdrift with noise increments drawn from a Stu-dent distribution with 3 or 7 degrees of freedom.Figure 17 shows that fat tail noise increments donot change the long horizon errors very much.While the IMA(1,1) model produces a parallelshift of the errors at medium to long horizons,the Student noise increments generate larger er-rors mostly at short horizons.

23

1 2 5 10 20 50

12

510

2050

100

200

Forecast horizon

()

IMA(1,1)RWD, NormalRWD, t(df=3)RWD, t(df=7)

Figure 17: Effect of fat tail innovations on errorgrowth. The figure shows the growth of the meansquared normalized forecast errors for four models,showing that introducing fat tail innovations in arandom walk with drift (RWD) mostly increases er-rors only at short horizons.

D Procedure for selecting

the autocorrelation pa-

rameter

0.0 0.4 0.8

1.0

1.4

1.8

2.2

Z m*

5 10 15 20

0.24

50.

255

w(

)

w* = E(w())

Figure 18: Estimation of as a global parameter

We select in several ways. The first methodis to compute a variety of weighted means forthe i estimated on individual series. The mainproblem with this approach is that for sometechnology series the estimated was very closeto 1 or -1, indicating mis-specification or esti-

0.0 0.2 0.4 0.6

2

01

2

ln(

IMA^

RW

D^

)x10

0 1210

5 10 15 20

01

23

4

ln

(IM

A^ R

WD

^) x

100

EmpiricalSimulation: meanSimulation: 95% C.I.

Figure 19: Using the IMA model to make better fore-casts. The right panels uses = 0.25

mation problems. After removing these 8 tech-nologies the mean with equal weights for eachtechnology is 0.27 with standard deviation 0.35.We can also compute the weighted mean at eachforecast horizon, with the weights being equal tothe share of each technology in the number offorecast errors available at a given forecast hori-zon. In this case, the weighted mean w() willnot necessarily be constant over time. Figure 18(right) shows that w() oscillates between 0.24and 0.26. Taking the average over the first 20periods, we have w =

120

20=1 w() = 0.25,

which have used as our main estimate of inthe previous section, in particular, figures 6, 7and 8. When doing this, we do not mean to im-ply that our formulas are valid for a system withheterogenous i; we simply propose a best guessfor a universal .

The second approach is to select in order tomatch the errors. As before we generate an arti-ficial data set using the IMA(1,1) model. Largervalues of imply that using the simple randomwalk model to make the forecasts will result inhigher forecast errors. Denote by ()empi theempirical mean squared normalized forecast er-ror as depicted in Figure 6, and by ()sim,the expected mean squared normalized forecasterror obtained by simulating an IMA(1,1) pro-cess 3,000 times with a particular global valueof . We study the ratio of the these two,averaged over all 1 . . . max = 20 periods, i.e.Z() = 1

20

20=1

()empi()sim,

. The values shown in

24

Figure 18 (left) are based on 2000 artificial datasets for each value of . The value at which|Z 1| is minimum is at m = 0.63.We also tried to make forecasts using the

IMA model to check that forecasts are improved:which value of allows the IMA model to pro-duce better forecasts? We apply the IMA(1,1)model with different values of to make fore-casts (with the usual estimate of the drift term) and study the normalized error as a functionof . We record the mean squared normalizederror and repeat this exercise for a range of val-ues of . The results for horizons 1,2, and 10 arereported in Figure 19 (left). This shows that thebest value of depends on the time horizon .The curve shows the mean squared normalizedforecast error at a given forecast horizon as afunction of the value of assumed to make theforecasts. The vertical lines show the minima, at0.26, 0.40, and 0.66. To make the curves fit onthe plot, given that the mean squared normal-ized forecast error increases with , the valuesare normalized by the mean squared normalizedforecast error for = 0, that is, obtained whenassuming that the true process is a random walkwith drift. We also see that as the forecast hori-zon increases the improvement from taking theautocorrelation into account decreases (Fig. 19,right), as expected theoretically from an IMAprocess.

E Deviation of the empirical

CDF from the theoretical

Student t

In this section we check whether the deviationsof the empirical forecast errors from the pre-dicted theoretical distribution are due to sam-pling variability, that is, due to the fact thatour datasets provide only a limited number offorecast errors.

We consider the results obtained when pool-ing the forecast errors at all horizons, up to themaximum horizon max = 20. We construct theempirical CDF of this distribution, split its sup-port in 1000 equal intervals, and compare theempirical value in each interval k with the the-oretical value, taken from the CDF of Studentdistribution with m1 degrees of freedom. Thedifference in each interval is denoted by k.

Fre

quen

cy

2 4 6 8 10

020

040

060

0

Fre

quen

cy

0.0 0.2 0.4 0.6 0.8

050

015

00

()2

Fre

quen

cy

0.00 0.04 0.08

020

040

060

0

max

Fre

quen

cy

0 5 10 15

020

060

010

00

Fre

quen

cy

0.0 0.4 0.8 1.2

020

060

0

()2

Fre

quen

cy

0.02 0.06 0.10

020

040

0

max

Figure 20: Expected deviations from the Student dis-tribution. The histograms show the sampling distri-bution of a given statistic, and the thick black lineshows the empirical value on real data. The simu-lations use = 0.25 (3 upper panels) and = 0.63(3 lower panels)

Three different measures of deviation areused:

k |k|,

k(k)2, and maxk. These

measures are computed for the real empiricaldata and for 10,000 simulated similar datasets,using = w = 0.25 and =

m = 0.63. The re-sults are reported in Figure 20. It is seen that for = 0.25 the departure of the real data empiricalCDF from the theoretical Student CDF tendsto be larger than the departure expected for asimilar dataset. Considering

k |k|,

k(k)2,

and maxk, the share of random datasets witha larger deviation is 0.001,0.002,0.011. If we usea larger (m = 0.63), these numbers are 0.21,0.16, 0.20, so if we take a large enough wecannot reject the model based on the expected

25

departure of the forecast errors from the theo-retical distribution.

References

Alberth, S. (2008), Forecasting technology costsvia the experience curve: Myth or magic?,Technological Forecasting and Social Change75(7), 952983.

Albright, R. E. (2002), What can past tech-nology forecasts tell us about the future?,Technological Forecasting and Social Change69(5), 443464.

Alchian, A. (1963), Reliability of progresscurves in airframe production, Econometrica31(4), 679693.

Argote, L. & Epple, D. (1990), Learning curvesin manufacturing, Science 247(4945), 920924.

Arrow, K. J. (1962), The economic implicationsof learning by doing, The Review of EconomicStudies 29(3), 155173.

Bailey, A., Bui, Q. M., Farmer, J., Margolis, R.& Ramesh, R. (2012), Forecasting technologi-cal innovation, in ARCS Workshops (ARCS),2012, pp. 16.

Baker, E., Fowlie, M., Lemoine, D. & Reynolds,S. S. (2013), The economics of solar electric-ity, Annual Review of Resource Economics5, 387426.

Barbose, G., Darghouth, N., Weaver, S. &Wiser, R. (2014), Tracking the Sun VI: AnHistorical Summary of the Installed Price ofPhotovoltaics in the United States from 1998to 2012, NREL.

Benson, C. L. & Magee, C. L. (2014a), On im-provement rates for renewable energy tech-nologies: Solar PV, wind turbines, capacitors,

and batteries, Renewable Energy 68, 745751.

Benson, C. L. & Magee, C. L. (2014b), Quanti-tative determination of technological improve-ment from patent data, ESD Working PaperSeries 2014-29.

Blough, S. R. (1992), The relationship betweenpower and level for generic unit root tests infinite samples, Journal of Applied Economet-rics 7(3), 295308.

Boston Consulting Group (1972), Perspectiveson Experience, 3 edn, The Boston ConsultingGroup, Inc., One Boston Place, Boston, Mas-sachusetts 02106.

Box, G. E. & Jenkins, G. M. (1970), Time SeriesAnalysis: Forecasting and Control, Holden-Day, San Francisco.

BP (2014), BP statistical review of worldenergy.URL: http://www.bp.com/en/global/corporate/about-bp/energy-economics/statistical-review-of-world-energy.html

Chatfield, C. (1993), Calculating interval fore-casts, Journal of Business & EconomicStatistics 11(2), 121135.

Clements, M. P. & Hendry, D. F. (2001),Forecasting with difference-stationary andtrend-stationary models, Econometrics Jour-nal 4(1), 119.

Colpier, U. C. & Cornland, D. (2002), Theeconomics of the combined cycle gas turbine,an experience curve analysis, Energy Policy30(4), 309316.

Cooper, M. (2009), The economics of nuclear re-actors: Renaissance or relapse, Technical re-port, Vermont Law School.

26

Dutton, J. M. & Thomas, A. (1984), Treat-ing progress functions as a managerial op-portunity, Academy of Management Review9(2), 235247.

Funk, J. L. (2013), What drives exponential im-provements?, California Management Review55(3), 134155.

Funk, J. L. & Magee, C. L. (2014), Rapid im-provements with no commercial production:How do the improvements occur?, ResearchPolicy .

Goddard, C. (1982), Debunking the learn-ing curve, Components, Hybrids, and Manu-facturing Technology, IEEE Transactions on5(4), 328335.

Goldemberg, J., Coelho, S. T., Nastari, P. M. &Lucon, O. (2004), Ethanol learning curve, theBrazilian experience, Biomass and Bioenergy26(3), 301304.

Gordon, R. J. (1990), The Measurement ofDurable Goods Prices, University of ChicagoPress.

International Energy Agency (2014), Technol-ogy roadmap: Solar photovoltaic energy (2014ed.), Technical report, OECD/IEA, Paris.

Jamasb, T. (2007), Technical change theoryand learning curves: patterns of progress inelectricity generation technologies, The En-ergy Journal 28(3), 5171.

Koh, H. & Magee, C. L. (2006), A func-tional approach for studying technologicalprogress: Application to information tech-nology, Technological Forecasting and SocialChange 73(9), 10611083.

Koh, H. & Magee, C. L. (2008), A functionalapproach for studying technological progress:

Extension to energy technology, Technologi-cal Forecasting and Social Change 75(6), 735758.

Koomey, J. & Hultman, N. E. (2007), Areactor-level analysis of busbar costs for USnuclear plants, 19702005, Energy Policy35(11), 5630 5642.

Kost, C., Mayer, J., Thomsen, J., Hartmann,N., Senkpiel, C., Philipps, S., Nold, S., Lude,S., Saad, N. & Schleg, T. L. (2013), Levelizedcost of electricity: Renewable energy tech-nologies., Technical report, Fraunhofer Insti-tute for Solar Energy Systems.

Lee, Y. S. & Scholtes, S. (2014), Empiricalprediction intervals revisited, InternationalJournal of Forecasting 30(2), 217234.

Lieberman, M. B. (1984), The learning curveand pricing in the chemical processing in-dustries, The RAND Journal of Economics15(2), 213228.

Lienhard, J. H. (2006), How invention begins:Echoes of old voices in the rise of new ma-chines, Oxford University Press.

Lipman, T. E. & Sperling, D. (1999), Expe-rience curves for policy making: the case ofenergy technologies.

Makridakis, S. & Winkler, R. L. (1989), Sam-pling distributions of post-sample forecastingerrors, Applied Statistics 38(2), 331342.

McDonald, A. & Schrattenholzer, L. (2001),Learning rates for energy technologies, En-ergy Policy 29(4), 255261.

McNerney, J., Farmer, J. D. & Trancik, J. E.(2011), Historical costs of coal-fired electric-ity and implications for the future, EnergyPolicy 39(6), 30423054.

27

Moore, G. E. (1965), Cramming more compo-nents onto integrated circuits.

Moore, G. E. (2006), Behind the ubiquitous mi-crochip.

Nagy, B., Farmer, J. D., Bui, Q. M. &Trancik, J. E. (2013), Statistical basis forpredicting technological progress, PloS one8(2), e52669.

Nagy, B., Farmer, J. D., Trancik, J. E. &Gonzales, J. P. (2011), Superexponentiallong-term trends in information technology,Technological Forecasting and Social Change78(8), 13561364.

National Research Council (2009), PersistentForecasting of Disruptive Technologies., TheNational Academies Press.

Neij, L., Andersen, P. D., Durstewitz, M.,Helby, P., Hoppe-Kilpper, M. & Morthorst, P.(2003), Experience curves: A tool for energypolicy assessment, Environmental and EnergySystems Studies, Univ.

Nemet, G. F. (2006), Beyond the learningcurve: factors influencing cost reductions inphotovoltaics, Energy Policy 34(17), 32183232.

Nordhaus, W. D. (2007), Two centuries of pro-ductivity growth in computing, The Journalof Economic History 67(01), 128159.

Nordhaus, W. D. (2009), The perils of the learn-ing model for modeling endogenous techno-logical change, Technical report, National Bu-reau of Economic Research.

Perlin, J. (1999), From space to earth: the storyof solar electricity, Earthscan.

Sahal, D. (1979), A theory of progress func-tions, AIIE Transactions 11(1), 2329.

Sampson, M. (1991), The effect of parameteruncertainty on forecast variances and confi-dence intervals for unit root and trend sta-tionary time-series models, Journal of Ap-plied Econometrics 6(1), 6776.

Schilling, M. A. & Esmundo, M. (2009), Tech-nology s-curves in renewable energy alterna-tives: Analysis and implications for industryand government, Energy Policy 37(5), 1767 1781.

Sinclair, G., Klepper, S. & Cohen, W. (2000),Whats experience got to do with it? sourcesof cost reduction in a large specialty chemicalsproducer, Management Science 46(1), 2845.

Taylor, J. W. & Bunn, D. W. (1999), A quantileregression approach to generating predictionintervals, Management Science 45(2), 225237.

Thompson, P. (2012), The relationship be-tween unit cost and cumulative quantity andthe evidence for organizational learning-by-doing, The Journal of Economic Perspectives26(3), 203224.

Wetterstrand, K. (2015), DNA sequencingcosts: Data from the NHGRI genome se-quencing program (GSP).URL: www.genome.gov/sequencingcosts

Williams, W. & Goodman, M. (1971), A sim-ple method for the construction of empiri-cal confidence limits for economic forecasts,Journal of the American Statistical Associa-tion 66(336), 752754.

Working, H. (1960), Note on the correlationof first differences of averages in a randomchain, Econometrica 28(4), pp. 916918.

Wright, T. P. (1936), Factors affecting the costof airplanes, Journal of the Aeronautical Sci-ences 3(4), 122128.

28

Zhao, J. (1999), The diffusion and costs of nat-ural gas infrastructures.

29