ORIGINAL ARTICLE Open Access

Quantitative scenario design with Bayesianmodel averaging: constructing consistentscenarios for quantitative modelsexemplified for energy economicsMonika Culka

Abstract

Background: Scenario design is currently not a standardised process. The formulation of storylines representingdifferent dimensions (for example economic or societal developments) demands an investigation of assumptioncompatibility, coherence, and consistency. Scenario techniques that use expert opinion as the sole informationsource are particularly appropriate for personal decisions. Contexts where scenarios serve as decision support on asocietal level—for example in political decision-making—benefit from unbiased, fact-depicting, multi-dimensionalinformation that is available in statistical data.

Methods: The presented approach uses the well-established method of Bayesian model averaging for theformulation of consistent, transparent, and intuitively understandable quantitative scenario assumptions. Theseassumptions are used in quantitative models to produce outlooks and forecasts. Illustrated by the example ofquantitative energy models used to investigate developments of the energy system by scenario technique, theapproach contrasts with other scenario methods. Bayesian model averaging (BMA) is a method that allows for anevaluation of both system relation stability in terms of observable co-evolvement of phenomena in the past and offuture system states of interest based on expert opinion where past evolvements serve as a point of reference.

Results: The results are scenarios assessable with respect to (1) the consistency of scenario assumptions in terms ofstatistical confirmation, (2) the suitability of a quantitative model to represent the scenario, and (3) the statisticaluncertainty of the scenario for a given quantitative model. A transparent scenario construction process results intraceable assumption documentation (an exemplary communication is provided in the Appendix). Perhaps, themost important novelty of the approach is the possibility of communicating to decision-makers the associateduncertainty in easily understandable terms. The distinction between provable possible assumptions (based onstatistical evidence) and hypothetical assumptions is a novelty and significantly improves the aptitude of scenariostudy recipients to evaluate scenarios on their part.

Conclusions: BMA provides the possibility for decision-makers (and all recipients of outlooks based on scenariotechnique) to trace back results to assumptions and provide an evaluation of these assumptions in terms ofstatistical confirmation. As such, the approach adds to the currently limited methodological diversity in scenarioconstruction techniques.

Keywords: Scenario technique, Uncertainty modelling, Assumption consistency, Empirical adequacy

Correspondence: energy.environment.engineer@gmail.comNICA New Interdisciplinary Collaboration Association, 6330 Kufstein, Austria

Energy, Sustainabilityand Society

© The Author(s). 2018 Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, andreproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link tothe Creative Commons license, and indicate if changes were made.

Culka Energy, Sustainability and Society (2018) 8:22 https://doi.org/10.1186/s13705-018-0162-3

http://crossmark.crossref.org/dialog/?doi=10.1186/s13705-018-0162-3&domain=pdfhttp://orcid.org/0000-0002-8580-9452mailto:energy.environment.engineer@gmail.comhttp://creativecommons.org/licenses/by/4.0/

BackgroundFormulating scenarios is a relevant part of future re-search. This paper aims to contribute to the currentlylimited toolbox of scenario construction methodologies.The Bayesian model averaging (BMA) technique is awell-established methodology today and, as I will argue,is an appropriate conceptual setting for consistent sce-nario construction for application cases where (some)cause-effect relations are uncertain and the mathemat-ical representation in models should account for thatuncertainty. Exemplified for the case of energy model-ling in this paper, the idea of the BMA scenario tech-nique is that what is observed in the past (documentedby statistical data) is a proven possible state of the world.A state of the world that observably (re-)occurred in thepast is more probable and less uncertain than a state ofthe world that has not been observed before.However, in many circumstances, investigating unpre-

cedented situations is the very reason for creating sce-narios! BMA offers a way that uses “knowledge of thepast” about parts of the world—say, the number of un-employed people documented in statistical data—to for-mulate expectations about these parts of the world(unemployment rate) in different states of accompanyingphenomena. It is important to understand that the stat-istical method tries to find relations of phenomenaexpressed as statistical data, based on similarities or dif-ferences in the changes these phenomena undergo. It isthe expected impact of an assumption on other assump-tions of the scenario, given the data record we consider.The technique is particularly suitable for scenarios thatfigure as assumptions for consequent processing inquantitative models, e.g. optimization models and simu-lation models.In contrast to most scenario techniques, BMA does

not primarily rely on expert judgement. I emphasise thatjudgement-based scenarios are suitable in different con-texts. In the case of energy scenarios, the need for tech-niques improving known difficulties associated withjudgement-based scenario design demands empirical evi-dence as a further source of information. In the follow-ing discussion, I will present the BMA method forconsistent scenario construction from a mainly concep-tual perspective. Constructing scenarios is generally bestperformed with a perspective on the specific applicationcase and purpose of the scenario. This implies that thescenario construction process should rely on differentmethodologies, in particular, qualitative techniques, toavoid an overemphasis of statistics. At the same token,applying only qualitative judgement-based approachesrisks neglecting evidence and promotes an apodictic ex-pert opinion. Choosing the appropriate methodologies ina scenario construction process remains the main taskof scenario designers.

The following discussion is a conceptual discussion. Incontrast to a technical presentation of a method, the focuslies here on arguing for Bayesian model averaging in thecontext of quantitative models. That means, for this paper,I explain how to make sense of BMA results in scenarioconstruction, and not primarily how to derive a BMA ana-lysis. For an example of a BMA analysis, I would like torefer to [1]. A detailed methodological discussion of BMAin terms of mathematical formulation and computationoptions is given in [2–9], to name just a few. Fragoso et al.[10] present a taxonomy of BMA literature by means of ameta-analysis of published works. While earlier works ofmine could be viewed as Fragoso et al.’s usage category“joint estimation”, the present discussion would match the“joint prediction” category, which is justifiably a separatecategory.1 Though the exemplary application case is againenergy-economic modelling, the inferences drawn fromapplying BMA for scenario design are novel.The focus of the paper is to present BMA as a technique

to compute scenarios. This is particularly suitable in con-texts where mathematical relations are approximations andthe modelled entities’ behaviour is uncertain. In contrast,the modelling of parts of the world that obey laws of natureis often straightforward in mathematical terms. A model ofa dropped ball is a precise mathematical relation forexample. The results computed with such a model depictempirically well-confirmed system states for known param-eters (e.g. gravitational pull) and variables (e.g. weight of theball). The computed model results are quite precise—ex-pectation for, say, the ball’s position at time t. It is insensibleto repeat a ball throw over and over to gather statisticaldata when we know a mathematical relation describing therelations precisely, a mathematical function. But when itcomes to scenarios depicting (also) human decision(s), forexample, scenarios in social sciences, then we lack a precisemathematical formulation as humans can decide differentlyat any time. The behaviour of humans and the consequent“behaviour” of relevant variables depicted in statistical data(e.g. GDP, trade balance, demand) can only be approxi-mated. This is what BMA, as a statistical method, does.BMA approximates with a view on the statistically evidentbehaviour in the past. Judgement-based scenario techniquesreflect the expert understanding of the phenomena’s behav-iour in the past.In the “Energy (economic) modelling as particular

context” section, I investigate the context of energy sce-narios. In the sections “Quantitative and qualitative sce-nario construction” and “Scenario definition anduncertainty evaluation”, I review multiple scenario con-struction techniques to contrast them with each otherand to relate them to the requirements for scenario con-struction in the energy modelling context. A basicprinciple depicts scenario design in terms of phenomenaand energy model boundaries. Using an example, my

Culka Energy, Sustainability and Society (2018) 8:22 Page 2 of 21

discussion of consistent scenario construction extends totwo cases: an existing energy model and the case of ascenario-adapted energy model in the “Consistent sce-nario construction for a given energy model and anadapted energy model” section. The BMA results are nu-merical assumptions necessary for quantitative energymodels as an input as discussed in the section “Results:consistent numerical value estimation”. The followingsection consists of a critical discussion of the approachand its limits in detail. Based thereon, I draw conclusionsand end with a brief summary. The Appendix is an ex-emplary communication of scenario assumptions.The hope is that this paper helps to acknowledge that

scenarios depicting a future world should also respectthe world of the past and the present. Here, BMA wouldbe one way to do so.

Energy (economic) modelling as particular contextThe energy system of a country is interrelated with dif-ferent societal aspects forming “systems” on their part.Economic, social, environmental, and governmental pol-icies influence the design and desired changes of an en-ergy system. Stakeholders are present in all societal“categories”, for example, industry, public, government,or non-governmental organisations. An adequate designand adaptation of the energy system to the changingneeds of a society are in the interest of all stakeholders.The priorities may, however, vary according to stake-holder objectives, planning, and societal duties.Changes to an energy system cannot be experimentally

tested, as compared to the design of a physical experi-ment. Implementing “new” policies is a delicate processthat needs to balance economic feasibility, societal accept-ance, industrial attractiveness, and political rigour. Inaddition, international agreements, such as the security ofsupply agreements or environmental protection agree-ments, demand for strategies that are respectful in regardto both the accepted duties and their practicability.Investigating potential consequences of policy mea-

sures for different stakeholders of an energy system interms of monetary, technical, environmental, and socialburdens has become a major concern of quantitative en-ergy modelling for policy advice. Assisting the impactevaluation for policy advice is a central role of quantita-tive energy modelling [11, 12]. To account for differentpossibilities, scenarios are developed representing a setof numerical assumptions interpreted in a narrative way,the so-called storyline. What the term “scenario” refersto is not clearly defined in the literature.Van Notten discusses 11 definitions and application ex-

amples for scenarios [13]. Lindgren debates paradoxicalsituations and practical indications of the scenario tech-nique [14]. Van Notten and also Lindgren accord to thescenario technique qualities as intuitiveness, creativity,

associational thinking, causal relation assumptions, andother possibly non-standardised characteristics. The mainobjective of scenarios is to create a set of assumptionsrepresenting a state of the world of interest, used for theevaluation of future developments [15]. Önkal et al. haveaddressed the difference between method-based statisticalforecasting and the scenario technique. According tothem, scenario technique reflects plausible futures basedon the reasoning of the scenario designer [16].In quantitative modelling, possible future states of the

energy system are limited to some defined input scenarios,also called storylines or key assumptions, what implies asubjective and decisive pre-selection of futures scrutinisedwith an energy model. This is a delicate process thatshould involve expert knowledge, and rigorous attentionmust be paid to plausibility. Individually stipulated as-sumptions may, in concert with other individually plaus-ible assumptions, amount to implausibility due toreciprocal assumption impact. An energy model, designedto represent an existing energy system, is typically appliedto investigate potential consequences for the target sys-tem, given things were as assumed in a scenario. However,due to the interrelated nature of the target system, experi-mental confirmation of scenario assumptions is limited, ifnot impossible. Therefore, the assumptions figuring in ascenario cannot solely be derived from intuitive scenariomethodologies, if the energy model results should repre-sent a provable possible or even probable energy systemstate. I refer to energy models as quantitative descriptionsof an existing energy system, e.g. [17–19]. The literatureon existing energy models is given for example in [20–22],or [12], where reviews and evaluations are published.The method proposed for scenario construction ad-

dresses the problem of scenario representation in energymodels and evaluates the scenario assumptions for a givenenergy model in terms of their empirical adequacy. Theempirical adequacy of an assumption is its propensity torepresent possible states of the world as confirmed by stat-istical evidence.2 In other words, I take consistent scenarioconstruction to mean that numerical assumptions areconsistent with statistically evident stable relations in thetarget system. The transparent documentation and com-munication of the assumptions’ statistical confirmationcan help recipients find their own opinion of a scenario.Energy model results are typically presented as energy

scenario studies, for example [23]. A consistent scenarioconstruction as an accompanying document is an uncer-tainty assessment, as presented in [24], as well as theconsequent predictive density computations, the scenar-ios. The scenarios come thus automatically with an un-certainty estimation for the specific energy model andthe specific scenarios computed with it.In other words, consistent BMA scenarios estimate a

quantitative (energy) model’s suitability to represent a

Culka Energy, Sustainability and Society (2018) 8:22 Page 3 of 21

scenario. Consistent scenario construction can assess sce-narios of a specific energy model in terms deemed rele-vant by Goodwin [25] p. 7: transparency (what are therelevant phenomena according to the data), the ease ofjudgement (how good are the relevant phenomena cap-tured in the energy model—both quantitative via posteriormodel probability (PMP) and qualitative via posterior in-clusion probability (PIP)), the versatility (the BMA can beapplied to many quantitative models), the flexibility (pro-vided statistical data are available, and different phenom-ena can be included in the statistical analysis), andtheoretical correctness (the mathematical core of theBMA is set, applying BMA means exercising that theoryon the data). I will return to these criteria by Goodwin inthe conclusion section and discuss the BMA methods’ ap-titude as a “formal strategy evaluation process within thescenario planning”.The general characteristic of scenario construction

that is specific to the energy modelling context is a tightconnection of the scenario to the actual world. In otherwords, scenarios modelling potential energy futures are(partially) used as a replacement of experiments (whichcannot be carried out) and serve as concrete guidance indecision support. This places requirements on theemployed scenario technique in terms of empirical ad-equacy, as the purpose of energy scenario studies is anevaluation of actual, possible, and plausible future op-tions, which decision-makers may have to consider.

Methods: quantitative and qualitative scenarioconstructionFor clarity, I would like to start with a clarification of theterminology used. I take a phenomenon to be either aphysically observable or an invisible socially emerged con-stellation of parts of reality that are naturally interrelated.Physically observable phenomena are quantifiable viameasurements and/or observation records. Social phe-nomena are observable and quantifiable via an interroga-tion and/or observation record. An observation record,also called empirical evidence, is in this case the statisticaldata. In fact, a phenomenon may exhibit different empir-ical evidence of itself. Statistical data have the advantageabove personal observations that they are collected sys-tematically, according to a method, and data observed thesame aspects of a phenomenon over time. This methodo-logical transparency of statistical data serves as commonground for different persons to speak about reality. How-ever, one must not think that statistical data describe orcapture a phenomenon exhaustively or even just appropri-ately. They are merely a basis allowing different people tospeak about the same aspects of the target system.In this discussion, an energy model is a mathematical

representation of an energy system with the aim to de-pict a real energy system simplified and idealised, but

nonetheless empirically adequate. The term energysystem refers to the part of reality that is (1) physicallyexistent in the world used to generate and deliver energy(e.g. electricity, heat); (2) economically associated withthe processes of generating, transporting, and consum-ing energy; (3) socially related to the effects induced byenergy consumption and access (e.g. fuel poverty); (4)related to environmental phenomena (e.g. change ofgaseous composition of the atmosphere due to energysector CO2 emissions); and (5) part of individual humanreality, i.e. a human is aware that the energy model andthe scenarios represent a part of her reality. The energysystem (ESS) is part of the world (WSS), and the worldis part of the universe.In Fig. 1, a subset illustrates being “part of” the larger

reality, to which I refer to as the target system. The energysystem is not naturally demarcated from any other systemof the world, and the world is not naturally separated fromany other system of the universe. The energy model de-picts parts of the energy system and all interrelated sys-tems of the world by stipulation, expressed in the energymodel design and energy model boundaries. Energy modelboundaries are an artificial demarcation between naturallyinterrelated phenomena. The energy model design isbasically the choice of input variables, parameters, andoutput variables representing the phenomena in WSS.Statistical data are quantitative, i.e. continuous or discreteor qualitative, i.e. categorical (nominal and ordinal)descriptions of phenomena in WSS according to thedefinition of the data collecting institution.3

I take a true data generating process to be a process thatcausally influences the numerical appearance of statisticaldata. A true data generating process can be artificial, forexample, if the data are created to test statistical methods,or natural, if known or unknown phenomena in WSScause the data recorded. For example, let our data be thecoordinates of a ball at a time. If the ball is thrown, thetrue data generating process is the force influencing theball, which “changes” the coordinates’ numbers—the datawe collect. The relation in WSS of the phenomena can,for the ball example, be described by a mathematical for-mulation using laws of classical mechanics. Depending onthe phenomena described by a statistical data point, thenumber of true data generating processes can vary andtheir causal status too. For example, the statistical numer-ical description of the phenomenon GDP is caused by, orcorrelated with, different phenomena in WSS as for ex-ample consumer satisfaction, trade balance, tax burdens,unemployment, etc. Statistical methods typically aim atidentifying what phenomena are influential. The proposedBMA method does so, too.Scenario construction has so far not been considered

as a scientific area itself, and to an extent it shouldhave been [26]. Some independently developed

Culka Energy, Sustainability and Society (2018) 8:22 Page 4 of 21

methodologies, techniques, and quality standards haverecently emerged. Research addressing the crucial roleof scenario construction and the difficulty in classify-ing the diverse techniques has been undertaken, forexample, by [27–29]. Scenario construction is the sys-tematic choice of numerical values for exogenous vari-ables (input variables) and parameters of an energymodel as assumptions. Scenarios constructed to deriverecommendations for decision support necessitate abalance between confirmed possibility and hypothet-ical assumption creation.Unfortunately, in energy-economic modelling, the

current practice of scenario construction is oftenopaque and unsystematic [30]. Sometimes, scenariosare defined, that is, agreed upon by modellers andsponsors. The consequently stipulated numericalvalues for assumptions translate the storyline. Withopaque scenario construction, a “result design” forsponsors is also possible, what is in my view scien-tific misconduct. Evaluating the representation qual-ity of the agreed storyline in a specific energy modelis rarely addressed. The proposed method does so bylooking at the energy model input variables’ abilityto represent the scenario. If the defined scenariosare not based on a systematic analysis of interrela-tions in the target system, the truth and legitimacyof the claim that energy model results represent a re-sponse of the energy system to the scenario cannot beevaluated. BMA scenarios for input variables allowfor the construction and evaluation of consistentscenarios based on observable energy system rela-tions in the past using statistical data.Methodologies and techniques reported in the literature

are presented in Table 1. The comparison of scenario tech-niques is based on the categorisation introduced by [31].The following series of arguments addresses the ques-

tion as to whether the proposed BMA technique is asensible addition to the purely qualitative approaches ofthe scenario method. To do so, I highlight some aspectsof opinion-based approaches which are implicit to them.I emphasise that these aspects are not erroneous inthemselves. They may serve as an advantage in scenariodesign in some contexts. The basic claim I argue for isthat scenario construction techniques which use opinionas a sole source of information fall short of empirical ad-equacy required for decision support and policy advice.Scenarios based on opinion

1. Lack a democratic perspective,2. Lack the possibility to evaluate the scenario quality,3. Suffer from detrimental psychological effects in the

context of decision support,4. Cannot reflect the target system’s complexity due to

limitations of human reasoning capacity.

I recall that the particular context of energy modellinguses scenarios as a basis for decision support, replacingexperiments. In other words, scenarios computed with en-ergy models may lead to political decisions influencingreal people in the real world. Scenario construction tech-niques which suffer extensively from the four points raisedin this section bias the futures investigated, which couldhave far-reaching and society-relevant consequences.Constructing scenarios for decision support with conse-

quences surpassing the realm of personal experience of thescenario constructor places in a sense an obligation on thescenario to account for the interests of all people affected. Ifthe information source of scenarios is an expert opinion, theconstructed scenarios are necessarily and inevitably biasedtowards the personal situation of the expert(s). In a democ-racy, however, possible (probable, plausible, and consistent)futures presented to decision-makers ought to envisionfutures respectful of all stakeholders. Statistical data as an in-formation source, in a sense, encode stakeholder choices andimplicitly reflect different interests, for instance, technologyacceptance, economic commitment to changes, ecologicalconcerns, institutional (personal, societal) priorities, etc.Many statistical data are available from trustworthy sourcesthat collect data non-discriminatory, regularly, reliably, meth-odologically sound, and freely available.Moreover, statistical data connect conceivable future

scenarios to relevant constraints. Relating the thinkable tothe feasible, and showing where changes are necessary torender scenarios feasible, can be achieved in a straightfor-ward manner based on statistical evidence. For example,the statistical information of average processing times forconstruction permits of electricity transportation facilitiescould constrain scenarios of generation capacity increaserealistically. As a transparent assumption in a scenario, itis at the same time the action recommended todecision-makers; the assumption that processing times forconstruction permits are stipulated shorter (or equal, lon-ger) than statistically evident in the scenario can be com-municated in detail.4

The second and the third point concern the problemthat expert elicitation as an information source for sce-nario construction cannot be evaluated in terms of quality.There are no standards as to who should be considered asan “expert” and no criteria for the status of being an ex-pert. There are no requirements for group design and en-vironmental design to prevent psychological effectsreported in the literature [32–35]. Although standardshave been proposed [36], it is not a common practice toaccompany the judgement-based scenario constructionwith a methodological assessment, and more importantly,the quality of the standards is itself a question of opinion.In contrast, statistical data can be evaluated in several di-mensions. Time, scope, collecting agency, post-processing,data arrangement, accessibility, and financing of the data

Culka Energy, Sustainability and Society (2018) 8:22 Page 5 of 21

Table 1 Comparison of scenario design methodologies and techniques adapted from [31]

Methodologicalapproach

Technique Keywords or a (very) brief description Source of information

Judgement Genius forecasting “Think the unthinkable” [55] Opinion

Visualisation Intuitive images are combined to scenarios that are in juxtapositionto analytical strategies, e.g. [56]

Opinion

Role playing A group judgement technique where individuals create a responseto a hypothetical situation which is considered as a scenario.Playing the devil’s advocate is an example of scenario forming withfocus on unprecedented or highly unlikely scenarios [57].

(Group) opinion

Coates and Jarratt For a given time frame and domain, four to six scenario themesregarding the most significant kinds of potential futuredevelopments based on judgement are formulated, cf. [58]

Opinion

Baseline scenario:the expected future

Trend extrapolation Measures existing trends and extrapolates effects into the future;both judgement and empirical analyses are possible. The Manoatechnique defines three strong trends which are analysed w.r.t.their implications separately, and in conjunction using across-matrix, cf. [26]

Opinion orstatistical data

Elaboration offixed scenarios

Incasting Based on an (extreme) state of the world participants judgepotential impacts in various respects as politics, economics, etc.Qualitative or quantitative for example life cycle assessmenttechnique [59]

Opinion/data

SRI matrix (StanfordResearch Institute matrix)

From a column-wise classification of fixed scenarios as for exampleworst case or expected future the dimensions (e.g. population,environment) are evaluated row-wise, cf. [60]

Opinion

Event sequences Probability trees andscenario trees

Different future conditions constitute individual paths which areassigned probabilities. Probability trees are used in risk management.A related technique is scenario trees where a reduction of probablepaths to relevant paths is carried out, cf. [61]

Opinion

Intuitive scenario building Probability trees are evaluated to identify characteristics that arecommon to several branches. Summarising these branches isconsidered as a way to create coherent scenarios, e.g. sociovision [62]

Opinion

Divergence mapping A set of events, derived from brainstorming, is aligned in differenttime horizons forming the storyline of a scenario. The relation ofearlier events to the later events is seen to be a plausiblesequence, cf. [63]

Opinion

Backcasting Horizon missionmethodology

Supposing a hypothetical situation (the scenario) was an actualsituation, the ways and necessary components at present areanalysed to achieve the scenario [31, 64]

Opinion, state-of-the-art technology data

Impact of futuretechnologies

Multiple future scenarios are the basis from which experts workbackward and identify necessary (technological) breakthroughs, e.g. [65]

Opinion

Future mapping An expert elicitation technique where pre-defined events andpre-defined end-states are arranged to investigate interrelationsand consequences, cf. [66]

Opinion

Dimensionsof uncertainty

GBN (Global BusinessNetwork)

Based on two dimensions of uncertainty and polarities, fourcombinations are seen to constitute plausible futures, cf. [62]

Opinion

Morphological analysisand field anomalyrelaxation

Multiple dimensions of uncertainty captured in columns arerelated to alternative events in rows. A scenario is created bythe alignment of alternatives of each column, cf. [67]

Opinion

Cross-impactanalysis

Interactive futuresimulation IFS

Based on a set of variables (descriptors) an assessment of theirmutual relevance based on expert judgement is carried out.Consistent scenarios are constructed in the sense that variablecombinations are computed that have been judged to be compatible.There are probabilistic versions of cross-impact analyses [37]

Opinion arranged bymathematical method

Modelling Trend impact analysis Based on a business-as-usual trend assumption, the impact of apotential event on that baseline scenario is evaluated indistinguished impact sequences (first depart from trend continuation,maximum impact, and effect integration), cf. [68]

Opinion(and statistical data)

Culka Energy, Sustainability and Society (2018) 8:22 Page 6 of 21

collection are transparent and often follow a methodo-logically rigorous process. It seems legitimate to considerthat scenarios based on statistical data are less susceptibleto personal interests, personal experience, and group dy-namics. Psychological factors (as consensual attitudes orauthority biases, in addition to cultural factors as pedigree,or gender prejudice, and environmental factors such asmeeting facilities, meeting location, travel times and hous-ing, as well as economic factors such as remuneration,funding, or nepotism) are not observable in statistical dataanalyses. The unclear quality of expert elicitation and thereported psychological phenomena involved in theconstruction of opinion-based scenarios gain a dramaticmomentum if we recall that these scenarios are presentedas possible (sometimes even consistent) futures todecision-makers.Using the technique of cross-impact analysis as an ex-

ample for a judgement-based scenario construction tech-nique, I would like to discuss the fourth point. However,my criticisms apply to all techniques based on expertopinion. Consistent scenario construction is a relevantprerequisite for the legitimation of energy model resultsand one approach addressing this issue is cross-impactanalysis [37, 38], reviewed by [39]. Briefly described, themethod presented in [37] defines the so-called descrip-tors figuring as a representation of context assumptionsfor a scenario. Experts are elicited to stipulate the recip-rocal influence of the descriptors and via an algorithmcompatible context assumption combinations are de-rived. Although cross-impact analysis (CIB) is preferableto an unsystematic assumption choice and numericalvalue stipulation, the method has some drawbacks.First, the number of so-called descriptors is limited

due to both practicability, cf. p359 [37], and reliance onexperts. In contrast, due to the Monte Carlo simulation,the number of potential influences (corresponding to de-scriptors) is not limited computationally using Bayesianmodel averaging. Expert knowledge is not required butcan be included through prior choice.

Secondly, the CIB methodology defines consistency ina particular manner based on expert judgement. The de-scriptors are evaluated with respect to their reciprocalinfluence, one-on-one, as assumed by the expert interro-gated. The basic principle describing the consistency isthe principle of compensation that says “two opposinginfluences on one state are to be judged as equallystrong if their effects can compensate each other. If it isto be estimated that one of the influences predominatesduring a confrontation, this one shall be judged higher,i.e. be given a higher number.” p.340 [37].Underlying the principle of compensation are three ar-

guable assumptions (1) dominance is generally valid, (2)dominance can be extended, and (3) dominance is pair-wise invariable if additional descriptors are simultan-eously considered in a scenario.Assumption 1 is a general statement for two descriptors

that can be for example “+ 1” meaning “weakly promotingdirect influence”, “− 3” meaning “strongly restricting directinfluence”, or “0” meaning “no direct influence”. Theproblem is that such an evaluation needs to be related ex-plicitly to another context (i.e. descriptor) and contextsmay vary in different scenarios, and for different experts.For example, if an expert generally judges a descriptor asweakly promoting another descriptor, she (implicitly) pre-supposes conditions where the statement is valid, what Icall that a state of the world. But the presupposed condi-tions are exactly those that are varied in different scenar-ios. In contrast, if we use statistical data and the BMAmethod, we evaluate the reciprocal influences of descrip-tors in many different states of the world, to be precise, allthose states of the world, our data record contains. This isin fact what statistical analyses do, regarding the rate ofchange of a variable (i.e. descriptor, influence) relative toall other variables. I call the property of a descriptor to beinfluential in many different states of the world stability. Itis advantageous to formulate scenarios based on stable re-lations in the target system as these relations are mostlikely to hold in scenarios too.

Table 1 Comparison of scenario design methodologies and techniques adapted from [31] (Continued)

Methodologicalapproach

Technique Keywords or a (very) brief description Source of information

Sensitivity analyses Given a model, exogenous variables or model parameters are varied.The changes of model results given varying input/parameterassumptions are evaluated as “sensitivity”. Often the one-parameter-at-a-time technique is employed, a kind of ceteris paribus approach [69]

(Opinion and)statistical data

Dynamic scenarios From brainstormed scenario themes a system is mapped usingcausal models. The variables figuring in different causal modelsare combined in a meta-model mapping the whole domain.The meta-model is analysed for different uncertainties involvedin the variables, cf. [70]

Opinion andstatistical data

Bayesian modelaveraging (BMA)

Scenarios are constructed from statistical data records of phenomenamost relevant for an input variable of a consequent quantitativemodel. Uncertainty is evaluated based on the explanatory powerof influencing phenomena most relevant in the historical record, cf. [24]

Opinion andstatistical data

Culka Energy, Sustainability and Society (2018) 8:22 Page 7 of 21

Assumption 2 implies that dominance of descriptorscan be extended to hypothetical states of the world. Ahypothetical state of the world is a state with unprece-dented conditions. In contrast, a provable possible stateof the world is a combination of occurrences in theworld observed in the past. If the scenario constructionis based on intuitive or hypothetically possible relationsin the target system, we are confronted with two kindsof uncertainty: (a) assumption uncertainty and (b) rep-resentation uncertainty. Now, (a) is a natural uncer-tainty for every future scenario and, in fact, it is thevery reason why we construct scenarios. Assumptionuncertainty arises because we do not know what willhappen in the future. So, if we assume a numericalvalue we stipulate an assumption for an input variable,we face assumption uncertainty as the value mightprove to be different in due course. In contrast, (b) rep-resentation uncertainty means that the relations in thetarget system are, at least partially, flexible and un-known. Intuitive cause-effect relations or expert opin-ion on reciprocal relations can be empirically adequate;however, the only way to evaluate the adequacy is tocompare the assumed relation with actual target systembehaviour in the past. This amounts to a statistical ana-lysis. The proposed method merely circumvents theintroduction of additional uncertainty due to poor sys-tem relation representation and straightforwardly usesthe stable relations that are statistically confirmed forthe observation period in the target system. BMA forconsistent scenario construction extends to hypothet-ical states of the world too but allows specifying that ascenario represents a hypothetical state and allows for aclear communication as to why and to which extent thehypothetical scenario differs from the observations inthe considered historical period.Assumption 3 is, in a sense, a ceteris paribus assumption

known to be an idealisation. It is related to assumption 1but now I mean consistency within the same scenario. Iwould like to give a simple, intuitive counterexample to thisassumption based on the interrelated nature of the energysystem with social, environmental, and political systems.Consider the three descriptors: gross domestic product,world tensions, and oil price, taken from the example in[37]. The experts are supposed to judge the influence of thegross domestic product on the world tensions, and so forth,pairwise. But if an expert was asked to assess the relevanceof the oil price on the world tensions given a high gross do-mestic product (a relaxed economic situation), it may bedifferent than when this relation is assessed given a lowgross domestic product (distressed economic situation).Whatever the expert’s subjective reasoning behind the sup-posed impact of one descriptor on another would be, itmust not necessarily hold true when a third, a fourth, etc.descriptor enters the picture (and is variable).

However, for a human being, also for an expert, it is dif-ficult to assess the strength of relations between the de-scriptors when the number of descriptors exceeds two orthree. I suppose the cause for this is that human’s reasonabout correlation and reasoning is difficult when relationsbecome multi-dimensional. In contrast, the statisticalBMA model can (and does) take dozens, even hundreds,of potential combinations of descriptors (influences) intoaccount and assesses their explanatory power with respectto all other descriptors in a model simultaneously. Thou-sands of such models are investigated in the MCMP sam-pler, which is not restricted by human capacity. And,perhaps most relevant, the explanatory power (equivalentto the “cross-impact judgements” of the experts) is notbased on human reasoning, but on statistical confirmation.Consistency is hence defined as non-contradiction withempirically confirmed states of the system (i.e. with statis-tical fact) rather than an expert opinion.In addition to being non-contradictory, the strength of

evidential support in a relation (given the data used) can beanalysed in principle in detail for any region, any time reso-lution, any historical period, and any type of statistical infor-mation by one person. In contrast, even if the group ofavailable experts has remarkably diverse backgrounds (whatbrings about other problems, e.g. language issues, incompat-ible implicit worldviews), this is for judgement-based scenar-ios in principle not possible. All experts employ humanreasoning. I would like to remark that this is a fundamentaldifference to all techniques presented in Table 1 with infor-mation source opinion. To be clear, I do not say that expertserr in principle their assessment of relations, even if amulti-dimensional scenario is constructed from theirtwo-dimensional assessment. But we cannot assess the qual-ity of the “human black box” directly. Whatever the reason-ing behind the expert’s opinion of the assessed relationwould be, it should be empirically adequate, and this ad-equacy needs to be evaluated. Using statistical data in theBMA method, we can obviate the risk of empirical inad-equacy due to human reasoning naturally involving psycho-logical effects and computational limitations.Another drawback of a judgement-based scenario design

with a semi-formal ordinal categorical formulation is exem-plified here using cross-impact analysis. However, the criti-cism applies to all techniques where quantitative scales areused as a way to represent expert judgement as an informa-tion source, rather than using a quantitative scale as an in-formation source. In contrast, the quantitative values ofstatistical data represent statistical evidence, i.e. observableempirical facts. Reciprocal effects and dominance in thesemi-formal formulation are represented for example as “+1” or “− 2”, “medium” or “low”, and it is unclear accordingto what procedure these formalisations are translated intovalues acceptable by an energy model. Even in case theassumed relations were right, the scenario formulation of

Culka Energy, Sustainability and Society (2018) 8:22 Page 8 of 21

numerical assumptions seems to not be based on a meth-odologically stringent interpretation procedure. In otherwords, the method invites interpretation opacity and uncer-tainty in the numerical value stipulation necessary for anenergy model.5 Different modellers may interpret theimpact of the descriptor oil price “quantified” as + 2 on thedescriptor gross domestic product in completely differentnumerical value stipulations “fed in” the energy model. TheCIB method in fact provides “sets of consistent assump-tions” for a number of descriptors forming the scenario it-self. To me, it is unclear if the interpretation of the scenarioB1 (p. 343 [37]) consisting of moderate world tensions,medium borrowing industrial countries, strong cohesion ofOPEC, an oil price of 35–50$, and 2–3% world GDPgrowth would be interpreted by different energy modellersnumerically in the same way. If an oil price of 35$ is equallywell confirmed as an oil price of 50$, the “rough” scenariosare possibly too general for energy models being highlysensitive to numerical assumption changes.Secondly, how would “moderate world tensions” be nu-

merically interpreted? Thirdly, if it was not included in themodel numerically, how is the scenario-relevant context as-sumption (and its non-inclusion in the energy model)accounted for? The proposed BMA method for consistentscenario construction uses statistical data with explicit nu-merical assumptions for quantitative statistical data. For ex-ample, the descriptor GDP is not interpreted but simply isthe GDP in units, e.g. billion EUR [40]. The derived statis-tical data, as for example GDP per capita, inherit theirmeaning from the basic statistics and the transparentlypublished computation procedure by statistic agencies.Vague scenario formulations as “high GDP” or “moderateworld tensions” reflect the relative nature of every judge-ment. In the case of CIB, the judgement expresses the ex-pert’s opinion and the modeller’s interpretation stipulatingthe numerical value. In the case of BMA, the judgement ex-presses the statistical data used, e.g. the stipulated value liesin the third quartile of the data (in the highest 25%). If dataare transformed [41] for BMA analyses, a systematic rever-sal to derive a numerical value for scenario assumptions ispossible. In some contexts, categorical data or ordinatestatistical data are available. They can be included in thescenario construction, but in contrast to semi-formalquantification, BMA is not limited to them. Their interpret-ation is straightforward from the statistical sources, forinstance, defined indices6 or systematic data treatment as,e.g. seasonal adjustments.The empirical assessment’s importance is particular to

energy modelling and policy advice scenarios in general.The severe consequences of experimental trial-and-errorstrategies in terms of economic impacts on society de-mands for a scenario construction method that takesinto account assumption compatibility and strength ofevidential confirmation. As energy modelling is the best

“experiment” of the energy system we have, it is mean-ingful to make sure that the constructed scenarios relateto what was possible in the target system and to estimatehow much deviation, we could say “novelty”, a scenariointroduces compared to the states of the energy systemas documented in statistical data. In this way, we esti-mate and give credit to the efforts we take in our pursuitof hypothetical system states we dream of in scenarios.Expert knowledge is part of this process; however, evalu-

ating the relations of target system phenomena is not an ad-equate scope for expert judgement, as speculations aboutrelations are more adequately scrutinised with statisticalmethods. If in fact one phenomenon in the target systemshappens to systematically change in concert with otherphenomena, statistical methods can point to that – what-ever the reasons for the simultaneous changes are. Expertscan interpret changes and guess reasons for correlations ifthe true data generating process is unknown. But a guessremains a guess, even if performed by an expert. Pretendingexpertise on something one does not understand is notprofessionalism but unscientific conduct. And as we cannotdirectly evaluate expert understanding and the “correct-ness” of reasoning as confirmed by evidence we might aswell guess ourselves or directly consult evidence, as we doin statistical analyses.If an expert knows a true data generating process, expert

judgement is valuable for scenario design. For example, ifan expert knows of signed contracts to build a pipeline, sheknows a reason for a change. In a scenario thephenomenon “infrastructure capacity” can be adjusted ac-cording to the expert’s opinion in due time of the scenario.

Scenario definition and uncertainty evaluationThe first step when creating BMA-scenarios is defining (i.e.choosing upon known) phenomena in the target system wewant to make a scenario of for an energy model. I refer tosuch phenomena of interest as “scenario phenomenon”.Public invitations to tenders for modelling exercises oftendescribe the scenario phenomena of interest in detail. Forexample “the impact of unconventional gas production onthe electricity price of country X” could be a phenomenonof interest. But also phenomena, as for example “implica-tions for the energy system of a legal threshold for CO2emissions in country X”, could be subject of a scenario. Letme replace “country X” with “Nicastan”, an inexistentcountry, to improve legibility and intuitive understandingcoupled with the generality of the presentation.What interesting phenomena for a scenario depend on

the person who asks the question—the societal groupconcerned. For energy models, governmental stake-holders sometimes investigate consequences of debatedpolitical measures by scenarios. Industrial stakeholdersmight be interested in energy system developmentsgiven different investment decisions. Public stakeholders

Culka Energy, Sustainability and Society (2018) 8:22 Page 9 of 21

might be interested in tax burdens or security of supplyscenarios, to name just a few. For a consistent scenariowith BMA, the phenomena investigated are not limitedin number and nature (social phenomena, environmen-tal phenomena, economic phenomena) provided statis-tical data are available. I return to that limitation in the“Discussion and limits of the approach” section.We discern between modelled phenomena and context

phenomena defined by the model boundaries of a spe-cific energy model. The consistent scenario constructionis solely concerned with the aptitude of an energy modelto represent the scenario given the energy model design.The relevant parts of an energy model for scenario rep-resentation are the input variables and sometimes pa-rameters. For the presented method of consistentscenario construction, we do not need any judgement ofthe energy model’s quality, as we only look at an energymodel’s capacity to represent the scenario, and constructnumerical assumptions consistent with statistical dataand expert opinion. In other words, the consistent sce-nario construction as presented here is independent ofconsequent processing by an energy model, its mathem-atical formulation, and internal error propagation, whatmakes the method applicable to a large number of quan-titative modelling techniques. The “interface” of scenar-ios and energy models are typically input variables;therefore, the following example uses a typical inputvariable. Figure 1 depicts a schematic illustration of sce-nario phenomena, statistical data, energy model repre-sentation of the energy system (ESS), and the targetsystem (WSS), i.e. the “real” energy system.

The rectangle should be considered extending be-yond the graph as “the universe”; it is associated withall possible states of the universe, one at a time, ac-cording to the time increment we choose for the rec-ord. The solid ellipse depicts schematically “theworld”; it is a subset of the universe associated with allpossible states of the world. Dots in the WSS subsetindicate existing phenomena. The dotted ellipse indi-cates “the energy system”; it is a subset of the worldassociated possible states of the energy system. Thecircles indicate statistical data depicting (a collectionof ) phenomena in WSS as follows. Purple circles sche-matically depict phenomena of the world subset(WSS), explicitly outside the energy model boundaries.Blue circles depict phenomena of WSS and the energysystem subset (ESS); statistical data of these phenom-ena are accounted for in an energy model in the formof parameters. Solid purple circles depict phenomenaof WSS and the energy system subset (ESS); statisticaldata of these phenomena are accounted for in an en-ergy model in the form of input variables. Solid bluecircles depict phenomena of WSS and the energy sys-tem subset (ESS); statistical data of these phenomenaare accounted for in an energy model in the form ofoutput variables. The arrows show the BMA descrip-tion of phenomena, to be read as “explains” in the dir-ection of the arrow tip (Fig. 1).I created a fictitious scenario and elaborate the dif-

ferent steps for the example. The storyline of the sce-nario is translated to numerical assumptions for inputvariables. Suppose the government of Nicastan is in-terested in a low-cost scenario for natural gas becausein the real world, recent developments in unconven-tional gas production indicate a lasting period of lownatural gas prices. We want to investigate the effectsof such a low-cost period on the energy system inNicastan with the fictitious energy model My-model.Assessing the empirical adequacy of My-model is

our aim, expressed as a lower bound of uncertaintythat My-model is apt to represent this scenario. Find-ing the numerical assumptions, value that best de-scribes the scenario follows. Phenomena of interest inthis example are different target system phenomena,including the unconventional gas production. The for-mulation of the scenario is captured in the input vari-able “natural gas price” of My-model. So, if theenergy models’ assumption should reflect the bearingof unconventional gas production on natural gasprices in Nicastan, there should be statistical evidencefor that. The strength of the supposed relation be-tween the energy model input variable (the price fornatural gas) and the scenario-phenomenon of thetarget system (unconventional gas production) isassessed by statistical evidence.

Fig. 1 Schematic illustration of the world, the energy system, andthe representation of phenomena with BMA

Culka Energy, Sustainability and Society (2018) 8:22 Page 10 of 21

It is, in a straightforward sense, evaluating whether theconsidered scenario can be described by the energymodel at all (and how well), based on the energy modeldesign. The input variable of My-model represents manyphenomena; with BMA, we can find out how relevantthe one phenomenon we are interested in (the uncon-ventional gas production) is. The uncertainty assessmentestimates how relevant the relation of scenario phenom-ena and energy model input variables is, based on obser-vations. For statistical data of phenomena outside theenergy model boundary, it is a quantitative assessmentof scenario uncertainty as defined by Walker et al. [42].This is a fundamental assessment because it concernsthe grounds of justification for any energy model resultclaiming to illustrate the scenario.I would like to emphasise that hypothetical energy system

state scenarios (i.e. not designed to represent energy systemstates observed in the past) are not in conflict with usingstatistical data for scenario construction. Both energymodels and scenarios are designed to describe possible en-ergy system states. Statistical data are evidence for a systemstate to be possible. Equivalent to energy model calibration,consistent scenario construction uses the evidential basis toderive statements about the possibility of scenarios. This isof interest for scenarios describing unprecedented energysystem states (hypothetical scenarios) because it providesan expectation based on what we know to be possible.The statistical data are used to analyse relations in the

target system of relevance for the scenario representa-tion in an energy model. To do so, classical statisticalmethods could be used. These methods have some dis-advantages, for example, biases in the choice of explana-tory variables by the scenario constructor. Using Baysianmodel averaging (BMA) [2, 3] allows for both expertopinion and statistical likelihood and reduces the risk ofmodel misspecification.The mathematical formulation representing a general

relation between the influences (also called explanatoryvariables) and the dependent variable is chosen, where theinput variable of the energy model (the natural gas price)figures as the dependent variable in the BMA model. Evi-dentially, production data of unconventional gas is consid-ered as an explanatory variable, but for an empiricallyadequate representation, statistical data of any kind whichis suspected to be relevant can be included. Our aim is tofind out what phenomena in the target system the inputvariable actually represents, according to statistical evi-dence. We employ BMA to evaluate the relevance of dif-ferent phenomena for the input variable, taking fulladvantage of the fact that BMA “sorts out” influences thatcannot contribute to explaining the dependent variable.Defining the prior distribution also allows us to take intoaccount the expert opinion on the number of explanatoryvariables expected to influence the dependent variable.

The BMA results are models containing different explana-tory variables. The models are ranked according to theirexplanatory power which is expressed as a posteriormodel probability (PMP). The relevance of individual ex-planatory variables is expressed as posterior inclusionprobability (PIP). These two values per se already encodehighly relevant information for scenario design. In a sense,they are the empirically confirmed counterpart to the ex-pert opinion of scenario design by cross-impact analysis.In contrast to the assumptions of cross-impact analysis, itis observable that the inclusion or exclusion of variables isnot reciprocally balancing. Even for the simplest case ofone explanatory and one dependent variable (comparableto CIB), we find that changing the role of the variablesdoes not always lead to reciprocal balancing. The balan-cing is even less observable, given a growing number ofexplanatory variables. However, the CIB idea that influ-ences are “impacting” each other according to the rela-tions of phenomena in the target system is also capturedin the BMA models. Unfortunately, the true data generat-ing process is often unknown, and in systems influencedby human decision, individual relations may be difficult tofind in statistical data. What we seek to analyse is thestrength of “stability” in the relations of influencing phe-nomena vis-à-vis the dependent variable.Briefly described, the BMA method takes statistical data

of all influences (i.e. context phenomena of the targetsystem, also called explanatory variables) and uses aMarkov Chain Monte Carlo importance sampling methodto “build” different models. “Different models” are justequations arranging explanatory variables. The explanatorypower of a model is assessed, and the sampler builds an-other model and assesses the explanatory power of thatmodel for the dependent variable. The number of explana-tory variables forms the “model space” defined as the pos-sible combinations of explanatory variables. For example, ifan energy model input variable (the dependent variable) issuspected to be influenced by 18 phenomena, the modelspace (218) contains 262,144 different models that could de-scribe the relations in the target system. A stipulated priordistribution depicts the scenario constructor’s opinion onhow many influences she considers relevant. If the con-structor has no expertise at all, a flat prior (non-informativeprior) reflects ignorance,7 and the models are determinedbased on statistical evidence, called likelihood. The poster-ior model probability (PMP) is proportional to the productof prior model probability and the marginal likelihood of amodel. The marginal likelihood of a model in the modelspace is the probability of the data given the specific model.With increased computational power and advanced import-ance of sampling techniques, the employment of BMA hasrisen since the 1990s considerably. Today, differentready-to-use options for mathematical software are avail-able which relinquishes the need for programming skills

Culka Energy, Sustainability and Society (2018) 8:22 Page 11 of 21

and tedious construction of adequate software. For the ex-ample in [1], I used the R package BMS by Zeugner [43],which offers abundant possibilities of BMA specificationand a complete set of standard features sufficient for a con-sistent scenario construction.The BMA approach amounts to saying “It is by at least

93% uncertain that My-model can describe a low-costnatural gas period due to developments in unconven-tional gas. The scenario is introduced in My-model viathe input variable “natural gas price”, but low naturalgas prices are also explained by …”. In the following sec-tion, two options for consistent scenario constructionare discussed: the consistent scenario construction andthe representation quality assessment for (1) a given en-ergy model and (2) an adapted energy model. The lattergenerally allows for a better representation of scenarios,as the model is adapted to account for the most relevantphenomena influencing the scenario phenomenon.

Consistent scenario construction for a given energymodel and an adapted energy modelTypically, a scenario is introduced to an energy modelby input variable adjustment; in My-model, it was thenatural gas price. Having gathered statistical data of allinfluences, we suspect to be related to the natural gasprice and the scenario phenomenon “unconventional gasproduction” we have investigated the empirical adequacyof the scenario in the fictitious My-model. The BMAanalysis delivers an indication of which phenomena arerelevant for the input variable and how relevant theseare relative to each other (and all phenomena we sus-pected being relevant).We can improve the scenario representation quality of

an energy model by changing the energy model design.In particular, we can decrease the energy models’ repre-sentation uncertainty of a scenario phenomenon by in-cluding the relevant input variables and/or parametersin the energy model. To do so empirically adequate, weneed to find out which phenomena in the target systemhave influenced the scenario phenomenon in the past. Ifour intention is to create a consistent scenario, ratherthan justifying that our input variable assumption repre-sents a scenario, the consistent scenario constructionmethod is suitable.In consistent scenario construction for adapted energy

models, we apply a different reasoning. In the target sys-tem, the emergence of a scenario phenomenon is oftenonly possible if different related phenomena develop in aspecific way. In other words, the scenario phenomenon,due to the interrelated nature of the target system, ne-cessitates other phenomena’s occurrence in a specificway. In the example, the scenario phenomenon of highunconventional gas production necessitates high naturalgas prices; otherwise gas production is not profitable.

We denote the BMA variable representing the scenariophenomenon as the dependent variable and span themodel space over all influences suspected to impact thephenomenon, including input variables and parametersof the energy model as explanatory variables. In thatway, we investigate what potentially explains the sce-nario phenomenon’s emergence in the target system. Ifwe consistently construct a scenario, our aim is to“present to the model” a world in which the scenariophenomenon is present in the way we are interested in(i.e. high unconventional gas production). To do so em-pirically adequate, we depict the scenario phenomenon“indirectly”, supposing its emergence is dependent onthe influences to be in a certain way.A phenomenon is “presented to the model” in a coherent

and consistent way by an adequate choice of numericalvalues for input variables and the introduction of relevantinput variables and parameters in the energy model, calledenergy model adaption. The more relevant input variablesand parameters an energy model contain, the better ourrepresentation of the scenario would be. Again, we haveBMA select for us which influences are systematically in-creasing or decreasing the observable value, but now, ourdependent variable is the scenario phenomenon. All wehave to do is be creative in our suspicions of what targetsystem phenomena influence the scenario phenomenonand perform statistical data handling.Including the data of energy model input variables as ex-

planatory variables allows us to evaluate how well we canalready represent the scenario phenomenon in the energymodel. We can evaluate this quite precisely using the PIP ofthe influences that are energy model input variables relativeto phenomena not modelled in the energy model. Also, wefind what other influences in the target system are relevantfor the emergence of the scenario phenomenon to efficientlyadapt the energy model with the objective to increase theempirical adequacy of scenario representation. Figure 2 iscomparable to Fig. 1 but depicts the different reasoning ofconsistent scenario construction in contrast to input variablejustification (Fig. 2).Let us suppose we want to construct a scenario repre-

senting an increase of unconventional natural gas pro-duction with My-model; using statistical data for thequantities of unconventional natural gas produced asthe dependent variable, we can find out what phenom-ena in the target system are most influential. As ex-planatory variables for BMA, we use both My-modelinput variable data and data outside My-model bound-aries (we suspect to influence the unconventional gasproduction). We can investigate as many influences onthe phenomenon as we want to make a scenario. Thebest BMA model in terms of posterior model probabil-ity is represented in Table 2 (all data and variables arefictitious).

Culka Energy, Sustainability and Society (2018) 8:22 Page 12 of 21

Using the PMP of the best BMA model, it is possible toevaluate the probability that the phenomenon of our sce-nario can be described with the influences (let us assumea fictitious PMP of 16%). My-model does not contain allphenomena to describe the scenario with the highestprobability. We can base our decision whether to adaptthe model to better represent the scenario on that evalu-ation on the BMA results. Using the influences’ PIP, wequickly find which energy model adaptions are most rele-vant to increase the scenario representation quality. Theenergy model adaptions may concern both input variablesand parameters, depending on the energy model design. Ifnot all influences are used to construct a scenario, thelower bound of uncertainty needs to be adjusted to thePMP of the BMA model consisting of the influences thatare used. This is a BMA model with higher uncertainty ifit is not the best BMA model in terms of PMP. The sce-nario construction with all relevant influences is a sce-nario representation according to statistical evidence ofthe scenario phenomenon with least uncertainty for theproven possible states of the world (observed values).

Using the consistent scenario construction approach, wepay attention to the interrelated nature of the targetsystem and design our energy model to represent the sce-nario phenomenon empirically adequate. This approach isalso applicable for phenomena, as for example enactmentof a law. This process may at first sight be self-sufficient inthat it is a human decision governed by human sover-eignty, not grounded in statistically observable interactionwith other target system phenomena. Upon secondthought, we may find that the decision of actually enactingthe law indeed necessitates a specific target system state.For example, the scenario of a law prohibiting theemissions of CO2 exceeding some defined quantity maynecessitate a minimum quantity of CO2 emissions, to berelevant at all. For human decision-governed scenarios, aninvestigation of most relevant influences on the decisionimproves the representation of a scenario.

Results: consistent numerical value estimationWith BMA, we can truly construct scenarios. Having iden-tified the most influential phenomena, we insert the stipu-lated values for the time period we want to project in theexplanatory variable data record. I would like to remarkthat this is in the same instance a transparent assumptiondocumentation. We use the BMA model of choice to com-pute the predictive density for the dependent variablebased on the stipulated values for the influences. Thismeans, expressed for readers of a scenario study, “the ad-justed input values of the energy model correspond statis-tically to an unconventional gas production of xy”.The PMP of the BMA model determines the lower

bound of uncertainty for the scenarios. In addition, wedispose of statistical criteria, for example, we can assesswhether the predicted numerical value of the scenariophenomenon is within the double standard deviation ornot. To be clear, the procedure is (1) stipulation ofvalues for influences; we simply fill in the data recordfor the future period of the scenario. We can orientateon historical “highs and lows”, and include expert know-ledge, as mentioned before if the expert knows of signedcontracts, the additional available capacity in the year ofexpected operation is entered as data. Next step (2) iscomputing the predictive densities for the scenario

Table 2 Fictitious evaluation of scenario representation for My-model

BMA model-dependent variable Influences Relevance of influence (PIP) Influence modelled in My-model

Unconventional natural gas production Natural gas price 0.99 Yes

Oil price 0.85 Yes

Conventional natural gas production 0.84 No

Electricity consumption 0.62 Yes

LNG infrastructure investment 0.59 No

Gas storage capacity 0.45 Yes

Fig. 2 Schematic illustration of the reasoning for consistentscenario construction

Culka Energy, Sustainability and Society (2018) 8:22 Page 13 of 21

phenomenon. Given the stipulated values, we computethe numerical value of the scenario phenomenon. Indoing so, we can explicitly explain both the numericalvalue of the scenario phenomenon (due to observed re-lations) and the representation quality of the scenario inthe energy model, as captured in the input variable(s).Finally, (3) communicate a complete and transparentdocumentation of the scenario construction.An exemplary communication for a scenario con-

structed according to case (b) BMA scenario construc-tion for an adapted energy model is available in theAppendix.

Discussion and limits of the approachIn this section, I would like to discuss the limitations ofthe approach and address difficulties I experienced whencarrying out the method. However, I may be unaware ofsome problems and limitations, so I do not claim thisdiscussion to be exhaustive.A practical limitation of the approach is its extensive

use of statistical data. This naturally involves both accessto statistical data and availability of statistical data in thefirst place. Today, many statistical data are freely avail-able from official sources, for instance, Eurostat orUSA.gov, and often data can be bought. Researchers en-tertaining energy models are likely to already dispose ofa significant amount of relevant data for calibration pur-poses. Efforts made to improve access to statistical dataoften prove to be advantageous in more respects thanconsistent scenario construction. It is my opinion that fi-nancial burdens for subscriptions, in particular for re-search collaborations or larger scientific institutions, arevindicated by the potential impact scenario studies canhave, if used for decision support.Data availability is different from data access. Unprece-

dented phenomena may have no data record at all. In thiscase, it is indispensable to transparently communicate thatthe scenario is incommensurable to any known energysystem state. The response of the energy system to a com-pletely unprecedented scenario phenomenon is highly un-certain as the assumption of well-established andlong-standing observable relations to hold is uncertain. Inless dramatic cases, some methods may be useful. Thefirst method is indirect phenomenon description with dif-ferent influences data so that as many suspected data gen-erating processes as possible are included in the analysis.Phenomena described with statistical data of various sci-entific disciplines allow for a multi-dimensional descrip-tion of the phenomenon if researchers are not primarilyfocussed on their scientific discipline.Another method applicable in some contexts is data

scaling. Scaling is not suitable for data known to dependon spatial or temporal relations in the system where theyare collected, unless that system is the target system of

the scenario modelling. For example, data of energy effi-ciency improvements of a technology product are scal-able, because the data are regionally independent, i.e. thesame technology product would have the same technicalefficiency everywhere in the world. In contrast, data ofconsumer behaviour of a region are not suitable for scal-ing, because the data depend on regional aspects such asincome, social system, political stability, or any culturalaspects of that region, to name just a few.A third method is data generation. The applicability of

data generation is dependent on the phenomena ofinterest. Consistent scenario construction provides aguideline to survey design, as we can find potentiallyrelevant phenomena for the scenario by statistical evi-dence. Say, a city wants a scenario of residential CO2emission reduction, but the data of the city is lacking.Based on a BMA analysis with data of a comparable city,we find what phenomena are statistically most relevantand hence what data are of primary interest in the city.A survey designed to generate the data for the scenariothen contains questions regarding the influence withhighest relevance in terms of PIP (e.g. primary heatingfuel in the residential housing sector) and questions onstatistically relevant data in terms of high PIP (e.g. heat-ing technology efficiency). Of course, other data sus-pected to be relevant, say as a particularity of the city,should also be generated. If data are partially available,the survey design should account for relevant influencesadditionally to or for in-depth surveys. Lastly, a methodto increase the data availability is open source policy ofanonymised data-managing enterprises or research insti-tutions. However, this is more a political question than amethod a researcher can apply ad hoc. Nonetheless, it isimportant to mention it and engage in the discussion.Some experts hold that some phenomena simply cannot

be represented statistically, and expert judgement is theonly way to evaluate such phenomena. Human psycho-logical phenomena or socially arising phenomena are typ-ical examples showcased. I agree with sincere reservations.Take the example of Weimer-Jehle in [37] “world tensions”.An expert evaluating the impact of world tensions may sub-jectively have a clear idea of what is meant. However, differ-ent experts are likely to have different interpretations of“world tensions” due to the vague definition that may leadto the incommensurability of expert opinions.I contend that for many psychological or social phe-

nomena, there are statistical data with a precise definitionof what is evaluated by the statistic and how the data aregenerated. Using such data enables recipients of scenariostudies and modellers to understand how a phenomenonis interpreted in a scenario. Statistical data, for example,national warfare expenditure [44] or arms trade data8 [45],or the United Nations Office for Disarmament AffairesUNODA databases, can provide, in my opinion, a suitable

Culka Energy, Sustainability and Society (2018) 8:22 Page 14 of 21

statistical representation of “world tensions”. In additionto the data for armed conflicts [46, 47], corruption indicessuch as the Corruption Perceptions Index (CPI) by Trans-parency International [48] or the World Bank’s CPIAdatabase [49] are available, to name just a few. Let ussuppose we construct a scenario, and the influence “worldtensions” were expressed in statistics. We could combinecorruption data and warfare expenditures of relevant na-tions for the scenario. Let us say for an exemplary scenarioin Nicastan, the country imports weapons from Germany.We could consult the United Nations register [50] andfind out how many weapons were exported in the pastyears to Nicastan. Our scenario is constructed such thatthe meaning of “world tensions” corresponds, for example,to an increase of arms imports from Germany by fourtimes the mean over the last years. We can even specifywhich weapons we include in our assumption and whatscope our assumption has (e.g. “transfers between UNmember states”). To me, this is a storyline understandableto recipients of scenario studies, and significantly less am-biguous than the CIB evaluation “strong”, “moderate”, or“weak” world tensions p. 339 [37]. For psychological andsocial phenomena, a large variety of data are available, forexample, the World Values Survey “is a global network ofsocial scientists studying changing values and their impacton social and political life” [51]. OECD social databases[52] document and allow access to social data, as well asthe United Nations Statistics Division UNDATA databases[53], or research institutions hosting databases [54].To reduce practical limitations computing “milestone

years” and interpolating between those may sometimes besuitable. However, it is an advantage of a BMA consistentscenario construction that the scenario design can signifi-cantly differ from linear paths. This is possible because thecomputation of the corresponding numerical value of thescenario phenomenon (predictive density) takes stipulatednumerical values for influences into account. Stipulatedvalues of influences can represent any “shock” or atypicaldevelopment, in consonance or individually, and the resultof the best BMA model will represent the expected valueof the scenario phenomenon based on the relations ob-served in the historical period considered. In this sense,intuitive scenario design has its place in BMA scenarios,as well as expert opinion.I would like to emphasise that using statistical data for

social and psychological phenomena does not mean thatconstructed scenarios are identical to past phenomena.Stipulating the numerical values of relevant influences al-lows for full creativity and exploration of hypothetical pos-sibilities. Using the BMA model to compute the predictivedensity means that interrelations and impacts in the pastare the basis of expectations for the future. Evaluating thebearing of an influences’ hypothetical value on other phe-nomena is always based on system relation assumptions—

be it in the form of expert opinion or observation record.Statistical data are evidentially confirmed system relations(and provable possible) what renders scenario construction(1) consistent with what we observed, (2) intersubjective,(3) systematic, (4) understandable, and (5) comparable. Iam aware that statistics can be manipulated and may notmirror the “true” state of the world. Nonetheless, I am con-vinced that they are a better record of fact than subjectiveobservations. In any case, the data are clearly explicable incontrast to ambiguous classifications (“high”, “low”, etc.) ex-perts can commit to.Today, scenario studies typically present storylines, but

lack of (1) transparent documentation of scenario assump-tions and implicit assumptions; (2) a (energy) model as-sessment of scenario adequacy, i.e. input variable aptitudeto represent the scenario; and (3) scenario assumptionevaluation (stipulation of numerical values). Omittingcommunication of that information averts scrutinising therepresentation quality of the scenario for a given (energy)model and the degree of confirmation for a scenario(proven possible or unprecedented hypothetical). In myview, providing that information is the (energy) scenariomodeller’s assignment because a scenario study recipienthas no means to retrieve that information from (energy)model results or a storyline narrative.The BMA method allows for the inclusion of any stake-

holder choices statistically evident. The behaviour of societymembers, consumers, the industry, or the government, isstatistically recorded in a non-discriminatory way in a multi-tude of dimensions, for instance, periodicity in time, geo-graphic area, social and cultural categories, or economicbenchmarks. The pedigree of these data, their collection,post-processing, financing, hosting, and availability are meth-odologically rigorous, transparent, and non-discriminative.Scenario construction by the BMA method does not obligethe scenario constructor to claim “expert knowledge” or haveskills superior to any person capable of handling statisticaldata. Some programs suitable to carry out a BMA scenarioconstruction are available without charges, for example, Rstatistical software package BMS [43]. The scenarioconstructor need not belong to some exclusive group of“experts”, where the quotes indicate the ambiguous status ofsuch an appointment, as there are no transparent qualitycriteria for being an expert.The opinion of a scenario constructor can be included

by numerical value stipulation. Opinion is systematicallyand methodologically levelled in the light of statisticaldata. In other words, the advantage of BMA over judg-mental scenario construction techniques is the embeddingof opinions in the relevant context’s empirical record.That levelling helps to alleviate psychological effects re-ported in the literature. In the BMA method, expert judge-ment is needed to choose the data of the phenomenasuspected to be influential, decide computational

Culka Energy, Sustainability and Society (2018) 8:22 Page 15 of 21

procedures (e.g. birth-death-sampler), define priors, andpre- and post-process data (e.g. outlier elimination, reduc-tion of results for interest groups). These choices are con-testable facts (not a “feeling”), subject to criticisms, andproduce testable results; if the expert changes her judge-ment and say another sampler routine, the result of BMAchanges.Lastly, the BMA scenario construction method can

better account for the real-world complexity of phe-nomena interactions than the limited cognitive capacityof human reasoners. However, statistical analyses arelimited in their capability of detecting relationships.Considering the evaluated uncertainty for scenarios asa lower bound expresses that awareness; an uncertaintyof at least x% corresponds to the best case of actuallyacknowledging the relevant relations of phenomena.The BMA analysis must not be considered as a toolcompletely characterising the phenomena present and(variably) interacting in the target system. Rather, themethod provides an additional “view” of the target sys-tem beneficial for scenario design.The distinction between provable possible assumptions

(based on statistical evidence) and hypothetical assump-tions is a novelty and significantly improves the scenariostudy recipients’ aptitude to evaluate the future scenarioson their part. The presentation of scenarios referencingtangible storylines (e.g. the assumption for the gas pricein the first year of projection is the mean of the statis-tical data period from 1995 to 2015), is in my view, morecomprehensible than storylines referencing abstract as-sumptions as for example moderate world tensions,medium borrowing industrial countries, and strong co-hesion of OPEC, cf. [37]). So, perhaps the most import-ant novelty of the approach is the possibility tocommunicate to decision-makers the associated uncer-tainty in easily understandable terms.

Conclusion and further researchI would like to conclude the discussion by brieflyhighlighting the advantages and disadvantages of the pro-posed approach. The motivation to advance a systematicscenario construction technique for quantitative models isvested in the current practice of opaque scenario design inmany disciplines, including energy-economic modelling.Models used in decision support generally aim to repre-sent the target system and the relations of phenomena ob-servable in the world, yet they fall short in assessing theirrepresentative quality for scenarios.Scenarios based on intuitive expert opinion or based on

mutual agreement of (energy) modellers and sponsors, riskbeing an inadequate representation of the scenariophenomenon, and most importantly, they defy evaluatingtheir empirical adequacy. This poses a problem for

recipients of energy scenario studies, because the results ofscenarios depend on a thorough design representing thephenomena of interest, in particular, if the results figure indecision support.The advantages of the presented BMA approach for

consistent model design are:

– Transparent scenario assumption documentation– Evaluation of the empirical adequacy of assumptions

via an uncertainty assessment– Evaluation of scenario assumptions as statistically

confirmed or hypothetical– Methodological rigour independent of subjective

expertise of the scenario constructor, although withthe possible inclusion of expert opinion and creativity

– Adaptability for individual energy models (specificinput variables, time resolution, geographic scope)

– Formulation of apprehensible scenarios referring to theobserved phenomena, e.g. “an economically flourishingperiod as observed in the years 2003–2005”, ratherthan general formulations as “high economic growth”

– A clear indication of phenomena influencing thescenario phenomenon in the target system (using PIP’s)

– A clear indication of efficient adaption of an energymodel with the aim of increased quality of scenariorepresentation (using PMP’s)

– The possibility to construct scenarios in astraightforward sense with concurrent evaluation ofthe scenario phenomenon’s numerical value and itsprobability based on empirical evidence (stipulationof numerical values of influences and consequentcalculation of the predictive density for a BMAmodel with given PMP)

– Inclusion of expert opinion (via prior distributionand stipulation of influence values in the projectionperiod) and statistical evidence (via statistical data ofthe historical period considered)

Opposed to these advantages are practical and concep-tual limitations of BMA scenarios. Completeness of thislist of limitations is not claimed.The prerequisites of the presented BMA approach for

consistent scenario construction are:

– The need to conceptually embrace that assessing theempirical adequacy of scenarios is important

– Statistical data gathering, handling, or generation,dependent on the phenomenon of interest for a scenario

– Basic understanding of ready-to-use software solu-tions for BMA, or programming requirements to de-velop BMA routines

– A clear definition of the primary scenario-phenomena– Creative and interdisciplinary selection of possible

influences on the scenario phenomena

Culka Energy, Sustainability and Society (2018) 8:22 Page 16 of 21

– Adaption and/or adjustment of a quantitative(energy) model if the relevant influences (asevaluated by PIP) are not yet part of the energymodel. The necessary workload for adaptions can beweighed against increased representation quality. Inany case, the actual representation quality of a givenenergy model should be communicated via theuncertainty assessment.

The tangible performance edge of Bayesian model aver-aging for decision-makers (and all recipients of outlooksbased on scenario technique) is the possibility of tracingback results to assumptions and communicate the as-sumptions’ statistical confirmation. But we should not takemy word for it, so I return to the criteria introduced byGoodwin for a “formal strategy evaluation process withinscenario planning” [25] and assess the BMA method forconsistent scenario construction in these respects.The BMA method meets the transparency criterion (“the

derivation of results can be understood”) both on methodo-logical and result levels. The result of the consistent scenarioconstruction is an explicit lower bound of uncertainty andan assessment of the lower bound’s legitimacy, in particularfor hypothetical assumptions. Data sources, assumptionvalue stipulations, and BMA technical choices leading to thelower bound can be