Comparative risk assessment of severe accidents in the energy sector

Peter Burgherr n, Stefan HirschbergPaul Scherrer Institut (PSI), Laboratory for Energy Systems Analysis, CH-5232 Villigen PSI, Switzerland

H I G H L I G H T S

� Accident risks are compared across a broad range of energy technologies.� Analysis of historical experience was based on the comprehensive database ENSAD.� OECD and EU 27 performed significantly better than non-OECD countries.� External costs of accidents are very small, but impacts can still be enormous.� No technology performs best for all risk indicators; thus tradeoffs are inevitable.

a r t i c l e i n f o

Article history:Received 9 May 2013Received in revised form28 November 2013Accepted 21 January 2014Available online 21 February 2014

Keywords:Energy-related severe accidentComparative risk assessmentDatabase ENSAD

a b s t r a c t

Comparative assessment of accident risks in the energy sector is a key aspect in a comprehensiveevaluation of sustainability and energy security concerns. Safety performance of energy systems can haveimportant implications on the environmental, economic and social dimensions of sustainability as wellas availability, acceptability and accessibility aspects of energy security. Therefore, this study provides abroad comparison of energy technologies based on the objective expression of accident risks forcomplete energy chains. For fossil chains and hydropower the extensive historical experience available inPSI's Energy-related Severe Accident Database (ENSAD) is used, whereas for nuclear a simplifiedprobabilistic safety assessment (PSA) is applied, and evaluations of new renewables are based on acombination of available data, modeling, and expert judgment. Generally, OECD and EU 27 countriesperform better than non-OECD. Fatality rates are lowest for Western hydropower and nuclear as well asfor new renewables. In contrast, maximum consequences can be by far highest for nuclear and hydro,intermediate for fossil, and very small for new renewables, which are less prone to severe accidents.Centralized, low-carbon technology options could generally contribute to achieve large reductions inCO2-emissions; however, the principal challenge for both fossil with Carbon Capture and Storage andnuclear is public acceptance. Although, external costs of severe accidents are significantly smaller thanthose caused by air pollution, accidents can have disastrous and long-term impacts. Overall, notechnology performs best or worst in all respects, thus tradeoffs and priorities are needed to balancethe conflicting objectives such as energy security, sustainability and risk aversion to support rationaledecision making.

& 2014 International Atomic Energy Agency. Published by Elsevier Ltd. All rights reserved.

1. Introduction

1.1. Scene setting

The origin of the term sustainability can be traced back to forestryin the 18th century when von Carlowitz (2000) in his book “Sylvi-cultura oeconomica” characterized a mode of cultivation, at which not

more wood was used than could grow again. With the publication ofthe so-called Brundtland-Report “Our Common Future” of the WorldCommission on Environment and Development (WCED, 1987) theconcept of sustainability or sustainable development became thefocus of a major worldwide discussion, and since the 1990s it hasbecome the dominant model of societal development (Noll, 2002).The textual connotation of the concept is still controversially debatednot only in politics but also in science, which is why there is no single,generally accepted definition of sustainability, although the one givenby the “Brundtland Commission” is now generally recognized as acertain standard. Controversial topics include for example the equal

Contents lists available at ScienceDirect

journal homepage: www.elsevier.com/locate/enpol

Energy Policy

http://dx.doi.org/10.1016/j.enpol.2014.01.0350301-4215 & 2014 International Atomic Energy Agency. Published by Elsevier Ltd. All rights reserved.

n Corresponding author. Tel.: þ41 56 310 26 49; fax: þ41 56 310 44 11.E-mail address: peter.burgherr@psi.ch (P. Burgherr).

Energy Policy 74 (2014) S45–S56

treatment of the three pillars or dimensions of sustainability(Levett, 1998), the degree to which different types of capital can besubstituted or not (Dietz and Neumayer, 2007; Hueting and Reijnders,2004; Rapp Nilsen, 2010), and the time period of concern (Kates et al.,2005).

Concurrently to the intensified discussion on sustainable develop-ment, the importance of accurately assessing accident risks in theenergy sector has been emphasized by different authors since the1980s (Fritzsche, 1989; Inhaber, 2004; Rasmussen, 1981). Neverthe-less, energy-related accidents were generally just addressed as part oftechnological accidents instead of a separate and in-depth analyticaltreatment (Fritzsche, 1992). To improve this situation and to establisha comprehensive and consistent framework for comparative riskassessment, the Paul Scherrer Institut (PSI) initiated in the early1990s a long-term research activity, with the Energy-related SevereAccident Database (ENSAD) constituting its quantitative foundation(Hirschberg et al., 1998).

In recent years, the comparative assessment of accident risks hasbecome an integral component within the broader context of energysecurity, sustainability and critical infrastructure protection (e.g.,Cherp and Jewell, 2011; Chester, 2010; Johansson, 2013; Sovacool,2013; von Hippel et al., 2011; Winzer, 2012; Yustaa et al., 2011). Theseoverarching topical areas also comprise overlapping zones, whichmake them all relevant in a comparative evaluation of energysystems, and thus require strengthening of interfaces as well as choiceof objective, transparent and adequate indicators to support informeddecision making (Gray and Wiedemann, 1999; Jeswani et al., 2010;Johansen and Rausand, 2014; Musango and Brent, 2011; Singh et al.,2009). Furthermore, energy security is conceptually closely linked tothe areas of resilience, vulnerability and risk governance (e.g., Coaffee,2008; Filippini and Silva, 2012; Kröger and Zio, 2011; Molyneaux et al.,2012; Renn et al., 2011).

There is a broad range of low carbon energy technologies thatcould contribute towards a more sustainable energy future andmitigation of climate change effects. Several studies have lookedat environmental, economic and social aspects of new renew-ables and nuclear technologies as well as decarbonizing fossilfuels by means of carbon capture and storage (e.g., Brook, 2012;Karakosta et al., 2013; Kharecha and Hansen, 2013; Lior, 2012;Torvanger and Meadowcroft, 2011). However, risks of severeaccidents are generally not analyzed in a detailed and compre-hensive manner, although their impacts can affect the environ-mental (e.g. land and water contamination), economic (e.g.damage and external costs) and social (e.g. human health andrisk aversion) dimensions of sustainability, as well as the avail-ability (e.g. supply disruption), acceptability (e.g. public percep-tion) and accessibility (e.g. import dependency and transit)dimensions of energy security. Furthermore, consideration ofaccident risks is also important with regard to different timehorizons because for example their consequences can be short (e.g. immediate fatalities) or long (e.g. latent fatalities, land andwater contamination) term.

1.2. Comparative assessment of accident risks

Since its initial release (Hirschberg et al., 1998), PSI's databaseENSAD and its associated framework for comparative risk assess-ment have been regularly extended and refined with respect toscope, coverage and content. New elements and methodologicaldevelopments include:

– consideration of new primary information sources to con-stantly improve data quality and the level of completeness;for example coal China (Burgherr and Hirschberg, 2007;Hirschberg et al., 2003) and oil spills (Burgherr, 2007;Burgherr et al., 2012b),

– coupling of ENSAD with Geographic Information Systems (GIS)to enable geo-statistical analyses and risk mapping (Burgherr,2009),

– application of a simplified level-3 Probabilistic Safety Assess-ment (PSA) for nuclear energy (Burgherr et al., 2008),

– estimation of external costs (Burgherr et al., 2004),– evaluation of new renewable technologies (Burgherr, 2011),– risk indicators for Multi-Criteria Decision Analysis (MCDA)

(Eckle et al., 2011; Roth et al., 2009) and– consideration of intentional attacks on energy infrastructure

and terrorist threat (Burgherr and Hirschberg, 2009).

Due to the substantially growing interest for accident risks in theenergy sector and its above described embedding within the broadercontext of sustainability and energy security, an increasing number ofstudies have been published addressing a variety of aspects. Theseinclude analyses of specific energy chains and/or infrastructure typessuch as coal mining (Chen et al., 2012; Maiti et al., 2009; Saleh andCummings, 2011), oil spills (Kontovas et al., 2010; Psarros et al., 2011),offshore installations (Vinnem, 2011), refineries (Okoh and Haugen, inpress; Szklo and Schaeffer, 2007), risks of new renewables (e.g.photovoltaics (Fthenakis and Kim, 2011), wind power (Mabel et al.,2010), and bioethanol (Delzeit and Holm-Müller, 2009)), and com-parative studies (e.g., Colli et al., 2009; Sovacool, 2008). Finally, Felder(2009) compared PSI's database ENSAD (Hirschberg et al., 1998) withthe Major Energy Accident (MEA) database (Sovacool, 2008), and alsoprovided some recommendations for future energy accident research.However, in the case of ENSAD his analysis was based on the firstrelease (Hirschberg et al., 1998), thus not taking into account any ofthe subsequent developments.

The main objectives of this article are the following: (1) toprovide a detailed overview of the framework for comparative riskassessment based on ENSAD, (2) to present a range of riskmeasures covering the various consequences and impacts ofenergy-related accidents, (3) to compare in an objective andquantitative manner the strengths and weaknesses of a broadportfolio of energy technologies, and (4) to support decisionmaking processes towards a more sustainable energy future.

2. Methodology

2.1. Overview of methodological framework

Fig. 1 provides an overview of PSI's comprehensive and inte-grated framework for comparative risk assessment, at the center ofwhich is PSI's Energy-related Severe Accident Database (ENSAD).While the initial scope was limited to technological accidents,more recently also accidents triggered by natural hazards, andintentional attacks on energy infrastructure including terroristthreat were included (Burgherr et al., 2011, 2008; Burgherr andHirschberg, 2009). For fossil energy chains and hydropowerextensive empirical evidence from actual accidents contained inENSAD is used; however, in the case of hydropower the analysis iscomplemented by site-specific consequence modeling. For nuclearpower a site-specific level-3 Probabilistic Safety Assessment (PSA)is applied to estimate accident consequences. For new renewabletechnologies a so-called “hybrid” approach is utilized, whichcombines available accident data (mostly relatively scarce due tolimited historical experience) with chain-specific modeling andexpert judgment. Comparative evaluations of energy technologiesencompass a broad range of analytical methods, including:

– various consequence indicators (see also severe accident defi-nition in Section 2.2),

– frequency-consequence (F–N) curves,

P. Burgherr, S. Hirschberg / Energy Policy 74 (2014) S45–S56S46

– advanced methods such as Bayesian networks, fat-tail distribu-tion fitting, multivariate techniques and data mining usingtree-based models,

– coupling of ENSAD with Geographic Information Systems (GIS)to enable geo-statistical approaches and risk mapping,

– economic loss and external cost estimation and– evaluation of risk indicators within Multi-criteria Decision

Analysis (MCDA) to provide support to decision processes.

2.2. Boundary conditions and main assumptions

To ensure that the above described framework can be appliedconsistently to a broad range of energy technologies, a number ofassumptions and boundary conditions need to be defined in aclear and transparent manner.

2.2.1. Severe accidentIn the literature no common definition of the term severe

accident exists, i.e. different databases may differ with respect totheir specific scope and purpose (Burgherr and Hirschberg, 2008a).ENSAD clearly focuses on severe accidents because industry,decision makers, regulators and the general public are mostconcerned about these. Furthermore, a much higher level ofcompleteness and accuracy in accident reporting among countriescan be assured, which is a necessary prerequisite for worldwidecomparisons. Within ENSAD the following seven criteria andthresholds are used to define a severe accident:

(1) at least 5 fatalities or(2) at least 10 injured or(3) at least 200 evacuees or(4) extensive ban on consumption of food or(5) releases of hydrocarbons exceeding 10,000 metric tons or(6) enforced clean-up of land and water over an area of at least

25 km2 or(7) economic loss of at least 5 million USD(2000).

The completeness, accuracy and robustness of the differentconsequence indicators has been presented before (e.g., Burgherr

and Hirschberg, 2008a; Burgherr et al., 2008, 2004; Hirschberget al., 2004)

It should be mentioned that ENSAD also contains accidentswith smaller consequences, but these are not collected with thesame rigor and effort, and differences among countries arepotentially larger because of considerable differences in reporting.

2.2.2. Complete energy chainsWhen comparing risks among different energy chains, it is

pivotal that one looks at the complete chains because the risks tohuman health, the environment and society do not only arise fromthe actual power and/or heat production, but all stages of thechain (Burgherr and Hirschberg, 2008a; Hirschberg et al., 2004).For example, in the coal chain accidents in the mines are prevail-ing, for oil it is resource extraction and transportation of crude andproducts, and in the case of hydropower accidents are practicallylimited to dam failures during operation as well as construction(Burgherr and Hirschberg, 2008a, b). In general, an energy chainmay comprise the following stages: exploration, extraction, pro-cessing and storage, long distance transport, regional and localdistribution, power and/or heat generation, waste treatment, anddisposal. However, one should be aware that not all these stagesare applicable to every energy chain.

2.2.3. Evaluation periodIn this study, severe accidents in the energy sector are analyzed

for the period 1970–2008. The start year was chosen becauseenergy-related severe (Z5 fatalities) accidents distinctivelyincreased at the end of the 1960s, which is primarily due to thehigher volume of activities (Hirschberg et al., 1998), whereas theend year corresponds to the consolidated database status at theend of the European Union project SECURE (Burgherr et al., 2011).Thus the selected period of observation covers more than threedecades of historical experience.

2.2.4. Data normalizationEnergy chain comparisons should be based on understandable

and meaningful risk indicators that cover a broad range of aspects.Direct comparisons among energy chains can be facilitated bynormalized indicators. Therefore, indicator values are normallyexpressed per unit of electricity production. For fossil energychains the thermal energy is converted to an equivalent electricaloutput using a generic efficiency factor of 0.35, but depending onthe technology portfolio and study objectives one may also usetechnology-specific efficiencies. For nuclear and hydro power thenormalization is straightforward since in both cases the generatedproduct is electrical energy. The Gigawatt-electric-year (GWeyr)was chosen because large individual plants have capacities in theneighborhood of 1 GW of electrical output (GWe), which makes ita natural unit to compare technologies on a common basis(Hirschberg et al., 1998).

2.2.5. Regional aggregationIndicator results can be calculated at different spatial scales, e.g.

worldwide, country groups, individual countries or country sub-divisions. However, the level of resolution is normally determinedby the amount of data. Therefore, often the intermediate level ofcountry groups is chosen because it allows for differentiationbetween well-defined entities. Generally, indicators are providedseparately for three major country groups, namely the Organisa-tion for Economic Co-operation and Development (OECD), theEuropean Union (EU 27), and states that are not OECD members(non-OECD). It should be noted that some countries are membersin both OECD and EU 27, thus creating some partial overlap, whichis considered acceptable because both country groups are essential

Comparative Risk Assessment

Man-MadeTechnical

Na-Tech Malicious

Accident risks

Evaluation Data Set for Technology Comparison- Consequence indicators, F-N curves, Bayesian networks- GIS, geo-statistics, risk mapping- Economic loss and external cost estimates- Multi-criteria analysis, decision support

Technologicalaccidents

Terrorist threat,sabotage, theft,vandalism

Accidents triggeredby natural hazard

ENSAD

PSA

Hybrid

Energy-related Severe Accident Database

Simplified level-3 PSA

Statistics, modeling, expert judgement

Fig. 1. Comprehensive framework for comparative risk assessment in the energy sector.

P. Burgherr, S. Hirschberg / Energy Policy 74 (2014) S45–S56 S47

at a global scale. While OECD and EU 27 are comprised of mostlyhighly developed countries, non-OECD is considered a moreheterogeneous mix of emerging and developing countries. Thiscategorization is also meaningful because of the substantialdifferences in management, regulatory frameworks and generalsafety culture between countries (e.g., Burgherr and Hirschberg,2008a; Hirschberg et al., 2004). In the case of the Chinese coalchain it has been previously shown that a separate treatment isnecessary because it performs significantly different from theother non-OECD countries (Burgherr and Hirschberg, 2007;Hirschberg et al., 2003). Lastly, it should be mentioned thatassignment of countries to groups can also be done in a moresophisticated manner to elaborate further on differences betweenthe above mentioned main groups (Burgherr et al., 2012a). In thecase of nuclear, site-specific results using a simplified level-3 PSAare used. For renewable energy technologies except large hydro-power, estimates can be considered representative for developedcountries (e.g., OECD and EU 27). Calculations for new renewablesare based on the methodology described in Burgherr (2011);however, in the case of wind updated data for the years 2000–2012 were used.

2.3. Statistical analyses

First, an overview of the available ENSAD data (1970–2008) forthe various energy chains and country groups (OECD, EU 27, non-OECD) is given, considering severe accidents that resulted in atleast five fatalities or ten injured persons, although in subsequentanalyses the focus is on fatalities only.

Second, severe (Z5 fatalities) accidents in fossil energy chains(coal, oil, and natural gas) are analyzed, for which extensivehistorical data is available in the ENSAD database, including:

a. Investigation of the distribution of accidents across energychain stages and description of major accident causes toidentify chain-specific risks.

b. Calculation of cumulative curves of individual country shares tototal numbers of accidents and fatalities to examine country-specific patterns.

c. For each fossil energy chain a country comparison based on thethree risk indicators accident rate, fatality rate and maximumconsequences of a single accident was performed. For this,principal component analysis (PCA) was used, which has beenfirst introduced by Pearson (1901) and Hotelling (1933), but seeJolliffe (2002) for a detailed overview. In a nutshell, PCA is amathematical procedure that transforms a set of possiblycorrelated variables (here indicators) into a smaller numberof uncorrelated variables, the so-called principal components(PC). Computations were done with the ADE-4 software(Chessel and Dolédec 1996; Thioulouse et al. 1997).

d. Based on country data, average normalized accident (AR) andfatality (FR) rates (per GWeyr) as well as confidence intervals(5% and 95%) were calculated for OECD and non-OECDcountries.

It should be noted that for the analyses in c and d those non-OECD countries were excluded whose AR and/or FR values were atleast one order of magnitude higher than the for the other non-OECD countries, i.e. 3 were eliminiated for coal, 11 for oil, and 1 fornatural gas.

Third, fatality rates and maximum consequences were com-pared across a broad range of fossil, nuclear, hydropower and newrenewable technologies.

Fourth, external costs of severe (Z5 fatalities) accidents wereestimated for fossil and hydropower chains, and compared to ownand other literature values for nuclear. Assumptions for external

cost calculation were based on the results of the EU projectNewExt (for details see, Burgherr et al., 2004), i.e. EUR 1,045,000for the value of statistical life (VSL), and degrees of internalizationwere 0.8 (OECD, EU 27) and 0.5 (non-OECD) for occupationalfatalities, vs. 0.5 and 0.2 for public fatalities.

3. Results and discussion

The presentation and discussion of results is divided into fourdistinct sections, namely (1) the historical experience available inENSAD, (2) analysis of major fossil energy chains, (3) comparativeevaluation of accident risks for fossil, hydro, nuclear and new renew-able technologies and (4) estimation of external costs of severeaccidents.

3.1. Current status and content of ENSAD

In total, ENSAD currently comprises 32705 accident records. Ofthese 83.2% are classified as man-made, 16.3% as natural disasters,and 0.5% as conflicts. Among man-made accidents, 20245 areattributable to the energy sector, and of these 93.8% occurred inthe years 1970–2008.

According to ENSAD's severe accident definition, 3367 acci-dents resulted in at least 5 fatalities in the years 1970–2008, 968 inat least 10 injured persons, and 515 met both criteria. Tables 1 and2 provide an overview of severe accidents per energy chain andcountry grouping for fatalities and injured, respectively.

While ENSAD provides a comprehensive coverage of severeaccidents in fossil energy chains, the historical experience for severeaccidents in hydropower, particularly for OECD and EU 27, is lesspronounced. Therefore, additional information on hypothetical damfailure scenarios is included in Section 3.3. For nuclear energy a moredetailed coverage is provided in Section 3.3 including a summary ofsimplified level-3 PSA results as well as of the three core melt eventsthat took place at nuclear power plants so far. Finally, ENSAD currentlydoes not contain many severe accidents for new renewable

Table 1Summary of severe accidents (Z5 fatalities) per energy chain and country groupfor the period 1970–2008. Acc¼accidents, Fat¼ fatalities, and LPG¼LiquefiedPetroleum Gas (LPG).

Energy Chain OECD EU 27 non-OECD

Acc Fat Acc Fat Acc Fat

Coal 87 2259 45 989 2394a 38,672162 5788818 11,3021214 15,750

Oil 187 3495 65 1243 358 19,516Natural gas 109 1258 37 367 78 1556LPG 58 1856 22 571 70 2789Hydro 1b 14 1c 116 9d 3961

12 26,108e

Nuclearf – – – – 1 31Biofuel – – – – – –

Biogas – – – – 2 18Geothermal – – – – 1 21

a First line non-OECD total, second line non-OECD w/o China, third line China1994–1999, fourth line China 2000–2008.

b Teton dam failure (USA, 1976).c Belci dam failure (Romania, 1991).d First line non-OECD w/o China, second line China.e Banqiao/Shimantan dam failures (China, 1975) together caused 26,000 fatal-

ities.f Only immediate fatalities of the Chernobyl accident are shown here. See text

for a more detailed discussion of the nuclear chain.

P. Burgherr, S. Hirschberg / Energy Policy 74 (2014) S45–S56S48

technologies, which is why accident risk estimates cannot be based onempirical evidence alone.

3.2. Major fossil energy chains

In this section the focus is on coal, oil and natural gas energychains, for which ENSAD comprises extensive historical accidentdata for the period 1970–2008. In Fig. 2 accidents (left) andfatalities (right) are assigned to different energy chain stages. Inthe coal chain accidents during mining activities are by fardominating, which is also reflected when looking at fatalities.Overall, gas accidents are the main cause of mining accidents,although for example in China also water hazard leading to mineflooding with a share of 15% is a concern (Burgherr andHirschberg, 2007). In the oil and gas chains the majority ofaccidents are attributable to the transport & storage stage. Thesecond and third most common causes are extraction & explora-tion and refinery processing in the case of oil, and heating & powerplant and extraction & exploration for natural gas. Transportaccidents in the oil chain predominantly relate to ships andpipelines in long distance transport, whereas in regional and localdistribution it is road tankers. For natural gas, pipelines are themajor means of transport, except in the case of liquefied naturalgas (LNG) that is transported by specially designed ships. A more

detailed discussion of accident causes can be found in Burgherrand Hirschberg (2009).

Fig. 3 shows cumulative curves of individual country shares tototal accidents (a) and fatalities (b) in fossil energy chains. The coalchain exhibits a clearly different pattern than oil and natural gas,which is due to mining accidents in China that contribute 90%,while all other countries together account for the remaining 10%.Therefore, the Chinese coal chain needs to be analyzed separately(Burgherr and Hirschberg, 2007; Hirschberg et al., 2003). Ukraine,Russia and India are ranked second to fourth, followed by Poland,USA and Turkey as first OECD countries. In contrast, the cumulatedaccident curves for oil and natural gas show much less extremedistributions than coal. Curves for fatalities show similar patterns,except that for the oil chain the curve shows a steeper increasewith already the top three countries making up 50% of all fatalities,whereas for natural gas it is the top five. However, the two curvesare then approaching each other, and above the 75th percentilethis trend is actually reversed.

The evaluation of country specific patterns in fossil energychains was based on the three risk indicators accidents per GWeyr,fatalities per GWeyr and maximum consequences of a singleaccident, using data of the period 1970–2008. The results ofprincipal component analysis (PCA) showing the position ofcountries on the first two principal component (PC) axes are givenin Figs. 4–6.

Fig. 4 shows the positions of individual countries for the coalchain on the PC1� PC2 factorial map. The first two PC axesaccounted for 58% and 34.5%, of the total variation. Both accident(AR) and fatality rates (FR) were negatively related to PC1, withhigher AR values on the positive and higher FR values on thenegative PC2 half-axis, respectively. PC2 was best explained bymaximum consequences, i.e. higher values were attributable tothe negative half-axis. Overall, positions of OECD countries form arather clear cluster, except for Turkey where one very deadlyaccident in 1992 resulted in 272 fatalities, which is almost threetimes greater than the second largest. EU 27 countries (see inset)also distinctly cluster with only Romania having a substantiallyhigher FR. Non-OECD countries exhibit a more heterogeneouspatterns, with some of them close to the bulk of OECD and EU27 countries. However, with exception of South Africa and Russiafor these countries only few accident data are available, whichmeans that their positioning may rather reflect an underreportingproblem than their actual performance. Finally, BRIC countries areclearly divided in two groups, i.e. China and India vs. Brazil andRussia because the former experienced significantly higher max-imum consequences, and particularly in the case of China a highFR was observed.

Fig. 5 displays the positions of countries for the oil chain, withPC1 and PC2 explaining 59.5% and 34% of total variation. Accident

Table 2Summary of severe accidents (Z10 injured) per energy chain and country groupingfor the period 1970–2008. Acc¼accidents, Inj¼ injuries, and LPG¼Liquefied Pet-roleum Gas (LPG).

Energy Chain OECD EU 27 non-OECD

Acc Inj Acc Inj Acc Inj

Coal 27 761 15 531 211a 515544 141417 468107 2262

Oil 170 8285 62 2128 194 16,411Natural gas 133 5239 25 866 88 13,486LPG 81 11,490 22 1147 58 4053Hydro 1b 800 – – 3 62Nuclearc – – – – 1 370Biofuel – – – – 1 21Biogas – – – – – –

Geothermal – – – – – –

a First line non-OECD total, second line non-OECD w/o China, third line China1994–1999, fourth line China 2000–2008.

b Teton dam failure (USA, 1976).c Only immediate injuries of the Chernobyl accident are shown here. See text

for a more detailed discussion of the nuclear chain.

Fig. 2. Relative shares of (a) accidents and (b) fatalities in the different stages of fossil energy chains for the period 1970–2008.

P. Burgherr, S. Hirschberg / Energy Policy 74 (2014) S45–S56 S49

and fatality rates are positively related to PC1 with FR on thepositive and AR on the negative PC2 half-axis, whereas maximumconsequences are related positively to PC2. Similarly to coal, OECDand EU 27 countries show a tighter clustering than non-OECDcountries. The non-OECD countries overlapping with the OECDcluster are again those for which only few accident data areavailable. For the oil chain, BRIC countries group close togetherwith the exception of Brazil. This can again be attributed to asingle large accident, i.e. a pipeline explosion and fire in 1984 thatdestroyed a shanty town southeast of Sao Paulo killing 508 people.

Fig. 6 shows the positions of countries for the natural gas chainon the PC1� PC2 factorial plane, accounting for 64.4% and 34.4%,respectively. The fatality rate and maximum consequences arepositively related to PC1 and PC2, respectively, whereas in the caseof the accident rate there is a contribution to both axes although

stronger to PC1. The OECD and EU 27 country groups are againrather compact, with one exception each, namely South Korea(single accident with 109 fatalities in 1995) and Greece (high ARand FR). The isolated position of China is primarily due to a verylarge accident in which a well explosion caused 243 fatalities,whereas in the case of Philippines the reason is again limited data.Non-OECD countries show again more variation for natural gas,however, there is a stronger overlap with OECD and EU 27 than inthe case of oil and coal chains, which for most countries is againdue to limited accident data.

In a last step, average, normalized accident and fatality ratesbetween OECD and non-OECD countries were compared (Fig. 7).Both rates were clearly lower in OECD for the coal and oil chains,whereas for natural gas only fatality rates differed. For all threechains confidence intervals were much smaller for OECD than

0

20

40

60

80

100

0 10 20 30 40 50 60 70 80 90

Chi

naU

krai

neR

ussi

aIn

dia

Pol

and

Turk

eyC

olom

bia

Pak

ista

nS

outh

Afri

ca

US

A

Japa

n

Nigeria

USA

JapanRussia

China

India

ItalyUK

MexicoIran

USA

Russia

China

Mexico

GermanyFrance

ItalyAlgeria

KazakhstanPakistan

Cum

ulat

ive

perc

enta

ge o

f acc

iden

ts (%

)

0

20

40

60

80

100

0 10 20 30 40 50 60 70 80 90

Number of Countries

Coal Oil Natural Gas

Cum

ulat

ive

perc

enta

ge o

f fat

aliti

es (%

)

China

UkraineRussia

India

JapanTurkey

USATaiwan

Mozambique

Nigeria

USA

Colombia

Egypt

China

India

Afghanistan

Philippines

Mexico

Brazil

USA

Russia

China

Mexico

GeorgiaVenezuela

Lebanon

Japan

South Korea

Zimbabwe

Serbia & Montenegro

Kazakhstan

Fig. 3. Cumulative curves of (a) accident and (b) fatality shares by country for the period 1970–2008. Open and filled symbols denote OECD and non-OECD countries,respectively.

P. Burgherr, S. Hirschberg / Energy Policy 74 (2014) S45–S56S50

non-OECD countries, indicating that OECD is more homogenous,thus supporting the results of PCA.

3.3. Comparative evaluation

Fig. 8 shows (a) fatality rates and (b) maximum consequencesfor fossil, nuclear, large hydropower and new renewable technol-ogies in OECD, EU 27 and non-OECD countries. Among fossilchains, natural gas performs best with respect to both metrics.

The fatality rate for coal in China is distinctly higher than for theother non-OECD countries (compare also Burgherr and Hirschberg,2007; Hirschberg et al., 2003). However, a comparison betweendata for 1994–1999 and 2000–2008 is indicative that China isimproving and slowly approaching the non-OECD level.

Among large centralized technologies, modern nuclear andOECD hydropower plants show the lowest fatality rates, but atthe same time the consequences of extreme accidents can be verylarge. Experience with hydropower in OECD countries points to

-4

-3

-2

-1

0

1

2

-7 -6 -5 -4 -3 -2 -1 0 1 2

Turkey

ColombiaPhilippines

Bosnia & Herzegovina

PakistanIran

Georgia

India

China

Serbia & Montenegro

PC1

PC2

-1

1

0 1

RussiaJapan

Mexico

KazakhstanThailand

Brazil

Romania

BulgariaIndonesia

Chile

South Africa

OECD

non-OECDOECD / EU27

non-OECD / BRICnon-OECD / EU 27

Fig. 4. Positions of individual countries for the coal chain in the PC1�PC2 factorial plane, based on three risk indicators calculated from ENSAD data for the period1970–2008.

-1

0

1

2

3

4

5

-2 -1 0 1 2 3 4 5

Brazil

China

India

Russia

Colombia

Egypt

Mexico

PC1

PC2

-1

1

-1 -0.2

UK

CanadaUSA

Italy

Venezuela

IndiaChina

Iran

Romania

Norway

SaudiArabia

Bahrain

Taiwan

SingaporeTurkey

Greece

Portugal

Ireland

OECD

non-OECDOECD / EU27

non-OECD / BRICnon-OECD / EU 27

Fig. 5. Positions of individual countries for the oil chain in the PC1�PC2 factorial plane, based on three risk indicators calculated from ENSAD data for the period 1970–2008.

P. Burgherr, S. Hirschberg / Energy Policy 74 (2014) S45–S56 S51

very low fatality rates, comparable to the representative PSA-based results obtained for nuclear power plants, whereas in non-OECD countries, dam failures can claim large numbers of victims.

To date, three core-melt events have occurred in nuclear powerplants, namely at Three Mile Island 2 (TMI-2, USA, 1979), Cherno-byl (Ukraine, 1986), and Fukushima Daiichi (Japan, 2010). At TMI,the collective effective dose to the public was about 40 person-Sv,on the basis of which one extra cancer fatality was estimated.However, 144,000 people were evacuated from the area aroundthe plant. For more information see Hirschberg et al. (1998). TheChernobyl accident resulted in 31 immediate fatalities. PSA-basedmaximum consequences including expected latent fatalities range

from about 9000 for Ukraine, Russia and Belarus to about 33,000for the whole northern hemisphere in the next 70 years(Hirschberg et al., 1998). According to a comprehensive study bynumerous United Nations organizations up to 4000 persons coulddie due to radiation exposure in the most contaminated areas(Chernobyl Forum, 2005). This estimate is substantially lower thanthe upper limit of the PSI interval, which, however, was notrestricted to the most contaminated areas.

Published health effects of the Fukushima Daiichi nuclearaccident show large variations, which is partially attributable todifferent assumptions and methods used. For example, Ten Hoeveand Jacobson (2012b) estimated an additional 130 (range 15–1100)

-3

-2

-1

0

1

2

3

4

5

-2 -1 0 1 2 3 4 5PC1

PC2

OECD

non-OECDOECD / EU27

non-OECD / BRICnon-OECD / EU 27

Greece

South Korea

Russia

Philippines

China

Turkey

Mexico

Japan

Qatar

Ukraine

Colombia

Saudi Arabia

Venezuela

IndiaAlgeria

Iran

PakistanArgentina

Indonesia Egypt Bangladesh

Fig. 6. Positions of individual countries for the coal chain in the PC1�PC2 factorial plane, based on three risk indicators calculated from ENSAD data for the period 1970–2008.

1.42E-2

1.46E-1

8.74E-3

7.05E-2

1.65E-2

2.92E-2

4.20E-1

2.31E+0

2.17E-1

1.69E+0

2.16E-1

6.94E-1

1.E-3

1.E-2

1.E-1

1.E+0

1.E+1

OECD non-OECD OECD non-OECD OECD non-OECD

Coal Oil Natural Gas

Accidents Fatalities

Acc

iden

ts a

nd fa

talit

ies

per G

Wey

r

Fig. 7. Comparison of fossil accident and fatality rates (per GWeyr) between OECD and non-OECD countries in the period 1970–2008. Averages and confidence intervals(5% and 95%) are shown.

P. Burgherr, S. Hirschberg / Energy Policy 74 (2014) S45–S56S52

worldwide cancer related latent fatalities, von Hippel (2011) about1000, and Beyea et al. (2013) about 600. See also the discussion in(Richter, 2012; Ten Hoeve and Jacobson, 2012a). In contrast,Vitázková and Cazzoli (2011) reported a much higher value of10,000, and some rather extreme sensitivity cases up to 300,000

latent fatalities. Finally, the World Health Organization (WHO,2013) recently published an authoritative report concerning healthrisks inside and outside Japan.

According to current knowledge, new Generation III reactorsare expected to have significantly lower fatality rates than

Fig. 8. Comparison of (a) fatality rates (with 5% and 95% confidence intervals) and (b) maximum consequences of a broad selection of energy technologies. Fossil andhydropower is based on the ENSAD database (period 1970–2008); for nuclear a simplified level-3 PSA is applied; and for other renewable sources a hybrid approach usingavailable data, modeling and expert judgment is used. Abbreviations: PWR¼pressurized-water reactor; EPR¼European Pressurized Reactor; CH¼Switerland; RBMK¼reak-tor bolshoy moshchnosty kanalny, a boiling watercooled graphite moderated pressure tube type reactor; PV¼photovoltaic; CHP¼combined heat and power; andEGS¼Enhanced Geothermal Systems.

P. Burgherr, S. Hirschberg / Energy Policy 74 (2014) S45–S56 S53

currently operating power plants because of various safety aug-menting systems, but maximum consequences could increase dueto the much larger radioactive inventory of a 1600 MW EPR, asindicated by the results of a simplified level-3 PSA (Burgherr et al.,2011, 2008; Roth et al., 2009).

As mentioned above historical experience for Hydropower inOECD and EU 27 countries suggest rather limited consequences of14 and 116 fatalities, respectively. Additional analyses based onempirical and theoretical approaches suggest that a hypotheticaldam failure could have much more dramatic consequences in theorder of up to 11'000 fatalities assuming zero pre-warning time(Burgherr and Hirschberg, 2005, and references therein) However,this result is strongly dependent on the model chosen and theactual pre-warning time, among various other factors. For exam-ple, a sufficient pre-warning time of two hours would reducefatalities to 2–27. Additionally, potential consequences have to beviewed under consideration of the frequency of occurrence of suchan event, which for Swiss dams is in the range 10�5 to 10�4 eventsper dam year (Hirschberg et al., 1998).

All other renewable technologies exhibit distinctly lower fatal-ity rates than fossil chains, and are fully comparable to hydro andnuclear power in highly developed countries. Concerning max-imum consequences, those renewable sources clearly outperformall other technologies because their decentralized nature stronglylimits their catastrophic potential. However, it should be notedthat current analyses of risks associated with new renewableshave limited scope and do not include probabilistic modeling ofhypothetical accidents. This may have bearing, particularly onresults for solar PV. Furthermore, future large-scale storage ofintermittent electricity produced by PV, solar-thermal or windusing hydrogen may add another risk component (Dubois et al.,2013) that is not considered in current analyses.

3.4. External cost of centralized technologies

Table 3 shows external costs of fossil energy chains and hydro-power in OECD, EU 27 and non-OECD countries for the period 1970–2008, using ENSAD data. The external cost value for nuclear repre-sents a calculation for a Swiss power plant (Hirschberg et al, 1998),whereas estimates for new renewables are restricted to thosetechnologies, for which sufficient data was available.

Among fossil energy chains, OECD and EU 27 countries gen-erally exhibit significantly lower external costs than non-OECD,except for natural gas, where the difference is distinctly lesspronounced. For the coal chain, values for China are about oneorder of magnitude higher than for other non-OECD countries andeven two orders of magnitude compared to OECD and EU 27. Thevalue for hydropower in non-OECD is strongly driven by theextremely deadly accident of the Banqiao/Shimantan dam failures(China, 1975) that resulted in 26'000 fatalities. If this accident isexcluded then the external cost estimate for non-OECD decreasesby about one order of magnitude.

The external costs shown in Table 3 are for immediate fatalitiesof severe (Z5 fatalities) accidents, however, in the case of nuclearthey are dominated by latent fatalities. PSA-based external costsfor an accident at a Swiss nuclear power plant have been assessedat 1.02E-3 EUR-cents/kWh (1.2E-3 USD-cent/kWh)1, with 5th and95th percentiles at 8.88E-5 and 3.24E-3 EUR-cents/kWh (1.0E-4and 3.8E-3 USD-cent/kWh), respectively (Hirschberg et al., 1998).It should be noted that these results include radiation-inducedhealth effects that are totally dominated by latent cancer fatalities.Additionally, Hirschberg et al. (1998) used for the value ofstatistical life (VSL) 4 million USD (3.41 million EUR) compared

to the value of 1 million EUR established in the EU project NewExt(Burgherr et al., 2004). Other studies claim significantly higherexternal costs for nuclear, for example Laes et al. (2011) report1.04E-2 EUR-cents and Rabl and Rabl (2013) even 3.80E-1 EUR-cents/kWh, both of which are significantly higher values than theSwiss estimate. However, due to differences in methodologies andassumptions, these values are difficult to compare.

Table 3 also provides results for selected new renewables forwhich external costs could be calculated based on available fatalitydata in ENSAD. For wind onshore and geothermal estimated valueswere in the same range as for hydro in OECD countries, whereasbiogas was in the same order of magnitude as for natural gas inOECD and EU 27 countries. However, one should also keep in mindthat external costs for new renewables are based on currentlyavailable experience that is less comprehensive than for largecentralized technologies. In contrast to the present study, Rabl andRabl (2013) report a significantly higher external cost due toaccidents for wind augmented by NGCC (natural gas combinedcycle) of 4.0E-1 EUR-cents/kWh, and the 4.87E-2 EUR-cents/kWhfor wind by Rentizelas and Georgakellos (in press) are also clearlyhigher. Again it is difficult to establish concrete reasons for thesedifferences, but it seems that these two studies included healthimpacts from air pollution into their estimates. For example,Larsson et al. (2014) provide a broad overview of external costsquantified in different studies. Although their comparison did notexplicitly address external costs from severe fatal accidents, itsuggests that external costs from impacts on human health,climate change, and other environmental issues dominate.

Following the Fukushima nuclear power accident the discus-sion on accident costs has recently been addressed from differentviewpoints, including additional aspects such as social and institu-tional factors, risk targets and threat beliefs (Al Shehhi et al., 2013;Aoki and Rothwell, 2013; Hartmann et al., 2013; Kampanart et al.,

Table 3External costs of different energy technologies are given in EUR-cents per kWh.Values for fossil energy chains and hydropower in OECD, EU 27 and non-OECDcountries are based on ENSAD data for the period 1970–2008. Nuclear results arefor a Swiss power plant including radiation-induced health effects that are clearlydominated by latent cancers (for details see Hirschberg et al., 1998). New renew-ables include wind onshore Germany, biogas and geothermal, and estimates can beconsidered representative for OECD and EU 27.

Chain Region External cost[EUR-cents/kWh]

Coal OECD 2.90E-4EU 27 3.33E-4Non-OECD w/o China 3.44E-3China 1994–1999 3.53E-2China 2000–2008 1.54E-2China 1994–2008 2.02E-2

Oil OECD 3.86E-4EU 27 3.92E-4non-OECD 8.58E-3

Natural Gas OECD 3.50E-4EU 27 3.90E-4non-OECD 9.56E-4

Hydro OECD 1.50E-5EU 27 4.90E-4non-OECD 6.71E-2non-OECD w/o Banqiao/Shimantan 9.10E-3

Nuclear CH 1.02E-3*

Wind Onshore GER 1.50E-5

Biogas OECD / EU 27 1.26E-4

Geothermal OECD / EU 27 1.57E-5

n Original value 1.20 USD-cents/kWh; exchange rate 1 USD¼0.85355 EUR.

1 Exchange rate 1 USD¼0.85355 EUR.

P. Burgherr, S. Hirschberg / Energy Policy 74 (2014) S45–S56S54

in press; Srinivasan and Gopi Rethinaraj, 2013; Vitazkova andCazzoli, 2013).

4. Conclusions

PSI's comparative risk assessment approach provides a com-prehensive framework for the objective and quantitative evalua-tion of severe accidents in the energy sector, with the databaseENSAD as the core element.

This study demonstrates how accident risks (e.g. in terms offatalities) can be effectively evaluated for different regionalaggregates within the same energy chain and across technologies.Differences among country groups and technologies wereexplored using various risk indicators representative of technicalperformance (e.g. accident and fatality rate) and socio-economicimpacts (e.g. maximum consequences and external costs).

Among the major centralized technologies, expected accidentrisks are by far lowest for hydro and nuclear, whereas performanceof fossil energy chains is significantly worse. On the other hand themaximum consequences of low-frequency hypothetical severeaccidents, representing a measure of risk aversion, are by farhighest for nuclear and hydro (provided population density down-stream from the dam is high), in the middle range for fossil chainsand very small for new renewables, which are currently lesssensitive to the issue of severe accidents, although their large-scale utilization may add new risk aspects such as storage risks tobuffer and regulate intermittent production, and geopoliticalaspects when large renewable capacities are installed in less stableregions (e.g. North Africa).

External costs of severe accidents are rather insignificant whencompared to the effects of global warming and air pollution(Burgherr and Hirschberg, 2008a). Nevertheless consequences ofaccidents in terms of human health (e.g., fatalities) as well as onthe environment and society should not be neglected, since theycan have large-scale and long-term impacts (e.g. DeepwaterHorizon, Fukushima).

Our analysis of accident risks supports the notion that inaddition to decentralized new renewable technologies also cen-tralized, low carbon technologies such as nuclear and CCS couldplay an essential role towards a more sustainable energy futureand the mitigation of climate change effects. However, the majorchallenge remains to address people's safety concerns with regardto low-probability high-consequence accidents that are oftendriven by aspects of risk aversion and public acceptance, i.e.catastrophic leakage of CO2 storage or a nuclear accident with alarge release of radioactivity (NEA, 2012; Torvanger andMeadowcroft, 2011).

In summary, a comprehensive risk assessment across a broadrange of energy technologies should consider a set of riskindicators that cover different risk aspects and perspectives.Consequently no technology performs best or worst in all respects,i.e. trade-offs, compromises and priority setting are necessary todefine a consensus energy technology portfolio, whichultimately is a societal decision to which science can provideobjective and transparent inputs to foster informed and rationaledecision making and policy formulation (Bruine de Bruin andBostrom, 2013).

Acknowledgments

The analyses presented here are based on the database ENSADupdate that was partially performed within the CollaborativeProject SECURE (Security of Energy Considering its Uncertainty,

Risk and Economic implications), Contract no. 213744 of the 7thFramework Programme of European Commission.

References

Al Shehhi, A.S., Chang, S.H., Kim, S.H., Kang, H.G., 2013. Analysis of economics andsafety to cope with station blackout in PWR. Ann. Nucl. Energy 59, 63–71.

Aoki, M., Rothwell, G., 2013. A comparative institutional analysis of the Fukushimanuclear disaster: lessons and policy implications. Energy Policy 53, 240–247.

Beyea, J., Lyman, E., von Hippel, F.N., 2013. Accounting for long-term doses in“worldwide health effects of the Fukushima Daiichi nuclear accident”. EnergyEnviron. Sci. 6, 1042–1045.

Brook, B.W., 2012. Could nuclear fission energy, etc., solve the greenhouseproblem? The affirmative case. Energy Policy 42, 4–8.

Bruine de Bruin, W., Bostrom, A., Assessing what to address in science commu-nication. Proc. Natl. Acad. Sci. USA, 110, 2013, 14062–14068.

Burgherr, P., 2007. In-depth analysis of accidental oil spills from tankers in thecontext of global spill trends from all sources. J. Hazard. Mater. 140, 245–256.

Burgherr, P., 2009. Using GIS and Multivariate Analyses to Visualize Risk Levels andSpatial Patterns of Severe Accidents in the Energy Sector. In: Martorell, S.C.,Guedes Soares, C., Barnett, J. (Eds.), Safety, Reliability and Risk Analysis: Theory,Methods and Applications. Taylor & Francis Group, London (UK).

Burgherr, P., 2011. Accidents and Risks (Chapter 9.3.4.7). In: Sathaye, J., O. Lucon, A.Rahman, J. Christensen, F. Denton, J. Fujino, G. Heath, S. Kadner, M. Mirza,H.Rudnick, A. Schlaepfer, A. Shmakin, 2011: Renewable Energy in the Context ofSustainable Energy (Chapter 9). In IPCC Special Report on Renewable EnergySources and Climate Change Mitigation [O. Edenhofer, R. Pichs-Madruga, Y.Sokona, K. Seyboth, P. Matschoss, S. Kadner, T. Zwickel, P. Eickemeier, G.Hansen, S. Schlömer, C. von Stechow (Eds.)],. Cambridge University Press,Cambridge, United Kingdom and New York, NY, USA.

Burgherr, P., Eckle, P., Hirschberg, S., 2012a. Comparative assessment of severeaccident risks in the coal, oil and natural gas chains. Reliab. Eng. Syst. Saf. 105,97–103.

Burgherr, P., Eckle, P., Hirschberg, S., Cazzoli, E., 2011. Final report on severe accidentrisks including key indicators. SECURE Deliverable No. D5.7.2a. SECURE Project– Security of Energy Considering its Uncertainty, Risk and Economic Implica-tions, Brussels, Belgium.

Burgherr, P., Eckle, P., Michaux, E., 2012b. Oil tanker transportation risk: drivingfactors and consequence assessment, In: Proceedings of the 11th InternationalProbabilistic Safety Assessment and Management Conference & EuropeanSafety and Reliability Conference (PSAM11 & ESREL2012). Helsinki, Finland.

Burgherr, P., Hirschberg, S., 2005. Comparative assessment of natural gas accidentrisks. PSI Report No. 05-01. Paul Scherrer Institut, Villigen PSI, Switzerland.

Burgherr, P., Hirschberg, S., 2007. Assessment of severe accident risks in theChinese coal chain. Int. J. Risk Assess. Manag. 7, 1157–1175.

Burgherr, P., Hirschberg, S., 2008a. A comparative analysis of accident risks in fossil,hydro and nuclear energy chains. Hum. Ecol. Risk Assess. 14, 947–973.

Burgherr, P., Hirschberg, S., 2008b. Severe accident risks in fossil energy chains: acomparative analysis. Energy 33, 538–553.

Burgherr, P., Hirschberg, S., 2009. Comparative Risk Assessment for EnergySystems: A Tool for Comprehensive Assessment of Energy Security. In: Voeller,J.G. (Ed.), Wiley Handbook of Science and Technology for Homeland Security.John Wiley & Sons Inc., Hoboken, NJ, USA.

Burgherr, P., Hirschberg, S., Cazzoli, E., 2008. Final report on quantification of riskindicators for sustainability assessment of future electricity supply options.NEEDS Deliverable no. D7.1 – Research Stream 2b. NEEDS project “New EnergyExternalities Developments for Sustainability”, Brussels, Belgium.

Burgherr, P., Hirschberg, S., Hunt, A., Ortiz, R.A., 2004. Severe Accidents in theEnergy Sector. Final Report to the European Commission of the EU 5thFramework Programme “New Elements for the Assessment of External Costsfrom Energy Technologies” (NewExt). In: Friedrich, R. (Ed.), 2004. DG Research,Technological Development and Demonstration (RTD), Brussels, Belgium

Chen, H., Qi, H., Long, R., Zhang, M., 2012. Research on 10-year tendency of Chinacoal mine accidents and the characteristics of human factors. Saf. Sci. 50,745–750.

Chernobyl Forum (IAEA, W., UNDP, FAO, UNEP, UN-OCHA, UNSCEAR, World Bank,Governments of Belarus, the Russian Federation and Ukraine), 2005. Cherno-byl's legacy: health, environmental and socio-economic impacts and recom-mendations to the governments of Belarus, the Russian Federation and Ukraine.The Chernobyl Forum: 2003–2005. Second revised version. IAEA, Vienna,Austria.

Cherp, A., Jewell, J., 2011. The three perspectives on energy security: intellectualhistory, disciplinary roots and the potential for integration. Curr. Opin. Environ.Sustain. 3, 202–212.

Chessel, D., Dolédec, S., 1996. ADE version 4: hypercard stacks and quick basicMicrosoft programme library for the analysis of environmental data, user’smanual. Lyon, France: Ecologie des Eaux Douces et des Grands Fleuves,Université Lyon 1.

Chester, L., 2010. Conceptualising energy security and making explicit its polysemicnature. Energy Policy 38, 887–895.

Coaffee, J., 2008. Risk, resilience, and environmentally sustainable cities. EnergyPolicy 36, 4633–4638.

P. Burgherr, S. Hirschberg / Energy Policy 74 (2014) S45–S56 S55

Colli, A., Serbanescu, D., Ale, B.J.M., 2009. Indicators to compare risk expressions,grouping, and relative ranking of risk for energy systems: application withsome accidental events from fossil fuels. Safety Science 47, 591–607.

Delzeit, R., Holm-Müller, K., 2009. Steps to discern sustainability criteria for acertification scheme of bioethanol in Brazil: approach and difficulties. Energy34, 662–668.

Dietz, S., Neumayer, E., 2007. Weak and strong sustainability in the SEEA: conceptsand measurement. Ecol. Econ. 61, 617–626.

Dubois, J., Hu, G., Poggi, P., Montignac, F., Serre-Combe, P., Muselli, M., Hoguet, J.C.,Vesy, B., Verbecke, F., 2013. Safety cost of a large scale hydrogen system forphotovoltaic energy regulation. Int. J. Hydrog. Energy 38, 8108–8116.

Eckle, P., Burgherr, P., Hirschberg, S., 2011. Final report on Multi-Criteria DecisionAnalysis (MCDA). SECURE Deliverable No. D6.2. SECURE Project – Security ofEnergy Considering its Uncertainty, Risk and Economic Implications, Brussels,Belgium.

Felder, F.A., 2009. A critical assessment of energy accident studies. Energy Policy 37,5744–5751.

Filippini, R., Silva, A., 2012. Resilience analysis of networked systems-of-systemsbased on structural and dynamic interdependencies, PSAM 11 & ESREL 2012,Helsinki, Finland, pp. 10.

Fritzsche, A.F., 1989. The health risks of energy production. Risk Anal. 9, 565–577.Fritzsche, A.F., 1992. Severe accidents: can they occur only in the nuclear produc-

tion of electricity? Risk Anal. 12, 327–329 (Editorial).Fthenakis, V.M., Kim, H.C., 2011. Photovoltaics: life-cycle analyses. Sol. Energy 85,

1609–1628.Gray, P.C.R., Wiedemann, P.M., 1999. Risk management and sustainable develop-

ment: mutual lessons from approaches to the use of indicators. J. Risk Res. 2,201–218.

Hartmann, P., Apaolaz, V., D'Souza, C., Echebarria, C., Barrutia, J.M., 2013. Nuclearpower threats, public opposition and green electricity adoption: effects ofthreat belief appraisal and fear arousal. Energy Policy 62, 1366–1376.

Hirschberg, S., Burgherr, P., Spiekerman, G., Cazzoli, E., Vitazek, J., Cheng, L., 2003.Assessment of Severe Accident Risks. In: Eliasson, B., Lee, Y.Y. (Eds.), IntegratedAssessment of Sustainable Energy Systems in China. The China EnergyTechnology Program – A Framework for Decision Support in the Electric Sectorof Shandong Province. Alliance for Global Sustainability Series, Vol. 4. KluwerAcademic Publishers, Amsterdam, The Netherlands, pp. 587–660.

Hirschberg, S., Burgherr, P., Spiekerman, G., Dones, R., 2004. Severe accidents in theenergy sector: comparative perspective. J. Hazard. Mater. 111, 57–65.

Hirschberg, S., Spiekerman, G., Dones, R., 1998. Severe accidents in the energysector – first edition. PSI Report No. 98-16. Paul Scherrer Institut, Villigen PSI,Switzerland.

Hotelling, H., 1933. Analysis of a complex of statistical variables into principalcomponents. J. Educ. Psychol. 24 (417–441), 498–520.

Hueting, R., Reijnders, L., 2004. Broad sustainability contra sustainability: theproper construction of sustainability indicators. Ecol. Econ. 50, 249–260.

Inhaber, H., 2004. Risk Analysis Applied to Energy Systems. In: Cleveland, C.J. (Ed.),Encyclopedia of Energy, Volume 5. Elsevier, Amsterdam, The Netherlands,pp. 469–482.

Jeswani, H.K., Azapagic, A., Schepelmann, P., Ritthoff, M., 2010. Options for broad-ening and deepening the LCA approaches. J. Clean. Prod., 18.

Johansen, I.L., Rausand, M., 2014. Foundations and choice of risk metrics. Saf. Sci. 62,386–399.

Johansson, B., 2013. A broadened typology on energy and security. Energy 53,199–205.

Jolliffe, I.T., 2002. Principal Component Analysis. Springer Series in Statistics, 2nded. Springer, New York, USA.

Kampanart, S., Yuki, I., Shogo, T., 2014. Cost per severe accident as an index forsevere accident consequence assessment and its applications. ReliabilityEngineering and System Safety, 123, 110–122.

Karakosta, C., Pappas, C., Marinakis, V., Psarras, J., 2013. Renewable energy andnuclear power towards sustainable development: characteristics and prospects.Renew. Sustain. Energy Rev. 22, 187–197.

Kates, R.W., Parris, R.M., Leiserowitz, A.A., 2005. What is sustainable development?Goals, indicators, values, and practice. Environ.: Sci. Policy Sustain. Dev. 47,8–21.

Kharecha, P.A., Hansen, J.E., 2013. Prevented mortality and greenhouse gas emis-sions from historical and projected nuclear power. Environ. Sci. Technol. 47,4889–4895.

Kontovas, C.A., Psaraftis, H.N., Ventikos, N.P., 2010. An empirical analysis of IOPCF oilspill cost data. Mar. Pollut. Bull. 60, 1455–1466.

Kröger, W., Zio, E., 2011. Vulnerable Systems. Springer Verlag, London, UK.Laes, E., Mesken, G., van der Sluijs, J.P., 2011. On the contribution of external cost

calculations to energy system governance: the case of a potential large-scalenuclear accident. Energy Policy 39, 5664–5673.

Larsson, S., Fantazzini, D., Davidsson, S., Kullander, S., Höök, M., 2014. Reviewingelectricity production cost assessments. Renew. Sustain. Energy Rev. 30,170–183.

Levett, R., 1998. Sustainability indicators – integrating quality of life and environ-mental protection. J. R. Stat. Soc., Ser. A (Stat. Soc.) 161, 291–302.

Lior, N., 2012. Sustainable energy development (May 2011) with some game-changers. Energy, 40.

Mabel, M.C., Raj, R.E., Fernandez, E., 2010. Adequacy evaluation of wind powergeneration systems. Energy, 5217–5222.

Maiti, J., Khanzode, V.V., Ray, P.K., 2009. Severity analysis of Indian coal mineaccidents – a retrospective study for 100 years. Saf. Sci. 47, 1033–1042.

Molyneaux, L., Wagner, L., Froome, C., Foster, J., 2012. Resilience and electricitysystems: a comparative analysis. Energy Policy 47, 188–201.

Musango, J.K., Brent, A.C., 2011. A conceptual framework for energy technologysustainability assessment. Energy Sustain. Dev. 15, 84–91.

Noll, H.-H., 2002. Towards a European system of social indicators: theoreticalframework and system architecture. Soc. Indic. Res. 58, 47–87.

Nuclear Energy Agency (NEA), 2012. The Role of Nuclear Energy in a Low-carbonEnergy Future. OECD/NEA, Paris, France.

Okoh, P., Haugen, S., 2013. Maintenance-related major accidents: classification ofcauses and case study. Journal of Loss Prevention in the Process Industries, 26,1060–1070.

Pearson, K., 1901. On lines and planes of closest fit to systems of points in space.Philos. Mag. 2, 559–572.

Psarros, G., Skjong, R., Vanem, E., 2011. Risk acceptance criterion for tanker oil spillrisk reduction measures. Mar. Pollut. Bull. 62, 116–127.

Rabl, A., Rabl, V.A., 2013. External costs of nuclear: greater or less than thealternatives? Energy Policy 57, 575–584.

Rapp Nilsen, H., 2010. The joint discourse ‘reflexive sustainable development’ –from weak towards strong sustainable development. Ecol. Econ. 69, 495–501.

Rasmussen, N.C., 1981. The application of probabilistic risk assessment techniquesto energy technologies. Annu. Rev. Energy 6, 123–138.

Renn, O., Klinke, A., van Asselt, M., 2011. Coping with complexity, uncertainty andambiguity in risk governance: a synthesis. Ambio 40, 231–246.

Rentizelas, A., Georgakellos, D., 2014. Incorporating life cycle external cost inoptimization of the electricity generation mix. Energy Policy 65, 134–149.

Richter, B., 2012. Opinion on worldwide health effects of the Fukushima Daiichinuclear accident by J.E. Ten Hoeve and M.Z. Jacobson. Energy Environ. Sci. 5,8758–8759, http://dx.doi.org/10.1039/c2ee22019a.

Roth, S., Hirschberg, S., Bauer, C., Burgherr, P., Dones, R., Heck, T., Schenler, W., 2009.Sustainability of electricity supply technology portfolio. Ann. Nucl. Energy 36,409–416.

Saleh, J.H., Cummings, A.M., 2011. Safety in the mining industry and the unfinishedlegacy of mining accidents: safety levers and defense-in-depth for addressingmining hazards. Saf. Sci. 49, 764–777.

Singh, R.K., Murty, H.R., Gupta, S.K., Dikshit, A.K., 2009. An overview of sustain-ability assessment methodologies. Ecol. Indic. 9, 189–212.

Sovacool, B.K., 2008. The costs of failure: a preliminary assessment of major energyaccidents, 1907–2007. Energy Policy 36, 1802–1820.

Sovacool, B.K., 2013. An international assessment of energy security performance.Ecol. Econ., 88.

Srinivasan, T.N., Gopi Rethinaraj, T.S., 2013. Fukushima and thereafter: reassessmentof risks of nuclear power. Energy Policy 52, 726–736.

Szklo, A., Schaeffer, R., 2007. Fuel specification, energy consumption and CO2

emission in oil refineries. Energy 32, 1075–1092.Ten Hoeve, J.E., Jacobson, M.Z., 2012a. Reply to the opinion on worldwide health

effects of the Fukushima Daiichi nuclear accident by B. Richter, Energy Environ.Sci., 5, 8760, 10.1039/c2ee22658h.

Ten Hoeve, J.E., Jacobson, M.Z., 2012b. Worldwide health effects of the FukushimaDaiichi nuclear accident. Energy Environ. Sci. 5, 8743–8757.

Torvanger, A., Meadowcroft, J., 2011. The political economy of technology support:making decisions about carbon capture and storage and low carbon energytechnologies. Glob. Environ. Change 21, 303–312.

Vinnem, J.E., 2011. Evaluation of offshore emergency preparedness in view of rareaccidents. Saf. Sci. 49, 178–191.

Vitazkova, J., Cazzoli, E., 2013. Common risk target for severe accidents of nuclearpower plants based on IAEA INES scale. Nucl. Eng. Des. 262, 106–125.

Vitázková, J., Cazzoli, E., 2011. Estimate of consequences from the Fukushimadisaster, Nordic PSA Conference, Stockholm, Sweden.

von Carlowitz, H.C., 2000. Sylvicultura oeconomica: Anweisung zur wilden Baum-Zucht/Hannss Carl von Carlowitz; bearb. von Klaus Irmer, Angela Kiessling –

Reprint der Ausgabe Leipzig: Braun, 1713. TU Bergakademie Freiberg, Freiberg,Germany.

von Hippel, D., Suzuki, T., Williams, J.H., Savage, T., Hayes, P., 2011. Energy securityand sustainability in Northeast Asia. Energy Policy 39, 6719–6730.

von Hippel, F.N., 2011. The radiological and psychological consequences of theFukushima Daiichi accident. Bull. At. Sci. 67, 27–36.

World Commission on Environment and Development (WCED), 1987. Our CommonFuture. Oxford University Press, Oxford, UK.

World Health Organization (WHO), 2013. Health risk assessment from the nuclearaccident after the 2011 Great East Japan Earthquake and Tsunami, based on apreliminary dose estimation. World Health Organization (WHO), Geneva,Switzerland.

Winzer, C., 2012. Conceptualizing energy security. Energy Policy 46, 36–48.Yustaa, J.M., Correa, G.J., Lacal-Arántegui, R., 2011. Methodologies and applications

for critical infrastructure protection: state-of-the-art. Energy Policy 39,6100–6119.

P. Burgherr, S. Hirschberg / Energy Policy 74 (2014) S45–S56S56