Secure Grid '11 After-Action Report

  • Upload
    hhrust

  • View
    218

  • Download
    0

Embed Size (px)

Citation preview

  • 8/2/2019 Secure Grid '11 After-Action Report

    1/28

    The views contained in this report are those expressed during the tabletop exercise by participants and do

    not reflect the official policy or position of National Defense University, the Department of Defense, or

    the U.S. Government.

    EXERCISEREPORTANDSUMMARYOFFINDINGSOnOctober45,2011,leadingfiguresfrom

    industry,stateandfederalagencies,theArmedForces,andotherrelevantorganizationsmetat

    NationalDefenseUniversitysInstituteforNationalStrategicStudiesinWashington,D.C.to

    conductandparticipateinacrisissimulationofseveresolarweathereffectsonthenations

    power

    grid.

    Their

    purpose

    was

    to

    explore

    issues

    pertaining

    to

    how

    private

    and

    governmental

    agencieswouldreacttowidespreaddamagetothegrid,howtheymightcooperateduringsuch

    acrisis,andtoexplorewhatstepscouldbetakentomitigatetheeffectsofsevereevents.

    SECUREGRID11ELECTRICALGRIDCRISISTABLETOPEXERCISE

    InstituteforNationalStrategicStudies

    NationalDefenseUniversity

    October45,2011

  • 8/2/2019 Secure Grid '11 After-Action Report

    2/28

    i

    SponsoringandHostingOrganizationsDepartmentofHomelandSecurity

    UnitedStatesNorthernCommand

    NationalDefenseUniversity

    ForinformationregardingSecureGrid,pleasecontactthebelowPOC:

    Dr.RichardB.Andres

    EnergySecurityChair,InstituteforNationalStrategicStudies

    ProfessorofNationalSecurityStrategy,NationalWarCollege

    NationalDefenseUniversity

    (202)6854427,[email protected]

  • 8/2/2019 Secure Grid '11 After-Action Report

    3/28

    ii

    Whileaseverestormisalowfrequencyoccurrenceevent,it

    hasthepotentialforlongdurationcatastrophicimpactstothe

    powergridanditsaffectedusers.Theimpactscouldpersistfor

    multipleyears

    with

    potential

    for

    significant

    societal

    impacts

    and

    witheconomiccoststhatcouldbemeasurableintheseveral

    TrillionDollarsperyearrange.

    AnOverviewoftheNationalAcademyofSciences

    ReportonSevereSpaceWeatherandthe

    VulnerabilityoftheUSElectricalPowerGrid

    MetatechCorporation

  • 8/2/2019 Secure Grid '11 After-Action Report

    4/28

    iii

    TableofContentsSecureGrid11Overview............................................................................................................................. 1

    ParticipatingOrganizations........................................................................................................................... 3

    Objectives..................................................................................................................................................... 6

    Background................................................................................................................................................... 7

    TabletopExercisePart1:Prevent(T0+7hrs).............................................................................................. 9

    TabletopExercisePart2:Protect(T0+20hrs)........................................................................................... 13

    Tabletop

    Exercise

    Part

    3:

    Respond

    (T0

    +

    20

    hrs)

    ........................................................................................

    17

    Recommendations...................................................................................................................................... 21

    AppendixSecureGrid11ScenarioBackground...................................................................................... 24

  • 8/2/2019 Secure Grid '11 After-Action Report

    5/28

    1

    SecureGrid11OverviewSecureGrid11isthethirdinanongoingseriesofexercisesthatNationalDefenseUniversitysInstitute

    forNational

    Strategic

    Studies

    (INSS)

    has

    conducted

    in

    conjunction

    with

    the

    Department

    of

    Homeland

    SecurityandU.S.NorthernCommandonthevulnerabilityoftheNorthAmericanElectricGrid.Previous

    exercisesexaminedtheabilityofpublicandprivateinstitutionstoworktogethertopreventand

    mitigatecyberandphysicalattacksonthegrid.Thisyearsgameexaminedthethreatposedtothegrid

    byextremesolarweather.Thistypeofeventisconsideredalowprobabilityhighimpactevent,thatis,

    whiletheprobabilityofaneventislowinanygivenyear,thepotentialharmitcoulddototheUnited

    Statesandothercountriesishigh.

    Theexercisescenariowasbasedonaspaceweathereventwithimpactssimilarinnaturebutlesssevere

    thanthosedescribedbytheNationalAcademiesofScience(NAS)2009ReportExtendedSummary

    Severe

    Space

    Weather

    Events

    Understanding

    Societal

    and

    Economic

    Impacts.

    The

    NAS

    report

    describedapossiblegeomagneticdisturbance(GMD)similartoasuperstormwhichoccurredin1921.

    ThisGMDproducedamagneticfieldchangerateof4800nanoTeslasperminute(nT/min)at50degrees

    geomagneticlatitude.TheNASreportexplainsthatsuchaneventoccurringtodaycouldresultin

    widespreadpoweroutageslastingforseveralyears.Bycomparison,thisexercisepostulatedaless

    severeeventproducingfewerlongtermpoweroutagesbutstilloccurringatanunprecedentedscale

    andduration.Additionalscenariobackgroundisprovidedintheappendixtothisdocument.

    SecureGrid11washeldonOctober45,2011atNationalDefenseUniversityinWashingtonD.C.and

    attendedbyleadingfiguresfromindustry,stateandfederalagencies,andthescientificcommunity.The

    exerciseentailedacrisissimulationwiththegoalofexploringhowprivateandgovernmentagencies

    wouldrespondtoasolarstormcausingwidespreaddamagetothegrid,howtheymightcooperate

    duringsuchacrisis,andtoexplorewhatstepscouldbetakentomitigatesuchsevereevents.The

    scenariosintheexerciseweredesignedbyateamofexpertsfromindustry,theDepartmentof

    HomelandSecurity,U.S.NorthernCommand,andtheNationalOceanicandAtmospheric

    Administration.

    Thetabletopexercisewasconductedinthreemodulesusingaguidedseminarstructuretoexplore

    relevantquestionsandissues.AfteropeningcommentsonthemorningofOctober4th,participantswere

    assignedtofourseparatebreakoutgroupscontainingadiversemixofrepresentatives.Eachgroupwas

    instructedtoaddressaspecificsetofbroadquestionsineachstage,whichconsistedofanearlywarning

    stage,an

    event

    occurrence

    stage,

    and

    apost

    event

    damage

    assessment/consequence

    management

    and

    mitigationstage.Groupsweregiventwohourstoaddresseachstageoverthecourseofthefirstday.

    Thesecondday,October5th,consistedofpostexerciseanalysis,whichbroughteachteamintoplenary

    sessiontodebrieftheentiregroupontheirfindings,insights,andrecommendations.

  • 8/2/2019 Secure Grid '11 After-Action Report

    6/28

    2

    Ontheseconddayoftheevent,allparticipantswereabletohearanddiscussthefindingsofthe

    individualgroups.Theafternoonoutbriefsessionprovidedaforumtopresentthegamesresultsto

    seniorofficials.

    Althoughthedetailsofthelessonslearnedandrecommendationsthatresultedfromeachgroups

    participationand

    immersion

    in

    the

    experience

    are

    discussed

    in

    detail

    below,

    the

    most

    important

    findingsaresummarizedhere.

    Thebroadestconsensusreachedbytheparticipantswasthatinaneventassevereastheonepositedby

    theexercise,preventionispreferabletoresponseandrecovery.Improvementsinearlywarning,

    especiallythereplacementoftheAdvancedCompositionExplorer(ACE)satellite,hardeningofcritical

    equipment,stockpilingofreplacementtransformers,andimprovedresilienceinthegridwereidentified

    ascrucialcomponentsofanyefforttominimizetheprobabilityoflongtermgridoutagesresultingfrom

    aGMD.

    Thebiggestunresolvedquestionconcernsthetypeofdamagesuchaneventmightcausetocriticalgrid

    components,especially

    large

    extremely

    high

    voltage

    (EHV)

    transmission

    transformers.

    Although

    the

    effectsofgroundinducedcurrent(GIC) ontransformershavebeenstudied,significanttechnical

    uncertaintysurroundsquestionsregardingwhethertypicalinservicetransformerswould

    catastrophicallyfailinGMDconditions.Thisfactorcomplicateseffortstoaccuratelyestimatethe

    possibleextentofequipmentdamageandrelatedgridfailurethatmightoccur.EHVtransformersare

    verylarge,extremelyheavy,andhighlydifficulttotransporttotheremotesiteswheretheyaretypically

    installed.ThenumbersofexistingsparesarenotsufficienttorecoverfromaGMDthatdamagesa

    significantfractionofthenationalorinternationalfleet.Globalsupplychainswouldbeunableto

    producereplacementEHVtransformersatarateneededtomeetdemandiflargenumberswere

    suddenlyrequiredtorecoverfromaGMDaffectingseveralindustrializedcountries.

    Extensivestockpilingofreplacementtransformersisahighlyexpensiveproposition,bothintermsof

    manufactureandstorage.Further,givenacatastrophicevent,thejustintimeproductionsystems

    currentlyinplacewouldbevulnerabletothecascadingeffectsofabroad,longtermpoweroutage,

    makingitdifficulttoobtainreplacementspostevent.Thesetwofactorscreateaconundruminwhich

    stockpilingpriortoaneventisexpensiveandproblematic,whilereplacementposteventwouldalsobe

    highlyproblematicandextremelytimeconsumingwhentimeisoftheessence.Thus,neitheroptionis

    palatableforindustrytoaddress,particularlygiventhelowprobabilityofsuchaneventoccurring.

    Therefore,itwastheconsensusoftheexerciseparticipantsthatindustryespeciallyrequiresamore

    developedunderstandingofexactlyhowlikelysuchaneventis,aswellashowmuchdamageislikelyto

    occurfrom

    asevere

    GMD.

  • 8/2/2019 Secure Grid '11 After-Action Report

    7/28

    3

    ParticipatingOrganizationsOnehundredandtwentyeightindividualsparticipatedinSecureGrid11andweredrawnfromawidevarietyofprivateandpublicorganizations.Participantswereselectedtorepresentorganizations

    responsiblefor

    operating,

    regulating,

    and

    protecting

    the

    grid.

    These

    organizations

    include:

    AmericanPublicPowerAssociation

    AnalyticalServicesInc.(ANSER)

    Atmospheric&SpaceTechnologyResearchAssociates(ASTRA),LLC

    BCSIncorporated

    BoozAllenHamilton

    DefenseR&D

    Canada

    Centre

    for

    Security

    Science

    DepartmentofDefense

    OfficeofAssistantSecretaryofDefenseforHomelandDefenseandAmericaSecurityAffairs

    (OASDHD&ASA)

    OfficeoftheAssistantSecretaryofDefenseforOperationalEnergyPlansandPrograms(OASD

    OEPP)

    OfficeoftheSecretaryofDefenseforInstallationsandEnvironment(I&E)

    DepartmentofEnergy

    InfrastructureSecurityandEnergyRestorationDirectorate

    DepartmentofHomelandSecurity

    OfficeofInfrastructureProtection

    Science&TechnologyDirectorate

    EdisonElectricInstitute

    ElectricInfrastructureSecurityCouncil

    EmbassyofSweden

    Emprimus

    ExelonGeneration,LLC

  • 8/2/2019 Secure Grid '11 After-Action Report

    8/28

    4

    FederalEmergencyManagementAgency(FEMA)

    FederalEnergyRegulatoryCommission(FERC)

    FriiPwr,Ltd.

    GeoStrategicAnalysis

    Headquarters,U.S.ArmyCorpsofEngineers(HQUSACE)

    HooverInstitution,StanfordUniversity

    ICS/SCADAThreatFocusCellTeamLeaders

    IdahoNationalLaboratory

    InteragencyCoordinationDirectorate

    ISONew

    England

    NARUCStaffSubcommitteeonCriticalInfrastructure

    NationalAeronauticsandSpaceAdministration(NASA)

    NationalAssociationofStateEnergyOfficials(NASEO)

    NationalCommunicationsSystem

    NationalDefenseUniversity

    NationalOceanic

    and

    Atmospheric

    Administration

    (NOAA)

    NationalWeatherService

    NavalSeaSystemsCommandDahlgrenMissionAssuranceDivision

    NAVFACMobileUtilitiesSupportEquipment

    Navy,OPNAVN00X(NavalWarfareIntegrationGroup,andQDR)

    NorthAmericanAerospaceDefenseCommand(NORAD)andUnitedStatesNorthernCommand

    (USNORTHCOM)

    NORADandUSNORTHCOMInteragencyCoordination(IC)

    NORADandUSNORTHCOMJ7JointTrainingandExercises

    USNORTHCOMJ34(AssessmentsandCIPBranch)

  • 8/2/2019 Secure Grid '11 After-Action Report

    9/28

    5

    NorthAmericanElectricReliabilityCorporation(NERC)

    GeomagneticDisturbancesTaskForce(GMDTF)

    PHIServiceCompany

    PJMCorporation

    PotomacElectricPowerCompany

    ProgressEnergy

    PSEGServicesCorp.

    ReliabilityFirstCorp.

    SecuredSciencesGroup

    SpacePlasma

    Laboratory

    SSG,LLC

    StormAnalysisConsultants

    TheNationalAssociationofRegulatoryUtilityCommissioners(NARUC)

    ThePraemittiasGroup,Inc.

    TranstectorSystems,Inc.

    UnitedStates

    Air

    Force/Air

    Force

    Weather

    Agency

    (AFWA)

    UnitedStatesNuclearRegulatoryCommission(NRC)

    WhiteHouse

    NationalSecurityStaff

    OfficeofScienceandTechnologyPolicy(OSTP)

  • 8/2/2019 Secure Grid '11 After-Action Report

    10/28

    6

    ObjectivesIdentifyandunderstandthepotentialimpactofamajorgeomagneticstormontheNorthAmerican

    PowerGrid.

    ExploreandsocializewaystominimizetheimpactofamajorgeomagneticstormonUScritical

    infrastructure.

    Examineandassessprocessesandproceduresforrecoveringfromacatastrophicgeomagneticstorm.

    Clarifyand

    deconflict

    roles

    and

    responsibilities

    between

    public

    and

    private

    agencies.

  • 8/2/2019 Secure Grid '11 After-Action Report

    11/28

    7

    BackgroundThesunissubjecttoperiodsofincreasinganddecreasingsunspotactivityapproximatelyevery11years

    inwhatisknownasthesolarcycle.Sunspotactivityhasbeencorrelatedtotheoccurrenceofsolarflares

    andcoronal

    mass

    ejections

    (CMEs),

    which

    can

    cause

    geomagnetic

    disturbances

    (GMD)

    on

    Earth.

    AlthoughCMEsoccurmoreoftenduringthesolarmaximumwhentheincidenceofsunspotsis

    greatest,theycanstilloccuratanytimeduringasolarcycle.MostCMEsareharmlesslydirectedinto

    interplanetaryspacebutsometraveldirectlytowardsEarth.WhenaCMEmovingthroughspaceat

    millionsofmilesperhourinteractswithEarthsmagneticfield,thenormalfieldisquicklydisturbedand

    cancontinuetochangeformanyhoursuntiltheeffectsofthestormhavepassedandthefieldreturns

    toanundisturbedstate.AstheEarthsmagneticfieldchanges,DCelectriccurrentsareinducedinlong

    distancehighvoltagetransmissionlines.ThesetransmissionlinesareconnectedtogroundedEHV

    transformersdesignedtooperateonlywith60cycleACpower.GeomagneticallyinducedDCcurrent

    causesACpowertransformerstooperateabnormallyandcanresultinrapidoverhearingandfailureif

    conditionsare

    unfavorable.

    Most

    geomagnetic

    storms

    have

    relatively

    limited

    and

    temporary

    effects,

    however,strongerstormscanadverselyaffectcriticalinfrastructure,bothinspaceandonEarth.In

    additiontothegeomagneticeffectsalreadydiscussed,solarstormscantemporarilydistorttheEarths

    ionosphereadverselyaffectingradiocommunicationsandGlobalPositioningSystem(GPS)service.

    Surfacechargescanbuilduponsatellites,causingunhardenedelectronicsystemstomalfunctionor

    permanentlyfail.

    In1989astronggeomagneticstormcausedablackoutinQuebecjust90secondsaftertheCMEreached

    Earth.SinceNASAsACEspacecraftwasnotlauncheduntil1997,therewaslesseffectiveearlywarning

    availabletogridoperatorsinthisevent.Ittookninehourstorestorethe18,000MWnetworkafter

    repairingor

    replacing

    destroyed

    equipment.

    A

    geomagnetic

    storm

    in

    2003

    took

    down

    the

    Federal

    AviationAdministrations(FAA)GPSbasednavigationsystemfor30hoursanddamagedelectrical

    systemsfromScandinaviatoSouthAfrica.Theseandotherrecentstormsweresmallcomparedto

    knownsuperstormsin1859(Carringtonevent)and1921.The1859superstormfeaturedtwo

    consecutiveCMEsfirstobservedbyBritishastronomerRichardCarrington.WhenthefirstCMEslammed

    intoEarthabout18hourslater,therewasnopowergridthatcouldbedamagedbutmanytelegraph

    systemsfailedorcaughtfireacrossEuropeandNorthAmerica.

    Modernstormscancausemuchmoredamageanddisruptionthanearliereventsbecause

    infrastructuresaroundtheworldaremoreubiquitous,complex,andinterconnected.IfaCarrington

    level

    event

    where

    to

    occur

    today,

    it

    could

    cause

    enormously

    more

    damage,

    potentially

    knocking

    out

    largeportionsoftheworldselectricinfrastructureandcreatingmassiveblackoutsacrosstheglobe.If

    suchblackoutswerelonglastingtheycouldaffectphysicalinfrastructureincludingwaterdistribution,

    perishablefoodsandmedications,heating/airconditioning,sewagedisposal,telecommunications,and

    fuelsupplies.Accordingtothe2009NASreportthedamageinflictedbyamajorCMEcouldcosttheUS

  • 8/2/2019 Secure Grid '11 After-Action Report

    12/28

    8

    $12trillioninitsfirstyear.Althoughthe1859stormwasarareevent,stormswitharoundhalfits

    intensityoccurmuchmorefrequently.The1921superstormisthemostrecentexample.

    ConcernsaboutthesusceptibilityofUSelectricalinfrastructuretoGMDeffectsaresignificantcompared

    toothersimilarlyindustrializedcountriesforseveralreasons:

    MuchoftheUSislocatedinrelativeproximitytoEarthsmagneticnorthpolewhereGMDeffectsarestronger.

    Becauseofgeography,theUShasalargenumberoflongdistanceeastwestorientedhighvoltagetransmissionlineswhichactasgoodantennasforbestcasecouplingwithGMD

    producedelectromagneticfields.

    TheUShasaddedincreasingamountsofhighervoltagetransmissioninfrastructureineveryyearsincethelastGMDsuperstormin1921.Highvoltagelinestendtobelongerinlengthand

    thereforeactasbetterantennasforcouplingGICintothegridandpotentiallydamagingEHV

    transformers.

    Ifcatastrophic

    grid

    failure

    should

    occur

    as

    aresult

    of

    asolar

    storm,

    effective

    response

    and

    efficient

    recoverywilldependonthehighlyeffectivecooperationandcoordinationbetweenpublicandprivate

    entities.

  • 8/2/2019 Secure Grid '11 After-Action Report

    13/28

    9

    TabletopExercisePart1:Prevent(T0+7hrs)0945,October4,2011BrieftoPlayersNOAAs

    Space

    Weather

    Prediction

    center

    in

    Boulder,

    CO,

    reports

    that

    an

    intense

    eruption

    on

    the

    sun

    associatedwithalargesunspotclusteronthesunhasbeenobserved.Theyreportthatthisflarehas

    alreadyregisteredanR5levelontheNOAASpaceWeatherRadioBlackoutScale,oneofthebiggest

    flaresseeninthelastfortyyears.Aradiationstorm,approachingtheS4toS5levelontheNOAASpace

    WeatherRadiationScale,isexpectedtoaccompanythiseventinthenextseveralhours.TheCMEis

    extremelylargeandbothNOAAandNASAcomputermodelsshowthatitisheadingtowardsEarth.

    SomeEarthorbitingsatellitesarealreadybeingaffected.

    AverylargegeomagneticstormisexpectedattheG5level,NOAAsspaceweatherequivalentofa

    Category5hurricanealthoughabout4eventsofG5strengthoccurinatypical11yearsolarcycle.

    Higherlatitudesaretypicallymorevulnerabletothestorm,particularlyinproximitytolargebodiesof

    saltwater,whichactsasaconduitforgroundinducedcurrents.Theeffectsofthestrongeststormscan

    actuallybegreateratmidlatitudesandleaveextremenorthernorsouthernlatitudeslessimpacted.

    ImpactEstimateUpto130MAmericansinitiallyw/oelectricityforseveralhours

    Upto2monthstofullyrestorenormalpower

    Secondaryimpacts:

    Energy

    Communications

    Water/Sanitation

    Banking&Finance

    Transportation

    EmergencyServices

    GovernmentServices

    SummaryofTeamDiscussionsTheteamsweregiventwohourstodiscusstheramificationsoftheimpactestimateandtodiscover

    whereweaknessesinpreparationcapacitymayreside.Asrepresentativesfromvariousorganizations

    spoke,itbecameclearthatissuesofcoordinationwerelessofaconcernthanthoseofphysicalcapacity

    torespond.

    Given

    such

    adisaster,

    government

    agencies

    explained

    that

    only

    industry

    had

    the

    appropriateresourcestorespond.

    TimeframeofwarningTheprimaryissuesrepeatedacrosseachteamwasthetimelineofwarningsandtheresultantcapacityofindustrytomakemeaningfuldecisionswithinthattimeframe.Fromthetime

    theACEsatelliteprovidesfinalinformationabouttheanticipatedseverityofanevent,industrytypically

  • 8/2/2019 Secure Grid '11 After-Action Report

    14/28

    10

    hasabout45minutestomakedecisionsonhowtoprotectandpreservethegrid.Thistimeframeis

    marginallysufficientforsystemoperatorstoworkthroughmechanismstoidentifyandanalyze

    informationabouttheeventasitispasseddown.Importantactionscanbecompletedorinitiatedinthis

    timeframebutthereisessentiallynoroomforeventheslightestdelaysormistakes.Actionsofthistype

    canincludeunloadingorevensecuringsomelongdistancetransmissionlinesthroughavarietyof

    possiblemeasures

    in

    order

    to

    reduce

    the

    GIC

    that

    will

    enter

    the

    grid,

    and

    starting

    peaking

    plants

    to

    establishasituationwhereregionalloadsandgenerationaremorebalanced.

    Advancedwarningcancomeasearlyastwodayspriortoanevent,butthesewarningshavesignificantmarginsoferror,thusmakingthemdifficulttoactupon.

    AdvancedwarningsonmeasuredgroundsurgescomemorerapidlyoutofCanadathanfromUSagencies.

    Certainoperatingstepscanbetakenwithintwentyhoursofnotificationtoreducevulnerabilityof

    the

    system

    and

    the

    procedures

    for

    these

    steps

    are

    already

    in

    place.

    ImmediateStepsGiventhewarningtimeframes,thediscussionturnedtowhatstepscouldactuallybetakentomitigateimpendingdamagetothegrid.Anindustrywideconferencecallwouldoccur,similar

    tothecallsconductedbeforehurricanes.AlleightNERCregionswouldbebroughtin,withtheir

    respectivecoordinators,tochallengethemtoprepareforwhatevertypeofspaceweathermightbe

    approachingandanalyzewhattheimpactcouldbe.

    Loadsheddingandthepossibilityofatemporarygridshutdownwerediscussedaslikelycandidatesfor

    immediateaction.Loadsheddingincludedsuchconceptsasislandingorestablishingmicrogridswithin

    thesystemtoisolatepotentialproblemareasandreducetheneedforlongdistancetransmission.The

    mostcontentiousissuewastheideaofsimplyshuttingoffthegridforsomeperiodoftimetoprotectit

    fromextensivedamage.Itwasagreedthatthedecisiontotakesuchadrasticmeasureresidedentirely

    withtheutilities,butitwasclearthatthepotentialliabilityissuesassociatedwithafalsealarmscenario

    weretoogreatforagridshutdowntobeareasonableoption.Complicatingissuesinclude:

    Maintainingserviceandkeepingthegridoperatingisalongstandingdeeplyingrainedtoppriorityfromindustryspointofview.

    Inplaceproceduresarebasedongettingreadyonceanalertisreceived,buttherestoftheprocedures

    are

    not

    implemented

    until

    the

    grid

    is

    actually

    overloaded,

    thus

    in

    place

    procedures

    arereactive,notproactive.

    Fromtheperspectiveofparticipantsthatwerenotpartofautility,difficultyexistedinunderstandingtheproceduresthatutilitiesalreadyhaveinplacetodealwithcrises.

  • 8/2/2019 Secure Grid '11 After-Action Report

    15/28

    11

    Thegridisdynamic;itcanhavehighlystablevoltageatonemomentandpoorvoltagestabilityafewminuteslater,etc.,andasaresult,anymitigationstrategywillchangedailyorfaster.

    Thebasicstrategyofcommercialpowercompaniesistomaintainreliableserviceandavoidashutdownatnearlyallcosts.Undernormalconditionsthisapproachensurestheavailabilityof

    powerto

    utility

    customers

    but

    under

    severe

    GMD

    conditions

    this

    approach

    increases

    the

    probabilitythatsomeamountofdamagetooperatingequipmentwilloccur.

    Muchoftheburdenforrapidlyrespondingtoearlywarningfallsupontheutilitycompanieswhoareboundbyasetofstandardsthatprohibitthemfromdroppingcustomersundermost

    conditions.Ifforecastsareincorrectabouttheevent,theutilitywillhavetoanswerto

    regulatoryagencies.Thereisnocertaintyaboutwhatdamagewillbecausedandutilities,

    operators,engineers,andothersdonotagreeonhowmuchdamagemayoccur.Becausethere

    isnolegalprotection,utilitieshavelittleincentivetotakestrongpreventativesteps.

    Seasonalityisacriticalfactoraswell.Twotothreemonthswithoutelectricitywouldhavedifferentimpactsinahotsummerascomparedtoaharshwinter.Thisscenarioassumedthat

    theweatherwasmild,butshouldasimilaroutageoccurinsubfreezingconditions,the

    reactions,procedures,andoutcomeswouldbevastlymorechallenging.

    Therewasaconsensusamongexerciseparticipantsthatamoreaccurateassessmentoftheseverityand

    timingofGMDconditionsisrequired.Bettermodelingandsimulationcapabilitiescouldhelpinthis

    regard.Additionally,abettertechnicalunderstandingoftheeffectsthestormwillhaveonextremely

    highvoltage(EHV)transformersthemselvesisimperativesincethisisthemostcriticalfactoraffecting

    theduration

    of

    ablackout

    and

    recovery.

    TransformersTheprimarypieceoftheelectricgridtobeaffectedbyaneventofthemagnitude

    positedbytheexercisearetheroughly2000EHVpowertransformersinthebulktransmissiongrid.

    Excessivedirectcurrent(DC)flowingthroughalargealternatingcurrent(AC)transformerground

    connectionmaycauseoverheatingandfailure.Whilethetruenumberoftransformersthatcouldbe

    damagedbyasevereGMDisunknown,losingasignificantportionofthefleetwouldresultin

    widespreadlongtermpoweroutages.ThedamagesusceptibilityofanindividualEHVtransformeris

    influencedbyitsageandhealth,butalsobythelocalgroundconductanceandgeologyofthe

    transformersubstationlocationandconnectedtransmissionlinelengthandorientation.Factorssuchas

    thesecan

    combine

    in

    ways

    that

    can

    make

    some

    older

    transformers

    less

    vulnerable

    than

    some

    newer

    ones.

    Significantpoints:

    MostlargeEHVtransformersaremanufacturedoutsideoftheUnitedStates.IftherewasaneventthatimpactedtheU.S.andothercountries,andifkeypiecesofequipmentweredamaged

  • 8/2/2019 Secure Grid '11 After-Action Report

    16/28

    12

    andrequiredreplacement,itwouldtakealongtimetoreplacethem,possiblymonthsoryears,

    andtheU.S.mightnotbethefirstpriorityofthemanufacturingcountries.

    AnothercomplicatingfactorisreplacementEHVtransformertransportationtotheremotelocationswheretheywouldbeneeded.ManylargeEHVtransformerlocationsarenoteasily

    accessibleand

    these

    very

    large

    and

    heavy

    transformers

    require

    special

    transportation

    equipmentthathaslimitedavailability.Ifthereisalargemagnitudeevent,littlecapabilityexists

    torapidlyreplacetransformers.

    ThereistheSTEPprogram(SpareTransformerEquipmentProgram)whichinvolvessharingsparetransformersbetweenutilitiesthatarevoluntarysignatoriestothisagreement.This

    programwasdesignedtohandleshortnoticerequirementsforsmallnumbersofreplacement

    transformersfollowinganeventsuchasanisolatedterroristattackonafewsubstations.The

    questionregardinganadequatenumberofsparesforlargerscaleeventssuchasasevereGMD

    ismoredifficulttoanswerandthereiscurrentlynoindustrypositiononthissubject.

    Furthermore,theleadtimecanbeaslongsas12yearsforprocuringmajortransformers,many

    ofwhichareuniquelydesignedforaspecificgridlocationintermsofvoltageandpowerratings.

    Inthe1921levelscenariodescribedbytheNASreport,hundredsoflargetransformersmightneedtobereplacedintheU.S.anditwouldtakeanextremelylongtime,asonlyasmallfraction

    ofthevoltageclassesandtypesneededarecurrentlyproduceddomestically.

    IfaGMDsuperstormdoesproduceablackout,servicecanberestoredtomanylocationsusinglowervoltagesystemstoeffectivelybypassdamagedpartsofthehighvoltagegrid.Generation

    iswellconnectedatlowervoltagelevels.

    TheMilitaryAccordingtoUSNORTHCOM,itsbasesattheinstallationlevelarenotnormallyprovidedrealtimeadvancewarningofanimpendingGMD.Certainmissionsdirectlyimpactedbyspaceweather

    docurrentlyreceiveforecastsandwarnings.However,noticetoinstallationstoensurereadinessofthe

    supportinfrastructuremaybenecessaryaswell.Itisessentialthatbaseshostingcriticalmissionshave

    sufficientelectricalpowertosustainindependentcontinuousoperationsevenifthesurrounding

    commercialpowergridislost.Thoughmanyinstallationspossessdieselpoweredelectricalgenerators,

    theyhavelimiteddieselfuelstorage.Intheeventofawidespreadgridoutage,fuelsuppliescould

    dwindlerapidly,therebyimpactingtheabilityofinstallationstoconductprimarymissions.

  • 8/2/2019 Secure Grid '11 After-Action Report

    17/28

    13

    TabletopExercisePart2:Protect(T0+20hrs)1300,October4,2011BrieftoPlayersNOAAs

    Space

    Weather

    Prediction

    Center

    in

    Boulder

    has

    issued

    a

    G5,

    extreme

    geomagnetic

    storm

    warning.ThecoronalmassejectionthatleftthesunlessthantwentyhoursagohasreachedtheACE

    (AdvancedCompositionExplorer)spacecraftandisanextremelyintensecoronalmassejectionthathas

    theworstcasemagneticpolarityandwillbegintoaffectEarthsmagneticfieldinthenexttwentyto

    thirtyminutes.

    Theconsequencesofthiseventonthepowergridcertainlycouldbesignificant.Themostsignificantly

    impactedareaswilllikelybeintheNortheastincludingtheNortheasternCanadianprovinces,through

    NewEnglandintotheMidAtlanticStates,thecentralGreatLakesregion,andalsointhePacific

    Northwest.

    ImpactEstimateWidespreadpoweroutagespossiblylastingforlongerthaninanypreviouslargeblackout.

    SummaryofTeamDiscussionsThediscussionsforthesecondmoduleprimarilycenteredupontwomaintopics:communicationsand

    authority.Givenasignificantoutageafteraseverespaceweatherevent,issuesdiscussedincludedhow

    communicationswouldtakeplace,and,inparticular,howlocalauthoritieswouldcommunicatewiththe

    publictopreventpanicandmaintainlevelsoforderandpublicsafety.Theauthorityissuecenteredon

    whohadauthoritytoauthorizewidespreadprotectivemeasuresforthegrid,suchasashutdown.

    CommunicationsCascadingeffectswereofsomeconcernregardingtheproblemofcommunications

    duringamassivepoweroutage.Questionsregardinghowcommunicationinthatsituationwouldoccur

    wereaskedamidconcernsoverpublicpanicgivenalongtermoutageofpower.

    Landlinesandcellphoneservicewillverylikelybeoverburdenedordown.Itwasquestionedhowcommunicationswouldoccurinthiscase.Someutilitieshavededicatedcommunications

    lines,butthemajorityhavemigratedtocommerciallyprovidedservices.Thisisparticularly

    importantbetweenprivateandpublicsectors.Thenuclearindustryhasdedicatedlineswiththe

    NRC.

    Publicbehaviorandreactionscanproducedangeroussituationseveninnormalweatherevents.Itisnotknownhowthepublicwouldreacttowarningsabouthappeningsinouterspace.Itis

    unknownwhatrangeofresponsetoexpectfromthepublic(theycouldpanicorcoulddisregard

    warnings),andthoseresponseswillvaryfromplacetoplace.

  • 8/2/2019 Secure Grid '11 After-Action Report

    18/28

    14

    Giventhenominalthirtyminutetoonehourfinalwarningprovidedtoutilities,thereisaquestionregardingwhetherthereshouldalsobeprovisionsforwarningthepublicbeyondthe

    NOAAwebbasednotificationprocess. Itisnotclearwhatthepubliccoulddoinresponsetoa

    warningexcepttowaitandseewherepowerislostthentakeappropriateactionwhere

    possible.Spaceweatherisnotawellknownthreatamongthepublic.Itishardtogaugehow

    seriouslysuch

    warnings

    would

    be

    taken,

    particularly

    since

    NOAA

    typically

    issues

    four

    to

    five

    G5

    worstcaseGMDalertsinatypical11yearsolarcycle.

    Ensuringthatcleanwatersuppliesandsanitationareafirstpriorityneedstobecommunicatedtoallplayers,especiallythepublic.Attentionmustbegiventohealthandsecurityifoutagesare

    potentiallylongterm.

    Localauthoritiesneedtobegiventimelyinformation,particularlyincaseofalongtermevent,tominimizethepossibilityofcivildisorder.

    BeforeandduringHurricaneKatrina,theelectricutilitiesmadepreparations,toincludeconferencecalls,alerts,warnings,actions,procedures.Lossofcommunicationsexpectedto

    accompanywidespreadpoweroutageswouldmakerecoverycoordinationverychallenging,and

    asignificantproblemgiventhepotentiallyshorttimeframebetweenfinalwarningandthe

    possiblelossofpower.

    DefenseSupporttoCivilAuthorities(DSCA)maybeespeciallyimportantifalongtermpoweroutageoccurs.AlthoughDODactivitieshaverobustbackupandredundantcommunications

    equipment,thatdoesnotnecessarilymeanthelocalauthoritiesdoaswell.Coordinatedeffort

    mustbemadetoensurecommunicationchannelsareopenduringanevent.

    Industryneedstokeepgovernmentinformedaboutwiththestatusofthegridandanymajordifficultiespertainingtocommunications.However,therewassignificantdisagreementabout

    theabilityofcommunicationssystemstoremainfunctionalinapoweroutage.

    Betterunderstandingofthetimeframeforapossiblewindowofpanicisneededsothatauthoritiescangetinfrontoftheproblemandprepareaccordingly.

    FEMAhasemergencybroadcastresponsibilityandtherearebackupsystemstothataswell.Radios

    and

    other

    basic

    equipment

    might

    be

    used

    to

    provide

    information

    to

    the

    public,

    as

    internetandtelevisionwilleventuallybelostatmostlocationsinanextendedpoweroutage.If

    FEMAneedstoprovideradiosthentheywilldothatbutobviouslythiscanonlybedonein

    limitednumbers.

    Therewasalsosomediscussionaboutthepossibilityofestablishinganationalcoordinationcenter.

    Industryparticipantswerecriticalofthatidea,claimingthatthefederalgovernmentalreadyhastoo

  • 8/2/2019 Secure Grid '11 After-Action Report

    19/28

    15

    manyoperationcenters.Abetteralternativewouldbeaprocessratherthananopscenter.Thereis

    alreadyagooddealofmonitoringgoingon,wejustneedbettercommunications.

    AuthorityandRoleofGovernmentOneachofthefourexerciseteamsthequestionwaspointedlyasked,Whatcangovernmentdotohelputilitiesduringthiskindofemergency?Theresponsewasa

    unanimousnot

    much.

    It

    was

    quickly

    established

    that

    government

    has

    little

    to

    no

    authority

    to

    dictate

    toindustryhowitistohandlethesesituations.Manyoftheproceduresneeded,suchasprioritizationof

    criticalinfrastructure,arealreadyinplace:water,sanitation,andhospitalstopthatlist.Theprincipal

    questions,then,werewhetherthereneedstobeagovernmentalauthoritytoorderutilitiestoshut

    downorreduceloadingonthegridinordertoprotectit,whetherinsteadwhatisneededisprotection

    fromliabilityshouldutilitieschoosetodosothemselves,and,giventhatmostoftheregulatory

    responsibilityrestswiththestates,justwhatistheroleofFederalagenciesinthissituation?

    WhatareDOEandDHSdoing?Atthisstageoftheemergency,DHSandDOEwillbecloselyengaged,butwillprobablynottakeanydirectactionaffectingprivatelyownedutilities.Atthe

    time

    NOAA

    issues

    the

    alert,

    DHS

    and

    DOE

    will

    be

    following

    the

    situation

    but

    do

    not

    have

    authoritytorequireanyaction.

    Whatarethestatesdoing?InmanycasesstateemergencymanagementofficeswillprobablynotevenbeawareofthethreatofaGMDalertbeforeitcouldstarttoaffectthegrid.Those

    thatarewillbespinningupEmergencyOperationsCentersinpreparation.

    ForFEMAthefirstpriorityistoprotectlivesandproperty.FEMAsNationalResponseCoordinationCenterwouldbeinitiatingresponseplansatthispoint.Communicationbetween

    theFederalgovernmentandthestateswouldbeginaswell.FEMAalreadycommunicateswith

    theNOAA

    Space

    Weather

    Prediction

    Center

    in

    Boulder.

    VTC

    at

    20

    hours

    with

    all

    partners,

    with

    multiplelevelsofactivation.Forexample,Level1brings15disparateorganizationsand

    activatesdialoguechannels.Level2bringsESF2andESF12intothedialogue.FEMAwillnotbe

    tellingthemwhattodo,butusingthemasthemainpointsofcontactfortherestof

    government.ESF3,publicworkssupport,willprovidegeneratorsifneededandwillbegin

    preparationstomovethemtowheretheythinkthemostaffectedareaswillbe.FEMAwillnot

    bedirectlyinvolvedintellingindustryhowtocorrectproblems,buttheywillreachbacktofind

    outwhatproblemsneedtobesolvedfirst.FEMAdoeshaveadeskwhichincludesprivatesector

    personneltodeterminewhatservicesindustryandtheprivatesectorareabletoprovide.

    Industrysmajorcoordinationroleduringthistypeofcrisisistoevaluatethescaleandextentofaneventandtocommunicatewithgovernmentalagencies.

    AStaffordActdeclarationmightbeneeded.

  • 8/2/2019 Secure Grid '11 After-Action Report

    20/28

    16

    Secondaryconcernsneedtobeanalyzed.Whatfuelsuppliesdohospitalsandotheressentialserviceshave?Areairportsstilloperational?

    Theprimarymajorissuerequiringmitigationispotentialpublicpanic.Publicpanicmayleadtoarunonresources,socialunrest,eruptionofviolence,anddenigratetheabilityoflocal

    governmentto

    restore

    order.

    Emergency

    services

    are

    likely

    to

    be

    strained

    to

    capacity

    and

    lifesavingservicesdelayed.Localgovernmentofficialswillberesponsibletoeffectively

    communicateunfoldingeventstothepublicandtomaintainpublicorder.Itisimperativethese

    officialsareprovidedwiththepertinentinformationtofullyunderstandtheeventsand

    empoweredtocommunicatethatinformationtothepublic.

    Thequestionwasraisedbyoneparticipant,Shouldweshuteverythingdownfor24hoursinordertoprotectthegrid?Furthermore,doesanyonehavetheabilitytomakethatcallin

    regulation?Theunanimousanswerwas,no.Thegridinterconnectionauthoritiescouldtell

    powerplantstoshutdownbutthereisnoestablishedauthoritytodirectthisaction.The

    Presidenthastheabilitytoprotectutilitiesfromliabilitiesbutshouldauthoritybegiventothe

    Presidenttobeabletodirecttheshutdownofthepowergrid?Industryrepresentativesstated

    thatplantownersarebestequippedtomakethesedecisions.

    OtherItems: Anobviousstepthatgovernmentcoulddotohelputilitieswouldbetoprovideareplacement

    fortheACEspacecraftthatprovidesthemostimportantearlywarnings.Itisalreadyseveral

    yearsbeyonditsdesignlifetime.Areplacementspacecraftisscheduledforlaunchin2014.

    Arewepreparedtoactivateemergencycentersbasedonspaceweatheralerts?Areutilitiespreparedtotellthepublicthattheymaylosepowerduetowhatisgoingoninouterspace?

    Powerlossmeansdifferentthingsindifferentregions.Betterunderstandingofthisissueisneeded.Ruralcommunitiestendtobebetterpreparedforpowerlossthanurbanareas.

    Federalizetheriskratherthanthegrid.Thisapproachallowsregionaloperatorstocontinuetomaketimelydecisionstoprotecttheintegrityofthegridandminimizestheprobabilityof

    extendedblackouts.

    Someliabilityconcernsmaybemisunderstoodorinflated,ashospitalsandothermissioncriticalindustriestypicallyhavebackupgeneratorsavailableforemergencies.

  • 8/2/2019 Secure Grid '11 After-Action Report

    21/28

    17

    TabletopExercisePart3:Respond(T0+20hrs)1500,October4,2011BrieftoPlayersFEMAanditsfederalpartnersarerespondingtotheworstgeomagneticstormindecades.Thisglobal

    eventhasproducedwidespreadimpactsthroughouttheUnitedStates,Canada,andseveralother

    countriesinAsiaandnorthernEurope.NOAAisreportingthatthestormisessentiallyovernow,buta

    varietyofcriticalinfrastructureswereheavilydamagedduringthestormthoughthemostproblematic

    impactsareassociatedwiththepowergrid.

    TherearelargepoweroutagesintheNortheastandMidAtlanticStates,aswellasacrossthenorthern

    borderwheremostoftheCanadianpowergridisinterconnectedtotheU.S.Manycities,towns,military

    bases,andindustrialcentersarecurrentlywithoutnormalpowersupplies.Similarelectricaloutages

    havebeenreportedinnorthernRussia,Japan,Sweden,Norway,andtheUK.MajordisruptionstoGPS

    serviceandsometelecommunicationsoccurred,butthoseeffectsareexpectedtobetemporaryand

    theseservicesshouldreturntonormaloverthenextseveralhoursasthefinaleffectsofthestormfade

    away.

    Theglobalnatureofthiseventalsopresentsproblemssince,insomecases,thecomponentstoreplace

    damagedequipmentwillbeneededinmanycountriesatthesametime.Thatcanplaceaseverestrain

    ontheworldwidesupplysystem.Thescopeofthisproblemwillrequirefederal,state,andprivate

    partnerstoprovideassistanceinmanypartsofthecountry.

    DamageEstimate40MAmericansw/oelectricity

    About65%expecttoberestoredin8hours About25%notexpectedtoberestoredinlessthan2weeks Final10%notlikelytoberestoredinlessthan2monthsusingnormalsupplies

    SummaryofTeamDiscussionsThebulkofthediscussionforthethirdmodulecenteredonbackupgenerationcapabilities,system

    restoration,andcoordination.Itwasduringthissessionthatwaysgovernmentcanhelpindustrywere

    better

    identified

    and

    expressed.

    Given

    the

    unprecedented

    loss

    of

    dozens

    of

    US

    transformers,

    a

    gap

    discussedduringthefirstmodulewashowtogetreplacementtransformerstolocationswheretheyare

    neededgiventhedifficultyintransportingthem.Governmentcanaidconsiderablyinovercomingthis

    problem.Further,governmentcanplayasignificantroleinfindinganddeployingtemporarypower

    generationoptionsforisolatedcriticalinfrastructure.

  • 8/2/2019 Secure Grid '11 After-Action Report

    22/28

    18

    BackupGenerationgivenaneventofthismagnitude,someareasmaybecomeisolatedfromthegrid.Inthoseinstances,localgenerationforsometemporaryperiodisneededtokeepcriticalinfrastructures

    runninguntilthegridcanberepairedtoastatewhereitcanprovideconsistentpoweragain.Thereare

    anumberofFEMAownedtwomegawattelectricalpowersourceskeptavailableforhurricanesand

    otherdisasters.Capacityexiststomovetheseandquiteafewcustomerscouldberestoredwiththose

    powersources.

    This

    is

    an

    opportunity

    for

    the

    government

    to

    assist

    by

    identifying

    needs

    and

    supplying

    electricalpowergenerationcapabilitiesinatimelymannertomitigatelossoflifeandpropertydamage.

    Discussionfindingswereidentifiedasfollows:

    Thereareportablegeneratorsthatcomeincontainers.FEMAownsmanyofthesecontainerizedgenerators.

    Inthecommercialmarket,therearerentalgenerators,buttheymovewiththeglobalmarket,sothosemaynotbeavailableunlesstheUSpayssomekindofretainertokeepsomeportioninthe

    country.

    TherearesomegeneratorsintheUStowhichindustryhasexclusiverights.Apremiumispaidsothatindustryhasfirstrightstothem.80percentofthemobilemarketisalowervoltagethan

    thegridusesandwouldtypicallyneedastepuptransformertogettothepropervoltage.

    AneffectiveresponsewillalsorequireclosecoordinationatthehighestlevelsofDOD. Itisunknownhowmuchbackupgenerationrefineriesandotherlargecustomershave.Although

    manycompanieshavehardenedtheirsystemstovaryinglevelsinrecentyears,thispracticehas

    notbeen

    widespread.

    Many

    have

    purchased

    or

    leased

    generators

    for

    use

    in

    emergencies.

    Hospitalshavebackupgenerationbutonlyafewdaysoffuel.Thefuelresupplyproblemwould

    applyacrosstheboardforsuchbackupgeneration.

    Aprimaryproblemisthatthereisonlyenoughfueltorunbackupsforafewdays.Hereisanotherplacewheregovernmentcanhelpbyprioritizingfuelsupplytogenerationsystemsin

    placetemporarily.Forinstance,intheVancouverOlympics,thegovernmentaggregatedkey

    playersneedsforfuel(diesel)generationandfoundthatnotonlywastherenotenoughfuelin

    thisareaofCanadatorefuelthesupplybutalsonotenoughtransportationcapabilitytogetfuel

    inplace.TheUKexperiencedasimilarproblemin2007whenthereweremajorfloodsand

    difficultdecisions

    had

    to

    be

    made

    about

    prioritizing

    distribution

    sites.

    Floridahasmandatedthatallgasstationsalongevacuationrouteshavebackupemergencygenerationcapabilities.Thisstepmaybeagoodideatohelpmitigatethefuelproblemposedby

    anylongtermpoweroutage.

  • 8/2/2019 Secure Grid '11 After-Action Report

    23/28

    19

    SystemRestorationUltimatelythepowersystemhastocomebackonline.Governmentassistancewouldbeneededtohelpfacilitatetherestorationprocessandsomepotentialsolutionsareoutlined

    below:

    Easingofregulationsduringcatastrophicevents.Onewaythatgovernmentcouldhelpisthrough

    the

    issuance

    of

    waivers

    to

    existing

    regulations,

    especially

    interstate

    waivers

    of

    vital

    commodities,suchasfuel.TheFederalgovernmentcouldassistwithallowingacheaperfuelto

    beusedtofacilitatefasterrestoration.Permittingcouldalsobeutilizedtoeasetheroutes.Each

    statewouldhavetoapprovethesewaiversunlessthefederalgovernmentmandatedit.The

    movementoflargeheavytransformerswouldbeveryresourceconstrained;theyneedtomove

    incrediblyslowlyandthereareveryfewcapabilitiestofacilitatemovementofequipmentthis

    sizeandweight.

    Plannedrollingblackouts.Duringpowerrestoration,rollingblackoutsareapotentialmethodtorationpowersothattherearefewerlocationswithnopoweratall.Servicecanbeprovidedto

    somecustomers,

    but

    not

    everyone

    at

    once.

    The

    goal

    would

    be

    to

    provide

    the

    best

    available

    servicetonormalcustomersafterthemostcriticalservicesarerestored.

    Prioritizingpowerrestoration.Theremaybesevereeconomicdamageinsteelmillsandindustriessuchaselectronicsfabricationfacilitiesthatnormallyrequirecontinuouspowerto

    maintainsterilemanufacturingconditions.Duringtherestorationperiodtherewillbepower,

    butitwillbelimited.Noteveryonewillhavepowerallofthetime.

    CoordinationIssuesofcoordinationwereasignificantpointofdiscussionamongallthegroups,but

    theconsensuswastheprimarycoordinatingbodywouldbeFEMA,underDHS.DOEwouldcoordinate

    theenergy

    industry

    recovery,

    while

    DHS

    would

    deal

    with

    the

    overall

    federal

    response.

    It

    was

    made

    clear

    again,however,thatindustryisnotcontrolledbygovernment,andthatutilitieswerelargelyfreeto

    respondastheyseefit.Inanorganizationalsense,respondingtoalongtermpoweroutagewouldbe

    theresponsibilityofthestatewiththeFederalgovernmentmakingresourcesavailable.

    Intermsofauthority,thegreatestuncertaintiesarepriortotheevent,notpostevent.Onceaneventhasoccurred,communicationprocessesarewellestablished,thoughsignificantdoubt

    existsthattheseprocesseswouldfunctionsmoothlyinalargepoweroutage.Statesandutilities

    thathavemanagedmanyrealdisasterswillbebetteratdealingwiththemthanstatesthathave

    lessrealexperienceeventhoughthesamemechanismsareinplaceforalltypesofdisasters.

    Underemergencysituationssomegenerationprioritiesmaybefungible.IfOhioisabletosupplyallofitsnormalpowerrequirementsbutitsneighborstateIndianais75%blackedoutthen

    shouldOhiodenypowertosomeofitsownterritoryinordertoassistIndiana?Thereisno

    clearlyestablishedprocessformakingthistypeoftradeoffdecisionundercrisisconditions.

    Todaythesedecisionprocessesrestwithutilities.Stategovernmentsworkwithutilitiestokeep

  • 8/2/2019 Secure Grid '11 After-Action Report

    24/28

    20

    somepublicassetsrunning,suchashospitals,buttheutilitiesowntheirpowergridandcan

    makedeterminationsthemselves.Thereareplansfiledwitheachstateregardingprioritiesfor

    water,firestations,peopleonlifesupport,etc.Duringemergenciesitismadeknownaccording

    tothoseplanswhichsourcesarethemostcritical. In90percentofrestorations,utilitiesmake

    thedecisionaboutwhatgoesbackonlinefirst,basedoninstructionsandwhattheycan

    physicallydo.

    Utilities

    frequently

    have

    interruptible

    power

    supply

    agreements

    with

    large

    customersandtheconditionsoftheseexistingcontractscandeterminetheorderinwhich

    powerisrestoredtousers.ThereisnotalistthatisprovidedbytheFederalgovernment.In

    general,however,mandatesexistforhospitalsandothercriticalfacilities.

    Amajorpotentialgapinvolvesunderstandinghowlongthepowercouldbelost.Manycoordinationissuescanmoveforwardproperlyandconfidentlyifthatinformationisknown

    early.

    MilitaryThemilitaryhasacriticalfacilitieslistthatwillbeusedasthemilitarymanagesoutages

    affectingits

    bases.

    A

    considerable

    number

    of

    critical

    assets

    on

    military

    installations

    have

    generators

    or

    otherbackuppowersupplies.Manyinstallationscanprovidebackupelectricalpowertosustain

    continuousoperationsformissioncriticalfacilitiessuchascommunicationscenters,althoughthelong

    termsustainabilityoftheseassetsdependsontheavailabilityoffuelsupplies.

    Somebasesmayhavesomeexcesspowerthatcouldfeedbackintothegrid,butonlytemporarily.Inpastdisasters,basecommandershavecoordinatedthesedecisionswithutilities.

    Defensecasestudiesarebeingconductedtodiscovertheactualdurationthatbasescansustainthemselves.Thereissomedebateoverhowlongatypicalbasecansustainitself.Somesaya

    weekwould

    be

    avery

    optimistic

    estimate.

    Mission

    critical

    functions

    are

    often

    covered

    but

    day

    todayassetsarenot.Othersclaimedthatthemilitaryhasafairlyrobustprogramtoprovide

    fuelandhasarobustbackupsystemforcriticalfacilities.Someclaimedthatthemilitarybases

    couldruntheirgeneratorsforseveralmonthsifneedbe,butothersdoubtedthecapacitytoget

    thefuelinplaceneededtodothat.Anobviousexceptiontothislimitationwouldexistatmany

    militarybaseswheretherecanbemillionsofgallonsofaviationorshipboardfuelstoredon

    base.Generatorsthatcouldrunonthistypeofplentifulfuelinsteadofdieselwouldhavemore

    thanenoughsupplytoprovidepowertoabaseformanyweeksormonths.

  • 8/2/2019 Secure Grid '11 After-Action Report

    25/28

    21

    RecommendationsOnWednesday,October5th,thefourteamsgatheredtodiscusstheexerciseandsummarizetheresults

    fromthepreviousdaysefforts.Thefollowingsummarizesthesediscussions.

    ReplacementoftheACESatellite:TheACEspacecraftiscurrentlylivingonborrowedtime,havingwellexceededitsdesignservicelife.Areplacementneedstobescheduledasquicklyas

    possible.NASAandNOAAareworkingtolaunchtheDeepSpaceClimateObservatory(DSCOVR)

    spacecraftonaDODvehiclein2014.DSCOVRwillprovidetheindicationsandwarningsthatACE

    hasprovidedinthepast.

    Hardeningthegridisthepreferredsolution:Preventionispreferabletoresponseandrecovery.Appropriatemethodstoprovideeffectivehardeningtoprotecttransformersandothercritical

    gridinfrastructureinaGMDsuperstormneedtobeexamined,studied,developed,assessed,

    tested,and

    deployed.

    A

    national

    strategy

    is

    needed

    to

    determine

    how

    the

    costs

    of

    hardening

    thegridagainstspaceweathereventsshouldbefunded.Therearepromisingsignsthatmanyof

    themostcriticalgridsystemscouldbehardenedforlessthan$1Btotalcost.

    Improvementsinforecastingaccuracyandtimelinessarenecessary:AlthoughearlywarningsaretypicallyprovidedbyNOAA24to36hoursbeforeaCMEcanreachEarthandproducea

    GMD,thefinalmostimportantinformationisgeneratedbytheACEspacecraftonlyabout45

    minutesbeforeGICcanoccurinthegrid.Thistimeframeisshortfortakingeffectiveactionat

    morethan3000independentutilitiesandtransmissioncompaniesintheUSalone.Additionally

    othercountriesdependonthissameshortwarningtime.

    Gridshutdownisseenbyindustryasanunrealisticoption:ThereisnonationalauthoritytodirectagridshutdowntoprotectcriticalinfrastructureagainstGMDimpacts.Ifestimatescited

    inthe2008NationalAcademiesofSciencereportareaccurate,thecostsofnotshuttingdown

    thegridcouldeasilyexceedthecostofatemporaryshutdown.Iftheauthoritytoimplementa

    shutdownislefttoindustry,legalprotectionagainstliabilityclaimswouldalmostcertainlybe

    necessary.

    Untilthegridishardenedtopreventtransformerfailuresinlargenumbers,thecapacitytoreplacetransformersinlargenumbersovershorttimeframesneedsmajorimprovement:The

    United

    States

    has

    limited

    capability

    to

    manufacture

    the

    numbers

    and

    types

    of

    EHV

    transformersthatcouldbeneededfollowingaGMDsuperstorm;thereforethereisdependency

    uponforeignsuppliersforreplacements.Whetheratransformerisproduceddomesticallyisnot

    directlyrelevanttoitsusefulnessbutfollowingaGMDsuperstormforeignmanufacturerswould

    likelyberespondingtoordersfrommanyinternationalcustomersandtheUSmightnotbeseen

    ashighestpriority.Thislimitationcouldbeaddressedthroughacombinationofexpanded

  • 8/2/2019 Secure Grid '11 After-Action Report

    26/28

    22

    domesticmanufacturingcapacityorincreasesinstockpilereplacementspares.Inbothcases

    thereisnobusinesscaseforsuchactionfromindustrysperspective.Governmentwouldneed

    toidentifyanationalsecurityriskrequiringsuchactionbyindustryandfindareasonablewayto

    fundthesemeasures.Thiswouldbesimilartowhathappenedafter9/11whenanothernational

    securityriskwasidentified.Althoughtheairlinesandairportswereprivatelyownedthe

    governmentrequired

    changes

    including

    major

    airport

    security

    enhancements,

    hardened

    airline

    cockpitdoors,FederalAirMarshalls,changestocriticalairspacerestrictions,etc.

    Effortsmustbemadetobettereducatestateandlocalemergencymanagementofficials:Spaceweatherremainsapoorlyunderstoodtopictomostemergencyplannersandmanagers.

    Althoughitisconsideredalowprobabilityhighconsequenceevent,therehavebeentwo

    superstormsinthepast150yearsbutinbothcasestheworldwasfarlessdependenton

    electricitysuppliestoprovideadequatelivingconditionstobillionsofpeople.Thepublicdoes

    notneedtounderstandthedetailsofCMEs,GMDs,GICs,andspacecraftwarningsbutitdoes

    needtobebetterpreparedforlongtermpoweroutageswhethertheyarecausedbyspace

    weather,other

    natural

    disasters,

    deliberate

    attacks

    or

    simple

    equipment

    failure

    associated

    with

    aginginfrastructureandgrowingdemand.

    Amorecomprehensivestudylookingatcascadingeffectsisneeded:Theimpactontheeconomyofawidespreadandlongtermgridshutdownduetoaspaceweathereventishighly

    complexandpoorlyunderstood.Thepotentialeffectsonsocietyareevenlesswellknown.A

    betterunderstandingofthissubjectisneededatanationalandinternationallevel.

    Federal,state,andlocalemergencymanagementofficialsneedaccesstoanaccuratedatabaseofportablegenerationequipment:Shouldasignificanteventoccur,itislikelythatsomesectionsofthegridwouldremainoperationalinanislandedmode.Inordertokeepcrucial

    infrastructureintheseregionsonline,especiallywaterandsanitation,itwouldbehelpfulto

    knowwhereallportablegenerationassetsare,andtohavesomeonewithauthoritybe

    responsibleforprioritizingtheirdeploymentacrossstatelinesandbetweenutilityserviceareas

    whennecessary.

    Providetimelyinformationtolocalgovernmentsandmunicipalities:Theburdenofmaintainingpublicsafetyandcommunicationwiththepublicwilllargelyfallupontheshoulders

    oflocalauthorities.IndustryandFederalagenciesneedtoestablisheffectiveemergencybackup

    communication

    capabilities

    that

    can

    keep

    enough

    of

    the

    public

    informed

    even

    though

    normal

    powersuppliesmaybeunavailable.

    Revisitthetransformersharingplan:Fewoftheparticipantshadconfidencethatthecurrentplanwouldholduptoanemergencyofthismagnitude.Additionalcooperationandagreements

    shouldbeputinplace.TheexistingindustrySTEPprogramfortransformersharingisvoluntary

  • 8/2/2019 Secure Grid '11 After-Action Report

    27/28

    23

    andwasdevelopedtohandleemergentneedsforsmallnumbersofreplacementssuchasmight

    beneededafteranisolatedterroristattackonnomorethanafewsubstations.

  • 8/2/2019 Secure Grid '11 After-Action Report

    28/28

    24

    AppendixSecureGrid11ScenarioBackgroundTheSG11scenariowasdesignedtoexamineproblemssimilartothosehighlightedinthe2009National

    AcademiesofScienceExtendedSummaryReportSevereSpaceWeatherUnderstandingSocietaland

    EconomicImpacts.

    This

    figure

    taken

    from

    that

    report

    shows

    one

    scenario

    examined

    by

    the

    NAS

    based

    onaneventsimilartoanactual1921superstorm.

    ForSG11thefollowingassumptionswereselectivelymadeinordertoprovideanextremelychallenging

    butmoremanageablescenario:

    Theexercisegeomagneticdisturbance(GMD)producesgeomagneticinducedcurrents(GIC)whichquicklycausespowerblackoutswithinthetwooutlinedareasshownabove.

    About65%oftheelectricalloadsandgenerationintheseareascanberestoredwithin8hours. About25%oftheelectricalloadsandgenerationintheseareascannotberestoredbynormal

    meansinlessthan2weeks.

    Thefinal10%ofelectricalloadsandgenerationintheseareascannotberestoredbynormalmeans

    in

    less

    than

    2

    months.