Cyber-physical security for the microgridNew perspectives to protect critical power infrastructure

2

Microgrids have emerged as a powerful potential solution to a range of challenges facing consumers and distribution networks. They can offer resilient power delivery at times of broader distribution outage by self-islanding and relying on a range of local generation and storage assets. In addition, microgrids have the potential to integrate larger quantities of renewable generation than even a smart distribution network through highly localized optimization of distributed generation output, demand flexibility and storage assets. These characteristics make microgrids a potentially effective approach for smaller, isolated systems with limited connection to the grid.

But microgrids also create new vulnerabilities. The sophisticated distributed control of flexible assets and demand within a microgrid provide the promise of resilience; however, the increased penetration of monitoring and control capabilities open up the possibilities for breaches of security. The digital era will extend and stretch an organization’s boundaries, and microgrids provide an extreme example of this trend for distribution utilities.

3

Microgrids provide a level of system optimization through improved visibility and control that is far beyond that of a typical distribution network. However, at this stage the implementation of a microgrid is a significant investment undertaking, which means that early stage deployment is likely to be concentrated on customers who place high value on one or more of three areas.

Energy surety for mission critical processes

Traditionally, energy supply for sites with mission-critical functions such as semi-conductor manufacturing, hospitals and datacenters have been assured using on-site, commonly diesel-fired, back-up generation and uninterruptible power systems. The microgrid concept adds new components to these solutions such as demand flexibility, storage and renewable generation to produce a more reliable, cost-effective and sustainable solution. If a physical event or cyber-attack causes a power outage, microgrids can contain the impact by shedding non-essential loads and continuing to energize critical loads.

Physical and cyber security

Sites that are at particular risk from physical attacks and sophisticated cyber warfare, such as defense establishments, key government departments and nuclear installations, are prime candidates for microgrid deployment. Microgrids provide the ability to self-island from a distribution network that is under cyber- or physical-attack as well as use distributed, self-healing architectures to maintain energy delivery.

In the United States, directives from the highest level of government1

have elevated the concept of resilience in the face of cyber-attacks from being largely restricted to conversations in the field among emergency responders to becoming a pressing concern for compliance directors and corporate leaders.

Microgrids as the next evolutionary stage of energy delivery

The April 2013 Metcalf transmission substation physical attack in California confirmed fears about the ease of executing an intentional coordinated attack on critical power infrastructure. In the case of Metcalf, it was possible to reroute power and critical loads were not impacted. However, had power not rerouted, the result could have been havoc. Despite several programs available to supply spare transformers (e.g., EEI, EPRI, spares at asset owners, etc.), in general there is significant lag time to obtain highly-rated transformers. Furthermore, not all utilities are members of spare transformer programs. Microgrids are one way that owners of critical loads can be provided additional assurance that energy will be supplied.

4

Sustainability and renewables integration

The renewables carrying capacity of distribution networks can be rapidly eroded where deployment is high, resulting in the requirement for significant network investment to maintain reliability and quality. Microgrid deployment offers the potential to locally optimize demand, storage and renewables output, allowing greater volumes of renewables to be deployed without significant negative impacts on the rest of the distribution network. Consumers can access data about their usage, real-time electricity prices and pollution implications to help them manage their demand more effectively.

A microgrid comprises of multiple segments: the network for the microgrid itself; the microgrid and the installation it supports; the interconnections to the connected utility and the utility control system. Microgrids are neither strictly cyber (e.g., information technology) nor electromechanical (e.g., recloser, switch gear, etc.). Communications take place between cyber and electromechanical components, such as substations, controllers, meters, plug-In motor vehicles, distributed generation and control center servers (Figure 1).

Figure 1. High-level framework illustrating the microgrid partitions

Source: Accenture

customer house

in-home display

DSL/cable modem

DER - wind turbines and diesel generators

BESS and storage +/-

BESS and storage +/-

PHEV/storage +/-

DER - PV farm

DER - PV array

thermostat

industrial complex

distributed energy management system

microgrid fast (or static disconnect switch/point of common coupling

critical assets

DER - PV array

PHEV/storage +/-

DER - PV farm

5

It is clear that electricity utilities are a target for both physical and cyber security attacks. Microgrids are unlikely to be an exception. As the number of system components of the microgrid increases, so too does the reliance on the distributed, active control of the network increasing the potential impact of an attack.

Cyber-attacks may be as a result of human error or deliberate attempts to infiltrate along the high number of potential attack vectors within a microgrid. Should any system component or communication path become compromised, others could also be affected. For example, a compromised vendor with access to system breakers could open and close, connect to, or isolate the microgrid from the distribution substation and larger grid. Yet even several weeks after reports of the Heartbleed2 vulnerability, only a few vendors had announced any remediation plans.

Attacks originating in cyberspace may range from the disruptive and destructive such as distributed denial of service (DDOS), defacing bill-payer websites, damage to automated modules, or ‘bricked’ meters all the way up to the life threatening. For example, cyber terrorism could result in disruption to energy supplies leading to failure of critical services in healthcare and military sites or catastrophic failure of assets such as transformers. Typically, entry points through non-critical loads are easier targets for malicious attacks owing to their reduced security. Smart household devices can provide simpler attack vectors as they tend to have fewer protection controls due to their lower load criticality and cost pressures.

Cyber-physical security challenges for microgrids

6

Developers of microgrids have to understand the specific risks inherent in their particular system. For example, end consumers with smart meters are likely to be particularly concerned with confidentiality breaches of household and financial data, while certain manufacturers will generally place a greater emphasis on industrial espionage. Military site operators are likely to place a greater emphasis on the risks of weaponized malware, such as Stuxnet, that could disable the electricity supply. The cyber security measures deployed should reflect these customer-specific concerns.

Whatever the specific microgrid characteristics, it is essential that operators manage the frequent, high visibility cyber-scams that require consistent and rapid action, as well as maintaining constant vigilance for the sophisticated and more serious attacks that could bring down the network.

“On a scale of 1 to 10, with 10 being strongly defended, our critical infrastructure’s preparedness to withstand a destructive cyber-attack is about a three.”

US General Keith Alexander (Retired) Former Chief of the National Security Agency3

7

Microgrids (and smart grids) present a broad attack surface for cyber-criminals. Assets range from industrial network components such as programmable logic controllers and remote terminal units, all the way to mass market components such as smart meters and in-home displays.

Cyber-enabled components on a smart electricity network could easily reach the thousands, or even millions if end-consumers are part of the system.

To become more resilient, microgrid developers and operators need a comprehensive and holistic approach to cyber-physical security. The principles to protect critical power infrastructure are:

1. Harden the microgrid

2. Complete ongoing assessments of interconnection security controls

3. Plan and prepare for disaster recovery

4. Resource the security strategy

Four principles for microgrid security

8

The foundation of a more secure microgrid is in an appropriate system architecture. Typically, the design would involve identifying the critical systems through an assessment of all physical and logical assets that are connected across the microgrid.

System characterization, threat identification and vulnerability assessments allow the risks to be catalogued and managed as part of the initial microgrid design. For example, segmentation and isolation of systems on the basis of functional groups provides a smaller attack surface and reduce many vulnerabilities. Similarly the microgrid control system overall should be segmented from less trustworthy networks, including utility business networks, customer energy data portals, vendor and third party extranets, and the Internet.

When in operation, a hardened microgrid requires defense in depth with a tiered defensive strategy including protective measures covering the physical, network, application, data and human operator levels. The defense in depth provides tailored security approaches around each system or functional group. These solutions cover a very wide range, such as firewalls, intrusion monitoring, permissions, physical access controls and user training. Unfortunately, it is the nature of the cyber threat that the defense in depth measures will need ongoing review and upgrade as new threats evolve.

As well as the foundational components of system architecture and defense in depth, there are additional ways to harden the microgrid that warrant further discussion.

1. Harden the microgrid

9

Physical protection of microgrid components

Fully securing a distributed network such as a microgrid is not possible, but measures should be considered for selected assets that are highly vulnerable or accessible. Appropriate steps could include:

• Incident-response procedures (e.g., roles and responsibilities, playbooks, evidence preservation)

• Tamper-proof devices (e.g., seals, locks, accelerometers)

• Fences and walls, such as:

- Non-conductive fiberglass-reinforced panels to guard against bullets of central microgrid components

- Concrete walls eight feet or higher at the perimeter of central microgrid components.

• Surveillance (e.g., video analytics to detect anomalies near central microgrid components)

Mitigation of insider threats

Threats might emerge from employees acting maliciously or, just as likely, inadvertently from staff who lack sufficient training. Organizations that rely on microgrids for resiliency need heightened awareness to monitor the activities of enterprise network engineers, third-party vendors, field technicians and engineers. Mobile field force solutions can support in delivering high quality work and help to catalogue the work done on assets.

Inclusion of forensics

Critical segments of the microgrid should be architected to preserve digital evidence. Asset owners are advised to gather cyber intelligence and watch proactively for back doors and vulnerabilities unseen by point compliance and checklist efforts. It is also important to look for complex and chained patterns that might indicate initiation of an attack. Organizations would be wise to expand the scope of their vulnerability assessments or penetration tests. These can harness external sources of threat intelligence to understand and train for zero-day exploits and detect reconnaissance activity by active and terminated employees, by hackers, or attempts by trusted third parties to escalate network privileges.

10

Pursuit of interoperability and standardization

If advanced metering infrastructure and smart grid deployments experiences are representative, microgrids will likely be implemented in advance of the implementation of a standard security approach. Utilities must, therefore, rigorously assess deployments for risk and continuously evolve their capabilities. Security-control interoperability, along with standards and regulatory compliance, are microgrid prerequisites. Generally, however, standards remain ambiguous, and some of the regulations are looser than utility-company guidelines.

Strengthening the governance structure

The industry is moving toward increased organizational awareness and is hiring chief information security officers to oversee comprehensive and well-coordinated security. However, utilities still need to address a number of gaps:

• An incident response command structure should be formed between the utility and the microgrid asset owner

• A divide needs to be bridged between corporate IT, which manages enterprise IT systems (finance, email and human resources), and field operations and engineering, which manage the grid

• Procedures are needed to alert customers when power goes out and when secondary resources are being used

• In addition, plans are needed for dropping non-essential loads, and asking customers to use only essential resources

11

The secret is out; gaining access to a remote asset through an interconnection is not that difficult. The concept of an ‘air gap’ between business and industrial systems has increasingly been shown to be non-existent in most cases.

In the smart grid and microgrid the ‘air gap’ is all but removed due to the high level of integration required between the industrial control networks and the business systems. In the real world, the ‘air gap’ has been replaced by a firewall which allows communications into the critical operations systems.

Taking an example, a consumer’s home system for energy management is accessed through the internet.4 If a consumer’s in-house display is compromised, lights can be turned off and on remotely, thermostat settings can be altered, alarm systems modified, and circuit breakers can be internally tripped from constant cycling. However, the same cyber breach can migrate through to the smart meter head-end system and from there towards the distribution management system and SCADA that may need control of distributed generation or storage at the household.

Thorough assessment of the multitude of interconnection points is the only way to determine accurately which security countermeasures merit attention. As fundamental as it may sound, it is essential that system owners periodically assess the security measures they have in place to protect against microgrid cyber-physical assets. This should apply whether the system is regulated and requires an assessment or not. Scheduled, recurring security evaluations foster ongoing situational awareness of the microgrid’s cyber security posture, that is, the system’s present ability to identify and detect vulnerabilities and threats. Assessments also provide information necessary to protect against, respond to, and recover from attacks in order to reduce or otherwise manage risk to microgrid systems, owners and the utilities that manage the larger grid.

2. Complete ongoing assessments of interconnection security controls

12

Security assessments help the business address key considerations for implementing security architecture:

• What to protect, that is, microgrid infrastructure, information, processes, etc.

• Why the protection is important, in terms of security control and business enablement objectives

• How to achieve the protection, that is, technical and management security strategies (e.g. Public Key infrastructure, Network Segmentation, and/or Role Security strategies)

• Who to involve in security management, in terms of roles and responsibilities and trust relationships between microgrid asset owners, users, service providers, and customers

Employing structured and repeatable processes can facilitate the accurate and up-to-date representation of what is happening in and around microgrid cyber-physical assets. This also allows system owners to select, rationalize, and prioritize microgrid cyber security measures, particularly the allocation of budget for people and technical improvements.

13

United States/North America• NERC CIP 002-011 and CIP-014. Reliability Standards for

the Bulk Electric Systems in North America

• Framework for Improving Critical Infrastructure Cybersecurity

• NISTIR 7628. Guidelines for Smart Grid Cybersecurity

• Security Profile for Advanced Metering Infrastructure (AMI-SEC 2.1)

• System Security Protection Profile for the Gateway of a Smart Metering System

• NIST SP 800-82. Guide to Industrial Control Systems (ICS) Security

• NIST SP 800-53. Recommended Security Controls for Federal Information NISTIR 7176. System Protection Profile - Industrial Control Systems

• Regulatory Guide 5.71. Cyber Security Programs for Nuclear Facilities (US)

• US Department of Energy, Electricity Subsector Cybersecurity Risk Management Process

International• ISO/IEC 27000, Information technology — Security techniques — Information security management systems — Overview and vocabulary

• ISO/IEC 15408, Evaluation criteria for IT security (also known as “Common Criteria”)

• IEEE 1711. Trial-Use Standard for a Cryptographic Protocol for Cyber Security of Substation Serial Links (International)

• IEC 62210. Power system control and associated communications - Data and communication security

• IEC 62351. Data and communications security

• IEEE 1686-2007. Standard for Substation Intelligent Electronic Devices (IEDs) Cyber Security Capabilities

• IEC 62443-1-1 (formerly ISA-99). Security for Industrial Automation and Control Systems: Terminology, Concepts, and Models

• Privacy and Security of the Advanced Metering Infrastructure (Netherlands)

The following is a sampling of the purpose-built frameworks designed to facilitate in-depth evaluation of security controls for the kinds of cyber-physical assets (e.g. electric generation/transmission/distribution and Industrial/Process Control Systems) inherent in microgrid architecture:

Frameworks for evaluating security controls

14

No microgrid can be totally protected from cyber-attack so operators must develop robust mechanisms to identify attacks and recover the network.

In the early 1980s the US Department of Energy defined the phases of incident management: prepare, identify, contain, eradicate, recover and follow-up (PICERF). These phases are widely accepted by cyber incident responders, and by standards-setting, training and certification organizations.5 Microgrid security technologies should be considered in the ‘prepare’ and ‘recover’ phases.

One of the major challenges to managing the cyber-physical security of a smart distribution grid is identifying when an attack is impacting the physical operation of the network. The relatively sparse power data collected, such as voltage, current and frequency, on a full scale distribution grid makes it relatively easy for abnormal operating modes to be unidentified until asset failure. The microgrid has the advantage of a greater proportion of monitoring and control nodes across the network, making analysis such as state estimation a powerful tool to monitor the health of the system. Early warning of a security breach can provide a key opportunity to protect assets from failure and maintain some degree of energy supply.

The power of leveraging microgrids for disaster recovery planning resides in load dropping and islanding. Priorities need to be established prior to outages, as the European Medical Center case study illustrates, to sustain critical operations and recover swiftly.

3. Plan and prepare for disaster recovery

15

In addition, the webserver hosting the medical center’s website was targeted by a DDOS, as was the wireless network. The result was a power outage for the medical center, accredited by the Joint Commission International,6 and shutdown of the public-facing website and internal wireless network for transmitting patient data.

Fortunately, the center had already prioritized which loads in its microgrid were critical and determined that systems designed to support those loads were up to the task. The cyber assets (e.g., downed website and wireless network) were in the physical prioritization plan; however, they were being “DDOS-ed” and did not necessarily need to consume power.

Electrical load criticality was prioritized based on the potential risk to human life. The center rated electrical loads as “critical,” “life safety” and “non-essential.” Critical loads, for example, included task illumination, fixed equipment, and special power circuits serving functions related to patient care. At the other end of the spectrum, non-essential loads included restroom hand dryers, hallway vending machines, power to the floral-shop and illumination of the outpatient waiting room – and the two DDOS-ed systems.

The center at the time was using an emergency power system based on a diesel generator backup. Since this incident, the center has upgraded to biodiesel generators compliant with European standards.7

After the failure occurred, the microgrid controller transmitted a command to isolate the microgrid from the larger grid until upstream power could be restored and stabilized. Load shedding was automatic and, again, was performed according to pre-defined levels, with “non-essential” being the first to go. After upstream power was reestablished, the controller sent a notification signal to other microgrid components.

Because the most essential loads had been prioritized beforehand, the net result was maximum preservation of human life and safety. The lesson from this is that microgrid operators should study and detect subtle patterns of customer demand so loads can be sustained or—in the event of an outage—restored with minimal interruption.

Case study: Recovery at a European medical center

At a major academic medical center in Europe, an electrical substation had become vulnerable after a malware-infected laptop connected wirelessly to the substation capacitor bank and caused it to disconnect.

16

In much the same way that an operations security support team maintains the security posture of macrogrids, a microgrid also requires security operations and maintenance support.

Financial plans and budgets should include funds for sustaining the overall quality of the security services expected to meet the business requirements of the microgrid.

The number of staff will depend on microgrid size and integration complexity, but several key areas should be addressed:

• Incident response operations. 24x7 or 9 to 5 alerting and escalation system

• Security operations. Vulnerability scans, monitoring, patching, tuning, report generation, and IT and OT Coordination

• Security engineering. Implementation of mitigation control

• Information risk management and audit support. Documentation of security protocols and procedures, and assistance with internal and third-party audits

• Frequency of Reporting. Will management and industry reports be delivered on a weekly, monthly or quarterly basis?

• Compliance. Alignment with changing security regulations and requirements

• Awareness and training. Instruct personnel how to recognize and respond to security attacks

Typically, the staff to be hired and trained should have experience with control systems engineering and security, smart grid engineering and security, integration projects and proprietary energy system protocols and systems.

4. Resource the security strategy

17

Increasing adoption of microgrids stems, in part, from the Energy Policy Act of 2005 and Energy Independence and Security Act of 2007, which require facilities to be metered, and energy and water consumption to be managed.

The microgrid approval process is complex. If connected to the DoD local area network, microgrids must gain support from the Directorate of Information Management (DOIM), which tends to be staffed by IT experts rather than by electrical power engineers or smart-grid specialists.

Guidelines for defense and commercial installations differ in the area of non-IP devices that communicate using routable protocols. These devices are within the scope of the defense department’s risk management framework, and must comply with the DoD Information Assurance Certification and Accreditation Process (DIACAP) and NIST 800-53. Non-IP devices, however, remain outside the scope of critical infrastructure protection overseen by the North American Electric Reliability Corporation.

DIACAP must be adhered to for the accreditation boundary and components within it, including cyber communication protocols, software applications and system interconnections. Accreditation might cover only the front-end processor or the entire microgrid. In addition, a ‘certificate of networthiness’ must be in place for all software within the local area network.

Bases are likely to experience challenges acquiring the needed expertise in cyber security, and understanding the DIACAP risks relating to automation systems, smart grid, control systems and more. Given the complexity, receiving accreditation typically requires the help of engineers and smart-grid experts with years of experience in designing, building and installing relevant systems.

Interest in microgrids growing at US defense installations

A number of USA Department of Defense (DoD) installations are looking to deploy microgrids to reduce the chance of extended outages.

18

The microgrid represents the next performance level in terms of efficient and resilient delivery of electricity to customers.

However, leveraging microgrids is not only an engineering feat, but a physical and cyber-security challenge as well.

To tie all of the pieces together, a strong and coherent cyber security framework must be high on the organizational agenda. Only with a strong security framework leading to effective deployment will organizations be able to unlock the full protective value of microgrids.

Harnessing microgrid security for high performance

Follow Us@Accenture_Util

Accenture Utilities

VisitFor more information on the Accenture utilities, visit http://www.accenture.com/utilities

Please contact: DEG@accenture.com

About AccentureAccenture is a global management consulting, technology services and outsourcing company, with more than 358,000 people serving clients in more than 120 countries. Combining unparalleled experience, comprehensive capabilities across all industries and business functions, and extensive research on the world’s most successful companies, Accenture collaborates with clients to help them become high-performance businesses and governments. The company generated net revenues of US$31.0 billion for the fiscal year ended Aug. 31, 2015. Its home page is www.accenture.com.

About Accenture Smart Grid ServicesAccenture Smart Grid Services focuses on delivering innovative business solutions supporting the modernization of electric, gas and water network infrastructures to improve capital efficiency and effectiveness, increase crew safety and productivity, optimize the operations of the grid and achieve the full value from advanced metering infrastructure (AMI) data and capabilities. It includes four offering areas: Digital Asset Management, Digital Field Worker, Intelligent Grid Operations and Advanced Metering Operations.

References1 Executive Orders 13636 and Presidential Policy Directive 21, issued by US President Barack Obama

2 https://www.us-cert.gov/ncas/alerts/ TA14-098A

3 Reuters: http://www.reuters.com/ article/2013/06/12/us-usa- cybersecurity-idUSBRE95B10220130612 “NSA chief says U.S. infrastructure highly vulnerable to cyber-attack”

4 http://www.forbes.com/sites/kashmirhill/ 2013/07/26/smart-homes-hack

5 NIST, SANS.

6 JCI maintains standards designed for the assessment and accreditation of academic medical center hospitals in more than 90 countries. With regard to utility systems, Standards FMS.9 and FMS.9.1, respectively, indicate that “potable water and electrical power are available 24 hours a day, seven days a week, through regular or alternate sources, to meet essential patient care needs” and “the organization has emergency processes to protect facility occupants in the event of water or electrical system disruption, contamination, or failure.”

7 EN 14214, a European standard, describes requirements (e.g. allowable alcohol content) and test methods for biodiesel fuels.

16-0049

Copyright © 2016 Accenture All rights reserved.

Accenture, its logo, and High Performance Delivered are trademarks of Accenture.