Joe Weiss - Amsterdam Presentation 12-6-10

Applied Control Solutions Proprietary Information

Cyber Security of Industrial Control Systems

Smart Grid Security and

Privacy SeminarDecember 6, 2010

Joe Weiss, PE, CISM(408) 253-7934

[email protected]


Background • Industrial control systems (ICSs)

operate power, water, chemicals, pipelines, etc

• ICSs include SCADA/EMS, DCS, PLCs, RTUs, IEDs, smart sensors and drives, emissions controls, equipment diagnostics, AMI (Smart Grid), programmable thermostats, building controls,…


Brief History of ICS• 20 years ago – Isolated systems, with

non-networked cyber “dumb” devices • 10 years ago – Emergence of network

integration, with more capable “intelligent” cyber-vulnerable devices

• Today – Combination of modern, integrated networks interoperating with legacy systems creating increasingly cyber-vulnerable networks

• 10 years from now – Who knows? Expect further convergence of networked legacy, intelligent, and newer technologies, with even more cyber vulnerability


Control Systems Basics

Slide courtesy of Anixter © Proprietary 04-2009


Evolution 1- Panel based Controls

• Push Buttons• Single Loop

Controls• Stand Alone• No Networks• No Communication

From a cyber security standpoint this system is

isolated and “cyber-dumb” Slide courtesy of Anixter © Proprietary 04-2009


Evolution 2 - Legacy Electronic Controls • Proprietary Networks• Proprietary OS• No Ethernet• No Internet connections• No Intranet connections• “Security by Obscurity”

From a cyber security standpoint this system is cyber vulnerable



Evolution 3- Modern Technology• Ethernet everywhere• Wireless ‘in the rack’• Remote configuration• Windows & Linux OS• Commercial Off The

Shelf (COTS)

From a cyber security standpoint this system is

very cyber vulnerable



Common ICS Cyber Issues

• Minimal ICS cyber forensics• People and technology issues• Older vulnerabilities still effective against many ICSs• Recurring incidents with minimal guidance how to avoid

problems• Conflicting guidance on how to address problems• Lack of focus on the control system-unique issues


ICS Security Expertise Lacking

IT

IT Security

ICS Security Experts

ICSEngineering


Myths• The Internet and Microsoft are biggest ICS cyber threats• Using Windows and TCP/IP “make it IT”• External malicious threats are the biggest concerns• Firewalls make you secure• VPN / encryption use makes you secure• IDS will identify ICS attacks• Field devices can’t be hacked• Can’t use dial-ups or default passwords • You are secure if hackers can’t get in• More and better “widgets” can solve all our security problems• “If we keep our head down they won’t find us”


ICS Cyber Issues• ICS designs did/do not include

security – it’s a back-fit– Many new systems cyber vulnerable

• System integration with insecure systems

• Lack of ICS cyber forensics• Culture

– Operations considers security a pain


ICS Vendor Cyber Issues• Modern wind farms have interactive control

capabilities– Built-in WiFi, GPRS with sim-cards, RS232

comport for external RTU– Local Mini-SCADA with direct access to

regional control• Some smart grid vendors using bluetooth

and “embedded” modems • Other ICS vendors using wireless modems

* Wind Power Communications Security Concerns and Protection Gary Seifert, Idaho National Laboratory


ProSoft i-View

- Mobile iPhone/iPod touch/iPad application allows for remote monitoring and control of process values within an EtherNet/IP and/or Modbus TCP/IP network, utilizing a wireless 802.11 (WiFi) and/or cellular network connection - ProSoft i-View provides an interface for accessing and monitoring variables (tags) and memory of PLCs. Plant engineers, PAC/PLC software developers, and Maintenance personnel now have the ability for live monitoring and control of PAC/PLC based systems at any time, from everywhere. - ProSoft i-View wirelessly connects directly to the PLC without routing through servers or personal computers, using direct TCP/IP links between iPhones/iPods and PLCs, with minimal configuration. - Security is guaranteed through extensive use of passwords and the encryption and tunneling options that the TCP/IP and 802.11 technologies provide.


Big Push for Smart Grid

AMI Meter

Utility Back Office

Utility Substation

Customer Premise

Remote Access

SCADA


What is Smart Grid• Multiple answers

– AMI– Home automation– Substation automation– Plant automation– (e) some or all of the above

• What is common– 2-way communication – CYBER!


NIST Smart Grid Framework - Interconnectivity


IEC TC-57 View of Smart Grid - Communications


Unique Smart Grid Cyber Threats• Privacy• Vastly expanded threat space• Blurring of IT and ICS• Public awareness of vulnerabilities


What has happened since last year

• Deepwater Horizon off-shore oil platform disaster (11 killed)• Lake Havasu City water disruption• San Bruno natural gas pipeline rupture (8 killed)• Stuxnet• VxWorks vulnerability• BACnet OPC client vulnerability• Other non-public ICS cyber incidents


Stuxnet Implications• First targeted cyber attack against ICS

– Can be used to attack many Windows-based ICSs (not just Siemens)– Engineering attack on a process – Cannot be PATCHED or addressed by AV!!!

• ICS community may not be able to identify sophisticated attack– Sophisticated worm can do multiple functions

• Defeated 2-factor authentication– Root kit – not able to be seen – no “solutions” yet– Aspect able to be seen has “solutions”

• Demonstrated weaknesses– Disclosure process– Key management– Forensics– Gaps between IT and ICS

• Need gap analyses on ICS Cyber Security standards and guidelines


Common ICS Issues with Stuxnet

• Difficult to detect• Lack of knowing what was actually on ICS networks• Lack of detailed knowledge of ICS logic• Lack of ICS forensics requiring manual investigation and

engineering analysis• Use of thumb drives


ICS Cyber Incidents

• 180+ incidents world-wide– Most unintentional– Some malicious attacks– Impacts range from trivial to

major outages to deaths– Most not identified as cyber

• ICS incidents may not violate IT security policies


Targeted SCADA Attack

• Insecure system integration enabled targeted attack

• No SCADA servers or mapping system for two weeks

• 4 Man-months to recover• Minimal forensics• No information sharing with

local law enforcement, FBI, or ES-ISAC


Pipeline Rupture• June 1999 Bellingham, WA

- Killed three, injured eight- Significant property and environmental damage- Bankruptcy of Olympic Pipeline Co

• Minimal cyber forensics- Data erased- People went to jail


EMI in Industrial Control Systems

November 1999, the U.S. Navy was conducting exercises off San Diego during which, two commercial spectrum users experienced severe electromagnetic interference (EMI) to their Supervisory Control and Data Acquisition (SCADA) wireless networks operating at approximately 928.5 MHZ.

The San Diego County Water Authority (SDCWA) and the San Diego Gas and Electric (SDGE) Companies were unable to remotely actuate critical valve openings and closings as a result. This necessitated sending technicians to remote locations to manually open and close water and gas valves.

The cause of the EM interference was determined to be a Navy AN/SPS-49 radar operating off the coast of San Diego.


SCADA EMI Resulting In ANatural Gas Pipeline Failure

• Natural gas pipeline SCADA system located 1 mile from the Naval port of Den Helder, Netherlands

• EMI was traced to an L-band Naval radar coupling into SCADA• SCADA disturbance caused a catastrophic failure of roughly 36-inch

diameter pipeline, causing a large gas explosion– RF energy caused the SCADA system to open and close a relay at the radar scan

frequency (6-12 rpm), which was in turn, controlling the position of a large gas flow-control valve

– Resulting changes in valve position created shock waves that traveled down the pipeline causing pipeline failure


Reactor Coolant Pump

Nuclear Plant Cyber Incidents

- Inadequate policies- Lack of forensics- Failsafes worked!

- Same problems have affected many non-nuclear plant facilities


Browns Ferry and Hatch

• Browns Ferry Broadcast Storm– Too much communication traffic shut down variable frequency

drives shutting down main coolant pumps• Hatch Software Change

– Unknown connections led to software change creating conditions to close all condensate valves

No ForensicsNeither incident violated IT security policies!


DC Metro Crash

•June 22, 2009 DC Metro trains collided•9 dead, 52 injured•System consisted of sensors, RTUs, and SCADA•Previous unresolved problems•Lack of sensor data and alarms•November 29,2009 DC Metro train crash


Unintended Consequences

• A disturbance caused by the implementation of a device locking security tool resulted in the loss of SCADA services. The tool was being implemented in response to the NERC CIP standards.

From January-June 2009 NERC Disturbance Reports


Other Concerns• Lack of personnel certifications

– Neither PE nor CISSP adequate• Lack of university interdisciplinary courses

– Need in both computer science and engineering

• Lack of understanding/denial – Based on presentations, articles, and NERC CIP process


Recommendations• Get senior management buy-in• Understand what you have installed• Develop appropriate policies and procedures

– Use the NIST Risk Management Framework• Implement appropriate technologies that won’t affect system

performance or compromise safety• Make it a living program


Conclusions• Can not fully secure ICSs

– Worry about intentional and unintentional– Need to be able to recover

• Threats are real– Lack of forensics complicates recovery and prosecution

• Need appropriate knowledge and coordination– This isn’t IT but we need IT

Documents

Joe Weiss - Amsterdam Presentation 12-6-10