Introduction to Cybersecurity: Workshop and Response Exercises · replicate without user action Virus • Malicious program attached to a host file that runs/spreads when the host

Introduction to Cybersecurity: Workshop and Response Exercises

United StatesEnvironmental ProtectionAgency

DECEMBER 2019

Common Acronyms and Terminology

AV Antivirus AWWA American Water Works Association BIOS Basic Input/Output System C3 Voluntary Program DHS’s Critical Infrastructure Cyber Community CERT Computer Emergency Response Team CSET Cyber Security Evaluation Tool CSF Cybersecurity Framework DCS Distributed Control System DHS Department of Homeland Security DMZ Demilitarized Zone EO Executive Order FBI Federal Bureau of Investigation FOIA Freedom of Information Act HMI Human Machine Interface HW Hardware ICS Industrial Control System ICS-CERT Industrial Control Systems Cyber Emergency Response Team IC3 FBI’s Internet Crime Complaint Center IDS Intrusion Detection System IEC International Electrotechnical Commission IRP Incident Response Plan ISO International Standards Organization IT Information Technology I/O Input/Output LAN Local Area Network NCCIC DHS’s National Cybersecurity and Communications Integration Center NIST National Institute of Standards and Technology OPC Open Platform Communications OS Operating System OT Operational Technology PCII Protected Critical Infrastructure Information PLC Programmable Logic Controller PPD Presidential Policy Directive RAT Remote Access Trojan RTU Remote Terminal/Telemetry Unit SCADA Supervisory Control and Data Acquisition Subnet Subnetwork SW Software USB Universal Serial Bus VLAN Virtual Local Area Network VDH Virginia Department of Health VPN Virtual Private Network VRF Virtual Routing and Forwarding WSUS Windows Server Update Services

CybersecurityWater Sector Threat Overview

Speaker Bio

Over 9 years of professional experience performing cybersecurity risk assessments utilizing ISO 27001, IEC 62443, and NIST standards both domestically and internationally, as well as conducting secure architecture reviews and OT cybersecurity trainings.

Consulted for clients across many sectors/markets, including water, oil & gas, building, civil, pharmaceutical, and defense.

Education

M.S., Cybersecurity, University of Maryland University College B.S., Information Sciences and Technology, The Pennsylvania State University

Certifications

Global Industrial Cyber Security Professional (GICSP) Certified Information Systems Security Professional (CISSP) ISO 27001 Lead Auditor (BSI Group) Security+ CE Splunk Certified Architect

Experience Summary

Caitlin FerroLead Associate

Industrial Cybersecurity Specialist

2

1

“It has long been recognized that among public utilities, water supply facilities offer a particularly vulnerable point of attack to the foreign agent, due to its strategic position in keeping the wheels of industry turning and in preserving the health and morale of the American populace.”

3

Traditional Security Focus

MAN-MADE INCIDENTS

• Power outages• Spills• Construction activities and accidents

breaking water mains

NATURAL INCIDENTS

• Hurricanes• Ice storms• Droughts

• Responding to and recovering from…

4

2

“Also, to make SCADA systems cost-effective in the future, we no longer build special purpose operating systems for them. We put on standard vendor operating systems, with additional vulnerabilities that are well known. So now we have systems that are well understood, connected to the Internet, but still providing a rather critical function in the element itself.”

How SCADA Has Evolved

- Tom Longstaff

Computer Emergency Response Team (CERT)

Research Center at the Software Engineering Institute

5

DataHistorian

Common Terms• Industrial Control System (ICS)• Supervisory Control And Data Acquisition (SCADA)• Distributed Control System (DCS)• Programmable Logic Controller (PLC)• Human Machine Interface (HMI)• Remote Terminal/Telemetry Unit (RTU)• Input/Output (I/O)• Data Historian• Open Platform Communications (OPC) Server

PLC

HMI

https://nvlpubs.nist.gov/nistpubs/SpecialPublications/NIST.SP.800‐82r2.pdf

6

3

Water and Wastewater

Today’s SCADA Reliance

Electric Power Generation,

Transmission, and Distribution

Transportation

Oil & Gas Production and Distribution

Industrial Processes

Industrial Control Systems• term that encompasses

multiple types of control systems that support industrial production processes

• although different, often usedinterchangeably with the termSupervisory Control and DataAcquisition (SCADA) systems

7

Increased Number of Pathways

• Utility systems have become moreautomated (e.g., SCADA, on-line billpaying) to improve operational efficiency

• With the convenience of monitoringsystem status remotely, morewaterworks are putting their ICS systemsonline

• To achieve cost savings, vendors areincreasing their use of remote accesscapabilities for troubleshooting

• Business users are more frequentlyrequesting visibility into SCADAnetworks for monitoring utility operations

INTERNET

IT BUSINESSNETWORK

ICS/SCADANETWORK

8

4

Remote Access TechnologiesMobile Applications

Remote Access Software9

Remote Access Anywhere!

10

5

11

SCADA is an Attractive Target

WH

Y I

CS

? • Physical consequences• Economic or political goals• Hacker notoriety• Financial gains• Espionage

WH

OIS

A

TTA

CK

ING

? • Disgruntled employees/integrators• Ex-employees or integrators• Disgruntled customers• Thrill seekers• Nation-states• Terrorists

12

6

• Water utility “honeypot” SCADA systems deployed in 2013 toassess:

− Who/what is attacking devices and why

− If the attack performed on these systems was targeted and for what purpose it was targeted

SCADA Systems Online

• Within 28 days, 39 attacks from14 different countries occurred

– 12 attacks were unique andclassified as “targeted”

– 13 attacks were repeated andconsidered “automated”

Country breakdown indicating the number of attack attemptsSource: Trend Micro, Who’s Really Attacking Your ICS Equipment?13

Shodan

14

7

Common Attack MethodsW

orm • Malicious program that can self-

replicate without user action

Vir

us • Malicious program attached to a

host file that runs/spreads whenthe host is executed

Sp

ywar

e

• Malware that monitors useractivity and can capturekeystrokes, screenshots,authentication credentials,personal email addresses, webform data, internet usage habits,and other personal habits

Sn

iffe

r• Software that monitors data

traveling over a network

Ran

som

war

e

• Malicious software that encryptscomputer files to prevent accessuntil a ransom is paid

Ph

ish

ing

• Fake websites or e-mailmessages that look genuine

• Often ask users for confidential,personal data

• Sometimes contain links orattachments that trigger anotherattack method, such as virusesor ransomware

15

History of Water Sector Cyber EventsDate Location Threat

2000 Australian WWTP-Maroochy Shire

Former disgruntled employee hacks into the system releasing 264,000 gallons of raw sewage.

2006 Pennsylvania An employee’s laptop was breached and used to install both a virus and spyware in a computer system at a water treatment plant.

2011 Springfield, IL Water Utility

Off-site vendor log-in created confusion of a possible cyber attack originating from Russia.

2013 Rye, NY Bowman Avenue Dam

Iranian hacker group infiltrated control systems for the Bowman Avenue Dam through cellular modem

2016 Not Disclosed Hackers breached a water utility and manipulated systems responsible for water treatment and flow control.

2016 Michigan Ransomware seized corporate computers at a Michigan water and electric utility. It encrypted files and forced the utility to shut down critical resources.

2016 Florida A hacker managed to steal customer information (i.e., credit card data) from 70 customers of a small water utility (~17,000 customers) in Florida.

2018 North Carolina Hackers encrypted several systems within a combined water and wastewater utility. The Authority had to rebuild their databases and computer systems.

2019 Florida A municipal employee opening a corrupted email (which infected the utility’s SCADA system) forced a Florida city to pay nearly $600,000 to the hackers.

2019 Texas In a multi-city ransomware attack, a Water Department’s treatment plant’s computer system was compromised, forcing the utility into manual mode. 16

8

Ransomware in the Contiguous U.S.

Link to Map

17

Consequences of Attacks

BU

SIN

ES

S IM

PA

CT

S • Loss of data and data integrity• Loss of plant security• Operational denial or disruption of business services or production• Regulatory intervention and legal liability• Monetary losses• Costs to the utility and region• Brand/reputation damage• Physical damage, injury, or loss of life• Erosion of public confidence

18

9

Security Advisories and Alerts

Note: ICS‐CERT is part of the National Cybersecurity and Communications Integration Center (NCCIC) 19

• ICS-CERT Monitor (November/December 2017)– announced the NCCIC went through an organizational

realignment to consolidate and enhance the effectivenessof its mission-essential functions, which includes changesto the structures of the ICS-CERT, NCC, and US-CERTdivisions

NCCIC Realignment

• National Cybersecurity and Communications Integration Center (NCCIC)– 24x7 cyber situational awareness, incident response, and cyber risk management center– shares information among public and private sector partners to build awareness of cyber

and communications vulnerabilities, threats, incidents, impacts, and mitigations

• NCCIC partners with Granicus– Granicus provides a digital subscription system to help users stay informed– users can receive the NCCIC Year in Review, NCCIC Monitor, ICS-CERT Alerts, ICS-

CERT Advisories and ICS-CERT Announcements product release notices directly to theiremail Inbox for free

Source: November/December 2017 ICS‐CERT Monitorhttps://ics‐cert.us‐cert.gov/sites/default/files/Monitors/ICS‐CERT_Monitor_Nov‐Dec2017_S508C.pdf 20

10

Top Six Weaknesses From 2014-2017

Source: November/December 2017 ICS‐CERT Monitorhttps://ics‐cert.us‐cert.gov/sites/default/files/Monitors/ICS‐CERT_Monitor_Nov‐Dec2017_S508C.pdf

21

Top Six Weaknesses of FY2017

Source: November/December 2017 ICS‐CERT Monitorhttps://ics‐cert.us‐cert.gov/sites/default/files/Monitors/ICS‐CERT_Monitor_Nov‐Dec2017_S508C.pdf

22

11

2019 Verizon Data Breach Report

Source: Verizon 2019 Data Breach Investigations Reporthttps://enterprise.verizon.com/resources/reports/dbir/

23

2019 Verizon Data Breach Report

Source: Verizon 2019 Data Breach Investigations Reporthttps://enterprise.verizon.com/resources/reports/dbir/

24

12

• Prioritizing cybersecurity to show that it’s asimportant as competing needs

• Cyber risk is not always included in overall riskmanagement program

• Water sector technology is constantly advancing• Attacks and risks are changing• Control systems govern sensitive operations• No two utilities are alike

Challenges to Increasing Cybersecurity across the Sector

25

13

Cybersecurity Drivers & Resources

Session Overview

• Cybersecurity Drivers– America’s Water Infrastructure Act

• Cybersecurity Resources– WaterISAC 15 Cybersecurity Fundamentals for Water and

Wastewater Utilities– EPA Cyber Incident Action Checklist– AWWA Tool

2

14

America’s Water Infrastructure Act 2018

Overview

New requirements for drinking water utilities:

• Risk and Resilience Assessment• Emergency Response Plan

EPA Factsheet, available at: https://www.epa.gov/waterresilience/overview‐new‐risk‐assessment‐and‐

emergency‐response‐plan‐requirements‐community

4

15

Cyber Components for Risk and Resilience Assessment

As specified by AWIA § 2013:• Electronic, computer, or other automated systems (including thesecurity of such systems) which are utilized by the system

• The monitoring practices of the system (including networkmonitoring)

• The financial infrastructure of the system (meaning accounting andfinancial enterprise IT systems operated by a utility, such ascustomer billing and payment systems).

5

Cyber Components for Emergency Response Plan

As specified by AWIA § 2013:1. Strategies and resources to improve the resilience of the system,

including the physical security and cybersecurity of the system

2. Plans, procedures, and equipment for responding to a malevolent act ornatural hazard

3. Actions, procedures, and equipment to lessen the impact of amalevolent act or natural hazard, including alternative source water,relocation of intakes, and flood protection barriers

4. Strategies that can be used to aid in the detection of malevolent acts ornatural hazards that threaten the security or resilience of the system.

6

16

For more information on AWIA

•Certification deadlines

•Certification process

•Certification review and revision

•Tools and Resources–VSAT–ERP Guidance

https://www.epa.gov/waterresilience/americas-water-infrastructure-act-risk-assessments-and-emergency-response-plans

7

Cybersecurity Resources

17

15 Cybersecurity Fundamentals for Water and Wastewater Utilities

• Overview of important securitymeasures

• Links to additional informationabout each measure

• Free resource

waterisac.org/fundamentals

9

15 Cybersecurity Fundamentals

1. Asset inventories

2. Risk assessments

3. Control system exposure

4. User access controls

5. Physical access

6. Independent safetysystems

7. Vulnerabilitymanagement

8. Cybersecurity culture

9. Policies and procedures

10. Threat detection

11. Incident and disasterplanning

12. Insider threats

13. Supply chain security

14. IoT, IIoT

15. Information sharingcommunities

10

18

Reporting to Members• Security & Resilience Updates (SRUs)

– Twice-per-week (Tuesdays and Thursdays)

– Includes sections on cybersecurity incidents, threats, and tools and upcoming events

• Cyber Threat Web Briefings

– Monthly

– Presenters from DHS NCCIC, cybersecurity firms,and WaterISAC

• Threat Notifications and Advisories

– As necessary

– Provide members with actionable information on urgent cybersecurity threats and incidents

WaterISAC (cont.)

11

• Customizable checklist to help utilities prepare for,respond to and recover from a cyber incident.

EPA’s Incident Action Checklist

https://www.epa.gov/sites/production/files/2017‐11/documents/171013‐incidentactionchecklist‐cybersecurity_form_508c.pdf 12

19

AWWA Guidance and Tool

• Water sector guidance thatprovides a consistent andrepeatable result

• Developed by a panel of utilityrepresentatives, vendors,consultants and federal agencies

• Facilitates compliance with AWIA

• Consistent with NISTCybersecurity Framework

• Aligns with other AWWA practicesand standards

13

Accessing the Tool

• Search for:AWWACybersecurity

• Log‐in or register for anAWWA account

14

20

Reporting Incidents

Unified Message for Reporting to the Federal Government

Report Cyber Events

Threat Response Asset Response

FBI Field Office Cyber Task Forceshttp://www.fbi.gov/contact‐us/field

National Cybersecurity and Communications Integration Center (NCCIC)888‐282‐0870 or [email protected]

United States Computer Emergency Readiness Teamhttps://www.us‐cert.gov/report

FBI Internet Crime Complaint Center (IC3)http://www.ic3.gov

16

21

Water Information and Sharing Center (WaterISAC)

– By telephone: 866-H2O-ISAC

– By email: [email protected]

http://www.waterisac.org/report-incident

Report Cyber Events

17

• DHS Cyber Security Analyst (Regional)

• DHS Protective Security Advisor

• Local Law Enforcement

Report Cyber Events

18

22

• PCII cannot:– Be disclosed through a Freedom of Information Act

(FOIA) request or through a similar request– Be disclosed in civil litigation– Be used for regulatory purposes

• PCII is specially marked and must be safeguarded

Protected Critical Infrastructure Information (PCII)The PCII program protects infrastructure information voluntarily shared with DHS to be used for homeland security purposes

http://www.dhs.gov/publication/pcii-fact-sheet

19

Additional Resources

23

• NIST Framework

• AWWA Cybersecurity Risk and Responsibility Guide

• EPA’s VSAT

• EPA’s ERP Guidance and Template

• EPA’s Baseline Information on Malevolent Threats

• NIST Standards

• ISO Standards

Additional Resources

21

24

Cybersecurity Services for the Water Sector

Klint WalkerCyber Security AdvisorRegion IVCybersecurity and Infrastructure Security Agency

December 4, 2019

2

25

3

Core CompetenciesCapacity Building

Incident

Management

Emergency Communications

Risk Assessment

Network Defense

Information Sharing

Partnership

Development

4

Focused on Critical InfrastructureCritical infrastructure refers to the assets, systems, and networks, whether cyber or physical, so vital to the Nation that their incapacitation ordestruction would havea debilitating effect onnational security, theeconomy, public health or safety, and our way of life.

26

5

National Critical Functions (NCF) –A Necessary Risk Management Evolution

“National Critical Functions” are the functions of government and the

private sector so vital to the United States that their disruption,

corruption, or dysfunction would have a debilitating effect on security,

national economic security, national public health or safety, or any

combination thereof.

COMPLETED

National Critical Functions Set

6

SUPPLY DISTRIBUTE MANAGE CONNECT

Exploration and Extraction Of Fuels

Fuel Refining and Processing Fuels

Generate Electricity

Manufacture Equipment

Produce and Provide Agricultural Products and Services

Produce and Provide Human and Animal Food Products and Services

Produce Chemicals

Provide Metals and Materials

Provide Housing

Provide Information Technology Products and Services

Provide Materiel and Operational Support to Defense

Research and Development

Supply Water

Distribute Electricity

Maintain Supply Chains

Transmit Electricity

Transport Cargo and Passengers by Air

Transport Cargo and Passengers by Rail

Transport Cargo and Passengers by Road

Transport Cargo and Passengers by Vessel

Transport Materials by Pipeline

Transport Passengers by Mass Transit

Conduct Elections

Develop and Maintain Public Works and Services

Educate and Train

Enforce Law

Maintain Access to Medical Records

Manage Hazardous Materials

Manage Wastewater

Operate Government

Perform Cyber Incident Management Capabilities

Prepare For and Manage Emergencies

Preserve Constitutional Rights

Protect Sensitive Information

Provide and Maintain Infrastructure

Provide Capital Markets and Investment Activities

Provide Consumer and Commercial Banking Services

Provide Funding and Liquidity Services

Provide Identity Management and Associated Trust Support Services

Provide Insurance Services

Provide Medical Care

Provide Payment, Clearing, and Settlement Services

Provide Public Safety

Provide Wholesale Funding

Store Fuel and Maintain Reserves

Support Community Health

Operate Core Network

Provide Cable Access Network Services

Provide Internet Based Content, Information, and Communication Services

Provide Internet Routing, Access and Connection Services

Provide Positioning, Navigation, and Timing Services

Provide Radio Broadcast Access Network Services

Provide Satellite Access Network Services

Provide Wireless Access Network Services

Provide Wireline Access Network Services

27

7

CISACybersecurity Advisor Program

8

CISA mission: Lead the Nation’s efforts to understand and manage risk to our critical infrastructure.

In support of that mission: Cybersecurity Advisors (CSAs):

• Assess: Evaluate critical infrastructure cyber risk.

• Promote: Encourage best practices and risk mitigation strategies.

• Build: Initiate, develop capacity, and support cyber communities-of-interest and working groups.

• Educate: Inform and raise awareness.

• Listen: Collect stakeholder requirements.

• Coordinate: Bring together incident support and lessons learned.

Cybersecurity Advisor Program

28

CSA Deployed Personnel

9

CSA Offices

10

Cybersecurity and Resilience

10

29

11

Threat Actors Are Sophisticated…

12

But They Don’t Always Need To Be

30

13

But They Don’t Always Need To Be

…

14

Against an Expanding Attack Surface

31

15

So What?

• Attackers are simply getting better than most defenders.

• But you still have a responsibility to appropriately manage risk and serve your customers.

• Your best effort is within your control.• Don’t let “paralysis by analysis” take hold.

• Measure your cybersecurity posture against established standards.

• Manage improvements and work on “operational resilience.”

16

• How do you know if your cybersecurity efforts are going well?

• Do you plan your cybersecurity activities?

• Do you adhere to a cybersecurity standard of practice?

• What’s at risk? Have you identified the potential consequences if your systems are compromised or unavailable?

• Have you planned for cyber incident management and exercised that plan?

• Can you sustain operations of critical processes following a significant cyber incident?

How Resilient Are You?

32

17

• Resilience is about:

• Preventing disruptions from occurring, and

• Responding quickly and recover from disruptions in its most critical business processes.

Operational Resilience

18

• Identifying critical services and mitigating risks,

• Planning for and managing vulnerabilities and incidents,

• Performing service-continuity processes and planning,

• Managing IT operations,

• Managing, training, & deploying people,

• Protecting and securing important assets, and

• Working with external partners.

Operational Resilience in Practice

Operational resilience emerges from what we do, such as:

33

19

Working toward Cyber Resilience

Follow a framework or general approach to cyber resilience. One successful approach includes:

IdentifyServices

Create Asset Inventory

Protect & SustainAssets

Manage Disruptions

Exerciseand Improve

Identify and prioritize services

Identify assets and align assets to services and inventory assets

Establish risk management, resilience requirements, control objectives, and controls

Establish continuity requirements for assets and develop service continuity plans

Define objectives for cyber exercises, perform exercises, and evaluate results

Process Management and Improvement

20

CISA Cybersecurity Services

20

34

21

Sampling of Cybersecurity Offerings

Preparedness Assistance:

• Information / Threat Indicator Sharing• Cybersecurity Workforce Development, Training and Awareness• Cyber Exercises and “Playbooks”• National Cyber Awareness System• Vulnerability Notes Database• Information Products and

Recommended Practices

22


Preparedness Assistance:

• Cybersecurity Advisors• Advisory Services• Assessments• Working group collaboration• Best Practices • Incident assistance coordination

• Protective Security Advisors • Assessments• Incident liaisons between government and

private sector• Support for National Special Security

Events

35

23


Preparedness Assistance: Assessments

• Cyber Resilience Reviews (CRR)

• Cyber Infrastructure Survey

• External Dependency Management Review

• Phishing Campaign Assessment

• Vulnerability Scanning

• Risk and Vulnerability Assessments (aka “Penetration Tests”)

• Cyber Security Evaluation Tool (CSET)

• Validated Architecture Design Review (VADR)

24

• Cyber Resilience Review (Strategic)

• External Dependencies Management (Strategic)

• Cyber Infrastructure Survey (Strategic)

• Cybersecurity Evaluations Tool (Strategic/Technical)

• Phishing Campaign Assessment (Technical)

• Vulnerability Scanning / Hygiene (Technical)

• Validated Architecture Design Review (Technical)

• Risk and Vulnerability Assessment (Technical)

Range of Cybersecurity Assessments

TECHNICAL(Network-Administrator Level)

STRATEGIC(C-Suite Level)

36

25

• Periodic assessments are essential for resilience, helping you:

• Measure your cybersecurity efforts

• Manage improvements over time

Criticality of Periodic Assessments

26


Incident Response Assistance*

• Remote / On-Site Assistance• Malware Analysis• Hunt and Incident Response

Teams• Incident Coordination

*Subject to ongoing national priorities

37

27

Incident Management

27

28

1. Get leadership support for incident management planning.

2. Establish an event-detection process.

3. Establish a triage-and-analysis process.

4. Establish an incident-declaration process.

5. Establish an incident-response and recovery process.

6. Establish an incident-communications process.

7. Assign roles and responsibilities for incident management.

8. Establish a post-incident analysis and improvement process.

Resource: CRR Supplemental Resource Guide, Incident Management.

Incident Management Planning Helps Mitigate Effects

38

29

Federal Incident Response, 1 of 2

Federal Incident Response

• Threat Response: Attributing, pursuing, and disrupting malicious cyber actors and malicious cyber activity. Conducting criminal investigations and other actions to counter the malicious cyber activity.

• Asset Response: Protecting assets and mitigating vulnerabilities in the face of malicious cyber activity, reducing the impact to systems and data; strengthening, recovering, and restoring services; identifying other entities at risk; and assessing potential risk to broader community.

30

Federal Incident Response, 2 of 2

Threat Response Asset Response

Federal Bureau of Investigation

855-292-3937 or [email protected]

CISA Integrated Operations Center

888-282-0870 or [email protected]

Report suspected or confirmed cyber incidents, including when the affected entity may be interested in government assistance in removing the adversary, restoring operations, and recommending ways to further improve security.

U.S. Secret Service

secretservice.gov/contact/field-offices

Immigration and Customs

Homeland Security Investigations

866-347-2423 or ice.gov/contact/hsi

Report Internet Crimes:

FBI Internet Crime Complaint Center

ic3.gov

39

31

Malware Analysis

To submit malware:

• Email submissions to NCCIC at: [email protected]

• Send in password-protected zip file(s). Use password “infected.”

• Upload submission online: https://malware.us-cert.gov

32

CSA Contact Information

Klint WalkerCyber Security Advisor, Region IV

[email protected]

CyberAdvisor Cyberadvisor @hq.dhs.gov

Questions?

40

CybersecurityVirginia Waterworks Assessment Findings

10+ years of professional experience, including as an ICS/SCADA security consultant across the oil & gas, nuclear energy, pharmaceutical, defense, and water/wastewater critical infrastructure sectors both within the U.S. and internationally.

Specialized in Systems Security Engineering, Security Test and Evaluations, and Risk Assessments for SCADA and ICS as well as enterprise-level IT systems.

Education

M.S., Cybersecurity, University of Maryland University College B.S., Information Technology, George Mason University

Certifications

Certified Information Systems Security Professional (CISSP) Global Industrial Cyber Security Professional (GICSP) Certified Ethical Hacker Computer Hacking Forensic Investigator ISO 27001 Lead Auditor (BSI Group) Project Management Professional Splunk Certified Architect

Experience Summary

Kyle MillerSenior Associate

Industrial Cyber Security

Speaker Bio

41

Assessment Logistics• Sponsored by EPA Region III and Virginia

Department of Health (VDH)

• Assessments conducted 2014, 2015, and 2016

• 30 waterworks across six VDH regions serving atotal population of approx. 1,740,000 people

– Smallest participating waterworks serves approx. 700 people

– Largest participating waterworks serves approx. 289,000 people

Year 1 Year 2 Year 32013-2014 2014-2015 2015-2016

14 New Sites 10 New Sites6 New Sites

6 Return Sites

Assessment Logistics On-site assessments reviewed both control systems and

connected business management systems

Measured cybersecurity Maturity (Against Modified ISO27001) and Qualitative Risk

There is no charge to the waterworks that are assessed;however, waterworks must prepare for the assessmentand actively participate in on-site activities

Not Performed

Initiated Performed Managed Optimized

Lowest Maturity Highest Maturity

42

Assessment Methodology

Assessment Outputs Each utility received a detailed assessment

report with their findings and recommendedfixes and mitigations

Reports focused on low and no costsolutions to widespread challenges ICS environments are unique and no one

solution works for all sites

As always, remember to work withvendors/integrators prior toimplementing any system changes

• Only aggregated, anonymous resultswere shown to EPA and VDH

43

Maturity Level Description

Not Performed No action has been taken

InitiatedBasic actions are performed, but frequency is ad‐hoc

PerformedBasic actions are performed regularly and have been documented

Managed

Best practices are performed regularly, have been documented, but there are more effective actions that could be taken

Optimized

Best practices are performed regularly, have been documented, and the most secure methods are utilized

Results: Overall Maturity

Results: Maturity Level by Control Family

44

Results: Risk Level by Control Family

1 2 3 4 5 6 7 8 9 10 11 12 13 14

Risk Level

Risk Level by Control Family

Control Family

Control Families

1. Access Control 8. Organization of Information Security

2. Communication Security 9. Info Security Aspects of Business Continuity Mgmt.

3. Operations Security 10. Supplier Relationships

4. Physical and Environmental Security 11. Incident Management

5. Asset Management 12. Compliance

6. System Acquisition, Development and Maintenance 13. Cryptography

7. Human Resource Security 14. Information Security Policies

Prevention and Mitigation Strategies

45

1. Access Control

1. Access Control

Findings

88%

Prevalence

Shared User Accounts

93%Unnecessary Administrative Privileges

52%Default Usernames and Passwords

74%Inadequate Password Complexity

89%Inadequate User Session Lock

46

1. Access Control

Findings

Shared User Accounts

Unnecessary Administrative Privileges

Default Usernames and Passwords

Inadequate PasswordComplexity

Inadequate session lock

Low Cost Solutions

Provide individual role‐based accounts to operators, engineers, vendors, etc.

Restrict admin privileges to only necessary users and only use for admin functions

Remove default accounts and change default passwords in SW, PLCs, and network HW

Utilize stronger password complexity (At least enough to stop a guess)

Implement time‐based revert to read‐only ornon‐sensitive changes features

2. Communications Security

47


Findings

43%

Prevalence

Flat Networks Between SCADA and Business Systems

47%Internet Availability on SCADA Systems

16%Direct Internet Facing PLCs, RTUs, etc.

75%Inadequate Remote Access Controls

62%Firewalls Not Present or Inadequately Configured


Findings

Flat Networks Between SCADA and Business

Internet Availability on SCADA Systems

Direct Internet Facing PLCs, RTUs, etc.

Inadequate Remote Access Controls

Firewalls Not Present orInadequately Configured

Low Cost Solutions

Disconnect internet from SCADA systems and physically separate SCADA traffic from

business traffic unless absolutely required.

Check for yourself online (e.g. Shodan).

Where physical separation is not possible, take advantage of existing technology to

restrict traffic (subnets, VLANs, DMZs, VRF).

Restrict access, use two‐factor VPN (e.g. text),and implement access restrictions.

Benefit from SCADA traffic predictability to configure ingress and egress filtering.

48

3. Operations Security


Findings

71%

Prevalence

No or Considerably Outdated Anti‐Malware

61%Outdated/Unsupported Operating Systems

89%Outdated Security Patches and Firmware

90%No Security Monitoring or Centralized Logs

54%No Formal Backup Process

17

49


Findings

No or Considerably Outdated Anti‐Malware

Outdated/Unsupported Operating Systems

Outdated Security Patches and Firmware

No Security Monitoringor Centralized Logs

No Formal Backup Process

Low Cost Solutions

Install anti‐malware to alert on detection. Use application whitelisting where possible.

Prioritize systems and conduct upgrades, updates, and patches where possible (OS, 3rd

party, firmware, etc.). Utilize redundancy to test updates and modifications.

Configure logging on capable devices and utilize a centralized logging facility.

Backup and locally keep backups of programming and configurations.

4. Physical and EnvironmentalSecurity

50

4. Physical and Environmental Security

Findings Prevalence

59%Remote Assets Lack SufficientPhysical Security Controls

Remote assets (such as a pump stations and water towers) typically have some SCADA related equipment present, which provides an attacker who is able to gain physical access to these areas the ability to tamper with the SCADA sensors, or to use trusted communication channels to gain access into the main SCADA system.

4. Physical and Environmental Security

Findings

Remote Assets Lack Sufficient Physical Security Controls

Low Cost Solutions

Ensure remote site locking mechanisms and alarms are functioning and utilized.

Utilize door, cabinet, or hatch sensors to alertsoperators of potential physical tampering

with SCADA related equipment, especially at remote sites.

Keep a log of employees issued keys and badges in order to ensure they are collectedwhen employment ends.

51

5. Asset Management

5. Asset Management

Findings Prevalence

72%Inadequate Asset Inventory is Maintained

80%Inventory Not Compared with Vulnerability

Databases

93%Removable Media Not Properly Managed

52

5. Asset Management

Findings

Inadequate Asset Inventory is Maintained

Inventory Not Compared with Vulnerability

Databases

Low Cost Solutions

Document details for all assets including makes, models, location, connectivity,

software, firmware, end‐of‐life date, and IP address. Ask for in procurement.

Register for organization and government programs which provide situational

awareness and alerts of known vulnerabilitiesand compare your inventory.

Removable Media Not Properly Managed

Restrict USB and removable media usage on all SCADA systems through BIOS or SW.

Conclusion• The state of waterworks’ ICS cybersecurity requires significant

improvement

• Many of the solutions come back to basic cyber hygiene - ICSSecurity 101

• The challenges across different critical infrastructure sectorsaren’t that different

• Several low-cost measures exist that can greatly reduce risks

• Cybersecurity risk assessments capture only a snapshot in timeand should be considered a process – not a project

• New vulnerabilities are being discovered in operating systemsand third party software every day

• Integrate security into the safety culture

• Be proactive

53

CybersecurityIncident Response Exercises

Agenda• Introduction to the Exercise

• Scenario #1

• Scenario #2

• Cybersecurity Incident Response

54

Introduction to the Exercise

Exercise Purpose and Scope

• This exercise focuses on owner/operator cybersecurityincident response and coordination with other internal andexternal entities regarding a potential attack

55

Exercise Objectives1. Explore and address cybersecurity challenges.

2. Increase awareness of the damage that can be caused bya cyber incident on a business or control system.

3. Explore internal and external relationships essential to thesuccess of organizational cyber incident management.

Exercise Structure• This exercise is a moderated, scenario driven discussion.that allows participants to interact and discuss the responseto a significant incident.

• Each exercise walks through a series of events or injectsthat occur over a period of time.

• Questions will follow each inject to facilitate discussion.

56

Exercise Guidelines• This is an open, low-stress, no-fault environment. Varyingviewpoints, even disagreements, are expected.

• Decisions are not precedent-setting and may not reflect youor your organization’s final position on a given issue. This isan opportunity to discuss and present multiple options andpossible solutions.

• Assume cooperation and support from other respondersand agencies.

• Problem-solving efforts should be the focus; “issue”identification is not as valuable as suggestions andrecommended actions.

Small Group Instructions

• Organize into small groups.

• Discuss the questions presented by the facilitator.

• Select a spokesperson to present your responsesto the entire group.

57

Scenario #1

Scenario #2

58

Tabletop Exercise Tool for Drinking and Wastewater UtilitiesEPA has just updated their Tabletop Exercise (TTX) tool for drinking and wastewater utilities. The TTX tool provides users with the resources to plan, conduct and evaluate tabletop exercises. This 2018 version of the TTX tool contains 12 customizable all-hazards scenarios (e.g., natural disasters, man-made incidents) that will assist utilities to practice, test and help improve emergency response plans and procedures.

You can download the TTX tool here: https://www.epa.gov/waterresiliencetraining/develop-and-conduct-water-resilience-tabletop-exercise-water-utilities

Cybersecurity Incident Response

59

What is Cybersecurity Incident Response?

“Cyber incident response is the way in which an organization responds to a perceived cyber-related incident that may impact ICS owner assets or their ability to operate. ”

– ICS-CERT

Why do we need Cybersecurity Incident Response?• More frequently, control systems and their devices arebecoming more interconnected

• Some control systems are accessible through the internet

• Tools like Shodan have been produced to easily identifycontrol system devices

• In addition to taking steps to prevent incidents, we mustalso be prepared to respond to them

60

Cyber Incident Reporting

• DHS has published the “CyberIncident Reporting: UnifiedMessage for Reporting to theFederal Government” on how toreport cyber incidents and who tocontact

• Refer to Section 2 of yourworkbook and turn to the“Reporting Incidents” sectionfor more information

Source: https://www.dhs.gov/sites/default/files/publications/Cyber%20Incident%20Reporting%20United%20Message.pdf

Incident Response Life Cycle

PreparationDetection & Analysis

Containment, Eradication, & Recovery

Post‐Incident Activity

Reference: NIST SP 800‐61, Computer Security Incident Handling Guide

61

Preparation (1 of 2)

How do you prepare for incidents?• Create an Incident Response Plan (IRP)• Keep up-to-date contact information (e.g. law

enforcement, ICS-CERT)• Familiarize yourself with PPD-41: Cyber Incident

Coordination• Exercise your IRP regularly and walk through

scenarios• Take regular backups for easy restoration

Preparation Detection & AnalysisContainment, Eradication, & Recovery


Preparation (2 of 2)

How can I prevent incidents?• Conduct regular risk assessments• Implement host and network based security• Conduct user awareness training• Maintain accurate details about your system (e.g.

topology)• Follow best practices or standards (e.g. AWWA

Guidance, NIST Cybersecurity Framework)



62

Preparation Resource

Review guidance issued by the Nation Cybersecurity and Communications Integration Center (NCCIC) on how to prepare for cyber incidents

https://ics-cert.us-cert.gov/sites/default/files/FactSheets/ICS-CERT_FactSheet_Cyber_Incident_Analysis_S508C.pdf



Detection & Analysis (1 of 2)



How can incidents occur on my system?• External/Removable media• Connection to the internet• Insecure network devices• Poor password practices• Loss or theft of equipment• Improper use by employees

63

Detection & Analysis(2 of 2)



How will I know if something is wrong?• Anti-virus software detects an issue• Unexpected software is installed• Passwords have been changed on their own• Security settings are altered or security software

is uninstalled• User files or log files disappear

Containment, Eradication, & Recovery (1 of 2)

•How can you prevent it from spreading?–Decision should be made on how to contain (e.g.

shut down, disconnect network, do nothing)– Depends on scope, magnitude, and impact of the

incident

•How can you remove the infection?–Remove the infection using appropriate removal

tools–Identify and mitigate exploited vulnerabilities



64

Containment, Eradication, & Recovery (2 of 2)

•How can you restore operations?–Validate your backups are clean/operational and

use them to recover impacted systems–Continue to monitor to ensure no undetected

threats remain



Post-Incident Activity(1 of 2)



• How can you prevent a future incident?– Conduct a risk assessment to see if other areas could

use improvement– Implement security improvements to lock down these

areas

• How can you improve your incident response?– Discuss with all participants what worked and didn’t work

in the response– Develop thorough “Lessons Learned”– Update policies and procedures to fix shortcomings– Conduct training activities

65

Post-Incident Activity(2 of 2)



• Who can you contact to spread awareness to others?– ICS-CERT / NCCIC– WaterISAC– Cyber Security Advisor (Regional)– Protective Security Advisor– InfraGard– Fusion Center– Local Law Enforcement

• Purpose was to “govern the Federal response to anycyber incident”

• Key features of the directive:1. Response actions and resources are based on the

risks posed to national security, the economy, orpublic health and safety

2. Formation of the Cyber Response Group and CyberUnified Coordination Group

• Designates Department of Justice, Department ofHomeland Security, and Office of the Director ofNational Intelligence as lead agencies.

PPD-41: Cyber Incident Coordination

66

Federal agencies undertake the following when responding to any cyber incident:

1. Threat Response – law enforcement and investigativeactivity

2. Asset Response – technical assistance to affectedentities to protect assets, reduce impacts and mitigatevulnerabilities

3. Intelligence Support – building situational threatawareness

PPD-41: Cyber Incident Coordination

67

CybersecurityProgram Development Overview

• Evaluate cyber risks as part ofan all-hazards riskassessment

– AWWA J100-10 (RAMCAP)

– EPA’s VSAT

• Utilities should:

– Conduct/update an all-hazards risk assessmentevery five years, or

– After any significant change inprocesses or operations

How do I manage my cybersecurity risks?

68

1. Document your IT systems, plans,policies, and procedures

2. Compare current cybersecurity practiceswith recognized best practices

3. Develop your cybersecurity riskmanagement plan and mitigate risks

Take a 3-step Approach to Developing a Cybersecurity Risk Management Plan:

• IT Systems (e.g., architecture, networks, processcontrol systems, application security)

• Plans (e.g., incident identification and response,recovery, business continuity, risk management)

• Policies (e.g., physical security, personnel andoperational security, access control)

• Procedures (e.g., remote access, encryption,education, auditing, asset management)

Step 1: Document Existing Systems

69

Identify recommended controls that apply to your utility and see how they compare to what you are already doing. Tools and standards that can help you include:

• AWWA’s Cybersecurity Guidance &Tool

• DHS’s Cyber Security Evaluation Tool(CSET)

• ISO 27001

Step 2: Compare Your Practices withRecognized Best Practices

Step 3: Develop your CybersecurityRisk Management Plan and Mitigate Risks

Your plan should: Your mitigation actions should:

Document cybersecurity weaknesses

Be done based on available time and budget

Prioritize weaknesses on risk level Focus first on the highest risks :

Provide remediation options (short and long term)

• Are there benefits to fixing several lowrisk areas instead of only one high riskarea?

Estimate remediation costs (e.g., manhours)

• Can fixing low risk areas reduce thelikelihood that a high risk area can beexploited?

Be communicated to staff

70

Contact Information

Will Keefer

Horsley Witten Group, Inc.

(508) 833-6600

[email protected]

Kyle Miller

Booz Allen Hamilton, Inc.

(703) 984-1893

[email protected]

Caitlin Ferro

Booz Allen Hamilton, Inc.

(703) 984-1552

[email protected]

Gemma Kite

Horsley Witten Group, Inc.

(508) 833-6600

[email protected]

71

Documents

Introduction to Cybersecurity: Workshop and Response Exercises · replicate without user action Virus • Malicious program attached to a host file that runs/spreads when the host