Using Cyber SecurityAssessment Tools on

Industrial Control Systems (ICS)

Dale PetersonDigital Bond, Inc.

peterson@digitalbond.comTwitter: @digitalbond.com

ICS Security Assessments

• Digital Bond performed our first ICS security assessment in 2000 … 15 years ago

• Digital Bond performs assessments on live / operational / running critical infrastructure ICS– Power plants, pipelines, water treatment,

chemical manufacturing, transportation• Digital Bond uses scanning tools• And we have never caused an

unacceptable impact to operations

Assessment Types

• Asset Owner / ICS End User Assessments– Is the ICS deployed and maintained in a good

security practice configuration?– Are known vulnerabilities remediated / fixed?– This presentation covers Asset Owner

Assessments• Assessments for Vendors / New Purchases

– Attempts to find new, 0day vulnerabilities– Very advanced testing, uses some commercial

and free tools, but also a lot of custom code– Digital Bond Labs does these, see more

tomorrow

Asset Owner Assessments

• Architecture Review• Configuration Inspection• Physical Inspection• Policy and Procedure Review and Audit• Interview (very important for determining

risk)

and

• Online Scanning/Testing/Exploits

Current State of ICS Security

• Many organizations are just beginning to worry about ICS security– They may have a poorly configured firewall– They may have some anti-virus running– Little else in the way of ICS cyber security

• ICS protocols and PLC’s are insecure by design– They lack basic security such as authentication– Access = compromise– Impact is limited to engineering and

automation skill

Efficient Risk Reduction

What should I do next?Where should you spend your next ¥ or

hour of time on ICS cyber security to get the maximum risk reduction or improvement in security posture?

• Assessment should provide a list of actions prioritized by efficient risk reduction

• Companies have limited ability to add security

Prioritization

• Threat– Very difficult to determine– Typically look at the accessibility of the

device/system• Vulnerability

– Assessment can clearly identify this• Impact

– This is the most important factor– Don’t waste time on small impact risks, eg

serial connected panels– Talk to the Operations team, what would

happen if …

Even the most basic, simple, non-intrusive scan of

a PLC or ICS application can cause a denial of service condition.

TRUE!

Example 1

• Safety PLC– Simple port scan of a safety PLC caused it to

crash, and it did not recover when rebooted– Additional scanning found a port that was used

to load new firmware did not have authentication or even check parameters

– Any activity on the port started a firmware update process

– PLC needed to be completely reloaded to recover

Example 2

• Redundant Pair of Real Time Servers – Issues read and write commands to PLC’s– Provides data and forwards commands from

HMI / Operator Stations• Scan of Standby Server … no problem• Scan of Hot/Active Server … crash and

failover

You cannot and should not use security scanning tools on an

operational ICS because they can cause important things to crash.

False!

How To Scan ICS

• Staging area or lab– Some sites have non-operational systems to

test• Leverage redundancy

– An ICS should not have a single point of failure– Many operator stations / HMI– Hot and standby servers

• Select best testing time– Many processes have key times weekly or daily

were a computer or device outage is more difficult to handle

Questions For Operations: 1. Is it acceptable if computer x crashes during the testing window? 2. Can you recover the system in an acceptable time frame if it crashes.

Answer: Yes … schedule scan

• You have a recovery issue– Don’t touch that because the guy who knew

how it worked is no longer with the company– What is your Recovery Time Objective (RTO)?– Do you have a proven ability to meet your RTO?

or• You have a single point of failure

– Missing redundancy– We can never reboot or have an outage of a

Windows NT, XP, 2003, 2008, 7 … FRAGILITY

Answer: No … important security finding

Create Your Scan List

• Work with Operations to identify one of each time of computer or device

• Find a sample that you can scan, assuming it may go down, without having an unacceptable impact to Operations– Always assume it will go down– Things are much better than 10 years ago

Scanning Tool Categories

• Basic Enumeration (what is it?)• Full featured scan (1000’s of tests)• Basic, random data fuzz testing• Secondary application testing

– Web servers, databases• Exploit proof of concept

Basic Enumeration

• Almost all recommend Nmap– It’s free and fast– Many claim it is more accurate– The results are reasonable size and good for

reference• Nmap tells you

– What TCP/UDP ports are open– What application and version is running on a

port– What operating system is running

• When not to run Nmap

Project Redpoint

• Digital Bond research project (free)– https://github.com/digitalbond/Redpoint– Also being integrated into Nmap download

• Nmap Scripting Engine (NSE) scripts– Send legitimate ICS commands to enumerate

specific ICS devices and applications – Identify ICS on the corporate network– Great for creating and maintaining inventory– Digital Bond tries to create new script

whenever we encounter a new ICS computer or device

BACnet

Broad Based Security Scanner

• Nessus from Tenable Network Security• Nexpose from Rapid 7• Retina from Beyond Trust• DeepDiscovery from Trend Micro

Or

• Scanning as a service, Qualys

Example: Nessus

• Credentialed Scanning• Learn the Product• Security Audit

Broad Based Security Scanner

• New plugins (tests) are created for each vulnerability or patch

• Nessus has over 75,000 plugins – Not all will be applicable – Not all will run in default config

Credentialed Scanning

• Inspect system with the same rights as an Administrator or root user

• More accurate– Patches: registry check vs. response to packet

• Less intrusive / less likely to crash computer– Port scan vs netstat

• A lot more information– Installed software, running services, users,

group policy info, USB usage, …– Look at the information level results

Adding Credentials

Security Patching

• ICS scans often identify many missing patches– Microsoft security patches– 3rd party / application software security patches– Security software security patches, eg anti-virus– Even ICS security patches

Question: What is the security finding?Answer: Ineffective security patching

program

Security Patching in ICS

• Good security practice is to apply patches in a reasonable time after available– IT / corporate network typically 30 days– Best in ICS is typically quarterly / 90 days

Question: Can you go from little or no security patching to applying all patches every 90 days?

Think Efficient Risk Reduction

Prioritized Security Patching

• Priority 1 – Computers accessible from corporate or external network– Monthly … should be a small number of

computers that are not required for operation• Priority 2 – Computers accessible from

Priority 1 computers– Quarterly … attackers will compromise Priority

1 computers and pivot• Priority 3 – Everything else

– Annual … maintain supported system

Controversial

• If you can do better, great– Shorter patching windows are better security,

but– We see many owner/operators fail in patching

• Select some achievable plan, succeed, and then shorten patching window

• Also … if an attacker can reach a Priority 3 computer he can compromise the ICS even if it is patched … ICS is insecure by design

Know Your Scanner

• These are complex, full feature products• Default scan configurations will miss a lot

of what you want to know in an assessment

• Take a class from the vendor or skilled teacher

Nessus Example 1

• Oracle Default Passwords

Nessus Example 2 – USB Usage

• USB Drive Usage

Compliance Audit

• Identify an optimal security configuration for OS and all ICS applications

• Develop an audit file for the scanner• Use the compliance plugin• Digital Bond Bandolier Project

– Funded by US Department of Energy

Adding the Audit File

• About 200 operating system (OS) audit tests

• Number of ICS application tests vary

Audit File Example

• Folder Permissions• ICS applications install software in one or

more folders– Read, write and execute permissions for the

folders should be least privilege– Permissions are often set to Everyone

• Vendor should define optimal security config– Ideally provide a document and audit file– Modify as necessary for your policies &

environment

Random Data Fuzzing

• ICS vendors historically only performed positive testing– Does the application or device perform properly

when receiving a legitimate command or packet

• Hackers, scanners, new applications may send something unexpected – Will the application/device handle the “error”

properly– Or will it crash

• This is a crude test– Not intelligent fuzzing that the vendor should

perform

Secondary Testing

• May not be necessary– Usually required after an ICS security program

has been running for 2 to 3 years– An attacker will take the easiest path to

success• Specialized tools and techniques

– Web application testing– Database testing– Password cracking– Man-in-the-middle / ARP spoofing

Proof of Concept Exploits

• If assessor is uncertain if vulnerability can be exploited– Should be attempted to accurately determine

risk– Denial of service vs. remotely run code

• Prove the danger of missing security patches / default credentials / other vulnerabilities– Show the Operator Station on your laptop– Attack compromise and pivot

How Many Assessments?

What if you have 50 or 100 factories or plants?

Should you perform an assessment at eachfactory or plant?

Recommendation

• Pick 3 to 5 different sites– Pick a variety of size and types of plants– Select a representative sample– Perform assessments on the samples

• Identify the common high priority findings• Define a common set of required security

controls– Not too much in the first year

• Define how the controls will be audited• Add additional controls in years 2, 3, …

Questions