CSC 382: Computer SecuritySlide #1 CSC 382: Computer Security Software Security Testing

CSC 382: Computer Security Slide #1

CSC 382: Computer Security

Software Security Testing


Topics

1. Why is Testing Important?

2. White Box vs Black Box Testing

3. Creating a Test Plan

4. Fault Injection– Source-based Fault Injection– Environment Fault Injection– Input-based Fault Injection

5. Fuzz Testing


Why is Testing Important?

Navy shoots down Civilian Airliner.– 1988– US Vicennes shot down Airbus 320.– Mistook airbus 320 for a F-14.– 290 lives lost.– Why: Software bug – “cryptic and misleading

output displayed by the tracking software.”



“I Love You” virus cost $8.75 billion.2

– Result of Microsoft decision to automatically execute binary attachments in Outlook.



1. Complexity– Continually increasing.– Windows 3.1 (3mloc) to Windows XP

(40mloc)

2. Extensibility– Plugins.– Mobile code.

3. Connectivity– Network access.– Wireless networking.


Software Complexity

5-50 bugs per/kloc8

– 5/kloc: rigorous quality assurance testing (QA)

– 50/kloc: typical feature testing

System Lines of CodeMS Word 95 2 million

MS Windows 3.1 3 million

Boeing 777 7 million

Space Shuttle 10 million

Netscape 17 million

MS Windows XP 40 million


Software Connectivity

• Internet connectivity nearly ubiquitous.– 802.11 wireless networking.– Bluetooth wireless personal area networking.– Embedded devices often networked.

• Can your cellphone get a virus?– Symbian Bluetooth virus released in 2004.

• Can your automobile get a virus?– BT discovery reveals embedded BT devices in

high-end cars.


Testing Caveat

"Testing can establish the presence of errors, but never their absence."

–Dijkstra


White and Black Box Testing

White Box Testing– Testing guided by the source code and design

documentation.– May be approximated by decompiling or

disassembling binary code.

Black Box Testing– Testing without access to system code.


Creating a Test Plan

Base Test Plan on Threat Model1. Decompose application into components.

2. Identify entry points for each component.

3. Use STRIDE classification to determine types of tests to perform for each entry point.

4. Prioritize testing effort based on risk evaluation for each threat.

5. Use threat trees and DFDs to guide implementation of specific tests.


1. Decompose Application

Threat Model provides list of application components.


2. Identify Entry Points

Threat model provides list of entry points.

Classify entry points by risk potential5.

Entry Point Characteristic PointsEntry point component runs with high privilege +2

Entry point component written in C/C++ +1

Entry point component written in VHL language -2

Entry point accepts arbitrary-sized input +1

Recipient buffer is stack-based +2

Entry point has no or weak access control +1

Entry point has good access control -2


STRIDE5 Threat CategorizationSpoofing

ex: Replaying authentication transaction.Tampering

ex: Modifying authentication files to add new user.Repudiation

ex: Denying that you purchased items you actually did.Information disclosure

ex: Obtaining a list of customer credit card numbers.Denial of service

ex: Consuming CPU time via hash algorithm weakness.Elevation of privilege

ex: Subverting a privileged program to run your cmds.


3. STRIDE Test Types

Spoofing– Attempt to force application to use no authentication.

– Attempt to force application to fall back to less secure authentication protocol.

– Can security tokens by replayed?

– Can security tokens be brute-forced?

Tampering– Is it possible to modify and rehash security token?

– Create invalid hashes, MACs, and signatures to verify that they are checked correctly.



Repudiation– Can you prevent logging or auditing?– Is it possible to create requests that create

incorrect data in event log? EOF, newline.

Information Disclosure– Attempt to access privileged data. Sniffer, file.– Kill process, then search disk for confidential

data left behind due to unclean termination.– Cause application to fail. Do error messages

reveal any useful information?



Denial of Service– Flood application with many requests.– Can malformed input crash the application?– Can external influences (low memory, disk, or

other resource limits) cause a failure?

Elevation of Privilege– Can you execute data as code?– Can an elevated privilege be forced to invoke a

shell?


4. Prioritize Testing Effort

Use risk evaluation from Threat Model.– Summing attack tree values based on categories from

DREAD evaluation system.

Read File

Get Password Network Access Physical Access

Search Desk Social Engineer Boot with CD Remove hard disk


Evaluate Risk with DREADDamage Potential

Extent of damage if vulnerability exploited.Reproducibility

How often attempt at exploitation works.Exploitability

Amount of effort required to exploit vulnerability.Affected Users.

Ratio of installed instances of system that would be affected if exploit became widely available.

DiscoverabilityLikelihood that vulnerability will be discovered.


5. Test Implementation

Attack Trees– Divide threat into testable conditions.

Data Flow Diagrams– Show how data moves from entry point to

system component impacted by threat.– Indicates security mechanisms to test/bypass.


Dynamic Analysis: Memory Checking

Tools like purify and valgrind check C/C++ code memory access at runtime.– Use of unitialized memory.– Memory access after free()/delete.– Out-of-bounds memory access.– Access to inappropriate areas of stack.– Memory leaks.– Overlapping source and dest pointers in memcpy() and similar functions.


Fault Injection

Fault Injection: Modifying internal or external environment of an application in an attempt to discover flaws.Source-based Fault Injection

Modifies application source code to produce faults.

Environment Fault InjectionModifies environment (memory, libraries, etc.) to

produce faults.

Input-based Fault InjectionModifies expected input to produce faults.


Source-based Fault Injection

Inject new code at compile-time or runtime toforce faults that would not normally occur.

– Cause functions to return unexpected values.– Set internal data to values that trigger faults.


Environment-based Fault Injection

Intercept application calls to OS (requests for memory, disk space, etc.) – Selectively deny resources.– Control system call responses.


Environment-based Example Attacks

Alter application files.– Create files using app’s expected filename.– Delete expected files.– Change permissions.– Replace files with links.– Use filenames longer than MAX_PATH.

Block access to shared libraries.– Does application crash?– If it doesn’t crash, what failures occur due to

lost functionality?


Environment-based Example Attacks

Limit memory, disk, and network resources.– How does app response to allocation failures?– Does app leak confidential info to swap?– Does no network prevent auditing/logging?

Manipulate registry key existence andpermissions on MS Windows systems.


Input-based Fault Injection

Send unexpectedly bad input to application.

Input should be construed broadly:– Command line arguments.– STDIN and GUI inputs.– Files.– Network sockets and RPC interfaces.– Environment variables.– Registry values.


Input-based Example Attacks

Overly large input.Can you trigger a buffer overflow?

Out of range values.Can you create an integer overflow?

Unexpected characters.Can you supply any metacharacters that will beused by application in an unsafe manner?

Check for default accounts.


Input-based Example Attacks

Modify input semantics.– How does system handle a parameter supplied

multiple times? Does it check input once, then use the other version of parameter?

– What does application do with unknown parameters/fields?

Use non-canonical resource names to checkfor directory traversal flaws.


Fuzz Testing

Black-box input based testing technique.– Uses random data.– Easily automated.– If application crashes or hangs, it fails.

Results of 1995 study9.– 15-43% of utilities from commerical UNIX

systems failed.– 9% of Linux utilities failed.– 6% of GNU utilities failed.– 50% of X-Windows utilities failed.


Browser Fuzz Testing

Michael Zalewski’s mangleme CGI14

– Generated streams of malformed HTML.– Most browsers crashed due to:

• NULL pointers.

• Memory corruption.

• Buffer overflows.

• Memory exhaustion.

– Exception: MSIE


SPIKE Fault Injection Tool

Network protocol fault injection.– Stateless protocols.– Stateful protocols.

Scripting language makes it easy to convertsniffed network packets into tests.

– Automatic length and checksum calculation.– Script permits user-defined varying of packets:

• Permute characters.

• Buffer-overflow attempts.


Web Proxy Fault Injection

Point browser at fault injection proxy.Fault injection proxy supports

– Cookie viewing and modification.– Form parameters viewing and modification.– Automated SQL injection attempts.– Automated buffer overflow attempts.– Brute-force password guessing.

Examples– SPIKE Proxy


Key Points1. The Importance of Testing.2. Black and White Box Methods.3. Test Plan based on Threat Model:

a) Entry points.b) Test types based on STRIDE.c) Prioritization based on DREAD.d) Implementation using Attack Trees.

4. Fault Injectiona) Source-based Fault Injectionb) Environment Fault Injectionc) Input-based Fault Injection

5. Fuzz Testing


References1. Matt Bishop, Introduction to Computer Security, Addison-Wesley, 2005.2. Computer Economics, “Malicious Code Attacks Had $13.2 Billion Economic Impact in 2001 ,”

http://www.computereconomics.com/article.cfm?id=133, 2002.3. Wenliang Du and Aditya Mathur, “Vulnerability Testing of Software System using Fault Injection,”

COAST TR98-02, 1998.4. Mark Graff and Kenneth van Wyk, Secure Coding: Principles & Practices, O’Reilly, 2003.5. Greg Hoglund and Gary McGraw, Exploiting Software: How to Break Code, Addison-Wesley, 2004.6. Michael Howard and David LeBlanc, Writing Secure Code, 2nd edition, Microsoft Press, 2003.7. Koziol, et. al, The Shellcoder’s Handbook: Discovering and Exploiting Security Holes, Wiley, 2004.8. Barton Miller, Lars Fredrickson, and Bryan So, “An Empirical Study of the Reliability of UNIX

Utilities,” ftp://ftp.cs.wisc.edu/paradyn/technical_papers/fuzz.pdf, 1990.9. Barton Miller, et. al., “Fuzz Revisited: A Re-examination of the Reliability of UNIX Utilities and

Services,” ftp://ftp.cs.wisc.edu/paradyn/technical_papers/fuzz-revisited.pdf, 1995.10. John Viega and Gary McGraw, Building Secure Software, Addison-Wesley, 2002.11. Jeffrey Voas and Gary McGraw, Software Fault Injection: Inoculating Programs Against Errors,

Wiley, 1999.12. David Wheeler, Secure Programming for UNIX and Linux HOWTO,

http://www.dwheeler.com/secure-programs/Secure-Programs-HOWTO/index.html, 2003.13. James Whittaker and Herbert Thompson, How to Break Software Security: Effective Techniques for

Security Testing, Addison-Wesley, 2004.14. Michael Zalewski, “Web browsers – a mini-farce,” Bugtraq mailing list posting,

http://www.securityfocus.com/archive/1/378632, 2004.

http://www.computereconomics.com/article.cfm?id=133

ftp://ftp.cs.wisc.edu/paradyn/technical_papers/fuzz.pdf

ftp://ftp.cs.wisc.edu/paradyn/technical_papers/fuzz-revisited.pdf

http://www.dwheeler.com/secure-programs/Secure-Programs-HOWTO/index.html

http://www.securityfocus.com/archive/1/378632

Documents

CSC 382: Computer SecuritySlide #1 CSC 382: Computer Security Software Security Testing