Upload
ankitmehta21
View
279
Download
0
Tags:
Embed Size (px)
Citation preview
Testing for Software Security
ECEN5053 Software Engineering of Distributed Systems
University of Colorado, Boulder
Testing for Software Security, Hebert Thompson, James Whittaker, Dr. Dobb’s Journal, November, 2002, pp. 24-34
November 18, 2002
Testing for SW Security, ECEN5053, University of
Colorado, Boulder
When is a security bug not like a bug? Traditional non-security bugs -- often defined as a
violation of a specification. Security bugs -- additional behavior, not
originally intended– Meanwhile, it is doing what it is supposed to do– Traditional techniques not good at finding– Even in inspections, tend to look for
• missing behavior• incorrect behavior
– Neglect to look for ... undesirable side-effects
November 18, 2002
Testing for SW Security, ECEN5053, University of
Colorado, Boulder
Intended vs. Implemented Behavior
Traditional faults
Intended Functionality
Actual Software Functionality
Unintended, undocumented, unknown functionality
November 18, 2002
Testing for SW Security, ECEN5053, University of
Colorado, Boulder
Traditional faults Incorrect
– Supposed to do A but did B instead
Missing– Supposed to do A and B but did only A.
November 18, 2002
Testing for SW Security, ECEN5053, University of
Colorado, Boulder
Security Bugs Side effects
– Supposed to do A, and it did.– In the course of doing A, it also did B
Monitoring for side effects and their impact on security can be challenging– Side effects can be subtle and hidden– Examples: file writes, registry entries, extra
network packets with unencrypted data
November 18, 2002
Testing for SW Security, ECEN5053, University of
Colorado, Boulder
Tools Commercially available tools
– Mutek’s AppSight
http://www.identify.com/products/appsightsuite.html– Holodeck Lite
http://se.fit.edu/holodeck/
(freeware developed by the authors using techniques similar to those in the handout Listing One to help easily monitor and obstruct common system calls)
– Write your own customized monitoring solution
November 18, 2002
Testing for SW Security, ECEN5053, University of
Colorado, Boulder
Ways in / sources of interaction with the environment Human interface -- UI
– Set of API’s get input from kbd, mouse, etc.– Concerns: unauthorized access, privilege
escalation and sabotage File system
– Provides data stored in binary or text format– Trusted to store sensitive data– Test how stored, retrieved, encrypted, managed
API -- input in form of return values of API calls Operating system kernel -- memory, file pointers,
time and date functions, etc.
November 18, 2002
Testing for SW Security, ECEN5053, University of
Colorado, Boulder
Memory is vulnerable Any information an app uses passes through
memory eventually– Encrypted -- usually ok– Decrypted and stored -- at risk of being read– Encryption keys, CD keys, passwords and other
sensitive information are eventually used in an unencrypted form• Must protect their exposure in memory
November 18, 2002
Testing for SW Security, ECEN5053, University of
Colorado, Boulder
Stress Stress testing for low memory and other faulty
operating conditions that may cause an application to crash
App’s tolerance to environmental stress can prevent – denial of service– situations where app may crash before
completing an important task like encrypting passwords• Once it crashes, the state of stored data is ... ?
November 18, 2002
Testing for SW Security, ECEN5053, University of
Colorado, Boulder
What won’t work Look at each method of input delivery Bombard that interface with input
Why that won’t work well enough Most revealing attacks require you to apply inputs
through multiple interfaces
Search said these categories can be used to expose vulnerabilities– Dependency attacks– Design-and-implementation attacks
November 18, 2002
Testing for SW Security, ECEN5053, University of
Colorado, Boulder
Attacking Dependencies Apps rely heavily on their environment to work
properly Not as overt as human input but there are lots of
sources of input– Like any input, if software receives a value
outside of its expected range, it can fail. Environment failures lead to calls on error-
handling code (if it exists)– Error handlers are the security weak point of an
application
November 18, 2002
Testing for SW Security, ECEN5053, University of
Colorado, Boulder
Testing error handlers Failures in the sw’s environment that exercise
these code paths are difficult to produce in a test lab situation– Tests that involve disk errors, memory failures,
and network problems are only superficially explored
– Illusion of security but ...• Servers do run out of disk space• Network connectivity can be intermittent• File permissions can be set improperly
Need to integrate failures into tests to evaluate
November 18, 2002
Testing for SW Security, ECEN5053, University of
Colorado, Boulder
Just do the impossible ... Create environmental failures
– Tamper with the application code– Simulate specific failing responses from the o.s.– Yeah, right
• Takes huge amount of time, effort, and expertise needed to simulate such a failure
• Determine where in the code the app uses these resources
• How to make appropriate changes to simulate a real failure
November 18, 2002
Testing for SW Security, ECEN5053, University of
Colorado, Boulder
Alternative approach Run-time fault injection
– Simulating errors to the app in a black-box fashion at run time
Advantage– Nonintrusive– Lets you test production binaries, not contrived
versions of the app that have phony return values hard-coded
November 18, 2002
Testing for SW Security, ECEN5053, University of
Colorado, Boulder
HOW Overwrite the first few bytes of the actual function
to be called in the process space and insert a JMP statement to fault injection code in its place.
Modify import address tables (For doing this in Windows environment, authors recommend Jeffrey Richter, Programming Applications for Microsoft Windows, 4th Edition, Microsoft Press, 1999)
November 18, 2002
Testing for SW Security, ECEN5053, University of
Colorado, Boulder
WHY? You redirect a particular system call to your own
impostor function Can log events
– Watch the application for file, memory, and registry activity
In control, you can forward a system request to the actual OS function or deny the request by returning any error message you choose.
November 18, 2002
Testing for SW Security, ECEN5053, University of
Colorado, Boulder
What about registry bugs? Security problem with the “registry” is trust When developers read from the registry, assume
– values are accurate– haven’t been tampered with maliciously– especially, if their code wrote those values in
the first place “Try and buy” -- users have limited functionality
or a time limit in which to try the software.– App can be unlocked if purchased/registered– App may check a registry key at startup
• protected with weak encryption or it’s a 1/0
November 18, 2002
Testing for SW Security, ECEN5053, University of
Colorado, Boulder
Test app’s use of corrupted files & names App reads/writes hundreds of files
– Handle bad data gracefully without• exposing sensitive information• allowing unsafe behavior
Test with the approach that can happen but is often not tested for– Change a file in a way not covered in
requirements• Numerical data --> add text characters
– “Successful” test results in denial of service• Crashes the application or the whole system• May expose data during the crash
November 18, 2002
Testing for SW Security, ECEN5053, University of
Colorado, Boulder
Test low memory/disk/network availability Deprive the app of resources to see robustness
under stress Rule of thumb -- block a resource when an app
seems most in need of it– memory when in an intense computation– disk errors -- introduce faults when doing
writes/reads (modify code in Listing One to intercept appropriate system functions)
November 18, 2002
Testing for SW Security, ECEN5053, University of
Colorado, Boulder
Attack (Test) Design and Implementation There are subtle security implications made during
the design phase Even if design is secure, choices made during
implementation can impact security Tests that expose these:
– Force all error messages– Seek unprotected test APIs– Overflow input buffers– Connect to all ports
November 18, 2002
Testing for SW Security, ECEN5053, University of
Colorado, Boulder
Force all error messages Purposes
– Try values that should result in error msgs to see how many are handled properly
– Make sure error msgs do not reveal unintended information
November 18, 2002
Testing for SW Security, ECEN5053, University of
Colorado, Boulder
Seek unprotected APIs Difficult to test complex apps by relying on the
APIs extended for normal users alone.– Many designers therefore include additional
hooks (extended but not published API) • These often bypass normal security because
they are intended for use by the good guys• Intended for custom test harnesses• They mean to remove them ... but then ...
they’ll be handy for the next release• So integrated into the code and testing
process, removal may destabilize the code If purposely left in, verify they are not trouble if
found
November 18, 2002
Testing for SW Security, ECEN5053, University of
Colorado, Boulder
Overflow input buffers Test ability to handle long strings in input fields
– Especially where long strings are entered into fields that have an assumed, but not enforced, length, e.g. ZIP codes, state names
API calls notorious for unconstrained inputs– GUI can filter inputs as they are entered– API parameters dealt with internally
• checks to ensure values are appropriate before they are used
– Most vulnerable -- seldom used or supporting legacy functionality
November 18, 2002
Testing for SW Security, ECEN5053, University of
Colorado, Boulder
Connect to all ports Sometimes apps open custom ports to connect
with remote servers– Create maintenance channels to automatic
updates– Relic from test automation– http://www.ntbugtraq.com/ for many cases
where these are left open and unsecured If this is done in the app
– Test the port in the release version– Check what kind of data flows through it
November 18, 2002
Testing for SW Security, ECEN5053, University of
Colorado, Boulder
Conclusion These attacks by testers can expose vulnerabilities
before release Only part of complete security-testing methods This area of research is in the infant stages