Copyright © The OWASP FoundationPermission is granted to copy, distribute and/or modify this documentunder the terms of the OWASP License.

The OWASP Foundationhttp://www.owasp.org/

The Power of Code ReviewIf you aren’t doing it, you really should

Dave WichersCOO, Aspect SecurityOWASP Board MemberOWASP Top 10 and ASVS Projects LeadOWASP Conferences Chairdave.wichers@aspectsecurity.comdave.wichers@owasp.org

OWASP - 2010

Background For This Talk

First off, I’m talking about Manual Code Review vs. Manual Pen

TestingBoth are tool assisted, of course.

Both techniques can use/leverage automated analysis tools

But I don’t think that affects my argument

- 2

OWASP - 2010

Why Do We Look For Vulnerabilities?

To find them? To find exactly where they are in the

app? To verify we don’t have them?

Which technique is best suited to answer these questions? Application Penetration Testing, or Code Review

- 3

OWASP - 2010

What do I Mean by Code Review vs. Pen Testing Code Review should review all

1. custom developed code, AND2. config files of the app, libraries,

frameworks, app server, etc.

App Pen Testing1. Tests the application from the

outside2. If you can get access to config files,

you should of course review them too

Ideally you are doing both! But let’s compare their strengths and

weaknesses - 4

OWASP - 2010

Both Have Their Advantages

Pen Testing Pros

Requires less specialized expertise

Easier setupEasier to performExercises the entire

app infrastructureProves

vulnerabilities

Code Review Pros -Easier toFind all the contentFind all instances of

certain types of flawsVerify controls are

correctVerify controls are

used in all the required places

- 5

OWASP - 2010

How do they stack up to the OWASP Top 10?

A1: InjectionA2: Cross-Site

Scripting (XSS)

A3: Broken Authentication

and Session Management

A4: Insecure Direct Object References

A5: Cross Site Request Forgery (CSRF)

A6: Security Misconfigurati

on

A7: Failure to Restrict URL

Access

A8: Insecure Cryptographic

Storage

A9: Insufficient Transport

Layer Protection

A10: Unvalidated

Redirects and Forwards

http://www.owasp.org/index.php/Top_10

- 6

OWASP - 2010

Finding Injection Flaws – Testing?

- 7

' ; droptable log --

Tool assistance is very helpful here.

OWASP - 2010

Finding Injection Flaws – Reviewing Code?

- 8

try{ String query = "SELECT employee.* FROM employee WHERE employer_name = '" + username + "' and employee_name = '" + subjectUsername + "'"; Statement answer_statement = session.createStatement(…); ResultSet answer_results = answer_statement.executeQuery(query); …} catch (SQLException sqle){ …

First, search for things like:• exec• SQL• J/ODBC

Then look for dangerous concatenation.

try{ String query = "SELECT employee.* FROM employee WHERE employer_name = ? and employee_name = ?"; PreparedStatement answer = session.prepareStatement( query, …); answer.setString( 1, username ); answer.setString( 2, subjectUsername ); ResultSet answer_results = answer.executeQuery(); …

Or safe use of Bind Variables.

OWASP - 2010

Injection Results

- 9

Uncover Flaws

Locate Flaws

Verify Lack of Flaws

Overall Winner

Code Review 5 5 5App Pen Testing

2 2 1Key: Ratings on a Scale from 1 (hardest) to 5 (easiest)

Check out the: http://www.owasp.org/index.php/SQL_Injection_Prevention_Cheat_Sheetfor how to avoid SQL injection vulnerabilities.

OWASP - 2010

Finding XSS Flaws – Testing?

- 10

<script>alert(document.cookie)</script>

<script>alert(document.cookie)</script>

? or <script>alert(document.cookie)</script>

OWASP - 2010

Finding XSS Flaws – Reviewing Code?

(Imagine this .jsp)<Table><TR><TD>

Full Name:</TD><TD> <%=user.getFirstName()%> <%=user.getLastName()%></TD><TD>

Display Name:</TD><TD> <%=user.getDisplayName()%></TD>

<Table><TR><TD>

Full Name:</TD><TD> <%=ESAPI.encodeForHTML(

user.getFirstName())%> <%=

ESAPI.encodeForHTML( user.getLastName())%>

</TD><TD>

Display Name:</TD><TD> <%=

ESAPI.encodeForHTML( user.getDisplayName())%>

</TD>

See

- 11

Look for unsafe user content inclusion.

Or output encoding to make inclusion of user content safe.

OWASP - 2010

XSS Results

- 12

Uncover Flaws

Locate Flaws

Verify Lack of Flaws

Overall Winner

Code Review 4 4 4App Pen Testing

3 2 2

Check out the: http://www.owasp.org/index.php/XSS_(Cross_Site_Scripting)_Prevention_Cheat_Sheetfor how to avoid XSS vulnerabilities.

OWASP - 2010

Finding AuthN & Session Mgt Flaws – Testing?Well …

- 13

http://www.stealmycustomersmoney.com/onlineStatement.do;jsessionid=7371FC8C8C4C228DECEDB0FCE5818789

sometimes its obvious

and lots of testing is easy

OWASP - 2010

Finding AuthN & Session Mgt Flaws – Code Review?

But sometimes its hard

- 14

<security-constraint>   <web-resource-collection>      <web-resource-name>User data</web-resource-name>      <url-pattern>/banking/user/*</url-pattern>   </web-resource-collection>   <auth-constraint>      <role-name>User</role-name>   </auth-constraint></security-constraint> …

bsmith:1045:Ejrt3EJUnh5Ms:Bob:Smithdwilson:1046:1MvGJq5Nqersjw:Dave:Wilson dwichers:1243:4jaM4lehUwp25N:Dave:Wichers

Server=172.145.34.2;Database=pubs;User ID=sa;Password=stealme"

Can you figure out all of the resources that require authentication and which don’t?

Can you tell if these hashes are properly salted?

Can you find all the stored credentials to make sure they are encrypted?

OWASP - 2010

Common AuthN & Session Mgt Reqts*

- 15

Requirement Pen Testing

Code Review

All non public pages require authenticationPassword field does not echo passwordAccount lockoutStrong credentials requiredSafe password create, change, recoverySupports idle and absolute timeoutsAuthentication events are loggedUser passwords are stored with salted hashService passwords are stored encryptedSessions properly invalidated on logoutLogout button easily visible on every pageSessions always protected with SSLSessions never included in URLSession tokens are sufficiently random

?

* - Requirements selected from OWASP ASVS

OWASP - 2010

Authentication and Session Mgt Results

- 16

Uncover Flaws

Locate Flaws

Verify Lack of Flaws

Overall Winner

Code Review 3 4 3App Pen Testing

4 3 3

??

Check out the: http://www.owasp.org/index.php/Authentication_Cheat_Sheetfor how to avoid authentication vulnerabilities.

OWASP - 2010

Finding Direct Object Reference Flaws –Testing?

- 17

https://www.onlinebank.com/user?acct=6065

OWASP - 2010

Finding Direct Object Reference Flaws –Code Review?

- 18

String query = "SELECT account_balance FROM accounts WHERE account_number = ?";PreparedStatement pstmt = connection.prepareStatement( query, … );pstmt.setInt( 1, account );

File fileToRetrieve = new File(usersuppliedfilename));

@Secured public Plan retrievePlan(PlanId pidId) {

… }

Is account authorized for this user? Maybe it was validated earlier.

String query = "SELECT account_balance FROM accounts WHERE account_number = ? and account_owner = ?";PreparedStatement pstmt = connection.prepareStatement( query, … );pstmt.setInt( 1, account );pstmt.setInt( 2, currentUser );

This time its clear that only the owner can retrieve this data.

Was this filename validated? Is the user authorized for this file?

@Secured invokes an authorization check that ensures the user is authorized for this plan. Is this annotation on every method thatrequires this protection? Its easy to check the code.

OWASP - 2010

Direct Object Reference Results

- 19

Uncover Flaws

Locate Flaws

Verify Lack of Flaws

Overall Winner

Code Review 4 5 4App Pen Testing

4 3 2

OWASP - 2010

Finding CSRF Flaws – Testing?

POST BODY:givenName=David&familyName=Wichers&nickname=DavidW&gender=M&crumb=B4k22G2EReX

crumb=g4bxSkzUEY2 Same form after logout and log back in.

http://www.somebank.com/atmwithdraw?acct=1756403&amt=300&csrf=h4Ul43xQ20

CSRF Testing is pretty darn easy. But if there are LOTS of pages and functions, it can be tedious to find and test them all.

- 20

OWASP - 2010

Finding CSRF Flaws – Code Review?

public void doFilter(ServletRequest request, ServletResponse response, FilterChain chain) throws IOException, ServletException { boolean requestAllowed = false; // Code is here to verify request requires CSRF Check // If not, set requestAllowed = true; // If so, check the following: String csrfReqToken = request.getParameter("tokenname"); String userCSRFToken =

(String) request.getSession().get("userCSRFtoken"); if (userCSRFToken.equals(csrfReqToken)) requestAllowed = true; However they verify them, they are pretty easy to check in code too.

However, in code, if this is checked in a chokepoint, like a filter, then its very easy to verify the entire app quickly, rather than having to check EVERY page on the site.

- 21

OWASP - 2010

CSRF Results

- 22

Uncover Flaws

Locate Flaws

Verify Lack of Flaws

Overall Winner

Code Review 4 5 4App Pen Testing

4 4 2

Check out the: http://www.owasp.org/index.php/Cross-Site_Request_Forgery_(CSRF)_Prevention_Cheat_Sheetfor how to avoid CSRF vulnerabilities.

OWASP - 2010

Finding Config Flaws – Testing?

Config, to me, involves configuration of OS, Web Server, App Server, DB Server, frameworks, etc.

If testers have access to config, then advantage is clearly with testers, but they usually don’t. Code reviewers frequently don’t either actually. Certainly not all of it.

Automated testing is frequently the most effective technique for detecting most product level config flaws.

Reviewing the actual configuration is extremely helpful too.

- 23

OWASP - 2010

Finding Config Flaws – Code Review?

Gathering all the config information for a reviewer is typically challenging.

Understanding what all the different configuration information means is also challenging.

Reviewing config is very effective when you understand what it all means.

But clearly, to me, advantage is with the testers here.

- 24

OWASP - 2010

Config Results

- 25

Uncover Flaws

Locate Flaws

Verify Lack of Flaws

Overall Winner

Code Review 2 5 2App Pen Testing

4 5 4

OWASP - 2010

Finding URL Based Access Control Flaws – Testing?To me this top 10 category is function level access control, not just URLs.

http://www.thebank.com/admin/getsitestats http://www.thebank.com/bankemployee/getacctbalance?cust=129843 http://www.thebank.com/bankemployee/main?button=deletecust&cust=23489

http://www.thebank.com/admin/main

POST BODY:button=setuserpwd&cust=84375&pwd1=ThEsecret1&pwd2=ThEsecret1&Token=m7G34lkw9Xr

- 26

Is each of these protected from unauthorized access?Force access from other user roles to verify.

Pretty easy to test.

But: Even if 1 role is prevented, are ALL unauthorized roles prevented?

OWASP - 2010

Finding URL Based Access Control Flaws – Code Review?@Secured public int getAccountBalance(int customerId) throws ServiceException { …

public void transferFunds(int srcAcct, int destAcct, int amount) throws ServiceException { …

@Secured public int createNewAccount(int customerId) throws ServiceException { …

- - - - - - - - - App 2 - - - - - - - public void addUser(String username, int customerId) throws ServiceException { if (isAuthorized((Integer) session.get("currentUser")) {

- 27

?

Checking a single function is about the same with code review.

Finding whether the controls are in place everywhere they are needed, and if they are correct is MUCH easier with code review.

OWASP - 2010

URL Based Access Control Results

- 28

Uncover Flaws

Locate Flaws

Verify Lack of Flaws

Overall Winner

Code Review 5 5 4App Pen Testing

4 4 2

OWASP - 2010

Finding Crypto Storage Flaws – Testing?

Practically all crypto storage flaws are very difficult to test for as they are generally invisible to the user Algorithm selected Strength of key Key protection Is data encrypted? Etc.

Not much can be done with straight pen testing Unless you can break into the server But, you may be able to get the configuration files or

even DB. If so, most of this can be reviewed without looking at the code. - 29

OWASP - 2010

Finding Crypto Storage Flaws – Code Review?

Typically very straightforward with code reviewAnd you typically also have access to the

config

- 30

Requirement* Pen Testing

Code Review

Strong algorithms are usedStrong keys are generatedCrypto keys and passwords stored safelyCrypto failures are loggedSources of randomness are truly randomSensitive data encrypted at restCerts used are validated to ensure trusted, not expired, not revoked, matches intended useDigital signatures are properly validated

?* - Requirements selected from OWASP ASVS

OWASP - 2010

Crypto Storage Results

- 31

Uncover Flaws

Locate Flaws

Verify Lack of Flaws

Overall Winner

Code Review 5 5 5App Pen Testing

1 1 1

Check out the: http://www.owasp.org/index.php/Cryptographic_Storage_Cheat_Sheetfor how to avoid cryptographic storage vulnerabilities.

OWASP - 2010

Finding Transport Security Flaws – Testing? Most items are easier to find via testing

Is SSL/TLS used for all network connections transmitting sensitive data?

Is the Server cert valid for this domain, not expired, not revoked, etc.?

Does the server support weak protocols? Does the server support weak ciphers?

- 32

OWASP - 2010

Finding Transport Security Flaws – Code Review?

But some may be easier via code reviewVerifying back end connections are also

properly secured.Verifying that all ‘client’ side SSL connections

generated by the application perform all the required certificate verification checks (e.g., trusted, valid, expired, revoked). And back end server side connections too.

Verifying that any such SSL negotiation failures are properly logged.

Note: If this is all done by products or libraries then this is just hard! - 33

OWASP - 2010

Transport Security Results

- 34

And the: http://www.owasp.org/index.php/Transport_Layer_Protection_Cheat_Sheetfor how to avoid transport layer security vulnerabilities.

See www.ssllabs.com for a battery of free and commercial SSL automated tests.

Uncover Flaws

Locate Flaws

Verify Lack of Flaws

Overall Winner

Code Review 2 3 2App Pen Testing

4 4 4

OWASP - 2010

Finding Redirect/Forward Flaws – Testing?

- 35

http://www.irs.gov/taxrefund/claim.jsp?year=2006& … &dest=www.evilsite.com

http://www.irs.gov/taxrefund/checkValid.jsp

POST BODY:year=2007&SSN=534684375&lastname=wichers&function=/taxrefund/approved.jsp

Finding the Redirect / Forward parameters can be a pain.Testing redirect parameters is really EASY.Testing for an unvalidated forward can be HARD.

OWASP - 2010

Finding Redirect/Forward Flaws – Code Review?public void doPost( HttpServletRequest request, HttpServletResponse response) {

try {String target = request.getParameter( "dest" ) );

...request.getRequestDispatcher( target ).forward(request, response);

}catch ( ...

...response.sendRedirect( target );...

- 36

Was target validated?

Finding redirects and forward in code is EASY.Verifying they are safe or not is pretty easy too.

OWASP - 2010

Redirect/Forward Results

- 37

Uncover Flaws

Locate Flaws

Verify Lack of Flaws

Overall Winner

Code Review 5 5 5App Pen Testing

3 3 2

OWASP - 2010

Results Summary

- 38

Uncover Flaws

Locate Flaws

Lack of Flaws

Advantage

# of End

PointsCode Rev

Pen Test

Code

Rev

Pen Test

Code

Rev

Pen Test

Injection 5 2 5 2 5 1 Code Review

Across App

XSS 4 3 4 2 4 2 Code Review

Across App

AuthN 3 4 4 3 3 3 Tie? 1Obj Ref 4 4 5 3 4 2 Code

ReviewAcross

AppCSRF 4 4 5 4 4 2 Code

ReviewAcross

AppConfig 2 4 5 5 2 4 Test A fewURL Access 5 4 5 4 4 2 Code

ReviewAcross

AppCrypto Storage 5 1 5 1 5 1 Code

Review A few

Transport

2 4 3 4 2 4 Test A few

Redirect 5 3 5 3 5 2 Code Review

A few

TOTALS 39 34 46 31 38 23

OWASP - 2010

Revisiting Our Initial Questions

- 39

To find them?• Result: Similar, but slight edge to code review.

• It’s a MYTH that code review is way more expensive. If you have people with the right skills, its actually faster AND more effective.

To find exactly where they are in the app?• Result: Clear advantage to code review

To verify we don’t have them?• Result: Also clearly the advantage goes to code review

OWASP - 2010

Some Success Stories

Large Financial Organization Working to Stamp Out XSS (30M lines of code)Manual code review to find XSS flawsRegex patterns developed to find similar flawsPatterns run and validated to find 1000+ XSS

flaws in subset of this codebase in 2 weeks

Large ISPManual code review to find XSS and SQL

InjectionHundred+ of flaws found and fixed in 2 man

weeks - 40

OWASP - 2010

Conclusion

The Ideal: Do both, … but sometimes you can’t.You should ALWAYS try.

Knowing where the vulns are in the code allows you to provide much better remediation guidance. Try to always report where in the code the vulnerability

is And how to fix the code

The advantage of code review grows significantly with size of the application size of the application portfolio the level of rigor of the assessment - 41