Upload
kevin-fealey
View
964
Download
8
Embed Size (px)
Citation preview
Aspect Security | 9175 Guilford Road, Suite 300 | Columbia, MD 21046 | www.aspectsecurity.com
Static Analysis Security Testing for Dummies… and You
OWASP LASCONAustin, TXOct 23, 2015
Application security that just works
©2015 Aspect Security. All Rights Reserved 2
WARNING
50 slides incoming.
We’ll be moving fast!
Come ask questions the end.
Application security that just works
©2015 Aspect Security. All Rights Reserved 3
ABOUT ME
Kevin FealeyPrincipal Consultant & Practice Lead,
Automation & Integration Services7 years AppSec experience, 2nd LASCON
@secfealzKey Interests:
• Process efficiency/effectiveness (DevOps, SecDev)• Open Source and Commercial Tools• Understanding your SDLC & security processes
Application security that just works
©2015 Aspect Security. All Rights Reserved 4
ABOUT YOU
Does your organization use SAST?• Commercial• Open-Source
How do you use SAST?• All manual scans• Automated in some way
‒ CI/IDE/Other
Application security that just works
©2015 Aspect Security. All Rights Reserved 5
WHY DO WE NEED TOOLS?
Development
Production
Security
• Manual security activities are bottlenecks for new features and bug fixes to get to production
• With the push for faster deployments, the bottleneck is tightening
• Business goals trump security needs• Lots of “Risk-based decisions”
• Security has to find a way to keep up (and catch up)
Application security that just works
©2015 Aspect Security. All Rights Reserved 6
APPSEC TOOLBELT
RAST, WAF, Manual Code Review, Pen Testing, Threat Models, Architecture Reviews,…
©2015 Aspect Security. All Rights Reserved 7
STATIC APPLICATION SECURITY TESTING TOOLS
Application security that just works
©2015 Aspect Security. All Rights Reserved 8
STATIC APPLICATION SECURITY TESTING TOOLS
Key benefit and drawback
Application security that just works
©2015 Aspect Security. All Rights Reserved 9
QUESTION OF THE DAY:
Q: Why do SAST tools have such a bad reputation?• Vendors are not educating their customers
‒ Tool vendors are focused on selling licenses, rather than providing guidance for a successful rollout (ie. implementation services)
‒ Most tools are marketed as low-touch silver-bullets› Leads to running a tool out-of-the-box, with no vulnerability
management program, education of developers, or communication plan
• Most current security tools are complex and noisy, but continue to evolve ‒ They are still better than manual assessments for scale, if tuned and
integrated properly
Application security that just works
©2015 Aspect Security. All Rights Reserved 10
AGENDA
1. Common SAST Questions1. Why does SAST scanning take so long?2. Why does SAST find so many false-positives?
2. Tailoring SAST, PMD as an example3. SAST Integration Scenarios
Application security that just works
©2015 Aspect Security. All Rights Reserved 11
BACKGROUND
Focus on statically typed & compiled languages• SAST on dynamically typed languages (Python, Ruby,
etc.) is much more difficult‒ Fewer tools available‒ Tools that exist are not as powerful
My bias is toward Java• Similar process for other languages
Generalizing how SAST works• Some tools may work differently
Application security that just works
©2015 Aspect Security. All Rights Reserved 12
SAST AT THE HIGHEST LEVEL
Source/
Byte Code
Model
Extraction
Intermediate Representati
ons
Analysis
Results
?Proprietary
Models
Application security that just works
©2015 Aspect Security. All Rights Reserved 13
SAST AT THE HIGHEST LEVEL
Source/
Byte Code
Model
Extraction
Intermediate Representati
ons
Analysis
Results
?Proprietary
Models
Analysis is performed on intermediate representations, not on source/byte code
Application security that just works
©2015 Aspect Security. All Rights Reserved 14
SAST AT THE HIGHEST LEVEL
Source/
Byte Code
Model
Extraction
Intermediate Representati
ons
Analysis
Results
Parser/Tokenizer per language Common rule/analysis engine
New Language Support = Translator + Rules
Application security that just works
©2015 Aspect Security. All Rights Reserved 15
HIGH LEVEL SAST PROCESS
Compilation
Scanner Model
Extraction
Pattern Matching
Control Flow
AnalysisData Flow Analysis
Translation Scan
Performed by SAST Tool
*The activities listed are not performed by all SAST tools. Some tools perform a subset of these – others perform additional activities.
find “password” in*.properties
out.write(request.
getParameter(“param”));
Database connection not
closed in ‘finally’ block
Syntactic / Semantic Checking
Application security that just works
©2015 Aspect Security. All Rights Reserved 16
OTHER TYPES OF ANALYZERS
Semantic •Unsafe function invocations
Structural •Pervasive issues that impact large portions of the application, like dead code and loggers not declared as static
Configuration •Checks configuration files for reasonable timeouts, etc.
Application security that just works
©2015 Aspect Security. All Rights Reserved 17
SAST WORKFLOW WITH JAVA
Compile Java Files
Compile JSPs
Perform Framew
ork Analysis
Pattern Matchi
ngAnalysis
Generate
Results
Translation Scan
*The activities listed are not performed by all SAST tools. Some tools perform a subset of these – others perform additional activities.
Compile-time vs Runtime issues
Dependency issues
Only supported
frameworks
Application security that just works
©2015 Aspect Security. All Rights Reserved 18
FRAMEWORK HANDLING (APPSCAN SOURCE)
©2015 Aspect Security. All Rights Reserved 19
TYPES OF ANALYSIS
Application security that just works
©2015 Aspect Security. All Rights Reserved 20
PATTERN MATCHING (APPSCAN SOURCE)
Accessible through the Security Analyst UI Modify/Create rule-sets (ex. Java, SQL) Or create new scan rules
Application security that just works
©2015 Aspect Security. All Rights Reserved 21
CONTROL FLOW ANALYSIS
XMLReader parser = XMLReaderFactory.createXMLReader("org.apache.xerces.parsers.SAXParser"); parser.parse(input);//Prevent XXE attacksparser.setFeature("http://apache.org/xml/features/disallow-doctype-decl",true);
Application security that just works
©2015 Aspect Security. All Rights Reserved 22
DATA FLOW ANALYSIS
Source Taint
Sink
Data Flow / Trace Nodes
Tainted Data
Vulnerability?
Application security that just works
©2015 Aspect Security. All Rights Reserved 23
TAINT ANALYSIS
Untrusted Source Taint
Sink
a.tainted = true
Vulnerability!
b.tainted = truec.tainted = trued.tainted = true
e.tainted = true
Application security that just works
©2015 Aspect Security. All Rights Reserved 24
STRING ANALYSIS (APPROXIMATED)
Source Taint
Sink
a.taint = a;
Vulnerability!
b.taint = b.substring(0, b.length()-4)
…
Application security that just works
©2015 Aspect Security. All Rights Reserved 25
LOST SINKS
Intermediate Representations (IRs) are not produced for dependencies
Lost Sink
Most SAST tools do not know what happens in externalMethodCall()Some tools allow for Lost Sink Resolution
• Is tainted data returned from this method?
Application security that just works
©2015 Aspect Security. All Rights Reserved 26
EVERY PATH IS CHECKED
•SAST does not know what will happen at runtime.•How many function calls happen when bar() is invoked? bar2()?
• SAST will check both.
©2015 Aspect Security. All Rights Reserved 27
ANSWER SUMMARY
Application security that just works
©2015 Aspect Security. All Rights Reserved 28
WHY DOES SAST SCANNING TAKE SO LONG?
1. The entire application is compiled2. The compiled code is translated to various models
1. Every line of code is translated to the appropriate IRs
3. Scanning is a multi-step process:1. Regex search finds low hanging fruit2. Semantic (context-aware) search finds use of dangerous functions3. The order of execution for every potential function is checked
(control flow analysis)4. [Nearly] Every possible path in the application is searched for
known dangerous patterns (data flow analysis)5. Findings are sorted/categorized to produce your report
Application security that just works
©2015 Aspect Security. All Rights Reserved 29
WHY DOES SAST FIND SO MANY FALSE-POSITIVES?
•Every potential data flow and control flow path is checked
• SAST tools do not know which logic will be executed at runtime• Paths that will never be executed will be checked
•SAST tools do not know which sources your business trusts
• Data from a database may be populated by an administrator, a malicious user, or another well-intentioned application
•Many SAST tools are architected with post-processing in mind, whether manual or automated
Application security that just works
©2015 Aspect Security. All Rights Reserved 30
INTERPROCEDURAL VS INTRAPROCEDURAL
Interprocedural:• Whole program analysis• Tracks variables across objects/procedures/functions• Most commercial tools
Intraprocedural:• Single procedure analysis• Most open-source tools• Most dynamically-typed languages scanners
©2015 Aspect Security. All Rights Reserved 31
PMD
Application security that just works
©2015 Aspect Security. All Rights Reserved 32
WHAT IS PMD?
PMD is a source code analyzer. It finds common programming flaws like unused variables, empty catch blocks, unnecessary object creation, and so forth. It supports Java, JavaScript, PLSQL, Apache Velocity, XML, XSL.
Additionally it includes CPD, the copy-paste-detector. CPD finds duplicated code in Java, C, C++, C#, PHP, Ruby, Fortran, JavaScript, PLSQL, Apache Velocity, Ruby, Scala, Objective C, Matlab, Python, Go.
-https://pmd.github.io/
Application security that just works
©2015 Aspect Security. All Rights Reserved 33
WHY DID I CHOOSE PMD?
PMD was not written to find security issues.
Many non-security tools that you are already using can be re-purposed or extended to provide security value.
QA Team
Application security that just works
©2015 Aspect Security. All Rights Reserved 34
SHOUT OUT
Only PMD security-focused ruleset I’ve found: https://github.com/GDSSecurity/GDS-PMD-Security-Rules
Application security that just works
©2015 Aspect Security. All Rights Reserved 35
PMD RULE ASSUMPTIONS
Assumptions:• Organization builds applications with Spring• Spring annotations are used to specify allowed HTTP
request method ‒ @RequestMapping(method=RequestMethod.POST)
• Functions lacking this annotation allow any HTTP method
• Security policy states that at least one HTTP method must be explicitly set for each entry point. Only HTTP GET and POST are allowed.
©2015 Aspect Security. All Rights Reserved 36
PMD: STEP 1Writing test cases
Application security that just works
©2015 Aspect Security. All Rights Reserved 37
SPRING MVC CONTROLLERS
Test Case #1: Correct Method
Test Case #2: Incorrect Method
Test Case #3: Missing Method
Application security that just works
©2015 Aspect Security. All Rights Reserved 38
SPRING MVC CONTROLLERS
Test Case #4: Correct Multiple Methods
Test Case #5: Incorrect Multiple Methods
©2015 Aspect Security. All Rights Reserved 39
PMD: STEP 2Generate and Analyze the AST
(Reverse-Engineering)
Application security that just works
©2015 Aspect Security. All Rights Reserved 40
WHAT ARE WE LOOKING FOR?
“RequestMapping” variable as a “Name” object
Child of “NormalAnnotation”
“method” variable
Type: “MemberValuePair”
Value we are looking for as a “Name” object
Child of “PrimaryPrefix”
©2015 Aspect Security. All Rights Reserved 41
PMD: STEP 3Write the Rule
Application security that just works
©2015 Aspect Security. All Rights Reserved 42
WRITING OUR RULE
Application security that just works
©2015 Aspect Security. All Rights Reserved 43
FINAL RESULT
Application security that just works
©2015 Aspect Security. All Rights Reserved 44
PMD REPORT
Not listed:CorrectRequestMethond.javaMultipleCorrectRequestMethods.java
Application security that just works
©2015 Aspect Security. All Rights Reserved 45
ALTERNATIVELY – XPATH ONLY
//NormalAnnotation /Name[@Image="RequestMapping"] [count(..//MemberValuePair//PrimaryPrefix/Name[@Image="RequestMethod.POST"]) < 1]
Doesn’t handle multiple methods – that was too hard.
©2015 Aspect Security. All Rights Reserved 46
WHERE DOES SAST FIT IN?
Application security that just works
©2015 Aspect Security. All Rights Reserved 47
SAST TIPS
SAST is meaningless without vulnerability management and remediation
• Any tool is only a part of an appsec program‒ People, Process, Technology
• Do not scale until you can have vetted processes• Developers can’t remediate effectively without support
SAST belongs in CI• Does not need to be run on every build
‒ Scan frequency based on application risk (nightly/weekly?)• Commercial SAST != developer tool
‒ Running of tool should be transparent to devs
Developers should never see false positives• Results should always be triaged/filtered
‒ False positives cause extra work for developers‒ False positives reduce confidence in the process
Application security that just works
©2015 Aspect Security. All Rights Reserved 48
MORE TIPS
Alternatives to vendor-supplied consoles• SonarQube, ThreadFix, GRC tools, etc.
Define continuous improvement for SAST• Regular modification of filters and rules• Tool configuration is an opportunity to strengthen
secdev
Application security that just works
©2015 Aspect Security. All Rights Reserved 49
ENTERPRISE ROLLOUT OF APPSCAN SOURCE: STRATEGY
Application PortfolioLess CriticalMore Critical
Cov
erag
e / A
ssur
ance
Scan
ScanScan
Full
Sca
n/R
evie
w
Remediation Guidance
Incr
ease
Cov
erag
e Reduce R
isk
• More time to review critical applications• More time to find and fix complex issues
©2015 Aspect Security. All Rights Reserved 51
APPENDIX
Application security that just works
©2015 Aspect Security. All Rights Reserved 52
OWASP BENCHMARK
As of 10/08/2015
• G: Open-Source SAST average (16.40%)
G
Application security that just works
©2015 Aspect Security. All Rights Reserved 53
TOOLS IMPROVE!
Results from September, 2015 – October, 2015
Application security that just works
©2015 Aspect Security. All Rights Reserved 54
COMMERCIAL SAST STRENGTHS AND WEAKNESSES
As of 10/08/2015
Application security that just works
©2015 Aspect Security. All Rights Reserved 55
VERY IMPORTANT
• Remediation requires more than just tools• Effective vulnerability management program• Training• Developer support
• Never deliver false-positives to developers
Application security that just works
©2015 Aspect Security. All Rights Reserved 56
COMMERCIAL SAST TOOLS TESTED
Checkmarx CxSASTCoverity Code Advisor (On-Demand and stand-alone version)HP Fortify (On-Demand and stand-alone versions)IBM AppScan SourceParasoft JtestVeracode SAST
Application security that just works
©2015 Aspect Security. All Rights Reserved 57
SCANNER MODEL EXTRACTION
Intermediate Representations
• For commercial tools, usually proprietary formats
• Common formats (per vendor) regardless of language
• Some tools build models for each file (intraprocedural); others for the whole application (interprocedural)
Modified AST, NST, Framework modeling, etc.
Application security that just works
©2015 Aspect Security. All Rights Reserved 58
SENSOR INTEGRATION FRAMEWORK
• Configure your Sensors• Configure your Publishers• Build job and see your results
https://github.com/aspectsecurity/sensor-integration-framework