Continuous Security Testing

© 2016 by Ray Lai, Confidential and Proprietary© 2016 by Ray Lai, Confidential and Proprietary

Continuous Security TestingRay LaiDec 7, 2016 (Version 0.2)

© 2016 by Ray Lai, Confidential and Proprietary

SynopsisThis session addresses the technology challenges of continuous security testing to “deliver securely,” and discusses best practices and tooling based on first hand experience in both enterprise and startup environment.


The Problem

How to complete security testing whenever a build is initiated

Develop

Security testDeploy

ImpactUS$10,000 / bug fixedUS$4M data breach (2016) Damage to brand


Security Testing: Now and Then

People• Professional/consultants

Technology• Commercial tools

Process• Central governance• Testing before SW release

Before/Now Desired State

People• Engineering community

Technology• Open source tools• UI and API security testing

Process• Self-serve• Continuous: at build/deploy

time


How


What is “Continuous Security Testing”?

Continuous Security Testing = New Tools?


Continuous Security Testing: Point of View

App Source Codes

Objectives• Efficiency• Feedback loop for developers

Security Metrics

Automated Security Testing

Web App Vulnerability Scanning

Threat Modeling

Secure Build

Build PipelineInput Output

Stakeholders• DevOps• Developers• Quality engineers• Security

• Static code analysis• Vuln scanning• App assessment• Pen test• Risk assessment• Compliance• Operations review• Defect reports• Threat modeling

Challenges: false positives/noise, long build time


Continuous Security Testing: Paradigm Shift

Testing approach• Network vs app security• Front-end vs backend;

integrated test

Tools• Enterprise (integrated), On-

premise

Execution• Testing separated from build

pipeline

Old New

Testing approach• Testing how apps and

systems are connected

Tools• Cloud-based, per pay use• Single-purpose, open source

Execution • DevOps: Test in build phase


How to Deliver Securely?

Exercise 1

You have just completed functional and security testing for your flagship app and platform. However, a security researcher informs you that they can reset user password via SMS, and take over a user’s account.

Options1. Extend test coverage

(resources)2. Enforce pen test before

release (professionals)3. Educate engineers (training)

How would you do differently to deliver securely?



Exercise 2

A security researcher reports a Cross-site Scripting (XSS) issue in your portal, with XSS examples in existing static Web pages. Your dev team argues that XSS on static Web pages is not harmful, and any fix will take them at least 1 month.

Options1. Validate if this is a fire drill

(risk management, education)2. Fix any XSS defect

(resources)3. Enforce pen test before

release (professionals)




Exercise 3

Amazon AWS notifies your security team that your standalone test VM (AMI) has been compromised. While your app team argues that it is an isolated instance, AWS finds traces that it was used to access other AMI instances without permission, and also becomes a DDOS agent.

Options1. Introduce security controls

and security testing for cloud apps (technology, resources)

2. Enforce pen test before release (professionals)

3. Raise awareness (education)




Recommendation (Past) Continuous Security Testing

Exercise 1 (password reset)

Threat modelingTest for trusted boundaries (connected security)

In build pipeline, spin an app instance to test for end-to-end flow.

Exercise 2 (XSS)

Resolve any XSS defectsRaise awareness

Include UI testing for XSS in automated test suite (build process).

Exercise 3 (compromisedAMI)

Cloud security testing –production pen testCloud security controls

In build pipeline, spin an AMI instance to test AWS security policies and controls.

This is what I would do differently...


Recommendation• Continuous security testing is a journey (various maturity levels)

– Identify your low hanging fruits and priorities– Building community is an art (e.g. contents, outcome and appropriate

engagement)• Be open mind about changes in execution approach

– DevOps -> build pipeline for security– Gradual shift from commercial, integrated tools to open source tools

• Be prepared for the next big wave– Emerging security challenges in IoT and block chain technologies


Appendix: Best Practices


What to Prioritize

https://techcrunch.com/2015/01/22/security-for-startups-in-10-steps/

Prioritize

Security culture

Secure platform

Your Website is front-door

Secure coding

Security controls

Plan for failurePhysical security

Be open with public

Email with multi-factor auth

Threats


Which Tools to Use

brakerman

Static Code Analysis / Vulnerability Scanners

Cloud / Infrastructure Security

Data SecurityConnected Systems

WhatWeb Web Scannerw3afOSSEC Arachni

findBugsFlawFinder

PMD


What to TestMajor Security Vulnerabilities Static Code

AnalysisWeb App Scan Pen Test

OWASP Top 10 Partial Partial PartialHeartBleed-type SSL Issues No No PartialDOS, e.g. due to memory leakage Partial

(Memory leakage)

Partial Partial

DDOS, e.g. bot attack No No PartialCloud hosting security issues No Limited YesNoSQL database security issues No No PartialIoT security issues No Varies VariesBlockchain security issues No No Varies


What to Test (cont’d)OWASP Top 10 Static Code

AnalysisWeb App Scan Pen Test

A1 Injection X X X

A2 Broken Auth and Session Management X X

A3 Cross-site Scripting (XSS) Partial X X

A4 Insecure Direct Object References Partial X

A5 Security Misconfiguration X X

A6 Sensitive Data Exposure Partial X

A7 Missing Function Level Access Control X Partial

A8 Cross-Site Request Forgery (CSRF) X X

A9 Using Components with Known Vulnerabilities

Partial

A10 Unvalidated Redirects and Forwards Partial X


How to Execute (Build Pipeline for Security)Web App

ScanDeploy to Production

Security Assessment

Deploy to Production

Build Pipeline

Penetration Test

Runtime Container

Security Test Client

Web App Scan

Before

New

Separate, serial events

Continuous integration process to create temporary containers for security testing


Security Build Pipeline (Example)

App Container

Jenkins Server

App Container

Docker Swarm

App Container

Security Scanner (Docker)

Shared Data Container / Data Volume

Apps

Data

Pre-requisites• Apps deployed in docker containers• App data (synthetic or scrubbed

production data) in shared data containers

• Parallelize testing tasks in Jenkins

Jenkins Setup• Create docker swarm (apps and data)• Launch docker swarm• Create security test client (security

scanner)• Launch security test client• Post test results in shared data

container or data volume• Shut down all docker containers

Gauntlt, ClamAV, Brakerman, OWASP ZAP


What to Look For: Web App Security Testing

Session Management• HTTPOnly and SECURE

flags (cookies)

UI and App• XSS, CSRF

Data• Injection vulnerabilities

Traditional Approach What’s New

API• API token robustness;

playback and Man-in-the-middle attacks

Data theft• Data privacy issues (in transit

or at rest)

NoSQL database• Access control issues


What to Look For: Platform and Cloud

Platform Infrastructure• Network security, e.g. port

scan, mis-configuration• Default passwords and

settings

Identity management• SSO; access control

Compliance• Security certification, e.g. PCI

Traditional Approach What’s New

Platform Stack• Timely patches• Core security, e.g.

Heartbleed bug• DDOS

Data security• Storage security, e.g. S3

Compliance• Public cloud (e.g. AWS)

security policies and controls


How to Measure Success

Defect Density Threat Model Custom Risk

Scores

OWASP Top 10

WhiteHat WSI Security Index

Cenzic HARM Scores

HP Fortify Scores

Internal Metrics

Financial management app has a defect density of 15.23 (5M LOC)

25 attack vectors Biz risks: 5.2Tech risks: 7.9

Tech risks reflect # of total vulnerabilities from scanning/testing

Industry

Example:

For benchmarking


How Do I Get Trained

Traditional Channels What’s New

Security Certification• CEH, CISSP, etc.

Conferences• RSA, OWASP AppSec,

BlackHat, DEFCON, etc.

Product training• E.g. Fortify

Online Resources for the Impatient

• Youtube• Udemy, Coursera, etc.

Community• Meetups or user groups, e.g.

OWASP