PROACTIVE SECURITY APPSEC CASE STUDYANDY HOERNECKE

HELOAndy HoerneckeApplication Security Engineer AppSec, Automation, Data Visualization

What We Will Cover• Background on Netflix

• Our Security Philosophy

• Walkthrough of Our Approach to AppSec

Terminology• Define technology terms:

• Application

• Instance

• ELB (Load Balance)

• AMI

• Security Groups

Netflix Primer• 100's of Developers

• Over 1,000 applications

• Hundreds of production pushes a day

• Over 50k instances

• Very Pro Open Source

• No Security Gates!

Continuous Delivery• Fast, Automated Deployment

• Immutable Platform

• Low Friction

The Challenge• Provide security in the environment described:

• No security gates

• Production Changes Rapidly

• Multiple Codes Bases (A/B Testing)

• Many Developers vs. 5 Member AppSec Team

How?

Act as enablers

not gatekeepers

Application developers are responsible for the security of their application.

Security is as important as: • functionality

• performance • availability

• scalability

Create paved paths, that are secure by default

Proactive Security• Know your environment & weaknesses and work to improve

• Find problems early and address them

• Monitor for anomalies and be prepared to respond

• Collect meaningful data and use it to improve

• Simplify make security the easy path

• Reevaluate your approach

• Share what you learn with others

Implementing Proactive Security AppSec Case Study*

* Note: Talk discusses new version of software yet to be open sourced

Goals1. Understand your environment

2. Inject automated security controls

3. Tie environment and security together

Goal 1 Understand Your Environment

1. Know the components of your environment

2. Understand connections

3. Monitor for changes

Defining The Environment

• Applications that make up and support the Netflix experience 1. Accessibility (How, Where, Who) 2. Functionality 3. Ownership 4. Risk Level 5. Security

Where do Applications Come From?

• Binaries

• Appliances

• SaaS

• Internally Developed (Source Code)

Where do Applications Come From?

SCMDevelopers Build Bake Deploy

1. Developers push code to SCM

2. Built into a package

3. Combined with BaseAMI to form a machine image

4. Deployed as an EC2 Instance

SCMDevelopers Build Bake Deploy

Dependencies BaseAMI

Source Code Package Baked AMI EC2 Instance

SCMDevelopers Build Bake Deploy

EC2 Instance

Cluster

Application

Cluster Cluster

EC2 InstanceEC2 Instance

ELB DNS Name

SCMDevelopers Build Bake Deploy

EC2 Instance

Cluster

Application

Dependencies BaseAMI

Source Code Package Baked AMI

DNS Name ELB

Penguin Shortbread

Penguin Shortbread• Specialized Branch of Scumblr • Tracks Applications and all their associated

metadata • Repositories • Committers • DNS Names • BaseAMI Information • Dependencies • More!

Penguin Shortbread• Individual tasks for gathering different pieces of

metadata • Tasks for Spinnaker, Github, Stash, Jenkins, etc. • Easy to customize, maintain, etc.

• Searching and filtering based on any information stored on the application. • Examples:

What application uses sketchy.netflix.com?What repos does Andy Hoernecke contribute to?

While we're at it...• Collect information about how risky an

application is

• Calculate a risk score

• Determine which applications posed the great risk and make decisions based on this

Security Monkey• Monitor for changes in AWS environment

• Get alerts for important changes

• Integrations with Scumblr/Penguin Shortbread

Goal 1 Understand Your Environment

1. Know the components of your environment

2. Understand connections

3. Monitor for changes

Goal 2 Automated Security Controls

1. Select and run tools

2. Aggregate data

3. Take action

SCMDevelopers Build Bake Deploy

Systems

Github

Stash

OpenGrok

Information

Source Code

Commit History

Committer

Owner Info

Security Tools/Services Static Analysis

SCMDevelopers Build Bake Deploy

Systems

Jenkins

Information

Packaged Application

Dependency Info

Security Tools/Services Static Analysis, Dependency Checking

SCMDevelopers Build Bake Deploy

Systems

Spinnaker

Bakery

Animator

Information

OS/Version

Animation Date

BaseAMI Info

Security Tools/Services Host Analysis/Hardening

SCMDevelopers Build Bake Deploy

Systems

Spinnaker

DNS

Security Monkey

Information

Application Name

DNS Names Security Groups

Security Tools/Services Dynamic Scanning, Runtime Analysis, Penetration Testing

Dirty Laundry• Evolution of Scumblr

Scumblr 1.0

queryquery

Results

Results

URLs

Scumblr 2.0• Extended the model with Metadata

• Added: • Generic Tasks • Task Ordering/Grouping • Customizable Views • Events

New vs. Old• Scumblr 1.0 Tasks:

Search Google Search TwitterSearch Facebook

• Example Scumblr 2.0 Tasks:1. Get list of Stash Repos2. Run Brakeman on Rails Repos 3. Save the Results and Send out Notifications

Pulling it Together• Dirty Laundry integrates with all our security tools

• Can track results based on a repo, a DNS name, an API endpoint, etc.

• With Penguin Shortbread, can fit things together

Action• Enhanced the ability to track status

• Added standard way to store/action vulnerability data

• Workflowable provides easy mechanism to create JIRA tickets, send out notifications, etc.

Goal 2 Automated Security Controls

1. Select and run tools

2. Aggregate data

3. Take action

Goal 3 Tie Environment and Security Together

1. Understand vulnerabilities in context

2. Prioritize security services and remediation efforts

3. Enable linking security risks with their source

4. Identify weak links and look for improvements

Coming Soon

Open Source• Netflix Open Source

• Scumblr • Security Monkey • Penguin Shortbread (soon) • Spinnaker • Animator • More: https://netflix.github.io/

• Arachni www.arachni-scanner.com • Dependency Check https://www.owasp.org/index.php/

OWASP_Dependency_Check • FindSecBugs http://find-sec-bugs.github.io/ • Brakeman http://brakemanscanner.org/ • Bandit https://github.com/openstack/bandit

Thanks!