Overcoming Security Challenges in DevOps

Taking a DevOps Approach to SecurityAlert Logic & Chef discuss overcoming security challenges in DevOps

Yvo Van Doorn

Manager, Solutions Architects

Chef Software

Justin Criswell

Cloud Solutions Architect

Alert Logic

Speaker Introduction

• Yvo Van Doorn• Manager, Solutions Architects• Chef Software, Inc.• @yvov• www.linkedin.com/in/yvovandoorn

• Justin Criswell• Cloud Solutions Architect• Alert Logic, Inc.• @JustinCriswell5• www.linkedin.com/in/justincriswell

Session Overview

More organizations are embracing DevOps to realize compelling business benefits such as faster yet safer feature release cadence, increased

application stability, and rapid response to shifting market conditions. However, security and compliance monitoring tools have not kept up and

often represent the single largest remaining hurdle to Continuous Delivery.

Topics covered in this session:•How DevOps Improves your Security Posture•Overcoming Challenges in DevOps Environments

DevOps Improves Security Posture

Mark Burgess, Father of Configuration ManagementAuthor of “In Search of Certainty”

Configuration Management

“We suffer sometimes from the hubris of believing that control is a matter of applying sufficient force, or a sufficiently detailed set of instructions.”

Automation and Convergent Infrastructure

“A system's desired configuration state can be said to be defined by fixed points. Most configuration management systems (e.g.: CFEngine, Chef, Puppet, PowerShell DSC) are based on this idea: they provide means to declare what must happen instead of requiring imperative workflows that prescribe what to do.”


The Birth of DevOps

Driving toward Immutable Infrastructure

"This is what I call disposable computing. Throw away a broken process rather than trying to fix it. Machines can be made expendable as long as the total software is designed for it. Not much of it is today, but we´re getting there. Nature shows that this is a good way of scaling services."


Infrastructure as Code

• Converge on a regular interval• Configuration Management is idempotent• All persistent changes must be in source control• Manual intervention discouraged• Out-of-band changes will be lost

Security & Compliance Implications

Continuous Delivery Patterns

Test Driven InfrastructureCosts Of Finding

Bugs in ProductionSecurity Issues

Continuous Security

Auditing & Compliance

End-to-EndVisibility

Disaster Recovery &

Business Continuity

Remediation & Fast

Resolution

Continuous Detection/Protection

Automated Configuration

& Scaling

Security Posture

v

OVERCOMING SECURITY CHALLENGES

Overcoming Security Challenges

• Challenges for Security Technology & Practice Today- AWS Shared Responsibility Model- Challenges remain for Customers

• Leveraging DevOps for Security- Best-Practices for blending DevOps with Security

• Toward Software-Defined Security- Embracing new reality of AWS Cloud Infrastructure

AWS Shared Responsibility Model

CustomerResponsibili

ty

FoundationServices

Hosts

• Logical network segmentation• Perimeter security services• External DDoS, spoofing, and scanning

prevented

• Hardened hypervisor• System image library• Root access for customer

• Access management• Patch management• Configuration hardening• Security monitoring• Log analysis

Apps

• Secure coding and best practices

• Software and virtual patching• Configuration management

• Access management• Application level attack

monitoring

• Network threat detection

• Security monitoringNetworks

CloudService Provider

Responsibility

Compute Storage

DB Network

2014: Security Top Cloud Pain PointSecurity

Pricing/Budget/Cost

Human Change Management

Security of Data, Control of Data Locality, Sovereignty

Compliance

Migration/Integration

Internal Resources/Expertise

Management

Lack of Internal Process

Vendor/Provider Issues

Organizational Challenges

Contractual/Legal Issues

Service Reliability/Availability

Network

Lack of Standards 4%

31%

17%

12%

11%

11%10%

9%

8%7.4%

7%

7%

7%

5%

5%

Application Security Technology Challenges

Network Changes

Host Identity Auto-Scaling

Application Security Technology Challenges

Traditional Security/Compliance is Slow

Mature DevOps Velocity is Fast

Security Practice does not Keep Up

Security at Odds with DevOps Velocity

InfoSec Ends Up Being Marginalized

“The problem for the security person who is used to turning around security reviews in a month or two weeks is they're just being shoved out of the game. There's no way with how Infosec is currently configured that they can keep up with that. So, Infosec gets all the complaints about being marginalized and getting in the way of doing what needs getting done.”

Gene Kim, former CTO of TripwireAuthor of “The Phoenix Project: A Novel About IT, DevOps”& “Helping Your Business Win”

Integrating Security with DevOps

• Leveraging DevOps practice for better Security- Prevent attack vectors with Immutable Systems

-Manage vulnerabilities with Base Images and Configuration Management

- Robust Auditing and Centralized Log Collection- Adopt strategy of Phoenix Upgrades- Embrace end-to-end Continuous Deployment

Manage Vulnerabilities with Base Images

Manage Vulnerabilities• Conduct normal vulnerability scanning• Identify Vulnerabilities that exist in Base

Images versus Application specific packages• Remediate at appropriate level as part of

Continuous Delivery process• Start with Hardened “secure by default” base

Results• Less work, done more reliably• Patching fits naturally into Phoenix Upgrades• Continuous Delivery allow frequent scanning

in test environments to have real value• Fixes potential vulnerabilities systematically

Prevent Attacks with Immutable Systems

Build secure base images that are representative of your infrastructure system base

Design file system layout to separate code from data, and

lock down to minimum required permissions. Should

expand to network as well

Leverage SANS Checklist and CIS Benchmark resources for

system level security best practices and guidance

Leverage configuration management tools to

standardized all software versions and configurations

Design SecureImmutable

Infrastructure

Centralize Robust Auditing & Logging

# This file contains the auditctl rules that are loaded# whenever the audit daemon is started via the initscripts.# The rules are simply the parameters that would be passed# to auditctl.

# First rule - delete all-D

# Increase the buffers to survive stress events.# Make this bigger for busy systems-b 1024

-a exit,always -S unlink -S rmdir-a exit,always -S stime.*……

Implement Local Auditing

#Sample syslog-ng configuration#Lots of configuration required

........

# Send *ALL* System Logs to Log Appliancedestination df_log_appliance_forward { tcp("my-log-appliance" port(514));};

log { source(s_all); destination(df_log_appliance_forward);};

Collect Important Logs

Centralize Log Collection for Search and Filtering

Adopt Strategy of Phoenix Upgrades

Embrace Phoenix Upgrades• Stand up new instances, don’t upgrade• Route traffic between old and new instances• Rich service metrics and automate rollback• Advanced routing can enable selective rollout

Results• Creates evergreen systems, avoiding

configuration drift and technical debt• Enforces refresh of all system components as

complete artifact, tested as a holistic system • Greatly reduces security risks when combine

with immutable instances and configuration management

Embrace Complete Continuous Deployment

End-to-End Continuous Deployment• Configuration Management (Chef), • Standardized Environment Images (Packer)• Environment/Subsystem Orchestration Layer • Production-like environments in Dev & Test

must include all components

Results• Hardened base images become part of the

standard development process• Continuous validation of secure configuration• Holistic system validation & testing

#Sample Alert Logic Chef NodeDef{ "name": "cloud-api-node", "versions": { "1.6.0": { "vm_type": "squeeze64", ], "install_phase": { "run_list": [ "[email protected]", ] },….

Leverage Configuration Management

Leverage Standardized Environment Images

Build an Orchestration Layer

#Sample Packer Configuration{"builders": [{ "type": "amazon-ebs", "access_key": "{{user `aws_access`}}", "secret_key": "{{user `aws_secret`}}", "region": "us-east-1", "source_ami": "ami-de0d9eb7", "instance_type": "t1.micro", "ssh_username": "ubuntu", "ami_name": "packer-ex {{timestamp}}" }]

Moving to Software Defined Security

• Significant Opportunity remains in front of us• Security remains a lot of work• Does not fit naturally into DevOps Roles

• AWS ready to Accelerate Security Technology- Leverage end-to-end visibility available- Protect automatically with real-time configuration- Transform periodic assessment into real-time automated responses

Leverage End-to-End Visibility

Leverage Detailed Visibility of AWS• AWS describe endpoints reveals environment• Cloud Trail shows incremental changes• Long-term AWS Config unifies this Information

Results• Enables visualization of entire environment• Can reason directly about exploitability• View of environment always up-to-date• No need to coordinate application changes

with security infrastructure

Protect with Automatic Configuration

Automated Protection• Add security infrastructure on-demand to

cover changes to environment• Autoscale that infrastructure based on traffic

seen in that environment environment

Results• Focus on application deployments and not

configuration of security infrastructure• Implements least-privilege configurations• Seamless experience that “just works” with

Security fitting naturally into DevOps practice

Transform Assessment to be Real-Time

Real-Time Protection• Assessment activities should become

continuous• Continuous Assessment “just happens” as

deployments occur• Zero-day coverage delivered automatically

Results• No more periodic assessment cycle• Security posture of environment available

immediately and automatically after deployments

• Security matches DevOps Velocity!

Contact Us

Justin CriswellCloud Solutions ArchitectAlert [email protected] @JustinCriswell5

Yvo Van DoornManager, Solutions ArchitectsChef Software, [email protected]@yvov

Thank you!

mailto:[email protected]

mailto:[email protected]