View
156
Download
0
Category
Preview:
Citation preview
Using Hypervisor and Container Technology to
Increase Datacenter Security PostureLinuxCon North America 2016 – Toronto Canada
#whoami – Tim Mackey
Current roles: Senior Technical Evangelist; Occasional coder• Former XenServer Community Manager in Citrix Open Source
Business OfficeCool things I’ve done
• Designed laser communication systems• Early designer of retail self-checkout machines• Embedded special relativity algorithms into industrial control system
Find me• Twitter: @TimInTech ( https://twitter.com/TimInTech )• SlideShare: slideshare.net/TimMackey• LinkedIn: www.linkedin.com/in/mackeytim
Attacks are Big Business
In 2015, 89% of data breaches had a financial or espionage motive
Source: Verizon 2016 Data Breach Report
Anatomy of a New Attack
Potential Attack
Iterate
Test against platforms
Document
Don’t forget PR department!
Deploy
Knowledge is Key. Can You Keep Up?
glibc
BugReported
July 2015
Vuln: CVE-2015-7547: glibc getaddrinfo stack-based buffer overflow
Knowledge is Key. Can You Keep Up?
glibc
VulnIntroduce
d
May 2008
glibc
BugReported
July 2015
CVE-2015-7547
CVE Assigned
Feb 16-2016
Low Security RiskVuln: CVE-2015-7547: glibc getaddrinfo stack-based buffer overflow
Knowledge is Key. Can You Keep Up?
glibc
VulnIntroduce
d
May 2008
CVE-2015-7547
CVE Assigned
Feb 16-2016
glibc
BugReported
July 2015
NationalVulnerabilityDatabase
VulnPublished
Feb 18-2016
Moderate Security RiskLow Security RiskVuln: CVE-2015-7547: glibc getaddrinfo stack-
based buffer overflow
Knowledge is Key. Can You Keep Up?
glibc
VulnIntroduce
d
NationalVulnerabilityDatabase
VulnPublished
YouFind It
May 2008
CVE-2015-7547
CVE Assigned
Feb 16-2016 Feb 18-2016
glibc
BugReported
July 2015
Patches Available
YouFix It
Highest Security RiskModerate Security
RiskLow Security RiskVuln: CVE-2015-7547: glibc getaddrinfo stack-based buffer overflow
1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 20150
500
1000
1500
2000
2500
3000
3500Open Source Vulnerabilities Reported Per Year BDS-exclusive nvd
Reference: Black Duck Software KnowledgeBase, NVD
Vulnerability Disclosures Trending Upward
Primary goals• Protect against BIOS and firmware
attacks• Protect cryptographic host state• Ensure valid hypervisor kernel • Validate launch of critical VMs• Attest to hosts’ trust state
Implemented by• Intel Haswell and newer• Cryptographic hashes stored in TPM
Intel TXT – Trusted Execution Protection - Foundational
Intel SMAP – Supervisor Mode Access Protection
Operating System Kernel User Mode Applications
Read Application MemoryWrite Application Memory
Read Kernel MemoryWrite Kernel Memory
Read Kernel Memory
Write Kernel Memory
Read Application MemoryWrite Application Memory
mov r8d,2Bhmov ss,r8wmov r9d,dword ptr [r13+3Ch]mov dword ptr [rsp],r9dmov esp,dword ptr [r13+48h]jmp fword ptr [r14]mov r14,rspmov word ptr [rsp+8],23hmov word ptr [rsp+20h],2Bhmov r8d,dword ptr [r13+44h]and dword ptr [r13+44h],0FFFFFEFFhmov dword ptr [rsp+10h],r8dmov r8d,dword ptr [r13+48h]mov qword ptr [rsp+18h],r8mov r8d,dword ptr [r13+3Ch]mov qword ptr [rsp],r8
Intel PML- Page Modification Logging
Intel PML- Page Modification Logging
Who changed the world?What in the world changed?When did the change occur?
Why did the world change?
Intel EPT – Extended Page Tables
Page 0…
Page 13553Page
13554…
…Page 126Page 127
…Page
64589Page
64590Page
64591
Page 0…
Page 217…
Page 31289……
Page 78924…
Page 97586…
0→64589 13553→12713554→6459
1
App Memory
OS MemoryTLB CR3
Virtual Machine
126→31289127→0
64589→97586
64590→21764591→7892
4
Host Memory
EPT
Hypervisor
Hypervisor Memory Introspection – Enabled by EPT
Implementation Overview• Critical memory pages are
assigned permissions in EPT• Exception handler defined in
hypervisor• Shadow EPT defined with
elevated privsProtects Against Attack Techniques
• Rootkit injection• Buffer overflow• API hooking
VM Kernel Memory Layout…
Kernel Code (R/X)Driver Code (R/X)
…
Driver Data (R/W)Kernel Code (R/X)Kernel Data (R/W)
…
126→31289 (R/X)127→0 (R/X)
64589→97586 (R/W)64590→217 (R/X)
64591→78924 (R/W)
EPT#1
126→31289 (+W)127→0 (+W)
64589→97586 (+X)64590→217 (+W)
64591→78924 (+X)
EPT#2 (Shadow)
Exception Handler
Guest Guest Guest Guest Guest
Critical Memor
y Access
Critical Memor
y Access
Critical Memor
y Access
Critical Memor
y Access
Critical Memor
y Access
Networking StorageCompute
Simplified Hypervisor Introspection Architecture Diagram
Xen Project Hypervisor
Control Domain (dom0)
Security Appliance(domU)
Memory Introspectio
n Engine
Direct Inspect APIs
Virtual Switches as Local Edge Protection – Silent Block
Guest VM
SSL access
Attack silently blocked
Virtual Switch RulesIngress:
HTTPS publicEgress:Dynamic port to originMySQL internalPrivate CIDR internal
Port 22 access
Virtual Switches as Local Edge Protection – Traffic Monitor
Guest VM
SSL access
Attack blocked with traffic log
Virtual Switch RulesIngress:
HTTPS publicEgress:Dynamic port to originMySQL internalPrivate CIDR internal
Port 22 accessovs Controller
Log SSH Port 22 accessCreate port mirror for attacker
Traffic Monitor
Virtual Switch RulesIngress:
HTTPS publicEgress:Dynamic port to originMySQL internalPrivate CIDR internal
Mirror: Port 22 to Traffic MonitorAll attacker traffic to monitor
Guest VM
Virtual Switches as Local Edge Protection – Quarantine
Guest VM
SSL access
Attack quarantined with full log
Virtual Switch RulesIngress:
HTTPS publicEgress:Dynamic port to originMySQL internalPrivate CIDR internal
Port 22 accessovs Controller
Log SSH Port 22 accessCreate port mirror for attackerQuarantine VM for attacker useTrigger replacement VM for farm
Traffic Monitor
Virtual Switch RulesIngress:
HTTPS attackerEgress:Dynamic port to origin
Mirror: Port 22 to Traffic MonitorAll attacker traffic to monitor
Container Use Cases
Application containers• Hold a single application• Can follow micro-services, cloud native design
pattern• Starting point for most container usage• Short lifespan, many per host
System containers• Proxy for a VM• Insulate against core operating system• Perfect for legacy apps• Long lifespan, few per host
MyS
QL
Tom
cat
ngin
x
Kernel
MySQLTomcatnginx
Kernel
Trust Container Source
Atomic Host
Atom
ic Ap
pAt
omic
App
Atom
ic Nu
lecu
leAt
omic
Nule
cule
RedHat Registry
MyS
QL
Redi
s
Jenk
ins
Docker Hub
Dock
er C
onta
iner
Dock
er C
onta
iner
Dock
er C
onta
iner
Dock
er C
onta
iner
Dock
er C
onta
iner
Third Party and Custom Problem: Who to trust, and why?
• Trusted source?• Unexpected image
contents• Locked application layer
versions (e.g. no yum update)
• Layer dependencies (monolithic vs micro-services)
• Validated when?
Determine Who Can Launch A Container
Container default is root access• RBAC/ABAC is orchestration
specificDocker Datacenter
• Universal Control Plane• RBAC – LDAP/AD/local users• Full/Restricted/View/None
Kubernetes• Authorization modules• Admission controllers
Define Sensible Container Network Policies
Docker default network is Linux BridgeAccess policy defined in iptables
• Based on Docker daemon startupExternal communication on by default
• -- iptables=off to disable iptables modificationInter container communication on by default
• -- icc=false to disable inter container communication• -- link=CONTAINER_NAME_or_ID:ALIAS with EXPOSE ports from Docker file• All inter-container/cross host communication is external
`docker network` command simplifies aspects of network design• Create user defined networks, including overlay networks• docker network create --driver bridge sql
Docker Networking - Example
Host
eth0/10.204.136.1
Cont
aine
rve
th0
Cont
aine
rve
th1
Cont
aine
rve
th2
Cont
aine
rve
th3
Cont
aine
rve
th4
Cont
aine
rve
th5
docker0NAT/ 172.16.1.0/24
iptables
Host
docker0
eth0/10.204.136.2
Cont
aine
rve
th0
Cont
aine
rve
th1
Cont
aine
rve
th2
Cont
aine
rve
th3
Cont
aine
rve
th4
Cont
aine
rve
th5
NAT/ 172.16.1.0/24iptables
Host
Kubernetes Networking - Example
Kubernetes Network
eth0/10.204.136.20
Pod
Cont
aine
r
Paus
e
Cont
aine
r
Cont
aine
r
veth0/10.204.136.21
Pod
Cont
aine
r
Paus
e
Cont
aine
r
Cont
aine
r
veth0/10.204.136.22
HostKubernetes Network
eth0/10.204.136.10
Pod
Cont
aine
r
Paus
e
Cont
aine
r
Cont
aine
r
veth0/10.204.136.11
PodCo
ntai
ner
Paus
e
Cont
aine
r
Cont
aine
r
veth0/10.204.136.12
Limit the Scope of Compromise
• Enable Linux Security Modules• SELinux
• --selinux-enabled on Docker engine, --security-opt=“label:profile”• AppArmor
• -- security-opt=“apparmor:profile”
• Apply Linux kernel security profiles• grsecurity, PaX and seccomp protections for ALSR and RBAC
• Adjust privileged kernel capabilities• Reduce capabilities with --cap-drop• Beware –cap-add and –privileged=false, and CAP_SYS_ADMIN
• Use a minimal Linux Host OS• Atomic host, CoreOS, RancherOS
• Reduce impact of noisy neighbors• Use cgroups to set CPU shares and memory
Control
Domain
NetworkingCompute Storage
Hypervisor
Container VM
Minimal OS
Understanding Scope of Compromise – Protect From the Inside
Cont
aine
rCo
ntai
ner
Cont
aine
r
Container VM
Minimal OS
Cont
aine
rCo
ntai
ner
Cont
aine
r
Secu
rity
Serv
ice
Cont
aine
r
Risk Mitigation Shrinks Scope of Compromise
Open source license compliance• Ensure project dependencies are understood
Use of vulnerable open source components• Is component a fork or dependency?• How is component linked?
Operational risk• Can you differentiate between “stable” and “dead”?• Is there a significant change set in your future?• API versioning• Security response process for project
7 of the top 10 Software Companies (44 of the top 100)
6 of the top 8Mobile Handset Vendors
6 of the top 10 Investment Banks
24Countries
250+Employees
1,800Customers
Who is Black Duck Software?
27
Founded
2002
8,500WEBSITES
350BILLION LINES OF CODE
2,400LICENSE TYPES
1.5MILLION PROJECTS
76,000VULNERABILITIES
• Largest database of open source project information in the world.
• Vulnerabilities coverage extended through partnership with Risk Based Security.
• The KnowledgeBase is essential for identifying and solving open source issues.
Comprehensive KnowledgeBase
Black Duck Hub Security Architecture
Hub Scan1 File and Directory Signatures
2 Open Source Component Identified
3
Hub Web Application
Black Duck KnowledgeBase
On Premises Black Duck Data Center
We Need Your Help
Knowledge is power• Know what’s running and why• Define proactive vulnerability response process• Don’t let technology hype cycle dictate security
Invest in defense in depth models• Don’t rely on perimeter security to do heavy lifting• Do look at hypervisor & container trends in security• Make developers and ops teams part of the solution• Focus attention on vulnerability remediation
Together we can build a more secure data center
Recommended