Scalable Cloud Security via Asynchronous Virtual Machine Introspection
Sundaresan (sunny) Rajasekaran, Zhen Ni, Harpreet Singh Chawla, Neel Shah, Timothy Wood and Emery Berger†.
The George Washington University †University of Massachusetts, Amherst
1
Introduction• Software will always be vulnerable to attacks.
• Existing techniques for prevention are slow to detect attacks.
• Need a way for cloud platforms to provide security functionality as a service.
2
Introduction• Software will always be vulnerable to attacks.
• Existing techniques for prevention are slow to detect attacks.
• Need a way for cloud platforms to provide security functionality as a service.
How can the cloud detect attacks inside a VM?How to provide strong security guarantees at low cost?
2
ScaaS• Scanning as a Service framework for security in
cloud data centers.
• Scans for a wide range of attacks within both application and the operating system.
• Uses an asynchronous checkpointing mechanism to replicate a VM’s memory onto a Scanner host for analysis.
• Uses VM introspection techniques to study the memory of the virtual machine.
3
Where do we stand?
4
Ove
rhea
d
window for vulnerability
Where do we stand?
4
Ove
rhea
d
window for vulnerability
MemorySafety toolseg: valgrind
Where do we stand?
4
Ove
rhea
d
window for vulnerability
MemorySafety toolseg: valgrind
VirusScanners.eg: McAfee
Where do we stand?
4
Ove
rhea
d
window for vulnerability
MemorySafety toolseg: valgrind
VirusScanners.eg: McAfee
ScaaS
Scanner Host
ScaaS Architecture
• VMs periodically send checkpoints to the Scanners for analysis.
• A Scanner host uses VM introspection techniques to search for evidence of vulnerabilities.
• Ensures integrity of Key Kernel data structures.
Primary Hosts
VMVM
ScaaS Agent
ScaaS Agent
5
Scanner Host
ScaaS Architecture
• VMs periodically send checkpoints to the Scanners for analysis.
• A Scanner host uses VM introspection techniques to search for evidence of vulnerabilities.
• Ensures integrity of Key Kernel data structures.
Primary Hosts
VMVM
ScaaS Agent
ScaaS Agent Check
points
5
VM VM VM VMCP
HistoryCP
HistoryCP
HistoryCP
History
Scanner Host
ScaaS Architecture
• VMs periodically send checkpoints to the Scanners for analysis.
• A Scanner host uses VM introspection techniques to search for evidence of vulnerabilities.
• Ensures integrity of Key Kernel data structures.
Primary Hosts
VMVM
ScaaS Agent
ScaaS Agent Check
points
Introspected VM
UnmodifiedKernel
AppApp
Syscall Table Slab
5
VM VM VM VMCP
HistoryCP
HistoryCP
HistoryCP
History
VM Checkpointing
VM execution timeline
Checkpoints
6
interval10 - 100 ms
VM
VM’s Memory is clean
t0
VM Checkpointing
VM execution timeline
Checkpoints
6
interval10 - 100 ms
VM
VM’s Memoryis dirtied
t0
VM Checkpointing
VM execution timeline
Checkpoints
6
interval10 - 100 ms
VM
VM’s Memoryis dirtied
Send dirty pages
..
t0
VMI Scanner
Replica RAM
VM Checkpointing
VM execution timeline
Checkpoints
6
interval10 - 100 ms
VM
VM’s Memoryis dirtied
Send dirty pages
..
t0
VMI Scanner
Replica RAM
VM Checkpointing
VM execution timeline
Checkpoints
6
interval10 - 100 ms
VM
VM’s Memoryis dirtied
Send dirty pages
..Kernel
Apachesshd
t0
VM Checkpointing
VM execution timeline
Checkpoints
6
interval10 - 100 ms
VM
VM’s Memoryis dirtied
t0
VM Checkpointing
VM execution timeline
Checkpoints
6
interval10 - 100 ms
VM
VM’s Memoryis dirtied
Send dirty pages
..
t0
VM Checkpointing
VM execution timeline
Checkpoints
6
interval10 - 100 ms
VM
VM’s Memoryis dirtied
Send dirty pages
..
VMI Scanner
t0
VM Checkpointing
VM execution timeline
Checkpoints
6
interval10 - 100 ms
VM
VM’s Memoryis dirtied
Send dirty pages
..
VMI Scanner
Kernel
Apachesshd
t0
Primary HostNetwork Buffer
VM
Client
Network buffering using Remus in Xen
• All network packets are buffered for each interval.
• The buffer content is released only at the end of the interval.
ith interval
7
ti-1 ti
Primary HostNetwork Buffer
VM
Client
Network buffering using Remus in Xen
• All network packets are buffered for each interval.
• The buffer content is released only at the end of the interval.
RX
ith interval
7
ti-1 ti
Primary HostNetwork Buffer
VM
Client
Network buffering using Remus in Xen
• All network packets are buffered for each interval.
• The buffer content is released only at the end of the interval.
TX
RX
ith interval
7
ti-1 ti
Primary HostNetwork Buffer
VM
Client
Network buffering using Remus in Xen
• All network packets are buffered for each interval.
• The buffer content is released only at the end of the interval.
TX
RX
TX
ith interval
7
ti-1 ti
Primary HostNetwork Buffer
VM
Client
Network buffering using Remus in Xen
• All network packets are buffered for each interval.
• The buffer content is released only at the end of the interval.
TX
RX
TX
ith interval
RX
7
ti-1 ti
Primary HostNetwork Buffer
VM
Client
Network buffering using Remus in Xen
• All network packets are buffered for each interval.
• The buffer content is released only at the end of the interval.
TX
RX
TX TXith interval
RX
7
ti-1 ti
Primary HostNetwork Buffer
VM
Client
Network buffering using Remus in Xen
• All network packets are buffered for each interval.
• The buffer content is released only at the end of the interval.
TX
RX
TX TX TX
ith interval
RX
7
ti-1 ti
Primary HostNetwork Buffer
VM
Client
Network buffering using Remus in Xen
• All network packets are buffered for each interval.
• The buffer content is released only at the end of the interval.
TX
RX
TX
RELE
ASE
TX TX
ith interval
RX
7
ti-1 ti
Primary HostNetwork Buffer
VM
Client
Network buffering using Remus in Xen
• All network packets are buffered for each interval.
• The buffer content is released only at the end of the interval.
TX
RX
TX
RELE
ASE
TX TX
RX
ith interval
RX
7
ti-1 ti
Primary HostNetwork Buffer
VM
Client
Network buffering using Remus in Xen
• All network packets are buffered for each interval.
• The buffer content is released only at the end of the interval.
TX
RX
TX
RELE
ASE
TX TX
RX RX
ith interval
RX
7
ti-1 ti
Primary HostNetwork Buffer
VM
Client
Network buffering using Remus in Xen
• All network packets are buffered for each interval.
• The buffer content is released only at the end of the interval.
TX
RX
TX
RELE
ASE
TX TX
RX RX
TX
ith interval
RX
7
ti-1 ti
ScaaS Execution
• Pause briefly at each checkpoint to be scanned for security vulnerabilities. • ScaaS says if it is safe to release the buffer. • If an attack is found, the VM can be rolled back and analyzed.
Inte
rval
i
ScaaS Execution Timeline
8
ScaaS Execution
• Pause briefly at each checkpoint to be scanned for security vulnerabilities. • ScaaS says if it is safe to release the buffer. • If an attack is found, the VM can be rolled back and analyzed.
Inte
rval
i
ScaaS Execution Timeline
8
ScaaS Execution
• Pause briefly at each checkpoint to be scanned for security vulnerabilities. • ScaaS says if it is safe to release the buffer. • If an attack is found, the VM can be rolled back and analyzed.
Inte
rval
i
ScaaS Execution TimelineIn
terv
al i+1
Paus
e an
d Sa
veSc
an C
heck
poin
tRe
leas
e O
utpu
ts
Async CheckpointBuffer Outputs
8
ScaaS Execution
• Pause briefly at each checkpoint to be scanned for security vulnerabilities. • ScaaS says if it is safe to release the buffer. • If an attack is found, the VM can be rolled back and analyzed.
Inte
rval
i
ScaaS Execution TimelineIn
terv
al i+1
Paus
e an
d Sa
veSc
an C
heck
poin
tRe
leas
e O
utpu
ts
Async CheckpointBuffer Outputs
8
ScaaS Execution
• Pause briefly at each checkpoint to be scanned for security vulnerabilities. • ScaaS says if it is safe to release the buffer. • If an attack is found, the VM can be rolled back and analyzed.
Inte
rval
i
ScaaS Execution TimelineIn
terv
al i+1
Paus
e an
d Sa
veSc
an C
heck
poin
tRe
leas
e O
utpu
ts
Async CheckpointBuffer Outputs
ScanFails
8
ScaaS Execution
• Pause briefly at each checkpoint to be scanned for security vulnerabilities. • ScaaS says if it is safe to release the buffer. • If an attack is found, the VM can be rolled back and analyzed.
Inte
rval
i
ScaaS Execution TimelineIn
terv
al i+1
Paus
e an
d Sa
veSc
an C
heck
poin
tRe
leas
e O
utpu
ts
Async CheckpointBuffer Outputs
Async CheckpointBuffer Outputs
ScanFails
Detectand
Replay
Rollback
8
Attack Detection and Response
• Forensic analysis: Do analysis that cannot be done on runtime.
• Rollback and Replay: Useful when using breakpoints that trigger errors such as buffer overflow.
• Honeypot mode: Resume and run in a sandbox.
9
Prototype Evaluation
• Prototype of ScaaS using Xen 4.5.2
• 1Gbps link between Primary and Scanner host.
• Checkpointing using Remus.
• VM introspection using libVMI.
10
Types of Scans• Process Black/White List Enforcer:
• Determines current running processes in a VM.Triggers errors depending on whether a target process is running or not.
• Memory Fingerprinter:
• Hashes the memory pages to compare against known good states. eg: sys call table, that doesn’t change that often.
11
0
0.2
0.4
0.6
0.8
1
0 20 40 60 80 100 120 140 160 180 200
Nor
mal
ized
Per
form
ance
Checkpoint Interval(ms)
httperfsudokut
sysbench
• Benchmarks vs. different checkpoint intervals • CPU intensive benchmarks perform well with longer intervals • httperf is a latency sensitive benchmark
• Longer the interval worse the performance.
Checkpoint overhead
12
• Performance change of application w.r.t. emulated scan costs.
• Normalized wrt to zero-cost scan • httperf costs worsens with scan cost
• as it has to hold buffer data for longer periods
0
0.2
0.4
0.6
0.8
1
16 64 256 1024 4096
Nor
mal
ized
Per
form
ance
Emulated scan cost(us)
httperfsudokut
sysbench
Emulated Scan cost
13
• Fingerprinter causes high overhead initially but becomes negligible as checkpointing interval increase.
0
20
40
60
80
100
0 20 40 60 80 100 120 140 160 180 200
Scan
ner C
PU u
sage
(%)
Checkpoint Interval(ms)
httperf-fingerprinthttperf
sudokutsysbench
CPU usage at scanner host
14
Conclusion• ScaaS: Framework for security Scanning as a
Service.
• Tool for attack detection and forensic analysis on memory.
• examining memory checkpoints for an attack.
• highly scalable and fast.
15
Discussion
• What types of attacks can we detect?
• Do we need to keep a history of checkpoints? Why? How?
• What is a reasonable cost for ScaaS?
16