1

Achieving QoS in Server Virtualization

Intel Platform Shared Resource Monitoring/Control in Xen

Chao Peng (chao.p.peng@intel.com)

2

Increasing QoS demand in Server Virtualization

• Data center & Cloud infrastructure

- Both public/private clouds call for it, especially for multiple tenants environment

- Key QoS requirements:

o Service availability

o Classes of service

o Bandwidth / throughput

• Network Function Virtualization (NFV)

- Intrinsic QoS demand from traditional ‘carrier-grade’ telecommunication environment

- Key QoS requirements:

o Service availability

o Classes of service

o Bandwidth / throughput

o Transmission delay / response time / interrupt latency

o Low noise ratio and error detection

3

1

1.5

2

2.5

3

3.5

4

4.5

sph

inx

3

mcf

bw

av

es

milc

sop

lex

om

ne

tpp

lbm

gcc

bzip

2

xa

lan

cbm

k

cactu

sAD

M

wrf

Ge

msF

DT

D

leslie

3d

asta

r

zeu

smp

libq

ua

ntu

m

ton

to

calcu

lix

de

alII

pe

rlbe

nch

go

bm

k

gro

ma

cs

h2

64

ref

sjen

g

ga

me

ss

na

md

hm

me

r

po

vra

y

Ap

p S

low

do

wn

vs.

Alo

ne

High Priority Application

Full Contention (No QoS): CPU2006 29x29

astar bwaves

bzip2 cactusADM

calculix dealII

gamess gcc

GemsFDTD gobmk

gromacs h264ref

hmmer lbm

leslie3d libquantum

mcf milc

namd omnetpp

perlbench povray

sjeng soplex

sphinx3 tonto

wrf xalancbmk

zeusmp

Low-Priority Applications (3 identical instances):

No QoS: Thread Contention

Data on Haswell Client (3GHz, 4 cores, 8MB cache, DDR3-1333, SPEC* CPU2006)

Resource contention causes up to 4X slowdown in performance

(Need ability to monitor and enforce cache/memory resource usage)

Example on Haswell Client

SPEC Pairs +

Linux OS

Cache/Memory Contention

Stats Max Avg Geomean

Full Contention 4.37x 1.37x 1.31x

4

The fundamental reason for QoS in virtualization

… is Shared resource contention.

• Physical CPU - Realtime scheduler

- CPU pinning/isolation for high priority VM

• Memory - Cache aware memory allocation

- Dedicated memory region for high priority VM

• Shared device - Dedicated device assignment

• Platform shared resource, such as LLC, memory bandwidth? - NO! Software can’t do anything for that.

5

Intel Platform Shared Resource Monitoring/Control

Socket

Core n Core 1

Core 0

Last Level Cache

Integrated I/O

CPM

PCI Express

Memory Controller

DRAM DRAM

DRAM

• Several Shared Resources (LLC,

Memory Controller) cause execution

contention

• No performance guarantees when

App’s / VM’s run in parallel

• Can be detrimental to Communications

workloads (NFV / SDN)

1

2

What is Platform QoS?

1. Cache Monitoring and Allocation

2. Memory Bandwidth Monitoring

3. Future Features (Under Research)

6

Platform QoS Vision Extending monitoring / enforcement across the platform

Code and Data Prioritization

Cache Monitoring

Cache Allocation

Mem

ory

Bandw

idth

Monitoring

Cache

Memory

Rich Set of Technologies Developed Along Cache

and Memory Vectors

7

• More New Features

available for Xen

Intel® Resource Monitoring/Control Technologies Roadmap

2016 2015 2014

Intel Xeon E5 v3

Family

Intel Xeon D-1500

Family

• Cache Monitoring

Technology (CMT) • Previous features, plus:

• Memory Bandwidth

Monitoring (MBM)

• Cache Allocation

Technology (CAT)

Xen 4.5 Xen 4.6

• Cache Monitoring

Technology (CMT)

• Cache Monitoring

Technology (CMT)

• Memory Bandwidth

Monitoring (MBM)

• Cache Allocation

Technology (CAT)

Future Processors

Xen 4.7

• Cache Monitoring

Technology (CMT)

• Memory Bandwidth

Monitoring (MBM)

• Cache Allocation

Technology (CAT)

• Code and Data

Prioritization (CDP)

• Previous features, plus:

• Code and Data

Prioritization (CDP)

• Future: Features under

consideration, feedback

solicited

8

Platform Quality of Service: Key concept -- RMIDs and CLOS

Resource Monitoring IDs (RMIDs) for the Monitoring features

Classes of Service (CLOS) for the Allocation features

Both organized as fields in IA32_PQR_ASSOC MSR

Resource Monitoring IDs (RMIDs)(Logical Construct)

ApplicationsThread(s) ApplicationsApplication(s) VM(s)VM(s)VM(s)

Flexible N:M Mapping

9

CMT Resource Marking

High-Level Monitoring Usage Flow

On Context

Switch*

RMID (for resource tracking)

CMT Resource Association

PQR

Register

Application

Memory Request

Tag with

RMID

Cache Subsystem

1

2

3

4

Monitoring counters

Monitoring counters

Monitoring counters

2 Monitoring counters

QoS Aware

Resource Monitoring

Set 1

Way 2

… … …

Way 16

Set 2

Way 2

… … …

Way 16

Set 3

Way 2

… … …

Way 16

.. .. ..

Way 2

… … …

Way 16

Set n

Way 1

… … …

Way 16

Memory

Request

PQR

RMID

PQR RMID identifies

application for QoS

resource-aware cache unit

Per Application

Monitoring Request

Event select for RMID

Read monitoring counter

CMT Resource Reporting

Event

request

Example: Occupancy,

Mem BW

10

Enumerate CAT

Configure class of

Service w/ bitmasks

QoS Enum/Config

On Context

Switch

Set Class of

Service in PQR

QoS Association

Config COS=2

Application

Memory Request

Tag with Cache

Class of service

Allocation

Cache Subsystem

QoS Aware

Cache Allocation Set 1 Way 2

… … … Way 16

Set 2 Way 2

… … … Way 16

Set 3 Way 2

… … … Way 16

.. .. .. Way 2

… … … Way 16

Set n Way 1

. . .

Way 16

Memory

Request

COS

Enforce mask

COS 1

COS 2

COS 3

COS 4

Mask1

Mask2

Mask3

Mask4

Transaction COS 1

COS 2

COS 3

COS 4

BitMask1 Mask1

Enforcement Target 2 BitMask2 Mask2

BitMask3 Mask3

BitMask4 Mask4

Architectural Implementation

dependent

High-Level Allocation Usage Flow

11

With CAT applied: Reduced Thread Contention

Important Thread=6MB isolated,

3 Low-Priority threads share 2MB

HP Mask:LP0 Mask:LP1 Mask:

LP2 Mask:

1

1.5

2

2.5

3

3.5

4

4.5

sph

inx3

mcf

bw

aves

milc

sop

lex

om

ne

tpp

lbm

gcc

bzip

2

xalancb

mk

cactusA

DM

wrf

Ge

msFD

TD

leslie3d

astar

zeu

smp

libq

uan

tum

ton

to

calculix

de

alII

pe

rlbe

nch

gob

mk

grom

acs

h264re

f

sjen

g

game

ss

nam

d

hm

me

r

po

vray

Ap

p S

low

do

wn

vs.

Alo

ne

High Priority Application

CAT Applied: 6MB Dedicated to HP App, LP Apps share 2MB

astar bwaves

bzip2 cactusADM

calculix dealII

gamess gcc

GemsFDTD gobmk

gromacs h264ref

hmmer lbm

leslie3d libquantum

mcf milc

namd omnetpp

perlbench povray

sjeng soplex

sphinx3 tonto

wrf xalancbmk

zeusmp

Low-Priority Applications (3 identical instances):

Stats Max Avg Geomean

Full Contention 4.37x 1.37x 1.31x

CAT Applied 3.44x 1.25x 1.22x

CAT Significantly reduces

contention

Remaining contention due

to memory bandwidth

Previous Contention Reduced Substantially!

12 12

Implementation in Xen - Overview

# xl help

…

psr-hwinfo Show hardware information for Platform Shared Resource

psr-cmt-attach Attach Cache Monitoring Technology service to a domain

psr-cmt-detach Detach Cache Monitoring Technology service from a domain

psr-cmt-show Show Cache Monitoring Technology information

psr-cat-cbm-set Set cache capacity bitmasks(CBM) for a domain

psr-cat-show Show Cache Allocation Technology information

More reference @ http://xenbits.xen.org/docs/unstable/misc/xl-psr.html

• Basic design for Xen Per-VM Monitoring/Allocation

Mechanism in hypervisor, policy in user stack

• XL commands:

13 13

Implementation in Xen - Monitoring

MBM:

$ xl psr-cmt-show total_mem_bandwidth <domid>

$ xl psr-cmt-show local_mem_bandwidth <domid>

$xl psr-hwinfo –cmt

Cache Monitoring Technology (CMT):

Enabled : 1

Total RMID : 63

Supported monitor types:

cache-occupancy

total-mem-bandwidth

local-mem-bandwidth

Step1: Check if CMT is supported

$xl psr-cmt-attach <domid>

Step2: Attach RMID to domain

$xl psr-cmt-show cache_occupancy

Total RMID: 63

Name ID Socket 0 Socket 1

Total L3 Cache Size 20480 KB 20480 KB

Domain-0 0 14240 KB 14976 KB

ExampleHVMDomain 1 4200 KB 2352 KB

Step3: Show monitoring data(LLC cache occupancy)

$xl psr-cmt-detach <domid>

Step4: Detach RMID from domain

14 14

Implementation in Xen - Allocation

$xl psr-hwinfo --cat

Cache Allocation Technology (CAT):

Socket ID : 0

L3 Cache : 12288KB

Maximum COS : 15

CBM length : 12

Default CBM : 0xfff

Step1: Check if CAT is supported

$xl psr-cat-cbm-set <domid> 0xff

Step2: Set custom cache Bitmask for domain

$xl psr-cat-show

Socket ID : 0

L3 Cache : 12288KB

Default CBM : 0xfff

ID NAME CBM

0 Domain-0 0xfff

1 HVM-guest 0xff

Step3: Show domain LLC cache allocation

$xl psr-cat-cbm-set <domid> 0xfff

Step4: Set default cache bitmask for domain

Next Slide: Besides xl commands(which display usage info on screen), let’s

explore some benefits / use cases…

15

CMT in Openstack

• Metering and monitoring

- Any VM using more than X% of LLC: indicates heavy workload and node is busy.

- All VM using less than X% of LLC: candidate for migration destination.

• Noisy neighbor detection

- Define alarm for noisy neighbor, including cache threshold and alarm action.

- Call alarm actions to do migration.

- Use CMT filter to choose one node with enough available cache to do migration.

Nova api(creat a flavor with “cmt”

in extra_specs)

Scheduler (choose the node with CMT

feature)

Compute (create/delete instance and

update the cmt_counter )

Nova api

16

CAT improves NFV interrupt latency

Significant improvements to interrupt-processing latency and predictability!

0

20

40

60

80

100

7 8 9 10 11

Pe

rce

nt

Dis

trib

uti

on

Interupt Latency (us)

Interrupt Latency -- With CAT

With CAT

Interrupt Latency with Aggressor Application (Fork Bomb) unpredictable with long tail

Cache Allocation Technology successfully contains the aggressor application

0

20

40

60

80

100

7 8 9 10 11

Pe

rce

nt

Dis

trib

uti

on

Interupt Latency (us)

Interrupt Latency -- No CAT

No CAT

See more @ Cache Allocation Technology Improves Real-Time Performance

17

CMT/CAT feedback loop

Start

Setup Policy,

and associate

RMID’s with

Applications

Monitor

Platform

Occupancy

Adjust CAT

Bitmask

Scale out?

YES

NO

Policy

Change

• Utilize Monitoring (CMT or MBM) and Allocation (CAT)

capabilities to build feedback loop to control efficient use

of shared resources

• Optimize Cache and Memory Bandwidth allocation based

on priorities

• A variety of throughput or prioritization-based algorithms

have been shown to provide benefit depending on use

model

18

Possible future work

• Feature Improvement

- HW: More RMID/CLOS.

- HW: Incremental feature improvements to reduce latencies.

- SW: RMID recycling in Xen (Linux already has this).

• Future Features

- Under consideration – Driven by customer feedback!

• Xen Scheduler improvement

- Intel Cache Monitoring: Current Status and Future Opportunities by Dario Faggioli

• Guest emulation

- Expose these technologies to guest so that guest and apps can take advantage of it.

• More usage models

- In program or in end-user scenarios.

19

Additional Resources

Specification & Reference

o Cache Monitoring and Cache Allocation Technology Landing page

o Architecture Software Development Manual(Vol 3b., Chapter 17.14 and 17.15, covers CMT, CAT,

MBM, CDP)

Blogs

o Part 1: Introduction to CMT

o Part 2: Discussion of RMIDs and CMT Software Interfaces

o Part 3: Use Models and Example Data

o Part 4: Software Supports and Tools: Intel's Cache Monitoring Technology: Software Support and

Tools

NFV

o NFV Delivers Packet Processing Performance with Intel

o Cache Allocation Technology Improves Real-Time Performance

o Packet Processing - Cache Monitoring Technology, Memory Bandwidth Monitoring & Cache

Allocation Technology

Xen

o Intel Cache Allocation Technology(CAT) design for XEN

o Intel Cache Monitoring: Current Status and Future Opportunities

Thank you!

20