Embed Size (px)
Citation preview
Real-‐Time Mul,-‐Core Virtual Machine Scheduling in Xen
Sisu Xi1, Meng Xu2, Chenyang Lu1, Linh Phan2, Chris Gill1, Oleg Sokolsky2, Insup Lee2
1Washington University in St. Louis 2University of Pennsylvania
Real-‐Time Virtualiza,on
Ø Consolidate 100 ECUs à 10 multicore processors. Ø Integrate multiple systems on a common platform à virtualization.
q Infotainment on Linux or Android q Safety-‐criHcal control on AUTOSAR q Example: COQOS, Integrity MulHvisor, Xen automoHve
Ø Must preserve real-time performance on a virtualized platform!
11/9/14 1
Source: hQp://www.edn.com/design/automoHve/4399434/MulHcore-‐and-‐virtualizaHon-‐in-‐automoHve-‐environments
Real-‐Time Cloud
Ø Internet of Things à Cyber-Physical Systems q Smart manufacturing, smart transportation, smart grid.
q Internet-scale sensing and control.
Ø Example: Intelligent TransportaHon q Data center collects data from cameras and roadside detectors. q Control the traffic signals and message signs in real-‐Hme. q TransportaHon informaHon feed to drivers. q SCATS @ Sydney: controlling 3,400 signals at 1s round-‐trip latency.
11/9/14 2
Virtualiza,on is not real-‐,me today
Ø Existing hypervisors provide no guarantee on latency q Xen: credit scheduler, [credit, cap]
q VMware ESXi: [reservation, share, limitation]
q Microsoft Hyper-V: [reserve, weight, limit]
Ø Clouds lack service level agreement on latency q EC2, Compute Engine, Azure: #VCPUs
11/9/14 3
Current platforms provision resources, not latency!
RT-‐Xen
Ø Real-time schedulers in the Xen hypervisor q Real-time performance for tasks running in virtual machines (VMs).
q Real-time performance isolation between VMs.
q Experimental study of real-time scheduling at the hypervisor level.
Ø Build on compositional scheduling theory q VMs specify resource interfaces.
q Real-time guarantees to tasks in VMs.
Ø Incorporated in Xen 4.5 as the rtds scheduler. q rtds: Real-Time Deferrable Server
q Release: 10th December 2014.
11/9/14 4
Xen Virtualiza,on Architecture Ø Xen: type-1, baremetal hypervisor Ø Domain-0: drivers, tool stack to control VMs. Ø Guest Domains running guest OS and applications.
Ø Scheduling hierarchy q Guest OS schedules tasks on VCPUs. q Xen schedules VCPUs on PCPUs. q Xen credit scheduler: round-robin with proportional share.
11/9/14 5
PCPUs
OS Sched
Xen Scheduler
OS Sched OS Sched
VCPU Real-‐Time Task
Composi,onal Scheduling
Ø Analytical real-time guarantees to tasks running in VMs. Ø VM resource interfaces
q Hides task-specific information q Domain: a set of VCPUs each with interface <period, budget> q Computed based on compositional scheduling analysis
11/9/14 6
Resource Interface Resource Interface
Resource Interface
Hypervisor
Virtual Machines
Workload Workload
Scheduler
Scheduler
Scheduler
Real-‐Time Scheduling Policies
Ø Global scheduling q Shared global run queue
q Allow VCPU migration across cores
q Work conserving – utilize any available cores
q Migration overhead and cache penalty
Ø Partitioned scheduling q Assign and bind VCPUs to cores
q Schedule VCPUs on each core independently
q Cores may idle when others have work pending
q No migration overhead or cache penalty
11/9/14 7
Scheduling a VCPU as a “Server”
8
Hme
T1 (10, 3)
T2 (10, 3)
0 5 10 15
0 5 10 15
0 5 10 15
Hme
Hme
Periodic Server (5,3)
Deferrable Server (5,3)
Budget
Budget back-‐to-‐back execuHon
not work-‐conserving
Run Queues
Ø Global: all cores share one run queue with a spinlock Ø ParHHon: one run queue per core
Ø A run queue q holds VCPUs that are runnable (not idle) q is divided into two parts: with budget; without budget q sorts VCPUs by priority (DM or EDF) within each part
11/9/14 9
RT-‐Xen: Real-‐Time Scheduling in Xen
Ø Single-core RT-Xen 1.0 [EMSOFT’11] Ø Single-core enhanced RT-Xen 1.1 [RTAS’12]
Ø Multi-core scheduling RT-Xen 2.0 [EMSOFT’14]
11/9/14 10
Fixed Priority (DM)
ParHHoned Scheduling
Dynamic Priority (EDF)
Periodic Deferrable
Periodic Deferrable
Deferrable Periodic
Deferrable Periodic
Sporadic
Polling Work
Conserving Periodic
Capacity Reclaiming Periodic
Global Scheduling
Experimental Setup
Ø Hardware: Intel i7 processor, 6 cores, 3.33 GHz q Allocate 1 VCPU for Domain-0, pinned to PCPU 0
q All guest domains use the remaining cores
q Software q Xen 4.3 patched with RT-Xen
q Guest OS: Linux patched with LITMUS
Ø Workload q Period tasks
q Allocate tasks à VMs
11/9/14 11
RT-‐Xen vs. Credit Scheduler
11/9/14 12
• Credit misses deadlines at 22% of CPU capacity.
• RT-Xen delivers real-time performance at 78% of CPU capacity.
RT-‐Xen: Scheduling Overhead
11/9/14 13
• Credit has poor max overhead due to load balancing.
• rt-global has extra overhead due to global lock.
RT-‐Xen: Theory vs. Experiments
11/9/14 14
• gEDF > pEDF empirically, thanks to work-conserving global scheduling.
• gEDF < pEDF theoretically due to pessimistic analysis.
1.1 1.5 1.9 2.3 2.7 3.1 3.5 3.9 4.3 4.7 0
0.2
0.4
0.6
0.8
1
Task Set Utilization
Frac
tion
of s
ched
ulab
le ta
skse
ts
gEDF (RT−Xen)pEDF (RT−Xen)pEDF (CSA Theory)gEDF (CSA Theory)
RT-‐Xen: Deferrable vs. Periodic Server
11/9/14 15
Work-conserving wins empirically! • gEDF+DS à best real-time performance.
• This is the rtds scheduler in Xen 4.5.
RT-‐Xen: How about Cache?
11/9/14 16
• gEDF > pEDF for cache intensive workload.
• Benefit of global scheduling outweighs migration cost.
• Shared cache mitigates cache penalty incurred by migration.
pEDF in guest OS
Conclusion
Ø Diverse applications demand real-time virtualization.
q Embedded system integration.
q Internet-scale control systems.
Ø RT-Xen supports real-time VMs on multicore processors.
q Efficient implementation of real-time scheduling policies in hypervisor.
q Leverage compositional scheduling theory.
Ø Insights from experimental study.
q pEDF > gEDF in theoretical guarantees, but pEDF < gEDF empirically.
q gEDF + DS performs best – work conserving wins empirically!
q With shared cache, benefit of global scheduling outweighs cache penalty.
11/9/14 17
Check out RT-‐Xen
11/9/14 18
RT-‐Xen
hQps://sites.google.com/site/realHmexen/
Incorporated in Xen 4.5 as the rtds scheduler To be released in December 14th, 2014!
git://xenbits.xen.org/xen.git xen/common/sched_rt.c
