System Virtual Machines

Ch 8.1~8.3

Liao Lingling

2006.10.23

System Virtual MachinesSystem Virtual Machines

2

ContentsContents

• Introduction• Key Concepts• Resource Virtualization

– Processors– Memory

3

Introduction (1/2)Introduction (1/2)

• Underutilized hardware -> time sharing– Allow multiple users to access a single computer system

simultaneously– Each user is given the illusion of having access to a full set

of system resources

• To create this illusion:– Multiprogramming operating system + process virtual

machine• Implement a process virtual machine for each application-level

program and switch resources among the programs on a time-shared basis

– System virtual machine:• Provide a similar illusion for complete systems

4

VirtualizingSoftware

Hardware

Applications

OSVirtualizing

Software

Hardware

Applications

OS

Process Virtual Machine System Virtual Machine

5

Introduction (2/2)Introduction (2/2)

• Advantages of system virtual machines:– Implementing multiprogramming– Multiple single-application virtual machines– Multiple secure environment– Managed application environment– Mixed-OS environment– Legacy application– Multiplatform application development– New system transition– System software development– Operating system training– Help desk support– Operating system instrumentation– Even monitoring– System encapsulation

6

ContentsContents

• Introduction• Key Concepts

– Outward appearance– State management– Resource control– Native and hosted virtual machine– IBM VM/370

• Resource Virtualization – Processors– Memory

7

Outward AppearanceOutward Appearance

• Outward appearance: – the illusion of multiple machines

• Two ways to do so:– Replication:

• replicate a subset of the hardware resources and share the rest of hardware

– Switch: • use a hardware switch or enter a special key sequence on the

key board to switch a subset of the hardware and share the rest of the hardware

8

State ManagementState Management

• The architected state of a computer is contained in and maintained by the hardware resources of the machine

• As operation on the guest’s state are performed, the state maintained on the host machine should be modified

• Two basic ways to do so:– Using indirection

• Hold the state for each guest in fixed locations in the host’s memory hierarchy with a VMM-managed pointer -> inefficiency

– By copying• Copy a guest’s state information to its natural level in memory hierarchy

• Choice:– The frequency of use– Whether the guest state being managed by the VMM is held in a

different type of hardware resource than on a native platform

9

Processor

Register BlockPointer

Processor

ProcessorRegister

VMM Memory

Register valuesfor VM3



VMM Memory




VM changes pointer when VM is activated

VMM copies register values when VM is activated

10

Resource ControlResource Control

• Conventional time-sharing systems:– E.g.: the interval timer

• System VM:– Provide the VMM with overall control– The VMM intercepts all accesses to so-called privileged resources

and emulates the operation of these resources at all times

• A fair scheme– Time allocation

• Large – not optimally utilized• Small – overhead

– The guest operating system on a virtual machine must be denied direct access to the interval timer facility

– The VMM must examine every guest setting of the timer interrupt and override the request value

11

Native and Hosted Virtual MachinesNative and Hosted Virtual Machines

• Role of the VMM:– Schedule and manage the allocation of hardware resources to the

various guest virtual machines• Shared physical resources include:

– The registers in the CPU– The real memory in the system– The various I/O devices attached to the system

• The catalog of system VM:– Native VM system

• The VMM is the only software that executes in the highest privilege level defined by the system architecture

– User-mode hosted VM system• The VMM utilizes the functions already available on the host OS to

control and manage resources desired by each of the virtual machines– Dual-mode hosted VM system

• Parts of the VMM are operated in a privileged mode and other parts in nonprivileged mode

12

Application

OS

Hardware

Guest Apps

Guest OS

VMM

Hardware

Guest Apps

Guest OS

VMM

Host OS

Hardware

Guest Apps

Guest OS

VMM

Host OS

Hardware

Nonprivilegedmodes

Privilegedmodes

Traditionaluniprocessor

system

NativeVM system

User-modehosted

VM system

Dual-modehosted

VM system

13

IBM VM/370IBM VM/370

• The chapter refer to the IBM virtual machine VM/370• VMM of VM/370 – control program (CP)• Single-user operating system – conversational

monitor system (CMS)• CP/CMS design separate the function of resource

management from the function of providing services to the user

14

ContentsContents


– Processors• Conditions for ISA virtualizability

• Recursive virtualization

• Handling problem instructions

• Patching of critical instructions

• Caching emulation code

• Efficient virtualizability of common instruction set

– Memory

15

Resource virtualization – Processors Resource virtualization – Processors

• Key aspect:– execute the guest instructions

• Two ways:– Emulation:

• Examining -> Interpretation or binary translation -> Emulating• Always possible

– Direct native execution on the host machine:• Possible only if the ISA of the host is identical to the ISA of the guest,

and even then only certain condition

• A basic design goal for system VMs with identical guest and host ISAs– A significant fraction of the instructions execute directly on the

native hardware

– Emulating the remaining instructions

16

Conditions for ISA Virtualizability(1/5)Conditions for ISA Virtualizability(1/5)• Host VM:

– Inherent inefficiency: the VMM relies heavily on services of the host operating system

• Discussion here only to native system VM• Assumptions:

– The hardware consists of a processor and a uniformly addressable memory

– The processor can operate in one of two modes, the system mode or the user mode

– Some subset of the instruction set is available only in the system mode

– Memory addressing is done relative to the contents of a relocation register

• Virtualized machine model: S=<E,M,P,R>– E: the executable storage; M: the mode of operation– P: the program counter; R: the memory relocation bounds register

17

Conditions for ISA Virtualizability(2/5)Conditions for ISA Virtualizability(2/5)• The catalog of instructions:

– Privileged instruction• One that traps if the machine is in user mode and does not trap if the machine

is in system mode• E.g.:

– Load Processor Status Word (LPSW):– Set CPU Timer (SPT)

– Special instructions• Control-sensitive instruction

– One that attemps to change the configuration of resources in the system• Behavior-sensitive instruction

– One whose behavior or results produced depend on the configuration of resources

– E.g.: » Load Real Address (LRA)» Pop Stack into Flags Register (POPF)

• Innocuous – Neither of above two

18

Nonprivileged

Privileged

Sensitive

Control sensitive

Behavior sensitive

Innocuous

Sensitive

19

Conditions for ISA Virtualizability(3/5)Conditions for ISA Virtualizability(3/5)• The functions of the VMM:

– Dispatcher– Allocator– A set of interpreter routines

Dispatcher

InterpreterRoutine 1

Allocator InterpreterRoutine 2

InterpreterRoutine 3

Privileged instruction

These instructions desire to change machine resources, e.g., load relocation bounds register

Instruction trap occurs




These instructions do not change machine resources but access privileged resources, e.g., IN, OUT, Write TLB

20

Conditions for ISA Virtualizability(4/5)Conditions for ISA Virtualizability(4/5)

• The properties of a true VMM:– Efficiency:

• All instructions that are innocuous must be executed natively on the hardware

– Resource control:• It should not be possible for guest software to directly change the

configuration of any system resources available to it

– Equivalence:• Any program executing on a virtual machine must behave in a manner

identical to the way it would have behaved when running directly on the native hardware with only a few exceptions

• Exceptions:– Performance can be slower due to the emulation– There may be a limitation on the extent of resources available– There may be differences in performance due to changed timing

relationships

21

Conditions for ISA Virtualizability(5/5)Conditions for ISA Virtualizability(5/5)

• Theorem 1:– For any conventional third-generation computer, a virtual

machine monitor may be constructed if the set of sensitive instructions for that computer is a subset of the set if privileged instructions

• Theorem is useful because it suggests a convenient way of testing an ISA to determine the feasibility of an efficient VMM

22

Recursive VirtualizationRecursive Virtualization

• Recursive virtualization:– The VMM itself would be running in the user mode under the

control of a copy of itself running in the privileged mode• Two effects:

– If the VMM itself has timing dependences, then its performance could be affected adversely when it runs in user mode within another virtual machine

– Each VMM layer uses up its own resources. If recursive virtualization is repeatedly performed, then these resource keep shrinking.

• Neither of these is a problem in practice– Number of levels is seldom beyond two

• Theorem 2:– A conventional third-generation computer is recursively virtualizable

if (a) it is virtualizable and (b) a VMM without any timming dependences can be constructed for it

23

Hardware

VMM

VirtualMachine

VirtualMachine

2nd-level VMM

VirtualMachine

VirtualMachine

Privileged Modes

Nonprivileged Modes

24

Handling Problem InstructionsHandling Problem Instructions

• Critical instruction:– Sensitive but not privileged

• It would be possible for a VMM to intercept critical instructions if all guest software were interpreted instruction by instruction

-> inefficiency

-> use the techniques related to ch.2 and 3:

Scanning and patching • Hybrid virtual machine system

– A virtual machine system in which some of the nonprivileged instructions must be emulated

25

Original Program Patched Program

Scanner and Patcher

Code patch for discovered

critical instruction

VMM

Control transfer, e.g., trap

26

Patching of Critical InstructionsPatching of Critical Instructions

• Compared with dynamic binary translation– Same:

• Both basic idea is to locate sections of code about to be executed and perform transformations on them

– Difference:• In binary translation, both the target code and the runtime execute in

user mode and hence a switch between them can be done via a jump• In the VM context, there is an actual change of privilege mode, which

means that the jumps to the runtime are replaced with system calls (traps) to the VMM

• To reduce overhead– The trap at the end of a scanned basic block can be replaced by

the original branch or jump instruction after all possible successor basic blocks have been patched

• To further reduce overhead– Instead of scanning single basic blocks at a time, the VMM can

scan through all branches that allow target computation

27

Caching Emulation CodeCaching Emulation Code• If interpretation of sensitive instructions is required frequently

-> high overhead• Solution:

– Cache the actions taken during the first interpretation

Patched Program

Code section emulated in code

cache

Translation Table

Control transfer, e.g., trap

Two critical instructions combined into a single

block

Block 1

Block 2

Code

Cache

Block 3

SpecializedEmulation Routines

Block 1

Block 2

Block 3

28

Efficient Virtualizability of Common Efficient Virtualizability of Common Instruction setInstruction set

• The System/370 architecture provide two modes of operation, a user mode and a privileged mode

• The CP for System/370 has to run in the privileged mode

• All sensitive instructions on the system/370 are also privileged

• CMS allows all applications to run in the “pseudo-supervisor”– Users can use any instruction whether privileged or not

29

Application

Privileged instructionstrap to OS

OS

Privileged instructions directly

executed

Hardware

Application

Privileged instructions trap to

VMM and reflected to OS

OS

Privileged instructions trap to VMM and emulated

in VMM

VMM


executed

HardwareA standard System A VM system

Application


in VMM

CMS


in VMM

CP


executed

System/370the CP/CMS

system

30

ContentsContents


– Processors– Memory

• Virtual memory support in a system virtual machine environment

• Virtualizing architected page tables

• Virtualizing an architected TLB

31

Virtual Memory Support in a System Virtual Memory Support in a System Virtual Machine EnvironmentVirtual Machine Environment

• A standard system:– Mapping: virtual address -> physical address

• A system VM:– Add a new level in the memory hierarchy– Virtual address -> real address -> physical address

• Real memory is a guest VM’s illusion of physical memory supported by the VMM

– The VMM maintains a swap space distinct from the swap spaces of each of the guests

• On contemporary platforms, page translation is supported by a combination of a page table and a translation lookaside buffer (TLB)– Architected page table

• The TLB is maintained and used only by the hardware– Architected TLB

• Hardware is unaware of the page table structure• Software-manage TLB

32

Virtual Page Real Page

--- ---

1000 5000

--- ---

2000 1500

--- ---

Virtual Page Real Page

--- ---

1000 Not mapped

--- ---

4000 3000

--- ---

Virtual Page Reall Page

--- ---

1000 500

--- ---

4000 3000

--- ---

VM 2 Real

Page

Physical Page

--- ---

500 3000

--- ---

3000 Not mapped

--- ---

VM 1 Real

Page

Physical Page

--- ---

1500 500

--- ---

3000 Not mapped

--- ---

5000 1000

--- ---

Real Map Table for VM1

Real Map Table for VM2

Page Table for Program 3



1000

2000

1500

3000

5000

1000

4000

500

3000

1000

4000

Not mapped

Not mapped to physical memory

500

1000

3000

Virtual Memory of Program 1 on VM 1



Real Memory of VM 1

Real Memory of VM 2

Physical Memory of System

33

Virtualizing Architected Page Virtualizing Architected Page Tables(1/2)Tables(1/2)

• Shadow page table:– Virtual-to-physical mapping kept by the VMM

– One for each of the guest VMs

– Used by hardware to translate virtual addresses and to keep the TLB up-to-date

• To make it work:– The page table pointer register is virtualized

– The VMM manages the page table pointer and has access to the virtual version of the register associated with each guest VM

– Access the page table pointer:• Read – return the guest’s virtual page table pointer• Write – update virtual version and update the table pointer to

corresponding shadow table

34

Page Table Pointer

Virtual Page Physical Page

--- ---

1000 1000

--- ---

2000 500

--- ---


--- ---

1000 Not mapped

--- ---

4000 Not mapped

--- ---


--- ---

1000 3000

--- ---

4000 Not mapped

--- ---

Shadow Page Tables Maintained by VMM

Shadow Page Table for Program 1 on VM1



Program 1 on VM1is currently active

35

Virtualizing Architected Page Tables(2/2)Virtualizing Architected Page Tables(2/2)

• Differences – consider the page fault handling– The VMM should not have a virtual-to-physical page mapping in a

shadow table if the guest OS does not have the same virtual page mapped to real memory in its corresponding virtual table-> equivalence

– When a page fault does occur, the page may or may not be mapped in the virtual table of the guest OS

• Mapped: – This page fault should be handled entirely by the VMM

• Not mapped:– Transfer control to the trap handler of the guest indicating a page

fault• Performance degradation

– Contiguous real pages may not be contiguous in physical memory– Optimization:

• E.g.: when ready to activate a new VM, gives priority to the virtual machine likely to have most of its active pages already mapped to physical pages

36

Virtualizaing an Architected TLBVirtualizaing an Architected TLB

• TLB must be virtualized by the VMM– The VMM maintains a copy of each guest’s TLB contents and also

manages the real TLB

• Manage the real TLB:– Simple way:

• VMM rewires the TLB whenever a guest VM is activated-> high overhead

– Use the address space identifiers (ASIDs) included in each entry of an architected, software-managed TLB

• Allow multiple processes to have address space mappings in the TLB simultaneously

• Each guest VM has a virtual ASID register and VMM maintain the real ASID register

• At any given time, the VMM maps certain guest ASID values to real ASID values for entries that are present in the real TLB

37

ASID

Virtual

Page

Real

Page

--- --- ---

3 1000 5000

--- --- ---

3 2000 1500

--- --- ---

7 4000 3000

--- --- ---

ASID

Virtual

Page

Real

Page

--- --- ---

3 1000 3000

--- --- ---

--- --- ---

--- --- ---

--- --- ---

--- --- ---

ASID

Virtual

Page

Physical

Page

--- --- ---

9 1000 1000

4 1000 3000

9 2000 500

--- --- ---

--- --- ---

--- --- ---

Virtual

ASID

Real

ASID

--- ---

VM1:3

9

--- ---

VM1:7

---

--- ---

VM2:3

4

ASID mapping:

Prog.1 – ASID 3

Prog.2 – ASID 7

ASID mapping:

Prog.1 – ASID 3

Virtual TLBs

Virtual TLB of VM1

Virtual TLB of VM2

ASID Map Table

Real TLB

38

SummarySummary


– Processors– Memory– Input/Output

• Performance enhancement• Case Study

Documents

System Virtual Machines