© 2010 VMware Inc. All rights reserved

Mobile I/O virtualization

Harvey Tuch, Staff Engineer, Mobile Virtualization Platform

January 25th 2012

Sponsored by MIT and VMware Academic Programs

VMware: www.vmware.com

VMware Labs: labs.vmware.com

2

Agenda

Smartphone hardware

I/O virtualization techniques

Examples from MVP hypervisor

• Storage virtualization

• GPS virtualization

• Telephony virtualization

3

Smartphone hardware

4

Computer System Organization

NIC

LAN

CPU

MMU

Memory

Controller

Local Bus

Interface

High-Speed

I/O Bus

Controller BridgeFrame Buffer

Low-Speed

I/O BusUSBCD-ROM

5

Evolution of mobile phone

to Mobile Personal Computer

6

Recent smartphone specs sample (source: pdadb.net)

6Copyright ® VMware, Inc. All Rights Reserved.

7 Copyright ® VMware, Inc. All Rights Reserved.

Hardware trends

Example smartphone circa 2012

• > 1GHz dual or quad core ARM Cortex-A9/A15 or Snapdragon

• 1GB SDRAM

• 512MB internal NAND Flash

• 32GB microSD

• Integrated GPS, > 10MP camera, 3G/4G (LTE), 802.11, Bluetooth

• 480x800 OLED

• Ice Cream Sandwich Android OS

Tablets, smartbooks, etc.

8

Smartphone physical hardware

• Processor core – ARM

• Power efficiency, battery life

• E.g. 8 DMIPS/mW (Cortex-A9)

• ARMv7 ISA

• Devices – highly integrated

• SoC organization

• Plethora of devices (IP blocks)

• Limited standardization

• Storage – solid state NAND

9

Agenda

Smartphone hardware

I/O virtualization techniques

Examples from MVP hypervisor

• Storage virtualization

• GPS virtualization

• Telephony virtualization

10

I/O virtualization techniques

11

Device virtualization goals

Present to guest a “virtual” platform (set of devices)

• Guest applications, middleware, OS expect standard set of mobile devices

• E.g. touchscreen, display, storage, GPS, Bluetooth, WiFi, GSM, GPU, cameras, accelerometers, audio, keyboard, etc.

Multiplex physical hardware between VMs

• Securely sharing physical hardware across domains

• Efficiency

• Fidelity

• Portability

• Interposition (checkpointing, live migration)

• Minimizing engineering effort (cost)

12

Device virtualization techniques

Device emulation

• VMM emulates registers and memory map of physical HW devices

Paravirtualization

• VMM provides optimized high-level API for guest devices

Pass-thru

• VMM provides guest direct access to physical HW devices

13

Device virtualization techniques – device emulation

VMM emulates registers and memory map of physical HW devices Leverage existing guest OS driver support

Interposable

✗Emulation overhead

✗VMM implementation cost

Hypervisor

Guest Operating System

ACME UART driver

ACME UART device model Physical FOOBAR UART

Write FIFO ‘a’

Read status register CSR

Write FIFO ‘b’

…..

Write FIFO ‘a’

Write FIFO ‘b’

Read status register FSR

…..

14

Device virtualization techniques - paravirtualization

VMM provides optimized high-level API for guest devices Higher performance possible

Simplified VMM

Interposable

✗Requires custom drivers/libraries per guest

Hypervisor

Guest Operating System

PV UART driver

PV UART device model Physical FOOBAR UART

Write shared buffer ‘abcdefghijklmno…’

…..

Write FIFO ‘a’

Write FIFO ‘b’

Read status register FSR

…..

15

Device virtualization techniques – pass-thru

VMM provides guest direct access to physical HW devices

✓Highest performance, even simpler VMM

✗I/O MMU required for safety (DMA danger)

✗Multiplexing HW devices requires extra HW support

✗Interposition hard (hybrid models separate control/data paths)

Hypervisor

Guest Operating System

ACME UART driver

Physical ACME UART

Write FIFO ‘a’

Read status register

Write FIFO ‘b’

…..

16

Bare-metal (Type-1) hypervisor architecture

Execute directly on physical HW

Preferred architecture for high-end servers

E.g. VMware ESX, Xen, Microsoft Hyper-V

17

Hosted (Type-2) hypervisor architecture

Executes alongside existing host OS, e.g. Mac OS X, Linux, Windows

Leverage host device drivers and resources management

E.g. VMware Workstation/Fusion, KVM, Parallels Desktop

18

VMware MVP system architecture

Personal applications run natively

Corporate applications run in a virtual machine

18

Copyright ® VMware, Inc. All Rights Reserved.

19

MVP as a hosted (Type-2) hypervisor

Hosted hypervisor benefits:

• Tackle SoC diversity

• Reduced TTM

• Zero virtualization overhead for personal phone.

• Almost all components can be downloaded OTA (app store).

19

Copyright ® VMware, Inc. All Rights Reserved.

20

Device virtualization stack

Frontend (RHS)

• Guest OS – VMM interface for virtual devices

Backend (LHS)

• Hypervisor implementation of virtual device semantics on host

21

Frontend virtualization – where to slice?

Virtualization Layer

R: 137G: 203B: 223

Operating System

Application

Libraries, middleware

R: 192G: 192B: 192

R: 217G: 84B: 30

R: 248G: 152B: 29

Device emulation

Device level PV

Library level PV

Application level PV

22

Backend implementation

Host user space (application level)

• Translate guest device I/Os to standard kernel/library calls (e.g. POSIX)

• E.g. write ‘a’ to /dev/ttyS0 for UART transmit

✓Portable, simple, language independent

✗System call overhead, latency

Host kernel

• Translate guest device I/Os to kernel level function calls

✓Lowest overhead, direct access to device subsystem

✗Kernel dependent, modularity issues, increased TCB

Hardware (pass-thru)

23

Agenda

Smartphone hardware

I/O virtualization techniques

Examples from MVP hypervisor

• Storage virtualization

• GPS virtualization

• Telephony virtualization

24

Storage virtualization

25

Smartphone storage devices

Internal storage

• NAND flash devices

• Software Flash Translation Layer (FTL)

• Limited size (256MB – several GB)

• Kernel, application code, libraries, middleware

External storage

• microSD cards

• Hardware FTL

• Up to 32GB today (2TB future)

• Economics of semiconductor scaling

• Optimized for cost, media workloads

• Application data (and some code)

26

Storage virtualization stack

27

VM image storage on SD cards

Why?

• Storage footprint of guest may be several GB (including checkpoint images)

Challenges

• Performance

• Reliability

• Security

28

VM image storage on SD cards - performance

FTL optimized for cost, media workloads

• Poor non-sequential I/O performance

• VM workloads exhibit far more non-sequential I/O than MP3 streaming, photos, videos, etc.

29

VM image storage on SD cards - performance

.

30

VM image storage on SD cards - performance

.

31

VM image storage on SD cards - performance

.

32

VM image storage on SD cards – other challenges

Reliability

• Dropped phone, whoops

• Battery dies

• Host software crashes (in particular at kernel level)

Security

• SD cards are FAT formatted for compatibility

• No access controls

• Malicious host apps can read/write VM images

33

Log structured filesystems (LSF)

An excursion back in time to… 1991…

• Slow seek latency on mechnical disks, CPUs getting faster

• Non-sequential file writes expensive

• Solution: every file update gets appended to a sequential “log”

• Maintain meta-data to later locate the most recent block for a file in the log

• Reads mostly hit in the page cache, don’t worry about non-sequentiality

• Mendel Rosenblum and John K. Ousterhout. The design and implementation of a log-structured file system. In ACM Symposium on Operating System Principles, 1991.

Benefits beyond performance…

• Meta-data only updated after new data is on disk

• Old data still exists earlier in log

• Crash recovery discovers latest valid block on disk

34

LSF example

35

LSF example

36

LSF example

37

LSF example

38

Storage backend - Logging block store (LBS)

Backend represents VM disk & checkpoint images with files on host

• Similar format to a LFS (performance, reliability)

• Split data and meta-data between internal & external storage

• Encryption & integrity checks on data file

• Meta-data and encryption keys located on protected internal storage (security)

39

LBS performance

.

40

Agenda

Smartphone hardware

I/O virtualization techniques

Examples from MVP hypervisor

• Storage virtualization

• GPS virtualization

• Telephony virtualization

41

GPS virtualization

42

GPS virtualization

43

Agenda

Smartphone hardware

I/O virtualization techniques

Examples from MVP hypervisor

• Storage virtualization

• GPS virtualization

• Telephony virtualization

44

Telephony virtualization

45

Telephony virtualization

GSM

• Dual SIM smartphones (geography dependent)

• Multiple International Mobile Subscriber Identities (IMSIs) per SIM (geo dep)

• GSM Alternate Line Service (ALS) (network dependent)

Network

• Call forwarding, ala Google Voice, Line2

Voice-over-IP (VoIP)

• Realistic with 4G/LTE

• Integration with enterprise Unified Commmunications (UC)