Upload
others
View
6
Download
0
Embed Size (px)
Citation preview
TM
Freescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc.,
Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink
and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc.
The Freescale Embedded Hypervisor
November, 2010
Jacques Landry
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc.,
Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink
and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc. 2
Agenda
►AMP Considerations
►Technical Overview of the Freescale Embedded Hypervisor
►ePAPR & Device Trees Overview
►Porting an OS to the Freescale Embedded Hypervisor
►Hypervisor Boot Flow
►Summary
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc.,
Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink
and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc.
AMP Considerations
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc.,
Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink
and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc.
Partitioning Multicore Systems
Multicore
System
Hardware
CPU
Shared
Cache
CPU CPU CPU
I/O
Interrupt
Controller
Memory
I/O I/O I/O I/O
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc.,
Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink
and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc.
Symmetric Multiprocessing (SMP)
Multicore
System
Hardware
CPU
Shared
Cache
CPU CPU CPU
I/O
Interrupt
Controller
Memory
I/O I/O I/O I/O
Linux®
App
AppApp
AppApp
App
Same OS image runs on all CPUs
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc.,
Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink
and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc.
Asymmetric Multiprocessing (AMP)
CPUCPU CPU CPU
Linux®
App
App
Memory
RTOS
App
App
Legacy OS
App
App
Memory
Memory Memory
I/O I/O I/O I/O
Multicore
System
Hardware
Shared
Cache
I/O
Interrupt
Controller
Different OS images runs on all CPU
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc.,
Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink
and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc.
Unsupervised AMP (Asymmetric Multiprocessing)
CPUCPU CPU CPU
Linux®
App
App
Memory
RTOS
App
App
Legacy OS
App
App
Memory
►Requires partitioning hardware resources:
• Private resources: CPUs, memory, I/O devices
• Shared resources: memory, devices
►Doing this cooperatively (all operating systems well-behaved) presents challenges
Memory Memory
I/O I/O I/O I/O
Multicore
System
Hardware
Shared
Cache
I/O
Interrupt
Controller
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc.,
Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink
and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc.
Supervised AMP
CPUCPU CPU CPU
partition
App
App
Memory
partition
App
App
partition
App
App
Memory
Memory Memory
I/O I/O I/O I/O
Hypervisor/Supervisor
Shared
Cache
I/O
Interrupt
Controller
Linux® RTOS Legacy OS
Multicore
System
Hardware
►Hypervisor Software• Analogous to role of an operating
system kernel in managing user processes
• More privileged than operating systems
• Enforces system security
• Manages globally shared resources
• Virtualizes some resources– e.g. interrupt controller, UART
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc.,
Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink
and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc.
Use Cases
►Virtualization enables running multiple OSes on a system at the same time
►Why do this?
• Oversubscription or underutilized resources Run multiple OSes on a single underutilized CPU
• Consolidation Multiple operating systems/partitions on a single multicore chip
Multiple homogeneous operating systems in an AMP configuration on multiple cores
• Divided workload–(e.g. control plane, data plane) Multiple operating systems, possibly heterogeneous, need to work securely and
seamlessly together. Isolation mechanisms are needed for safety, robustness. Efficient inter-partition communication mechanisms are needed for cooperation.
• Isolate untrusted software/sandboxes
• Migration Migrate functionality from legacy RTOS to another OS (e.g. Linux).
• Security Secure partition for sensitive security tasks (e.g. access rights control, rule
definitions, key storage/management)
CPU
Guest
OS
MemoryCPU
Memory
Guest
OS
CPU
Guest
OS
Memory
hypervisorhypervisor
Guest
OS
CPU
hypervisorhypervisor
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc.,
Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink
and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc.
Unsupervised AMP Considerations
►Security– all OSes must be trusted and well behaved because any OS can map any physical address
►How will global resources be initialized, shared, and/or used? How are global events handled?
►MPIC
• who initializes?
►PAMU
• who will initialize and set up PAACT table?
• How will PAACT configuration be specified/described?
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc.,
Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink
and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc.
Unsupervised AMP Considerations…continued
►CPC (Platform Cache) Partitioning• who initializes?
►Corenet Coherency Domains• How are they set up and initialized
►Scarce Resources– P4080 has only 2 dUARTs, GPIO pins
►Datapath Initialization• how are Fman, Bman, Qman, PME initialized?
►OS assumptions about physical address 0x0?
►Debugging• how to debug multiple OSes at the same time?• What communications channel? JTAG?
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc.,
Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink
and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc.
Unsupervised AMP Considerations…continued
►error management• who will set up configurable error parameters? (e.g. single bit ECC error
threshold) in DDR, CPC, etc
• where is the platform error interrupt (interrupt 0) routed to? How is it handled?
Error conditions: global DDR, CPC, CCM, internal SRAM
How are device errors handled? PCIE, Qman, Bman, Fman
• who handles PAMU access violations (interrupt 8)?
• If one OS crashes, how will recovery occur? /How will other OSes know?
►Boot• what is the boot sequence for starting all OSes? Do they all start at the same
time?
• Is there a 'primary' OS that takes care of global initialization?
• How will the sequencing work?
• How are secondary CPUs booted?
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc.,
Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink
and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc.
Technical Overview of the Freescale Embedded Hypervisor
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc.,
Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink
and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc.
Freescale Embedded Hypervisor
CPUCPU CPU CPU
partition
App
App
Memory
partition
App
App
partition
App
App
Memory
Memory Memory
I/O I/O I/O I/O
Hypervisor
Shared
Cache
I/O
Interrupt
Controller
Linux® RTOS Legacy OS
Multicore
System
Hardware
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc.,
Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink
and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc.
Freescale’s Embedded Hypervisor
►A small hypervisor for embedded systems based on Power Architecture®
technology (architecture version 2.06)
► Initial version focuses on static partitioning
• CPUs, memory and I/O devices can be divided into logical partitions
• Partitions are isolated from one another
• Configuration is fixed until a reconfigure and system reboot
• Not addressing problem of multiple operating systems on 1 CPU
►Uses the Embedded Hypervisor feature in the QorIQ/e500mc which makes
virtualization efficient
►Uses a combination of full-virtualization and para-virtualization which
provides good performance and minimal changes to guest operating
systems
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc.,
Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink
and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc.
Hypervisor Contrasts
Freescale Hypervisor
Implementation
Traditional Hypervisor
Implementation
Requirement: supervised AMP --
isolation, performance
Implications: No more than one OS
per core, OS has direct control of
high-speed peripherals
Requirement: high level of
virtualization-- solves problem of
under-utilized CPUs, plus isolation
Implications: more than one OS
per core, complexity, performance
implications
QorIQ™ P4080 hypervisor hardware assists
in meeting both requirement sets
Guest
OS
CPU
Guest
OS
CPU
Guest
OS
CPU
Guest
OS
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc.,
Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink
and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc.
Hypervisor Features
virtual CPU
(e500vcpu)services boot services
(ePAPR)
Emulation
(privileged instructions)
guest operating system
debug stub
Hypervisor
Debug console
hypercalls device tree
system hardware
UART
device
tree
direct
I/O
mux
UART
Doorbells
GPIOPIC
IOMMUByte
Channels
Partition
Mgmt
Operating System sees a virtual core plus hypervisor services
► Virtual CPU (like e500mc
minus hypervisor features)
► Services via hypercall
► Debug stub interface for
debugging guest operating
systems
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc.,
Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink
and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc.
MMU – CPU to Memory Access Control
CPU
I/O
CPU
Linux® RTOS
Memory Access
OK
I/O I/O
CPU CPU
Memory Access
Denied
►MMU is
controlled by
hypervisor
and restricts
all CPU
accesses to
physical
address
space
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc.,
Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink
and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc.
IOMMU – Device to Memory Access Control
CPU
I/O
CPU
Linux® RTOS
Access
Denied
Memory
I/OI/O
CPU CPU
Memory
IOMMUAccess
OK
► IOMMU
enforces I/O-
to- memory
accesses
►A key
component in
a securely
partitioned
system
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc.,
Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink
and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc.
Direct I/O
Guest
Hypervisor
driver
physical hardware
► PCI
► Serial RapidIO®
► DMA
► USB
► SD/MMC
I/O device is assigned and dedicated to partition
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc.,
Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink
and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc.
► Network interfaces
► Security
► Pattern matcher
Direct I/O thru Portal
Guest
Hypervisor
driver
physical hardware
portal
driver
portal
Guest A Guest B
P4080 data path portal is assigned and dedicated to partition
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc.,
Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink
and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc.
Virtual I/O – Hypervisor
Guest
Hypervisor
driver
physical hardware
► Interrupt
controller
► I2C
► GPIO
► Byte-channelsdriver
hypercall or emulation
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc.,
Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink
and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc.
virtual CPU
(e500vcpu)services boot services
(ePAPR)
Emulation
(privileged instructions)
guest operating system
debug stub
Hypervisor
Debug console
hypercalls device tree
system hardware
UART
device
tree
direct
I/O
mux
UART
Doorbells
GPIOPIC
IOMMUByte
Channels
Partition
Mgmt
Boot Services – ePAPR
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc.,
Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink
and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc.
►ePAPR (Embedded Power Architecture® Platform Requirements)
►Defines boot program to client program interface
• Boot program: firmware, hypervisor
• Client program: bootloader, hypervisor, OS
►Device tree
• Data structure that represents a partition‟s hardware and virtual resources
CPUs, memory, I/O devices, hypervisor-provided resources
• Address passed to boot CPU
►Multi-CPU boot architecture
• Single boot cpu
• Mechanisms to start secondary CPUs
Boot Services – ePAPR
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc.,
Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink
and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc.
Hypervisor Device Trees– HW, HV, Guest
u-boot
hypervisor
HW
Dev tree
Guest #2
dynamically
created and
loaded into
guest memory
Loaded into
hypervisor
memory by u-
boot. Hardware
dev tree /chosen
node points to
HV config tree
Guest #1
Guest Dev tree Guest Dev tree
HV config tree
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc.,
Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink
and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc.
Guest Device Tree
root
cpus
cpu0
cpu1
memory
virtual interrupt controller
shared memory
hypervisor
byte-channel
doorbell
example►A data structure used for
representing a partition‟s
physical and virtual devices
►See application note
AN3649 - “Understanding
Device Tree Files in
Multicore Hypervisor/LWE
Implementations”
► AN3649 contains specific
details on both hypervisor
and partition device trees
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc.,
Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink
and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc. 27
Core Enumeration
►From the perspective of a partition, the CPUs are contiguous.
However, the actual physical CPUs being utilized may be non-
contiguous.
►For example, the master Hypervisor tree may make physical CPUs
1, 3, and 5 available to a partition. However, from the perspective of
the partition the available CPUs are viewed as CPUs 0, 1, and 2.
►Additionally, no CPUs other than those explicitly made available to a
partition are visible from within the partition.
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc.,
Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink
and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc.
virtual CPU
(e500vcpu)services boot services
(ePAPR)
Emulation
(privileged instructions)
guest operating system
debug stub
Hypervisor
Debug console
hypercalls device tree
system hardware
UART
device
tree
direct
I/O
mux
UART
Doorbells
GPIOPIC
IOMMUByte
Channels
Partition
Mgmt
Virtual CPU
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc.,
Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink
and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc.
Virtual CPU
► The behavior of CPU facilities (instructions/registers/interrupts) as seen by an OS-- an e500mc minus the hypervisor extensions
► Full virtualization is used– i.e. OS is not hypervisor aware with respect to CPU behavior
• there are some exceptions to this general rule
► User and kernel mode privileged instructions and registers behave normally
► Hypervisor privileged instructions and register accesses trap to hypervisor and are emulated by the hypervisor
lwz r3,(r4)
addi r3,r3,1
sc
User mode
mfspr r3,spr_dec
Hypervisor
Decrementer
emulation
Kernel mode Hypervisor mode
system call
privilege
trap
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc.,
Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink
and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc.
virtual CPU
(e500vcpu)services boot services
(ePAPR)
Emulation
(privileged instructions)
guest operating system
debug stub
Hypervisor
Debug console
hypercalls device tree
system hardware
UART
device
tree
direct
I/O
mux
UART
Doorbells
PowerMPIC
ErrorByte
Channels
Partition
Mgmt
Hypervisor Services – hcalls
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc.,
Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink
and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc.
Hypervisor Services – hcalls
► Interrupt controller (MPIC)
►Byte-channels – character I/O stream
► Inter-partition signaling – doorbell
►Partition management
• Start/stop/image-loading
• Partition management interrupts
►Reset
►Power management– change clock frequency, power states
►Error Management
►Future
• CPU hot plug/unplug
• GPIO– supports partitioning of GPIO pins
• IOMMU– supports create/destroy mappings
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc.,
Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink
and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc.
VMPIC API
Hypercall Description
FH_VMPIC_SET_INT_CONFIG Configures the specified interrupt
FH_VMPIC_GET_INT_CONFIG Returns the configuration of the specified interrupt
FH_VMPIC_SET_MASK Sets the mask for the specified interrupt source
FH_VMPIC_GET_MASK Returns the mask for the specified interrupt source
FH_VMPIC_GET_ACTIVITY Returns a value indicating the activity status of an interrupt source,
regardless of whether an interrupt has been requested or is in
service.
FH_VMPIC_IACK Acknowledges an interrupt and retrieves the interrupt number.
FH_VMPIC_EOI Signals the end of processing for the highest-priority interrupt
currently in service.
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc.,
Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink
and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc.
Byte-Channels
►Byte-channel– a hypercall based character I/O channel
►Flexible endpoint configuration
• A physical UART on the QorIQ™ P4080
• Another byte-channel endpoint
• A byte-channel to UART multiplexer
• A hypervisor debug stub
• The hypervisor console
debug stub
Hypervisor
Debug console
Host
UART
Byte-
channel
mux
UART
partitionpartitionpartitionpartition
RS232
gdb (host)telnet telnet
Byte-channel
mux server
byte-channel
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc.,
Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink
and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc.
Debug Stubs
►Hypervisor provides an internal API that allows debug stubs to be
created and built into the hypervisor.
• Currently mutually exclusive from guest debug mode, where a guest
owns the CPU debug resources (debug interrupt and registers)
►Two stubs supported today:
• gdb
• TRK (Code Warrior)
►Each stub controls 1 CPU
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc.,
Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink
and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc.
CPUCPU CPU
partition
Memory
partition
Memory
Memory
Hyper-
visor
MPIC
OS
System
Hardware
Debug Stub Event Flow
OS
UART
MUX
stub stub stub
GDB remote serial protocolMUX server
GDB
Host
GDB
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc.,
Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink
and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc.
Inter-partition Signaling
►Mechanism by which
operating systems
can signal each other
►A one-way signal
with no payload
which results in an
external interrupt in
the destination
partition
►One-to-many, many-
to-one supported
Hypervisor
partitionpartitionpartitionpartitionreceive
endpoints send
endpoint
receive
endpoint
receive
endpoint
send
endpoint
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc.,
Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink
and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc.
Partition Management
CPUCPU CPU CPU
App
App
Memory
Memory
Memory Memory
I/O I/O I/O I/O
Hypervisor
Shared
Cache
I/O
Interrupt
Controller
Linux®
App
App
RTOS
App
App
Legacy OS
Multicore
System
Hardware
►Capabilities• Copy data to/from another
partition’s memory (e.g. loading OS
images)
• Starting other partitions
• Rebooting other guests
• Notifications– guest watchdog fires,
guest requests reboot, error
conditions
runningpartition partition partition
running runningstopped stopped
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc.,
Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink
and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc. 38
Privilege Levels – Guest State (GS) MSR bit
OS
App
App
partition
User
MSR[PR=1][GS=1]
Kernel/Supervisor
MSR[PR=0][GS=1]
Hypervisor
MSR[PR=0][GS=0]
Under Hypervisor
OS
App
App
partition
User
MSR[PR=1][GS=0]
Kernel/Supervisor
MSR[PR=0][GS=0]
Bare Metal
CPU
Memory
I/O
CPU
Memory
I/O
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc.,
Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink
and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc.
Standards
►power.org ePAPR• 1.0 complete in 8/2008
• Resource discovery (device tree)
• Multi-CPU boot
►power.org Embedded Virtualization Committee• Virtual CPU standard– the behavior of instructions and registers
under a hypervisor Working on RFC to the Power ISA– targeting 2.07
• Paravirtualization & standard hcalls Device tree related– hypervisor node
Shared page mechanisms
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc.,
Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink
and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc.
ePAPR & Device Trees Overview
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc.,
Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink
and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc.
Boot Program / Client program
►Boot Program
• Firmware
• Second stage
bootloader
• Hypervisor
►Client Program
• Second stage
bootloader
• Hypevisor
• Operating system
• Other bare metal
application
guest operating system
Hypervisor
device tree
Boot firmware
device tree
operating system
Boot firmware
device tree
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc.,
Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink
and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc.
ePAPR 1.0
► Power.org released the Power Architecture™ Platform Requirements (PAPR) specification in August 2006-- for desktop/server platforms
► ePAPR 1.0 for embedded systems addresses „boot services‟– how a boot program initializes hardware and boots a client program
► Benefits of standard interfaces
• Reduced OS porting effort and cost
• Enables to development of standard boot programs (firmware and hypervisors)
► Key areas
• State of machine when control is transferred to client (e.g. registers, MMU, state of interrupts)
• Device Discovery – definition of device tree
• Multi-cpu boot architecture
• ELF (Executable and Linking Format) for client programs
► Loosely related to IEEE 1275, and draws heavily on on-going work being done in the PowerPC Linux community
• (booting-without-of.txt)
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc.,
Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink
and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc.
Initial Machine State
►ePAPR defines the state of the hardware when control is transferred
to a client program
• Registers
• MMU
• CPUs
• Memory
• State of I/O devices– no interrupts or DMA
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc.,
Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink
and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc.
Initial State of Registers
Register Value
MSR PR=0 supervisor state
EE=0 interrupts disabled
ME=0 machine check interrupt disabled
IP=0 interrupt prefix-- low memory
IR=0,DR=0 real mode (see note 1)
IS=0,DS=0 address space 0 (see note 1)
SF=0, CM=0, ICM=0 32-bit mode
R3 Effective address of the device tree image.
R4, R5, R8,
R9
0
R7 shall be the size of the boot or secondary IMA in bytes
TCR WRC=0, no watchdog timer reset will occur
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc.,
Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink
and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc.
Initial Mapped Areas (IMA)
► A client program‟s IMA is a region of memory that contains the entry
points for a client program.
► Requirements:
• An IMA shall be virtually and physically contiguous
• An IMA shall start at effective address zero (0) which shall be mapped to a
naturally aligned physical address
• The mapping shall not be invalidated except by a client program‟s explicit
action
• The Translation ID (TID) field in the TLB entry shall be zero.
• The memory and cache access attributes (WIMGE) have the following
requirements:
WIMG unspecified
E=0 (i.e., big-endian)
• An IMA may be mapped by a TLB entry larger than the IMA size, provided the
MMU guarded attribute is set (G=1)
• An IMA may span multiple TLB entries.
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc.,
Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink
and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc.
Device Tree Overview
►A device tree is a tree data structure with nodes that describes the physical devices in a system
►Abstracts most hardware details out of the OS/client-- enables firmware to provide an OS with a complete description of the physical hardware in a system– devices, hardware address map, interrupt routing
• Previously OSes were required to have hardcoded information about system hardware.
• Provides a basis for booting an operating system under a hypervisor in a partitioned system
►Each device node has property/value pairs that describe the device
►All nodes have a “binding” which document required properties– the ePAPR documents bindings for some common devices
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc.,
Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink
and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc.
Examples
soc
serial
compatible = “simple-bus";
#address-cells = <1>;
#size-cells = <1>;
ranges = <0 e0000000 00100000>;
reg = <e0000000 00000200>;
compatible = "ns16550"
reg = <4600 100>
clock-frequency = <0>
interrupts = <a 8>
interrupt-parent = < &ipic >
/
cpus
cpu@0
cpu@1
ethernet
serial
soc
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc.,
Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink
and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc.
Device Tree details
►ePAPR defines requirements for:
• Logical structure of the device tree – node names, paths, properties
• Standard properties
• Hierarchy & routing of interrupts (including cascaded interrupt controllers)
• Representation of
CPUs
Memory
Caches
• Device bindings for
PCI
Open PIC and ISA interrupt controllers
Serial devcies
Network devices
Device Control Registers (DCR)
• Binary format of device tree DTB
• DTS syntax
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc.,
Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink
and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc.
Standard Properties
►compatible
►model
►phandle
►status
►#address-cells and #size-cells
►reg
►virtual-reg
►ranges
►dma-ranges
►interrupts
►interrupt-parent
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc.,
Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink
and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc.
UART Node Definition Example
serial@11c500 {
device_type = "serial";
compatible = "fsl,ns16550", "ns16550";
reg = <11c500 100>;
clock-frequency = <0>;
interrupts = <24 2>;
interrupt-parent = <&mpic>;
};
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc.,
Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink
and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc.
Device Tree Compiler
►The typical process of using a device tree in an embedded Linux
system is:
• An ASCII representation of the device tree is created in an a 'device tree
source' (DTS) file
• DTS is compiled into a binary 'device tree blob' (DTB) file using a device
tree compiler (DTC) tool
DTB format is specified in the ePAPR
• Firmware loads the DTB into RAM and passes a pointer to the DTB to
the Operating System kernel when it is started
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc.,
Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink
and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc.
Multicore boot architecture
►ePAPR describes specifics on how secondary CPUs are booted for
a system with multiple CPUs
►Default boot architecture
• The boot program releases all CPUs from hardware reset
• 1 CPU is designated to be the client program‟s boot CPU
• All other CPUs are secondary and are placed into loop where the
CPUs spin, waiting for a spin table field to change that directs them
where to go
• Control is transferred to the client program on the boot CPU
• When the client program is ready for secondary cores to start, it
releases them by writing the spin table field with the desired address
►The architecture allows for other custom-defined secondary CPU
release mechanisms as well
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc.,
Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink
and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc.
Device Tree and Virtualization/Partitioning
►Each OS in a partitioned system is presented with a device tree
describing that partition‟s subset of physical resources
• CPU cores
• Memory
• I/O devices
►Shared or virtualized resources would also be described
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc.,
Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink
and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc.
Porting an OS to the Freescale Embedded Hypervisor
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc.,
Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink
and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc.
Porting Overview
►Initial State and Boot
►CPU
►SOC Platform
►Additional Hypervisor-provided Resources Services
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc.,
Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink
and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc.
Initial State and Boot
►An OS must be minimally device
tree aware
►Interrupt numbers passed in the
device tree are handles used in
VMPIC API hcalls
►In order to boot multiple CPUs
the spin table release mechanism
must be used
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc.,
Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink
and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc.
Hypervisor Device Trees– HW, HV, Guest
u-boot
hypervisor
HW
Dev tree
Guest #2
dynamically
created and
loaded into
guest
memory
Loaded into
hypervisor
memory by u-
boot.
Hardware dev
tree /chosen
node points to
HV config tree
Guest #1
Guest Dev
treeGuest Dev
tree
HV config tree
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc.,
Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink
and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc.
CPU Considerations
►The virtual CPU as seen by an OS behaves as a normal OS would
expect, but there are some differences.
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc.,
Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink
and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc.
SOC Platform Devices
►The following devices are hypervisor-owned and should not be
accessed directly:
• Interrupt controller (see vmpic services)
The MPIC timers are not available for guest use
• Global Utilities
Power management
Clock control
Reset control
• Peripheral Access Management Unit (PAMU)
• DDR memory controllers
• CPC (Platform cache)
• CCS (LAWs, coherence domains)
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc.,
Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink
and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc.
Hypervisor Services
► Interrupt controller (MPIC)
►Byte-channels – character I/O stream
► Inter-partition signaling – doorbell
►Partition management
• Start/stop/image-loading
• Partition management interrupts
►Reset
►Power management– change clock frequency, power states
►Error Management
►Future
• CPU hot plug/unplug
• GPIO– supports partitioning of GPIO pins
• IOMMU– supports create/destroy mappings
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc.,
Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink
and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc.
Hypervisor Boot Flow
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc.,
Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink
and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc.
Hypervisor Device Trees– HW, HV, Guest
u-boot
hypervisor
HW
Dev tree
Guest #2
dynamically
created and
loaded into
guest
memory
Loaded into
hypervisor
memory by u-
boot.
Hardware dev
tree /chosen
node points to
HV config tree
Guest #1
Guest Dev
tree
Guest Dev
tree
HV config tree
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc.,
Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink
and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc.
U-boot
►U-boot initialization
• Configuration of physical memory map (LAWs)
• Probing/initialization of DDR
• Enabling all caches, including platform cache
• Configuration clocks
• Setting up LIODNs
• Sets up ePAPR spin table for secondary CPUs, releases secondary
CPUs
• Loads hypervisor image into memory
• Updates hardware device tree, loads it into memory
• Transfers control to the hypervisor
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc.,
Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink
and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc.
Hypervisor
►Hypervisor initialization
• PAMU initialization
• Coherence domain setup– set up LAWS, CSDIDs
• Error Configuration– e.g. single bit ECC error thresholds
• Driver initialization
DDR
CPC
CCM
UART
• Release secondary CPUs
• Partition instantiation/creation
The boot CPU for each partition takes care of partition creation
• Control transfer to guest OS
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc.,
Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink
and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc.
Hypervisor Relocation
Boot time view Runtime view
0x0 0x0
HV
HV
OS
TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc.,
Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink
and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc. 66
Summary
►A common usage model for multicore systems will be to run multiple
operating systems on a single processor– this requires partitioning.
►A hypervisor provides a good solution to enforce partition
boundaries and provide services to manage global resources (like
the interrupt controller).
►The Freescale Embedded Hypervisor in conjunction with hardware
features of the QorIQ™ P4080 provides an efficient solution for
partitioning – secure partitions with minimal overhead.
TM