View
536
Download
6
Category
Preview:
Citation preview
OSb: OSv on BitVisor
Yushi OmoteUniversity of Tsukuba
Takahiro ShinagawaThe University of Tokyo
(2014/11/21 BitVisor Summit 3)
Monday, November 24, 14
BitVisor
BitVisor
Hardware
OS
App App App
http://www.justis.as-1.co.jp
Monday, November 24, 14
OSv
VMM
Hardware
OSvApp
http://medical-care.feed.jp
Monday, November 24, 14
BitVisor
Monday, November 24, 14
BitVisor OSv
Monday, November 24, 14
BitVisor OSv
http://www.root.ne.jp/nishide/shs/
Monday, November 24, 14
OSv on BitVisor
BitVisor
Hardware
OSvApp
Monday, November 24, 14
OSv on BitVisor
BitVisor
Hardware
OSvApp
OSbMonday, November 24, 14
OSv on BitVisor
BitVisor
Hardware
OSvApp
OSb
What the fuckno Virtio !
Physical interfacesucks !
What the hell isPRO/1000 !
Monday, November 24, 14
OSv on BitVisor
BitVisor
Hardware
OSvApp
OSb
What the fuckno Virtio !
Physical interfacesucks !
What the hell isPRO/1000 !
You have no choice.
Virtio
Monday, November 24, 14
~ The Road to OSb ~
Monday, November 24, 14
• Boot Process
• Some Drivers
OSv Code Reading- MBR- Read local disk with INT13/42.- Load command line, boot loader, OSv kernel.- Get memory map with INT15/E820.- Setup segment descriptors/page tables.- Switch to 64-bit mode.- premain()- main()...
- Serial / VGA output.- SATA.- Virtio NIC/BLK/RNG/SCSI.- ACPI.- APIC....
OSv may run onphysical machine.(=BitVisor)
Monday, November 24, 14
Only one thing to fix
drivers/acpi.cc:
// Copy the root table list to dynamic memory if (!is_bitvisor()) { status = AcpiReallocateRootTable(); if (ACPI_FAILURE(status)) { acpi_e("AcpiReallocateRootTable failed: %s\n", AcpiFormatException(status)); return; } }
Triple faultSkip on BitVisor
Monday, November 24, 14
Hello world !
Monday, November 24, 14
OSb
Whenever updating OSv images,you update the entire Physical Disk.
Troublesome...
http://www.dreamstime.comhttp://a.fsdn.com/sd/articles/14/11/12/1946208-1.jpg
On BitVisor,
(especially for development)
Monday, November 24, 14
Network-boot of OSb
./scripts/run.py
Monday, November 24, 14
Network-boot of OSb
Hardware
./scripts/run.py
Monday, November 24, 14
Network-boot of OSb
Hardware
./scripts/run.py
ImageUpdate
Monday, November 24, 14
Network-boot of OSb
Hardware
./scripts/run.py
ImageUpdate
IPMI
Monday, November 24, 14
Network-boot of OSb
Hardware
./scripts/run.py
ImageUpdate
IPMIPXE Boot
Monday, November 24, 14
Network-boot of OSb
Hardware
./scripts/run.py
ImageUpdate
IPMIPXE Boot
BitVisor
Monday, November 24, 14
Network-boot of OSb
Hardware
./scripts/run.py
ImageUpdate
IPMIPXE Boot
BitVisor
OSvNetwork Boot
Monday, November 24, 14
Network-boot of OSb
Hardware
./scripts/run.py
ImageUpdate
IPMIPXE Boot
BitVisor
OSvNetwork Boot
App
Monday, November 24, 14
Network-boot of OSb
BitVisor
Hardware
OSvApp
Network Boot
IPMI
./scripts/run.py
TTY
ImageUpdate
PXE Boot
Monday, November 24, 14
Performance?
0
6.50
13.00
19.50
26.00
OSb Linux (Host, Bare-metal)
Elap
sed
Tim
e (s
ec)
for (12-billion times) val++;
Monday, November 24, 14
Virtio NIC
BitVisor
Hardware
OSvApp
PRO/1000...
No PRO/1000...Virtio NIC please...
Monday, November 24, 14
Virtio NIC
BitVisor
Hardware
OSvApp
Virtio NIC
PRO/1000...
Implementing Virtio
No PRO/1000...Virtio NIC please...
Monday, November 24, 14
How to implement?
BitVisor
Hardware
OSvApp
Virtio NIC
Finally, OurVirtualization?
Monday, November 24, 14
How to implement?
BitVisor
Hardware
OSvApp
Virtio NIC
Finally, OurVirtualization?
No,BitVisor’s Way!
Monday, November 24, 14
Para Pass-through Virtio
BitVisor
Hardware
Virtio Ring
PRO/1000 Ring
OSvApp
MSI-X etc.
Intercept only interesting I/Os!
PCI Functions
Pass-through!
Monday, November 24, 14
Faking PCI IDs 0x1AF4(Virtio Device)
0x1000(Virtio NIC)
0 (Legacy Virtio)
1 (Virtio NIC)
Monday, November 24, 14
Faking PCI BARsBAR0: IO spacefor Virtio = PRO/1000 IO space
Everything elsePass-through(e.g. MSI-X)
Monday, November 24, 14
Virtio NIC Operations
• Virtio Ring
• Packet Transmission
• Packet Reception
Monday, November 24, 14
Virtio RingGuest selects rings: * Transmission Ring * Reception Ring
BAR0(IO)
Memory space
Req. Req.Req.
Virtio Ring (Available + Used)
BitVisor emulatesthese registers
Monday, November 24, 14
Packet Transmission
OSv(Driver)
BitVisor(Virtio NIC)
Req.
PRO/1000
Req.
Transmission Ring
Req.
Req. Req. Req.
Notify!(PIO)
Req.
Req.
Reuse
Transmission Ring
VMExit
Monday, November 24, 14
Packet Reception
OSv(Driver)
BitVisor(Virtio NIC)
Req.
PRO/1000
Req.
Reception Ring
Req.
Req. Req. Req.
Interrupt
Req.
Req.
Reuse
Reception Ring
VMExit
Monday, November 24, 14
Netperf TCP_STREAM
0
250.00
500.00
750.00
1000.00
OSb Linux (Host, Bare-metal)
941.39
816.78
Thr
ough
put
(Mbp
s)
Monday, November 24, 14
VMExits are Costly...
OSv(Driver)
BitVisor(Virtio NIC)
PRO/1000
Interrupt
VMExit
Notify!(PIO)
VMExit
Guestmode
Hostmode
SwitchingOverhead
Monday, November 24, 14
Exitless Virtio with Dedicated Core
Req. Req.Req.Transmission
Ring
Req. Req.Req.Reception
Ring
Req. Req.Req.
Req. Req.Req.
Interrupt (IPI)
AlwaysGuestMode
AlwaysHostMode
Polling
ELVIS [Har’El et al. ATC’13]
Monday, November 24, 14
Implementation Summary
• Core Concealing
• Modifying ACPI MADT Table
• Notification PIO Pass-through
• Pass-through to ineffective PIO
• Interrupt
• Get vector # from MSI-X table and Send IPI
• (Or ask PRO/1000 to trigger)
Monday, November 24, 14
Netperf TCP_STREAM(Exitless Virtio)
0
237.50
475.00
712.50
950.00
OSb Linux (Host, Bare-metal)
941.39921.83
Thr
ough
put
(Mbp
s)
Monday, November 24, 14
Ping(Exitless Virtio)
0
0.04
0.08
0.12
0.16
OSb Linux (Host, Bare-metal)
0.16
0.12
Late
ncy
(mse
c)
Monday, November 24, 14
Still Under Optimization
• TODOs
• Interrupt Moderation
• Zero Copy/VMDq
• Advanced PRO/1000 Descriptor
• TCP Offloading
Monday, November 24, 14
Summary &Future Work
• Summary
• Running OSv on BitVisor
• Virtio NIC + Optimization
• Future work
• Further Optimization & Evaluation
• Other Virtio Devices (BLK, RNG...)
Monday, November 24, 14
Thank you !
http://www.root.ne.jp/nishide/shs/
Monday, November 24, 14
Recommended