A Fast and Flexible Hardware-based Virtualization

A Fast and Flexible Hardware-based Virtualization

Mechanism for Computational Storage Devices

Dongup Kwon, Dongryeong Kim, Junehyuk Boo, Wonsik Lee, and Jangwoo Kim

Department of Electrical and Computer Engineering, Seoul National University

Background: Computational Storage

• SSD-FPGA integration for near-storage processing

− Fast data transfers between the storage and computation units

− Programmable operators and on-chip interconnects in an FPGA

NVMeSSD

FPGA

PCIeSwitch

DRAM DRAM Controller

SSD Controller

On-chipSwitch

Operators

StorageDevice

Computational storage = SSD + FPGA + near-storage processing

2/21

Background: I/O Virtualization

• SW-based virtualization: Paravirtualization (VirtIO)

• HW-assisted virtualization: Passthrough, SR-IOV, FVM*

*FVM: FPGA-assisted Virtual Device Emulation for Fast, Scalable, and Flexible Storage Virtualization, OSDI 2020

I/O virtualization enables resource sharing between VMs.

VMVirtio Front-end Driver

HostSW

Virtio Back-end Driver

HW

Linux Kernel

KVM/QEMU

SSD FPGA

HW SSD FPGA

SR-IOV SR-IOV

IOMMU

Paravirtualization

VMSSD/FPGA Driver

Linux Kernel

SR-IOV

3/21

Outline

• Background

• Motivation

− SW-based virtualization for computational storage

• FlexCSV: HW-assisted Virtualization Stack

• Evaluation

• Conclusion4/21

SW-based Virtualization Approach

• SW emulation of SSD-FPGA integrated devices

• Host SW-level device resource allocation and scheduling

SW-based virtualization provides flexible virtual device construction mechanisms.

SSD FPGA

PCIeSwitch

VM

Device EmulationHostSW

HW

User Application

SSD FPGA

Resource Allocation

VM’s view of the virtual devices

5/21

Limitation #1:CPU-centric Device Emulation

• CPU-centric device orchestration & data transfers

• Cannot achieve full potential of near-storage processing

The bottleneck shifts to the SW components in a virtualized environment.

Percentage of Execution Time

SW HW(SSD+FPGA)

Hash

Filter

Grep

100%50%0%

VM

Device EmulationHostSW

HW

User Application

SSD FPGA

Resource Allocation

6/21

Limitation #2:Static Resource Allocation

• Static VM-to-HW resource allocation & scheduling

• Cannot achieve cost-effectiveness due to inefficientresource sharing

Static resource allocation incurs extra costs for the additional HW resources to meet QoS requirements.

.

FPGA

SSD

Operator

Operator

OperatorVMs

StaticVM-to-HW mapping

StorageDevice

7/21

Limitation #3:Coupled HW Architecture

• SSD-FPGA coupled designs & fixed provisioning

• Cannot provide flexible device/resource configurations

FPGA

SSD

StorageDevice

FPGA

SSD

StorageDevice

SSD-FPGA coupled architectures suffer from limited device scalability.

Operator

Operator

Operator

Operator

FPGA BW > SSD BW FPGA capacity < SSD capacity

Operator

Operator

Operator

Operator

8/21

Design Goals

Device Sharing

Device Scalability

High Performance

SW-basedVirtualization

Low Cost

Trap-and-emulate

Tightly-coupled architecture

CPU-centric orchestration

Static resource allocation

Design Goals

9/21

Outline

• Background

• Motivation


• Evaluation

• Conclusion

10/21

FlexCSV: SW/HW Architecture

• HW virtualization for computational storage

VM

HW

User Application

SSD

FlexCSV Engine

IOMMU

SSD FPGA SSD/DRAM Controllers

Switch

FlexCSV Engine (FPGA)

SR-IOV

Device Emulation

Resource Allocation

Operators

FlexCSV offloads a virtualization stack for computational storage devices.

11/21

FlexCSV: Key Ideas

FlexCSV

HW-assisted virtualization(including SR-IOV)

SSD-FPGA decoupled architecture

HW-level resource orchestration

Dynamic resource allocation

Device Sharing

Device Scalability

High Performance

Low Cost

Design Goals Key Ideas

12/21

Key Idea #1:HW-assisted Virtualization

• SR-IOV implementation in FlexCSV Engine

• SSD/FPGA sharing between VMs with direct HW access

FlexCSV Engine virtualizes itself through SR-IOV and offers device sharing.

FlexCSV Engine

⋯

Host OS VM 0 VM 1 VM n⋯

PF VF 0 VF 1 VF n

Virtualization Layer

FPGA FPGASSD SSDSSD

13/21

Key Idea #2:HW-level Orchestration

• NVMe extension for data processing requests

• Guest/host OS bypassing and direct data communications

FlexCSV Engine orchestrates SSD and FPGA operations without SW arbitration.

struct nvme {

u8 rw;

u64 rsvd;

u64 prp1;

u64 prp2;

u64 slba;

}

struct proc {

u64 op;

u64 src;

u64 dst;

u32 size;

u64 param;

} DRAM

SSD

Switch

Operators

FPGA

FlexCSVEngine

1

23

456

SSD ctrl

FPGActrl

SQ CQ

14/21

Key Idea #3:Dynamic HW Allocation

• Renaming of user-requested HW resources

• Efficient use of HW resources – High HW utilization

FlexCSV Engine implements HW renaming logic for dynamic resource allocation.

FPGA

Operator

Operator

Operator

VMs

FlexCSV Engine

ResourceAllocation

OperatorRenaming

SSD

15/21

FPGA

Key Idea #4: SSD-FPGA Decoupled Architecture

• Decoupled HW through board-level PCIe switches

• Scalable virtual devices with many PCIe-attached cards

FlexCSV Engine provides scalable and flexible device/resource configurations.

FlexCSVEngine

VM’s view HW implementation

PCIeSwitch

FPGASSD FPGASSDSSDFPGAFlexCSV

EngineFPGASSDSSD

SwitchNetwork

16/21

FlexCSV Prototype

• HW Prototype

−Supermicro Server 4029GP-TRT2

− Intel Optane 900P SSDs

− Xilinx U250 FPGA (FlexCSV Engine)

• SW Frameworks

− Ubuntu / Linux kernel v5.3

− KVM / QEMU v3.0

FlexCSV prototype is built on off-the-shelf HW devices and open-source SW.

Intel Xeon Gold 5118

XilinxAlveo U250

Intel Optane900P SSDs

17/21

Outline

• Background

• Motivation


• Evaluation

• Conclusion

18/21

Near-storage Processing Performance

• 8 FPGA benchmarks with direct SSD read & write

• Guest/host OS bypassing + fast data copy ➔ 2.4x speedup

XXX

0.0

0.5

1.0

1.5

2.0

2.5

3.0

Encrypt Decrypt Hash Aggregate Filter Grep KNN Bitmap

Read-and-Write

Speedup

Full SW FlexCSV (SW)Full SW SW + P2P FlexCSV Higher is better

FlexCSV achieves high performance through its HW-assisted virtualization.

19/21

QoS Evaluation with Oversubscription

• 2 operators + 4 VMs with different request rates

• Dynamic allocation + partial reconfiguration ➔ 2.4x better

0.0

0.2

0.4

0.6

0.8

1.0

0.54

14

24

44

84

QoS V

iola

tion R

atio

Static (worst) Static (best) Dynamic Dynamic + PR

LowHigh FPGA Resource Contention

FlexCSV achieves lower QoS violations through its efficient HW resource use.

Lower is better

20/21

Thank You!

A Fast and Flexible Hardware-based Virtualization Mechanism for Computational Storage Devices, ATC 2021

Dongup Kwon, [email protected], https://hpcs.snu.ac.kr/~dongup/

Intel Xeon Gold 5118

XilinxAlveo U250

Intel Optane900p SSDs SSD/DRAM Controllers

Switch

FlexCSV Engine (FPGA)

SR-IOV

Device Emulation

Resource Allocation

Operators

21/21

Documents

A Fast and Flexible Hardware-based Virtualization