Solros: A Data-Centric Operating System Architecture for Heterogeneous...

Solros: A Data-Centric Operating System Architecturefor Heterogeneous Computing

Changwoo Min, Woonhak Kang, Mohan Kumar, Sanidhya Kashyap,Steffen Maass, Heeseung Jo, Taesoo Kim

Virginia Tech, eBay, Georgia Tech, Chonbuk National University

April 26, 2018

Changwoo Min Solros: Data-Centric OS April 26, 2018 1 / 21

Cambrian Explosion of Processor Architecture

Specialization of general-purpose processors

Generalization of co-processors

Specialization of co-processors

Blazingly fast IO Devices

Blazingly fast storage/memory

Blazingly fast network

How to exploit the full potential of such hardware devices without pain?

System-wide performance

Ease of programming

Outline

1 Heterogeneous Computing Architectures

2 Solros: Split-Kernel ApproachSolros ArchitectureOperating System Services

3 Evaluation

Host-Centric Approach

Host OS controls co-processors and IO devices

Examples: OpenCL, CUDA

Application

Host processor

Mem Core

I/O deviceSSD / NIC

Co-processorApplication

Mem Core controldata

Application

Host processor

Mem Core

I/O deviceSSD / NIC

Mem Core

controldata

Application

Host processor

Mem Core

I/O deviceSSD / NIC

Mem Core

controldata

Application

Host processor

Mem Core

I/O deviceSSD / NIC

Mem Core

③ controldata

Application

Host processor

Mem Core

I/O deviceSSD / NIC

Mem Core

③ controldata

Problem

Redundant data communicationComplex to program and hard to optimize

Coprocessor-Centric Architecture

Co-processors control IO devices

Examples: Xeon Phi (Linux), GPUfs [ASPLOS13], GPUNet [OSDI14]

Application

Host processor

Mem Core

I/O deviceSSD / NIC

Application

Co-processor

Mem Core

controldata

Application

Host processor

Mem Core

I/O deviceSSD / NIC

Application

Co-processor

Mem Core

controldata

Application

Host processor

Mem Core

I/O deviceSSD / NIC

Application

Co-processor

Mem Core

controldata

Application

Host processor

Mem Core

I/O deviceSSD / NIC

Application

Co-processor

Mem Core

controldata

Problem

Significant effort required for porting IO stack to co-processorNot completely exploiting powerful host processors

Outline

1 Heterogeneous Computing Architectures

2 Solros: Split-Kernel ApproachSolros ArchitectureOperating System Services

3 Evaluation

Solros Goal

Ease of programming

Best use of processor architecture

System-wide optimization

Solros Goal

Ease of programming

Best use of processor architecture

System-wide optimization

Challenge

Co-processor needs IO abstraction

IO stacks is branch-divergent and difficult to parallelize

It needs system-wide information

Solros Architecture

Split-Kernel Architecture

Data-plane OS

Runs on a co-processorProvides IO abstractionDelegates actual IO operations to a control-plane OS

Control-plane OS

Runs on a host processorRuns actual IO stackPerforms system-wide coordination

Solros Architecture

Control-plane OS: actual OS service + system-wide coordination

Data-plane OS: thin communication layer to host processor

controldata

Application

Host processor

OS proxy

Mem Core

I/O deviceSSD / NIC

Application

Co-processor

OS stub

Core Mem

Policy

Solros Architecture

controldata

Application

Host processor

OS proxy

Mem Core

I/O deviceSSD / NIC

Application

Co-processor

OS stub

Core Mem

① Policy

Solros Architecture

controldata

Application

Host processor

OS proxy

Mem Core

I/O deviceSSD / NIC

Application

Co-processor

OS stub

Core Mem

Policy

Solros Architecture

controldata

Application

Host processor

OS proxy

Mem Core

I/O deviceSSD / NIC

Application

Co-processor

OS stub

Core Mem

②③

Policy

Solros Architecture

controldata

Application

Host processor

OS proxy

Mem Core

I/O deviceSSD / NIC

Application

Co-processor

OS stub

Core Mem

②③

Policy

Co-processor has OS abstraction with minimal effort

Best use of each of the fat and lean processors

Efficient global coordination among devices (policy)

Operating System Services

1 Transport service

2 Filesystem service

3 Network service

Operating System Services

1 Transport service

2 Filesystem service

3 Network service

Transport Service

High performance data transfer among devices are challenging:

Uniform data transfer among devices

High contention in massively-parallel co-processor

Asymmetric performance between host processor and co-processor

Transport Service

Our approach

Uniform data transfer ⇒ system-mapped PCIe window

High contention ⇒ combining, replication, interleaving, etc.

Asymmetric performance ⇒ flexibly configurable (host DMA enginevs. co-processor DMA engine)

Transport Service

Our approach

Uniform data transfer ⇒ system-mapped PCIe window

High contention ⇒ combining, replication, interleaving, etc.

Asymmetric performance ⇒ flexibly configurable (host DMA enginevs. co-processor DMA engine)

See details in the paper

Filesystem Service

Peer-to-peer operation

Buffered operation

Host processor

DMA engine

Application

Co-processor

File system stub①

File system proxy

File system

controldata

Zero-copy of data between co-processor memory and SSDMinimal data transfer

Filesystem Service

Buffered operation

Host processor

DMA engine

Application

Co-processor

File system stub①

File system proxy

File system

controldata

Filesystem Service

Buffered operation

Host processor

DMA engine

Application

Co-processor

File system stub① ③

File system proxy

File system

controldata

Filesystem Service

Buffered operation

Host processor

DMA engine

Application

Co-processor

File system proxy

File system

controldata

Filesystem Service

Buffered operation

Host processor

DMA engine

Application

Co-processor

File system proxy

File system

⑤ controldata

Filesystem Service

Buffered operation

Host processor

DMA engine

Application

Co-processor

File system stub①

File system proxy

File system

controldata

Buffer cache

Reduce storage IO by leveraging shared buffer cache among co-processorsAvoid performance anomaly of peer-to-peer communication over PCIe bus

Filesystem Service

Buffered operation

Host processor

DMA engine

Application

Co-processor

File system proxy

Buffer cache

File system

controldata

Filesystem Service

Buffered operation

Host processor

DMA engine

Application

Co-processor

File system proxy

Buffer cache

File system

controldata

Filesystem Service

Buffered operation

Host processor

DMA engine

Application

Co-processor

File system proxy

Buffer cache

File system

controldata

Filesystem Service

Buffered operation

Host processor

DMA engine

Application

Co-processor

File system proxy

Buffer cache

File system

controldata

④⑥

Implementation

Host: 2-socket Xeon processor (12 cores each)

Co-processor: 4 Xeon Phi (KNC, 61 cores, Linux, PCIe Gen 3x16)

Storage device: 4 NVMe SSD

NIC: 100 Gbps Ethernet

ModuleLines of code

Added lines Deleted lines

Transport service 1,035 365

File system ServiceStub 5,957 2,073Proxy 2,338 124

Network ServiceStub 2,921 79Proxy 5,609 34

NVMe device driver 924 25SCIF kernel module 60 14

Total 18,844 2,714

Evaluation

Questions:

Performance of Solros services

Impact on real-world applications

Performance of Solros Services

10 100 1000

(a) file random read on SSD

Phi-SolrosPhi-Linux

Latency(usec)

(b) TCP latency: 64-byte message

Phi-SolrosPhi-Linux

File IO performance: 19x faster than the stock Linux on Xeon PhiTCP latency (99 percentile): 7x shorter than the stock Linux on Xeon Phi

Performance of Solros Services

Phi-Linux Phi-Solros0

Phi-Linux Phi-Solros

(a) file random read on SSD

StorageBlock/Transport

File system

(b) TCP latency: 64-byte message

Proxy/TransportNetwork stack

Significant performance gain in data transportRunning IO stack on co-processor is slower

Real-world Application - Image Search

Image search engine is running on Xeon Phi

Image database is on NVMe SSD (shared read-only)

Image search queries are from network

0 100 200 300

Phi-SolrosPhi-Linux

Time(sec)

Phi-SolrosPhi-Linux

Solros performs 2x faster than stock Linux on Xeon Phi

Conclusion

Solros, a new operating system architecture for co-processors and fastIO devices

Control-plane, data-plane architecture allow:

Supporting high-level OS abstraction on co-processorEfficient global coordination among devicesNear ideal IO performance from co-processor

We will release source code soon

Image Search - Scalability

Increase the number of Xeon Phi to 4

1 2 3 4

The number of Xeon Phi co-processors

Solros load balancing mechanism achieves near linear scaling

Related work

Control-plane/data-plane OS: Arrakis [OSDI’14], IX [OSDI’14]

OS for heterogeneous systems: Helios [SOSP]09], M3 [ASPLOS’16],Hydra [ASPLOS’08]

IO support for GPU: PTask [SOSP’11], GPUfs [ASPLOS’13], GPUnet[OSDI’14]

Transport service

Host's physical address space

Host RAM PCIe Window mmio

NICDRAM mmio

Xeon PhiDRAM mmio

Host's virtualaddress space

App 1 App 2

mmioDevice's physical

address sapce

mapped to

(across devices)

mapped to

(in-host)

Network Service (TCP)

Outbound operation

Inbound operation

Host processor

Application

Co-processor

TCP stub

TCP proxy

Load balancer

TCP stack

controldata

A load balancer on a host distributes incoming TCP connections to one ofleast-loaded co-processors.

See details in the paper.

Discussion

Hardware support other than Xeon Phi

Two atomic instructions: transport serviceMMU: isolation among co-processor applications

Scalability of control-plane OS

Limited by scalability of OS service, PCIe interconnect, andperformance of IO devices

Real-world Application - Text Search

CLucene text indexing engine running on Xeon Phi

Text data is on NVMe SSD

0 50 100 150

Phi-SolrosPhi-Linux

Time(sec)

Phi-SolrosPhi-Linux (virtio)

Solros performs 19x faster than stock Linux (ext4/virtio) on Xeon Phi

Solros: A Data-Centric Operating System Architecture for Heterogeneous...

Documents

A Scalable Architecture for Autonomous Heterogeneous

Management architecture for heterogeneous IoT

A Network Architecture for Heterogeneous Mobile …daedalus.cs.berkeley.edu/new_papers/Architecture.pdfA Network Architecture for Heterogeneous Mobile Computing ... 2 A Network Architecture

Identity Centric Architecture Aligning SOA with NGN

Heterogeneous Embedded Network Architecture - DiVA Portal

Guide to heterogeneous system architecture (hsa)

PL-4090, Sequential Consistency for Heterogeneous-Race-Free: Programmer-centric Memory Models for Heterogeneous Platforms, by Brad Beckmann

A computational architecture for coupling heterogeneous

A SECURITY ARCHITECTURE FOR NET-CENTRIC ENTERPRISE ...people.cs.vt.edu/.../WebServices/DISA-Security-Architecture.pdf · A SECURITY ARCHITECTURE FOR NET-CENTRIC ENTERPRISE SERVICES

Heterogeneous System Architecture Overview

FPGA 10x10 Heterogeneous Architecture with OpenDwarfs onresearch.cs.vt.edu/seec/2015-08-workshop/SEEC_Anshuman.pdf · 10x10 Heterogeneous Architecture with OpenDwarfs on FPGA

Data-Centric Architecture for Space Systemsflightsoftware.jhuapl.edu/files/FSW09_Joshi.pdfData-Centric Architecture for Space Systems Rajive Joshi, Ph.D. ... • Same angst about software

Architecture-Centric Software Development

Heterogeneous Missions Accessibility Context and Architecture

HELPFUL: Flexible Architecture to Control Heterogeneous

A Reconfigurable Heterogeneous Microserver Architecture

Scalable Oriented-Service Architecture for Heterogeneous

Intercloud Architecture Framework for Heterogeneous … · Intercloud Architecture Framework for Heterogeneous Multi-Provider Cloud based Infrastructure Services Provisioning 3 Figure

Data Entitlement in an API-Centric Architecture

8 - Architetture Software - Architecture centric processes