November 22, 2019

UNICORE: A toolkit to automatically build unikernels

Gaulthier Gain Prof. Laurent MathyCyril Soldani

Grascomp Doctoral Day 2019

[first].[last]@uliege.be

Funded by the Horizon 2020 Framework Programme of the European Union under agreement No 825377

1/12

Virtual Machines vs Containers

Virtual Machines (VMs) Containers

Hardware

Operating System

Container Engine

Bins/Libs Bins/LibsBins/Libs

App1 App3App2

Hardware

Hypervisor

Guest OS Guest OSGuest OS

Bins/Libs Bins/LibsBins/Libs

App1 App3App2

‣ Each virtual machine requires its own underlying guest operating system as well as a hypervisor.

‣ It provides strong isolation, virtual machines are heavyweight since they require a full OS image to run.

1/12

Virtual Machines (VMs) Containers

Hardware

Operating System

Container Engine

Bins/Libs Bins/LibsBins/Libs

App1 App3App2

Hardware

Hypervisor

Guest OS Guest OSGuest OS

Bins/Libs Bins/LibsBins/Libs

App1 App3App2

‣ Each container shares the OS kernel which provides better efficiency than virtual machines.

‣ It results into a poor isolation and containers are subject to many vulnerabilities (bigger attack surface).

Virtual Machines vs Containers

2/12

Virtual Machines (VMs) Containers

Strong isolation

Heavyweight — Degrade performance

Poor isolation — A lot of exploits

Lightweight

Dilemma

2/12

Virtual Machines (VMs) Containers

Strong isolation

Heavyweight — Degrade performance

Poor isolation — A lot of exploits

Lightweight

Solution Unikernels!

Dilemma

!!

3/12

Virtual Machines (VMs) Unikernels

Hardware

Hypervisor

Bins/Libs

App1

Kernel

Bins/Libs

App2

Kernel

Hardware

Hypervisor

Bins/Libs Bins/Libs

App1 App2

Unikernels

‣ Unikernels are purpose-built: ‣ Thin kernel layer (only the features that the application

needs). ‣ Essential functions dead-code elimination. ‣ One application single address space (isolation).

4/12

Unikernels gains

! Well suited for cloud computing and IoT.

‣ Fast instantiation, destruction and migration time: ‣ Hundred of milliseconds.

‣ Small per-instance memory footprint: ‣ Few MBs or even KBs.

‣ High density: ‣ 10k instances on a single host.

‣ High performance: ‣ 10-40 Gbps throughput.

‣ Reduced attack surface.

5/12

Challenges

‣ Manual process requiring significant expert resources and

takes a lot of development time:

‣ Dependency analysis: gathers the right symbols, system

calls and shared libraries of an existing application.

‣ Rewrite and migrate libraries and kernel primitives.

‣ Multiple cycles consisting of measurement, programming

improvements and tweaking.

‣ The build process must be repeated for each target platform,

architecture and application.

The UNICORE project

‣ Motivations:

‣ Simplify creation, deployment and management of

unikernels.

‣ Concentrate efforts on a single base unikernel (follow-up

of the Unikraft project).

‣ Open-source toolkit that will enable secure and portable

unikernels deployment.

‣ Concept:

‣ Everything is a library.

‣ Monolithic software code is broken down into smaller

components (µlibs).

6/12

7/12

Two main components: Library pools

unicore_barex86_64

unikernel binaries

API

unicore_bareARM32

unicore_xenx86_64

unicore_xenARM32

unicore_kvmx86_64

unicore_kvmARM32

libx86_64arch.o libarm32arch.o libmipsarch.o

libbareplat.o libxenplat.o libkvmplat.o

arch lib pool

platform lib poolinternal lib pool

liblwip.o libvfs.o libc.olibnewlibc.olibfat.olibtcpip.o

libnetback.o

libnetfront.o

libbuddy.o

libheap.o

libpython.o

liberlang.o

drivers memory alloc. runtimes

network stack filesystems standard libs

external lib pool

App

main lib pool1. Library pools:

‣ Internal/External µlibs.

‣ Platform µlibs.

‣ Architecture µlibs.

2. Toolchain:

‣ Set of tools to

automatically build

unikernels.

‣ Generate binaries for

multiple platforms and

architectures.

Two main components: Toolchain

App Dependency analysis Dependencies

VMLibs

Kernel

Dec

ompo

se

µlib

µlib

Optimize

APIs

UNICORE Code Repo

µlib

µlibVerify

Build

TestReconfig

Generate optimized images

Help to break down existing libraries and kernel primitives into µlibs

Ensure that the features of the unikernel match those of an app running on a standard OS

Use the dependencies to select the relevant µlibs and produce unikernel

Analyse and extract dependencies

8/12

Dependency Analysis Tool

‣ The goal find a sufficient, but minimal superset of other

components that are required by an existing application to

work correctly.

‣ To gather information of an existing application, we

considered 2 mechanisms:

1. Static analysis: examining binary file without execution.

‣ Limited since applications can be stripped or obfuscated.

2. Dynamic analysis: analysing binary file by running it.

‣ Find all the execution path of application: unfeasible.

‣ Heuristic: high application code coverage (Tests with

expected inputs, fuzz-testing, … )

9/12

Automatic Build Tool

10/12

‣ The goal of the automatic build tool is to produce unikernel(s)

that can run the target application(s).

‣ The tool is divided into 2 components:

‣ A controller that uses the previous inputs to select the

right µlibs (from library pools).

‣ A build system to compile and link the unikernel into

target VM images.

‣ For now, sources of an existing application are required.

‣ Future work: Consider binary rewriting techniques.

‣ Port an existing binary application as unikernel.

103

104

100

101

102

103

Porting an existing application (SQLite)

1195Mb

125Mb

0.868Mb

SQLite dockerDebian VM (10.2.0-amd64)

Size

(Mb)

SQLite unikernel (QEMU+KVM)

Tim

e (m

s)SQLite dockerDebian VM

(10.2.0-amd64)SQLite unikernel (QEMU+KVM)

11235ms

2835ms

798ms

Fig1: Size (in megabytes) of the SQLite application on a Debian VM, a Docker container and a

Unikernel.

Fig2: Total life cycle* (in milliseconds) of the SQLite application on a Debian VM, a Docker

container and a Unikernel.

11/12*creation, boot, shutdown, and destroy times

12/12

Conclusion

‣ Unikernels:

‣ Replace the heavyweight virtual machines and insecure

containers.

‣ Require expert resources and time-consuming to develop.

‣ Not ready to be used Need an adequate tool.

‣ UNICORE:

‣ Based on Unikraft.

‣ Provides a toolkit to develop and deploy unikernels.

‣ Solution to quickly bring unikernels in the software

industry.

‣ Still at early stages.

!

‣ F. Manco, C. Lupu, F. Schmidt, Jose M., Simon K., Sumit S.,

Kenichi Y., Costin R., and Felipe H.. 2017. My VM is Lighter

(and Safer) Than Your Container. In Proc. of SOSP 2017.

‣ Unikernels.org. [n. d.]. Unikernels: Rethinking Cloud

Infrastructure. http://unikernel.org/.

‣ Xen Project. [n. d.]. Unikraft Development Team. https://

xenproject.org/developers/teams/unikraft/.

‣ UNICORE. [n. d.]. Quickly developing applications. http://

unicore-project.eu.

References

THANK FOR YOU ATTENTION

QUESTIONS?

Funded by the Horizon 2020 Framework Programme of the European Union under agreement No 825377