Policy-Based Scheduling:

Improving Resources Provisioning In OpenStack

May 12th, 2014Atlanta

Khanh-Toan TRAN (R&D Engineer)khanh-toan.tran@cloudwatt.com

Outline

Motivation Why Smart Placement ?

Current Openstack Provisioning Overview

Limitation

Proposed Policy-Based Scheduler

Usecases & Demo

Meta-scheduling

MOTIVATION

Smart Placement Expectations

For the clients perspective Contract guarantee

Transparency

Better experience

More services, more offers

Attractive price

For the cloud provider perspective Flexible scheduling

• Scheduling per client• Scheduling per resources• Scheduling per context

Capable of realizing different admin objectives and operations

Efficient and simplified control and management by policies

Smart Placement Big Picture

Smart Placement

Compute, storage and network nodes, etc

…

Datacenter 1 Datacenter 2 Datacenter n

Cloud provider operation

Clients

Environmental info

Client contract

Regulation

OpenStack OpenStack

High-level requirements

Environmental info

Infrastructure info

Secu

rity

polic

ies

OVERVIEW OF THE EXISTING

OPENSTACK PROVISIONING

Placement in Openstack: FilterScheduler

Filter_Scheduler

Nova-scheduler

Filter Weigh

Request

FilterFilter WeighWeigh

1 VM2VCPU, 2GB RAMService=gold

Filtering Weighing

Compute 18VCPU; 8Go

Compute 23VCPU; 4Go

weight = 8

weight = 4

Compute 18VCPU; 8GB RAM

Compute 23VCPU; 4GB RAM

Compute 34VCPU; 1GB RAM

Service=gold

Service=gold

Compute 48VCPU; 8GB RAM

Service=goldCompute 1

8VCPU; 8GB RAM

Compute 23VCPU; 4GB RAM

Compute 34VCPU; 1GB RAM

Compute 48VCPU; 4GB RAM

Openstack Nova: Scheduling Module : Driver

Default driver: FilterScheduler1. Filters out hosts that are not

admissible to hosts the VMs (Filter).

2. Weighs the remaining hosts following their capacity (Weigher)

FilterScheduler Limitations

Static scheduling Difficult to answer to admin’s various objectives

Ineffective in management in different situations (maintenance, failure, breakout, catastrophe…)

Lack of client context No consideration for each client requirements

Client contract is not taken into account

Lack of consideration for infrastructure Unable to provide fine-grain scheduling

Example:A cloud service provider wishes to apply the global Load

Balancing policy to the whole datacenter and a local Consolidation policy to a cluster of hosts.

Unfeasib

le

Two-step approach for the situation improvement

Short term Policy-based scheduling solution (incremental): Enhancing the

existing Nova-centric architecture.

Mid / Long term Meta-scheduling solution (disruptive): independent scheduling

service for OpenStack on top of Nova, Cinder and Neutron.

POLICY-BASED SCHEDULING(SHORT TERM)

Policy-Based Scheduling Module

Decoupling the scheduling logic and the execution domain

The scheduling logic is represented as Rule:

Target – Effect – Conditionwith correspondent Plugins.

Rules are stored in Policy Repository (PR)

Policy-Based Scheduling Engine (PBSE) consults the Policy Repository to find and execute the suitable rules

Policy-Based Scheduler (PBS) is the proxy between Nova-scheduler and PBSE

Nova-scheduler

Filter Weigh

Client request

FilterFilter WeighWeigh

Policy-Based Scheduler (PBS)

Policy-Based Scheduling

Engine (PBSE)Policy

Repository

PluginPluginPlugin

Policy-Based Scheduling

Nova-scheduler

Filter Weigh

Client request

FilterFilter WeighWeigh

Policy-Based Scheduler (PBS)

Policy-Based Scheduling

Engine (PBSE)Policy

Repository

PluginPluginPlugin

Target Rules

{« tenant: «Frank»}

{ «effect »: {«service_class»: « gold »}, «condition» : « all » }

Target Rules

« all » {«effect »: {« LoadBalancing»: «Ram » },«condition » : «all»}

« aggr: aggr-1 »

{« effect »:{« Consolidation»:«Ram»},«condition » : « all »}

Service_Class_Plugin

LoadBalancing_Plugin

Consolidation_Plugin

Policy-Based Scheduling

Availability-Zone-1

Aggregate-1

Server-01

Server-02

Availability-Zone-2

Aggr-2 Aggr-3

Load-Balancing

Consolidation

Server-03

Server-04

USE CASES & DEMO

Usecases and Demo

Use Cases 1: Respecting the regulation A French medical company wants that all their VMs and data are hosted in

France as requested by French authority

All VMs are initiated in France zone regardless of the choice of the users (e.g. employees)

Use Case 2: Enforcing the client contract A client signs a gold contract

All his VMs are initiated in the gold zone with high-end equipments (e.g. high freq. CPU, high-mem, SSD storage…)

Use Case 3: Reducing software licences A cloud provider signs a contract with an ISV which charges the license fee

based on the number of hardware

Admin wants to apply 2 policies:• A global Load-Balancing policy in all his datacenter to favor the load

sharing and limiting the overload• A local Consolidation policy in the zone dedicated to the software to

reduce the number of charged physical servers

META-SCHEDULING(MID / LONG TERM)

Meta-Scheduling

Policy-Based Scheduler blueprint:https://blueprints.launchpad.net/nova/+spec/policy-based-scheduler

Meta-Scheduling

Meta-Scheduling

Meta-Scheduling

Related works

Scheduler service: Gantt https://blueprints.launchpad.net/nova/+spec/scheduler-lib

https://blueprints.launchpad.net/nova/+spec/remove-cast-to-schedule-run-instance

Remove DB between scheduler and compute nodes

https://blueprints.launchpad.net/nova/+spec/no-db-scheduler

Instance Group API https://blueprints.launchpad.net/nova/+spec/instance-group-api-e

xtension

Support for Multiple Active Schedulers https://blueprints.launchpad.net/nova/+spec/multiple-scheduler-dr

ivers

SolverScheduler

https://blueprints.launchpad.net/nova/+spec/solver-scheduler

Thanks for your attention khanh-toan.tran@cloudwatt.com

Policy-Based Scheduler blueprint:https://blueprints.launchpad.net/nova/+spec/policy-based-scheduler