Cloud FederationsPatrizio Dazzi (ISTI-CNR) [Overall Presentation]

patrizio.dazzi@isti.cnr.it

Gaetano Anastasi (ISTI-CNR) [Hands-On]gaetano.anastasi@isti.cnr.it

contrail is co-funded by the EC 7th Framework Programme under Grant Agreement nr.

257438

http://contrail-project.eu

Presentation Outline

• Cloud Federations

• Contrail Federations

• Contrail Federation Architecture

• Federations and Resources

• A few details about the current status of Contrail Cloud Federations

• Introduction to the hands-on session

Cloud Federations

What is Cloud Computing ?

• Computing as a public utility– Clouds are the computing “power plants”

• Do not manage resources, just use them

• Choose the interaction model: SaaS, PaaS, IaaS

4

Principal Advantages of Cloud Comuting

• Advantages for Tenants:– Pay just what you get– Pay only when you get– Reduce costs for maintenance

• Advantages for Providers– maximizing the effectiveness of the shared

resources. – dynamically re-allocation depending on actual

usage

5

Anyway…

• Providers have:– A limited amount of resources– A limited range of resource types– Resources placed only in same countries

• Providers want:– to avoid that their customers leave them– To make (more) money

• Users want– To have all the resources they need (possibly more)– To pay less….

• How can we manage this ???6

Cloud Federations

• A Federation of Clouds is (maybe) the answer– By federating, providers can offer more

resources– More resources means…

• To be able to run bigger applications• To be more elastic• To provide (potentially more) different resources

– A federation (may) allow users to• Choose among a wider range of providers• Seamlessly use more providers for the same

application• Migrate from a provide to another

7

Federation of IaaS providers

• Infrastructure as a Service– provide computation, storage and network

• IaaS Providers– distinct Clouds may have different interfaces,

access rules, capabilities, prices, …

• Special Providers– Peculiar computational supports– Specific Storage capabilities– High performance networking

8

How Federation of IaaS impacts on PaaS and SaaS

• The background:– Platform and Software as a service offer an

even more complex landscape– PaaS and SaaS provider may be distinct from

IaaS ones

• IaaS Federation can– Allow properly instrumented PaaS and SaaS to

run on top of a huge amount of resources– Need to translate PaaS and SaaS requirements

• To address heterogeneity of providers• Enforce QoS guarantees

9

Unified Identity and Billing management

• Federations should be almost invisible from the point of view of users but:– User’s identity should be automatically

managed• In a cross-cloud fashion• Creating and managing proper identities in each

cloud (if needed)• Map by keeping proper capabilities and permissions

– Users have to be properly billed• Unified monitoring• Unified accounting

10

Security Mechanisms

• Authentication– Allow the access of users to Clouds– Exploit existing identity infrastructures

• E.g. OpenID

– Enact an identity federation

• Authorization– Access Control– Usage Control

• Particularly useful for long lasting applications

11

Cross-Cloud management of Service Level Agreements

• Cloud Applications have QoS requirements– Performance, reliability, security– Typically expressed as SLAs

• SLA management is a complex activity also in a single Cloud– Violations may occur and need to be managed

• Even more complex in a Federation of Clouds– Not all providers provide the same degree of

• guarantees• strictness• penalties• Compensation

– An overall coordination activity shall be performed

12

Last but not Least: Brokering

• Cloud Federations also behave as brokers– To find for each user’s application the best

Cloud(s) depending on• Application structure and requirements• Cloud reputation• Cost

– Splitting the user’s application in order to• Choose the best Cloud for each part of the

application• Allow a better exploitation of Specialized Clouds

13

14

Contrail Federations

Contrail Iaas Federation

A Contrail Federation integrates in a common platform multiple Clouds, both public and private ones.

To perform this task it provides:

• A common support for authentication, authorization and billing

• Mechanisms for policy definition, monitoring, and enforcing for all the QoS-related aspects

• An automated selection of providers depending on the user applications

• A support for resource provisioning to applications able to deal with heterogeneous sets of resources

Contrail development methodology

• Do not rebuild and reinvent– Contrail exploits Standards

• OVF• OCCI• OpenID

– Exploit existing platforms and functionalities• Open Nebula• SLA@SOI• Etc.

16

Develop a Federation support that integrates and actively coordinates SLA management provided by single Cloud providers

• Do not disrupt provider’s business model

• Allow exploiting a Federation seamlessly

• Federation Support must be scalable

Exploit Providers’ SLA support

Main Contrail Federation Innovations (1)

Federation = more than a simple broker, or portal, or decentralized cloud-bursting

– Interoperability

– Heterogeneous providers– Dynamically choosing best providers– At deploy time and at runtime– Allow to combine providers– Migration, elasticity– Security and privacy framework

19

Main Contrail Federation Innovations (2)

• Sophisticated SLA/QoS

– QoS via SLA– Via provider selection and integration– Enforcement mechanisms– Federation as a mediator and a 3rd party– Federation also acts as a coordinator

20

Contrail Federation Architecture

Simplified Blocks Architecture

Final Users ConPaaS

Cloud Federation

Cloud Provide

r 1

Cloud Provide

r 2…

Cloud Provide

r N

21

Distributed Federation Access Points

• Several Federation access points (FAPs)• FAP act as brokers, but share a common

view– Security, status of resources, users and

providers reputation

F

F

F

F

• Hierarchical structure• Common Policies• Detailed resource

allocation is on providers• AP may be hosted by

providers

Federation Interfaces

SLAAuth

AuthZFederation Core

Coarse-grain view of Contrail Federation Architecture

• Functionalities which extend horizontally in the platform

Provisioning Manager

Concrete Architecture

In the next slide

• Module View of a single Access point

• Interactions, some modules not shown

• Interfaces sit on top of this core

• Auth/AutZ mechanisms included

24

Contrail federation

25

Provisioning Manager

Provisioning Manager

26

Federations and Resources

Basic concepts

• What’s an application for IaaS?– A set of software entities which need to be

deployed on a suitable set of resources for execution

• The parties involved– User

• Who submit applications

– Provider• Any entity managing physical resources which may be

used to run applications

– Federation• The union of many providers under a common set of

APIs and functionalities, which can be exploited as a single Cloud

Application Description

• The requirements so far can be expressed as a Task Interaction Graph– An undirected graph G(N,E) can model a

distributed application– Each Node in N is a task (needs a resource)– Edges imply relations between tasks (need links)– Heavily labeled graph

• Nodes state all resource constraints and SLA measures

• Edges labeled with communication needs

– Cloud applications can gethard to describe

Types of resources

• Computation resources– Available VMs slots on top of physical hardware

• Storage resources– Shared filesystems– Shared Databases

• Networking resources– Virtual networks connect machine within an

application– Behaviour of the joining points between the

internet and the federation is important

How resources are measured ?

• Static specification by analogy with the physical counterparts– Memory size, CPU type and speed (peak)– Size, FS of storage– Nominal bandwidth and latency of networks

• Dynamic specification– Available computing power, memory– Actual (average, peak, used) storage speed– Observed in-Cloud bandwidth and latency– Observed bw/lat to the outside– Application-specific metrics

Provider’s View on resources

• Providers know exactly the layout and state of all physical (virtual) resources– Dedicated link bandwidths, physical memory…

• They can greatly optimize resource management– Turn off unused resources, exploit cheapest– They have their own goal (revenues)

QoS properties of resources

• Extend the set of characteristics to be measured on the platform

• Protection– Type of security mechanisms which are in place

• Auth. Protocols, Encryption mechanisms, Isolation

• Privacy– Guarantees offered by storage holder, network

infrastructure

• Geo-localization – Can have deep legal implications

• Power consumption– Overall power, power efficiency

QoS expressed via SLAs

• As the SLA is signed, the user should be able to trust resources from the platform– But not all Providers may offer the same

reliability

• How reliability can be measured ?– failures, SLA violations with different providers– lengthy task with poor reward for single users

Provider Reputation

• Management information– Available resources per kind– Features granted– Amount of users/apps ongoing– State of SLAs controlled by the federation– Static level of “trust” given from federation to

the provider

• Past History– History of SLA violation per user / type of app– Average level of satisfaction of SLA

Contrail approach to SLA

– Reuse SLA@SOI framework as a starting point

• Integration with Contrail internal interfaces and components

• Integration with domain-specific reasoning/monitoring plugins

– Extend SLA@SOI with:• Federation support• Integration with external providers (and their SLA

management systems)• Reputation model for providers• Cost-based QoS enforcement

Federation-level SLA Enforcing

• Federation will act as a SLA coordinator over providers– As much as possible the single providers is in charge of

the local SLA• Reduce reaction delay

– Federation evaluates the provider and the app

– Extend the SLA@SOI monitoring infrastructure to the federation

• Keep track of main application parameters

– Receive SLA violations from providers

– Reallocate some resources on a provider basis• May require a new negotiation between federation and provider

What if… a SLA is violated in a single provider scenario ?

The application is deployed on a single provider, which may still violate SLA

• Actions: – The provider resize the set of allocated

resources– If the application is no longer violating SLAs

• The application will be up and running and that’s it

– Otherwise• A renegotiation phase is conducted and if will not be

successfully the application will be terminated

…and what if the violation occurs in a multi-provider scenario ?

The application has been sent by the federation to a provider for the execution

and a SLA violation occurs

• Actions:– Previous slide shows what happens when the

provider is able to manage the violation by itself

– If it is not able, the federation can still migrate (part of) the application to a different provider

38Contrail S. School 2012 – P. Dazzi- Cloud Federations

Applications Running in Multiple Providers

Some applications could be also decomposed in parts and each deployed in a different provider• Actions

– resize part(s)• managed by providers

– migrate part(s)• may need to stop the application

– violate constraints• By leveraging the whole federation we push away the limit

where a violation is unavoidable

– rebalance constraints• By renegotiating with more providers, overall SLO may be

achieved• Increase resource availability where they are cheaper/more

available at the moment

Anyway SLA splitting is still an open issue

• Necessary to leverage SLA management at single providers

• How to derive a combination of SLA for separate parts of the application which allow to manage the application overall

• Not yet addressed in literature• Not hard if providers are specialized

– Compute, network, storage = SLA aspects

• As the user expresses SLA terms on parts of the application (task groups) this will become easier

Contrail Federation, the 3rd Player

• A Contrail Federation sits in the middle• Aims

– Serving the users– Exploit efficiently the providers

• If economic gain is sought, it comes form intermediation

• Can efficiently gather provider information– Gathers from all users and all providers– Can make better informed choices than users– Can afford to launch directed tests in doubt– Can trigger corrective actions impossible to

single providers

42

A few details about the current status of Contrail Cloud Federations

How it is used

• Federation Interfaces use REST• The Web portal and command line tools access

the REST layer• Different roles

– federation coordinator– cloud administrator– end user

• Main steps– account creation– SLA formation– Application upload (VMs and descriptor)– SLA negotiation and agreement– Deploy

43

What is implemented today

• User management– Registration, mapping are working– some integration activities are still ongoing

• SLA Management– Negotiation with providers counterpart is

working– A complete integration is expected in the next

release

• Application management– Applications can be submitted and executed– So far only one provider at a time can be used

for a single application

44

Aligning with standard formats

• OVF (Open Virtualization Format)– Open format for describing application– Allows to describe structured applications– Directly expresses only HW constraint and

deployment information• Partially overlaps with full SLA specification

– OVF is extendable– As of v1.1.0 it does not target Application

Management

• SLA@SOI provides a general formalization– Includes monitoring hierarchy and negotiation

• Exploit a combination of OVF and SLA@SOI mechanisms

“Securing” the implementation

• The current implementation of federation authentication– Support for OpenID– Additional details in Jens’s keynote

• The current implementation of federation authorization– Advanced support to access control– Ongoing work to finalize the support to usage

control

46

Ongoing work about advanced features

• Multi-criteria mapping algorithms– Genetic algorithms for application mapping

• Evolving mapping plans to achieve better allocations

• Application splitting– Interplay with SLA splitting

• Monitoring data aggregation and filtering

47

Introduction to the hands-on session

Contrail S. School 2012 – P. Dazzi- Cloud Federations 48

Sample Application

• OVF Representation• 2 Appliances

49Contrail S. School 2012 - M. Coppola - Cloud Federations

<VirtualSystem ovf:id="VirtualSystem1"> <VirtualHardwareSection> <Item>

<rasd:Description>Number of virtual CPUs</rasd:Description>

<rasd:ElementName>1 virtual CPU</rasd:ElementName>

<rasd:InstanceID>1</rasd:InstanceID><rasd:ResourceType>3</

rasd:ResourceType><rasd:VirtualQuantity>1</

rasd:VirtualQuantity> </Item>

<Item> <rasd:AllocationUnits>byte *

2^20</rasd:AllocationUnits><rasd:Description>Memory

Size</rasd:Description><rasd:ElementName>512 MB of

memory</rasd:ElementName><rasd:InstanceID>2</rasd:InstanceID>

<rasd:ResourceType>4</rasd:ResourceType>

<rasd:VirtualQuantity>512</rasd:VirtualQuantity> </Item>

Simplified Deployment Chain

• Users submit an OVF• Federation mapping phase:

– Which provider(s) for the application?

• Federation Application Submission • Provider Deployment

– Provisioning Manager: receives requests coming from the federation

– VEP: manages provider resources for deployment

• For simplicity do not consider SLAs

50Contrail S. School 2012 - M. Coppola - Cloud Federations

Thanks for you attention!

51

Questions?

Funded under: FP7 (Seventh Framework Programme)

Area: Internet of Services, Software & Virtualization (ICT-

2009.1.2)

Project reference: FP7-IST-257438

Total cost: 11,29 million euro

EU contribution: 8,3 million euro

Execution: From 2010-10-01 till 2013-09-30

Duration: 36 months

Contract type: Collaborative project (generic)

contrail is co-funded by the EC 7th Framework Programme