An introduction to Apache Mesos

Embed Size (px)

Citation preview

Apache Mesos

What is it ?

Beyond Hadoop

Resource Sharing

Mesos Intentions

Architecture

Users

[email protected]

Apache Mesos What is it ?

A cluster manager

Provides resource isolation / sharing

Run distributed systems on the same pool of nodesHadoop, Spark, Storm, Jenkins, K ...

Cluster monitoring via web UI

Tasks isolated via Linux Containers

Scalable 10k's

Written in C++

C++/Java/Python API's for new app dev

[email protected]

Apache Mesos What is it ?

Its fault tolerant Leader election via Zoo Keeper

Multi platformLinux , OSX , Open Solaris

It is now a full Apache project

A Master / Slave based system

[email protected]

Apache Mesos Beyond Hadoop

What requirements are there beyond Hadoop ? Cluster sharing / multi tenancy

Higher resource utilization

Faster response times

Resource sharing

A wider range of functionality

More out of the box functionality Graphical Cluster monitoring

Graphical drag and drop cluster manipulation

[email protected]

Apache Mesos Resource Sharing

[email protected]

Apache Mesos Intentions

Avoid multiple specialised clusters

Avoid the need for silo'ed clusters

Avoid the lower utilization that this brings

Offer the ability for multi tenancy

Which offersLower hardware / maintenance costs

Higher utilisation

Scalability

Higher fault tolerance

Program for the Data Center

[email protected]

Apache Mesos Architecture

[email protected]

Apache Mesos Users

Companies using Mesos Air bnb

Cloud Physics

HubSpot

Shopify

Twitter

Vimeo

Among many others

[email protected]

Contact Us

Feel free to contact us at www.semtech-solutions.co.nz

[email protected]

We offer IT project consultancy

We are happy to hear about your problems

You can just pay for those hours that you need

To solve your problems