Marconi: Queuing and Notification service for OpenStack

Preview:

DESCRIPTION

Marconi is a multi-tenant cloud queuing system written in Python as part of the OpenStack project. As message bus, it’s main goals are: performance, availability, durability, fault-tolerance and scalability. This talk aims to give the audience as much information as possible about Marconi’s architecture, design, patterns, performance, issues it faced and how / why it was built from scratch.

Citation preview

@flaper87 at EuroPython 2013

Queuing & Notification Service… made by and for OpenStack

@flaper87 at EuroPython 2013

What is it?

What isn't it?

Architecture

API v1

Who's this dude?

@flaper87 (a.k.a, Flavio Percoco):Very proud Red Hatter, who's also part of the MongoDB Master team and has contributed to Open Source since... forever.

One of those who thinks about programming when he's not programming.

member of

@flaper87 at EuroPython 2013

Queuing service Queues MessagesClaims

@flaper87 at EuroPython 2013

Messaging ServiceReq / RepPub / Sub

@flaper87 at EuroPython 2013

Very young projectBugsWIP IdeasMissing pieces

@flaper87 at EuroPython 2013

It is NOT a ...

Replacement for QPID, RabbitMQ, ZMQ

@flaper87 at EuroPython 2013

It is NOT a ...

Incubated, yet!

@flaper87 at EuroPython 2013

Plugin Based TransportsStorage

@flaper87 at EuroPython 2013

Transports HTTPZMQ

@flaper87 at EuroPython 2013

HTTP FalconNo container

@flaper87 at EuroPython 2013

Storages SQLiteMongoDB

@flaper87 at EuroPython 2013

Authentication Keystone

@flaper87 at EuroPython 2013

RESTFul CRUDCollections

@flaper87 at EuroPython 2013

RESTFul CRUDCollections

API Consistency planned throughout transports

@flaper87 at EuroPython 2013

FIFO Guaranteed

… don't run out there just to test it

@flaper87 at EuroPython 2013

ACCESSPaginatedLimitedStreamed

@flaper87 at EuroPython 2013

CUSTOMMax SizeMax Bulk...

@flaper87 at EuroPython 2013

QUEUESMetadata64b Name

@flaper87 at EuroPython 2013

MESSAGESTTLEcho

@flaper87 at EuroPython 2013

CLAIMSTTLGrace

@flaper87 at EuroPython 2013

BEST EFFORTBulksClaims

@flaper87 at EuroPython 2013

Some benefitsAPI over queuesService orientedEasy to scale

@flaper87 at EuroPython 2013

Q&A

Thankswe

're hi

ring

http://wiki.openstack.org/Marconi

Recommended