Messaging with AMQP and RabbitMQ

Eberhard Wolff Architecture and Technology Manager

adesso AG

Overview •  Why Messaging, AMQP and RabbitMQ •  Basic AMQP •  Exchanges •  More on Spring-AMQP

Why Messaging? •  Decoupling

– Data, no action i.e. receiver can react arbitrarily

– Asynchronous i.e. decoupled by time

•  Reliable – Message can be stored-and-

forwarded – Redelivery until message

processed •  Solves typical problems of

distributed systems

Component

Component Messages

Why Messaging? •  But: Requires different architecture •  Very different from calling remote

methods •  Asynchronous •  AJAX has the same model

•  See for example “Patterns of Enterprise Integration”

Why AMQP? •  Open standard protocol •  Standard wire protocol •  i.e. just one client library – no matter which

implementation you are using •  Less vendor lock in •  Efficient

– Binary wire protocol •  Support in all major languages •  Supported on most OS platforms

What about JMS? •  JMS has been the default for Java

messaging system for 10+ years •  But:

– Only standardized on the API level – Less flexible than AMQP

•  Mapping AMQP/JMS is being defined •  Some products support both

Why Rabbit? •  Because it has a kewl name •  Numerous protocols supported •  Most popular choice on EC2 •  Foundation for demanding systems e.g.

NASA’s cloud initiative Nebula •  Implemented in Erlang •  Clustering built in •  Currently in 2.8.7 •  Supports AMQP 0.8, 0.9, 0.9.1 •  1.0 as a prototype Plug In

Broad Support in RabbitMQ

Broad Support in the JVM Space

•  Grails Plug In •  Java Client •  Scala / Lift support

•  We will discuss Spring support in detail •  Spring AMQP project 1.1.2 •  http://www.springsource.org/spring-

amqp

Why Erlang? •  Originally designed for telephone

switches by Ericsson •  Much easier to develop scalable and fault

tolerant systems (by factors)

•  See Motorola presentation: http://www.slideshare.net/Arbow/comparing-cpp-and-erlang-for-motorola-telecoms-software

•  Good tool for reliable and scalable systems

Erlang‘s Model

Light weight

process with state

Monitor

Link to monitor, restart

Light weight

process with state

Light weight

process with state

Messages Messages

Why Erlang? •  Let it crash

–  If a process fails, it can be easily restarted – Different approach to fault tolerance – Otherwise lots of error handling

•  Message Passing in the Core – RabbitMQ is a messaging system…

•  Light-weight process model – Scales to massive numbers of connections

Basic AMQP

Very Basic AMQP •  Queues: Store messages •  Queues might be

– Durable: Survive server restarts – Exclusive: For one connection – autoDelete: Deleted if connection closes

•  All resources are dynamic •  Producer sends a message to a Queue

Code ConnectionFactory conFactory = new CachingConnectionFactory ("localhost");RabbitAdmin admin = new RabbitAdmin(conFactory);admin.declareQueue( new Queue("myQueue", false, true, true));RabbitTemplate template = new RabbitTemplate(conFactory);template.convertAndSend("myQueue", "Hi AMQP!");String receive = (String) template.receiveAndConvert("myQueue");Assert.assertEquals("Hi AMQP!", receive);

Spring’s MessageConverter •  Messages are binary data •  RabbitTemplate uses

MessageConverter to convert between objects and messages

•  E.g. JSON, Serialization, XML … •  Can also send binary data if preferred

Basics of AMQP •  Sending messages directly to queues is

not enough •  What about e.g. pub / sub?

•  Exchange: Route messages (stateless) •  Example used the default exchange

•  More dynamic, flexible and cleaner than JMS

AMQP  in  a  nutshell  Exchange routes message Stateless Usually created by producer No queue: Message discarded

X

Queues buffer messages Usually created by consumer

Binding binds an Exchange to a Queue

AMQP  in  a  nutshell  

AMQP protocol

Producer and Consumer might be written in Java, C#, Python, Ruby …

X

C

C

P

RabbitMQ AMQP protocol

Exchanges

Exchange: Route Messages •  The type of Exchange defines the

routing algorithm •  Binding provides selector for routing •  Exchange is addressed by name

•  Some standard types •  Can provide additional ones

X

Fanout Exchange •  Broadcast to all bound queues •  Fast •  Simple

•  amq.fanout is mandatory

•  To broadcast information

X

X

C

C

C

P

Fanout

Fanout Exchange X

Queue fanoutQueue = new Queue("fanoutQueue");admin.declareQueue(fanoutQueue);FanoutExchange fanoutExchange= new FanoutExchange("myFanout");admin.declareExchange(fanoutExchange);admin.declareBinding( BindingBuilder.bind(fanoutQueue). to(fanoutExchange));template.setExchange("myFanout");template.convertAndSend("Hi Fanout!");String receive = (String) template.receiveAndConvert("fanoutQueue");Assert.assertEquals("Hi Fanout!", receive);

Direct Exchange •  Routing based on one routing key •  amq.direct and the default Exchange (no

name) always exist

•  To send work orders to a specific worker

X

X

C

C

C

P

Direct Exchange

express

normal

express normal

Direct Exchange

Topic Exchange •  Routing based on routing pattern •  amq.topic is mandatory

•  E.g. for public / subscribe scenarios

X

Topic Exchange  

X C

C

P

Topic Exchange

order.*

invoice.*

order.DE invoice.USD

Headers Exchange •  Routing based on one or more headers and

an expression •  amqp.match is mandatory

•  Complex routing roles

X

Other Features •  Message can be persistent •  Request / response using correlations

possible

•  Redelivery / acknowledgement possible

•  Clustering with e.g. Linux HA possible •  ...or send message through multiple

channels and drop duplicates

More on Spring AMQP

The MessageListener •  So far: Calling receive() on

RabbitTemplate •  Needed: Something that is called when

a new message appears

Spring’s MessageListener Container

•  Spring provides lightweight containers to call MessageListeners

•  SimpleMessageListenerContainer •  Advanced scheduling and endpoint

management options available •  i.e. thread pools, concurrent consumers,

transaction handling

Spring's message-driven objects

•  MessageListener means the receiver depends on Spring API

•  Why not just a POJO?

Message-driven POJO

•  Takes a POJO and makes it a MessageListener

•  i.e. calls consume on Bean consume

<rabbit:listener-container connection-factory="connectionFactory“ message-converter="jsonMessageConverter"> <rabbit:listener ref="consumer" method="consume"

queue-names="my.amqp.queue2" /> </rabbit:listener-container>

Consumer code

•  No dependency on AMQP! •  But: What about the result of the method? •  Send to the Reply-To address given in

message properties with same correlationId as original method

@Componentpublic class Consumer { public String consume(String message) { return …; }}

Client Code

•  Message sent to destination with routing key •  Reply-To set to exclusive, autodelete, non-

durable queue •  Response received through Reply-To

converted and returned •  Easy request-response! •  Beware of potential latency

String response = (String) rabbitTemplate.convertSendAndReceive( "my.fanout", "", "test");

Web Messaging

Web Messaging •  Goal: Send / receive messages in the

browser •  With JavaScript •  I.e. JavaScript must receive message •  HTML5 browser: WebSockets •  Pre HTML5: long polling etc

SockJS •  Unifies Web Sockets, long polling etc •  Unified JavaScript API

•  Server component •  Can run embedded in RabbitMQ

STOMP •  Simple (or Streaming) Text Oriented

Message Protocol •  Very simple protocol for Message Oriented

Middleware

•  STOMP over SockJS •  See http://www.rabbitmq.com/blog/

2012/05/14/introducing-rabbitmq-web-stomp/

AMQP Java C# … Web Sockets

Long Polling

SockJS

STOMP JavaScript

http://127.0.0.1:55670/

JavaScript Code WebSocketStompMock = SockJS; var client = Stomp.client('http://localhost:55674/stomp'); … client.send('/topic/test', {}, data); … client.connect('guest', 'guest', function(x) { id = client.subscribe("/topic/test", function(message) { if (message.body) { alert("got message with body " + message.body) } }); });

AMQP: Topic exchange amq.topic with routing key test

Conclusion: AMQP •  Ubiquitous Messaging •  AMQP: Protocol standard •  Better scalability •  Dynamic resources

Conclusion: Spring AMQP •  Easy to use •  Flexible (e.g. message encoding) •  Allows scalable message handling •  Full support for AMQP and RabbitMQ

Conclusion: Web Messaging •  Simple API •  Works on older browsers •  Single server – also for messaging •  Erlang is very well able to handle the

many connections

More •  http://springsource.org/spring-amqp •  Also a .NET version available •  …and support in Spring Integration •  …and there is very similar Spring JMS support •  http://blog.springsource.com/2011/04/01/routing-

topologies-for-performance-and-scalability-with-rabbitm

•  https://github.com/zanox/rabbiteasy – CDI event / AMQP integration – Managed publisher and consumer