Upload
stormmq
View
2.076
Download
0
Tags:
Embed Size (px)
DESCRIPTION
StormMQ Introduction to AMQP, Dublin, in conjunction with IrishDev.com, 16th November 2010.
Citation preview
Agenda
A Real Application ! 20 min
Nibbles and Hands–On ! 60 min
Introduction to AMQP ! 20 min
Why Message Queuing? ! 25 min
Classic Architecture ! 10 min
History of Message Queuing
Manual Telegraphy Machine Assisted Telegraphy
1920s1911 - 192019th Century 1900s
Telegrams sent using“Store and Forward”
1900
1930s
History of Message Queuing
Telcos UseElectronic Telegraphy
1950s1940s 1960s
IBM System/360 with BTAM & QTAM
Message Switching
1964
First Electronic Mail Solutions
1965
Banking Users
1970s
IBM TCAM which is the first true solution
Retired 1990!
1971
Machine Assisted Telegraphy
History of Message Queuing
First Electronic Mail
FMCG & Utilities
1990s
IBM Launch MQSeries(now WebsphereMQ)
1992
Machine Assisted Telegraphy Corporates Large Websites YOU
Noughties Today
Sun Release Java JMS, Reinvigorating
Enterprise Messaging
2001
AMQP Working Group Formed by Investment Banks
2006
Cloud Enables and Drives StormMQ
Adoption
2009
The Integration Tag Soup
WS – File Transfer
!"#$%&'(#)#*#+%+
Message QueuingSOAP
SMTP
FTP
RSYNC
NTFS
!','-
!).$%&'/$.0%&1$%+
234%5)6!%$7%$
SOAMessage
OrientatedMiddleware
JMS
MQ
MQSeries
AMQP
Amazon SQS
Tibco
StormMQ
RabbitMQ
HTTP
TCP/IPDCE / RPC
CORBA
DCOM
REST
RMIXML-RPC
.NET Remoting
Remote Procedure Call
XML-RPC
The Integration Tag Soup
WS – File Transfer
!"#$%&'(#)#*#+%+
Message QueuingSOAP
SMTP
FTP
RSYNC
NTFS
!','-
!).$%&'/$.0%&1$%+
234%5)6!%$7%$
SOAMessage
OrientatedMiddleware
JMS
MQ
MQSeries
AMQP
Amazon SQS
Tibco
StormMQ
RabbitMQ
HTTP
TCP/IPDCE / RPC
CORBA
DCOM
REST
RMIXML-RPC
.NET Remoting
Remote Procedure Call
XML-RPC
Under Examination, though
Deployment
Intimate System Knowledge
Configuration
Admin
File Transfer
!"#$%&'(#)#*#+%+
Remote Procedure Call
Don’t Work in the Cloud
Scaling
Why Use it: Loose Coupling
BillingCatalogue
Shipping S-a-a-S Inventory
better position: Why do people
queuing? fonts
How do we connect them, without one outage or system change taking everything down like a pack of cards?
Why Use it: Loose Coupling
BillingCatalogue
Shipping S-a-a-S Inventory
better position: Why do people
queuing? fonts
How do we connect them, without one outage or system change taking everything down like a pack of cards?
Message Queuing lets Systems and Components exchangedata, events, commands and actions with one another
with no explicit knowledge or need for them to be online
!!!Until today, messaging is …
(ZeroMQ)
(SQS uses HTTP)
(MSMQ uses COM)
(memcached)
(Java JMSuses
Source)
(MQSeries / WebsphereMQ)
(Talarian, Rendezvous, etc)
Platform Restricted BespokeProprietary
!!!Until today, messaging is …
(ZeroMQ)
(SQS uses HTTP)
(MSMQ uses COM)
(memcached)
(Java JMSuses
Source)
(MQSeries / WebsphereMQ)
(Talarian, Rendezvous, etc)
However, AMQP fixes this
A common wire-level binary format and protocol
An explicit definition of a server (aka broker)’s
semantics
Open Means
Interoperable
That is good …
“AMQP will be to Messaging what HTTP was to the Web”
MRG
Clients run on any Platform Vendors are Interoperable
That is good …
“AMQP will be to Messaging what HTTP was to the Web”
MRG
Clients run on any Platform Vendors are Interoperable
64K
Eddie Velez
Quick Recap
✓
The fifth male memberof the A-Team, Frankie,
was played by
Question
Message Queuing connects systems and components. Is it ideal for the cloud?
Does Loose-Coupling make individualsystems more likely to suffer outages?
Yes
No
AMQP is Open.
This makes it suitable for programmingin C and Javascript?
!
✓
Agenda
Introduction to AMQP ! 20 min
Why Message Queuing? ! 25 min
Classic Architecture ! 10 min
A Real Application ! 20 min
Pizza and Hands–On ! 60 min
Store and Forward
Billing
“Could you send me a duplicate of my bill please”
REST
StormMQMessaging
Billing
!
Fire and Forget
Shipping
“Too many orders. The website is running like a dog.”
SQL Push SQL Pull
StormMQMessaging
Shipping
One-To-Many
“There’s a new widget in the Catalogue: Tell all the systems.”
CatalogueShipping
S-a-a-S Inventory
File Transfer
ETL
One-To-Many
“There’s a new widget in the Catalogue: Tell all the systems.”
CatalogueShipping
S-a-a-S InventoryStormMQMessaging
StormMQMessaging
Round-Robin
Billing
“How do we easily scale a massive batch job like Billing?”
EMail BillGenerate
StormMQMessaging
Publish-Subscribe (“Topics”)
“Shipments Sent, Delivered and Received”
Shipping
S-a-a-S InventoryBilling
RIASentSent or Returned
Agenda
Introduction to AMQP ! 20 min
Why Message Queuing? ! 25 min
Classic Architecture ! 10 min
A Real Application ! 20 min
Nibbles and Hands–On ! 60 min
Messaging: Which Jargon?
EMail(SMTP, POP3, IMAP)
VoIP(VoiceMail, XMPP)
Texting(SMS)
Instant Messaging(ICQ, MSN, Jabber)
Enterprise Service Bus(ESB)
Dynamic OO Languages(eg Ruby)
Message Queuing(MQ)
Message Queuing(MQ)
What do wemean by
Messaging?
Essential Terms
*Strictly speaking
a receiver polls for messages
a consumer has messages pushed
≣ ≊Send Publish Enqueue
≣ ≊ DequeueConsume*Receive
Connection
AMQP Client AMQP ServerTCP / IP Network
ConnectionVirtual Host
Connections and Channels
TLS “Shielding”
Channels
Each Channel is Independent:Effectively, a Virtual Connection
Basic AMQP
Basic AMQP: Connections
Open a Connection to a Virtual Host
Open a Channel
Send a Message
Receive a Message
Close Channel
Close Connection
You only need one channel!
"
#
$
%
&
'
Exchanges Route Messages
Exchanges route Messages to Message Queues
You send messages to an exchange, not a message queue
Exchange
MQ
A
MQ
B
How do Exchanges Route?
Exchange
MQ
A
The Exchange finds a Bindingmatching the Routing Key#
A Binding connects a Routing Keyto one or more Message Queues
$Every sent message has a
Routing Key"
The Exchange delivers theMessage to the Message Queue%
A message queue can bebound more than once to
one or more exchanges(
“string”
Types of Exchange
More rarely used exchange types include amq.headers and extensions
Message Queue Name
direct
Like a ‘Map’: All MQs bound with the
routing key receive copies of the message
“” (blank)amq.direct
fanout
Empty String
All MQs bound to the Exchange receive
copies of the message
amq.fanout
topic
Dotted
Bindings useglobbing expressions (wildcards) to route messages to MQs
amq.topic
A message queue can be bound more than once to an exchange;A message queue can be bound to more than one exchange
But a message queue will only receive a message sent once
Routing Key
RoutingBehaviour
Default Definitions
Point – to – Point One – to – Many Publish – SubscribeTypical Use
And there is more!
Transactions
More Message Properties
Custom Message Properties
Immediate Delivery
Additional Exchange Types
Auto Deletion
Fine-Grained User Permissions
QueuePurging
QoS
Agenda
Introduction to AMQP ! 20 min
Why Message Queuing? ! 25 min
Classic Architecture ! 10 min
A Real Application ! 20 min
Nibbles and Hands–On ! 60 min
What’s needed
Live Debugging Live Logging Live Statistics
Audit ComplianceStatistics Mining SMS Alerts EMail Alerts
Design of the Clients
intrusion
api.daemon.info.charon api.memory api.daemon.sshd api.warn
checksstatisticslog
Content-Type: application/json; charset=utf-8Timestamp: 2010-11-03T16:45:56.981ZPriority: Low / HighType: statistics_2011.10.01.1203Expiry: 5 mins
api.firewall.warn.in.portscan.tcp
checks
Handling Log Messages
api.*.*.warn Queue
#Queue
Timestamp
No Acks
QoS Prefetch
WebSockets
logMessageId
Routing Key
*.warnQueue
checks
Stat
istics
Arc
hitect
ure
is th
e Sa
me
From Checks to Alerts
#Queue
emergencyQueue
#Queue
checks
alerts
alerts
Contact Detailsnot in message
Message Aggregation /Suppression by alerts
Uses MessagePriority