Upload
jeremy-deane
View
270
Download
0
Embed Size (px)
Citation preview
Agenda
v EnvironmentSetup
v HelloBunny!
v AMQPFundamentals
v Exercises
v RabbitMQ Architecture
v RabbitMQ Security&Operations
v AMQPIntegrations
v Exercises
AdditionalEnvironmentSetup§ DownloadandinstallRabbitMQ (3.6.2+)
§MacStandalone orHomebrew§WindowsStandalone
§ Add$RABBITMQ_HOME/sbin toPATH
§ Commands(WindowsasaService)§ Start:rabbitmq-server -detached§ Status:rabbitmqctl status
§ Stop:rabbitmqctl stop
§ InstallManagementPlugin
§ VerifyManagementConsole(guest/guest):
rabbitmq-plugins enable rabbitmq_management
http://localhost:15672/
Mac&LinuxO/S– Update/etc/hosts
127.0.0.1 <<name of your laptop>>
EnvironmentSetup§ Download(Clone)CodeFromGithub
§ TestHarness-https://github.com/jtdeane/amqp-client§ AMQPConsumer- https://github.com/jtdeane/amqp-consumer§ AMQPProducer- https://github.com/jtdeane/amqp-producer§ AMQPComplexEventProcessor(CEP)- https://github.com/jtdeane/amqp-cep
Hello Bunny!
1. StartRabbitMQ
2. Buildandimportamqp-client intoIDE
3. SendAMQPMessage– RunasJUnit Test
4. ViewExchange,Queue,andMessageinManagementConsole
5. ReceiveAMQPMessage– RunasJUnit Test
mvn clean install
mvn test -Dtest=ProducerFunctionalTest
mvn test -Dtest=ConsumerFunctionalTest
http://localhost:15672/#/exchanges/%2F/test.direct
http://localhost:15672/#/queues/%2F/magic.greetings
amqp-client/src/test/java/ws/cogito/magic/messaging/ProducerFunctionalTest.javaamqp-client/src/test/java/ws/cogito/magic/messaging/ConsumerFunctionalTest.java
P2PHidden Costs
BottomLineSOAbyMarcRix
WebservicesAwebservicedoesNOT trulydecoupletheconsumerandprovider
P2PIntegrationsThecostofmaintainingP2Pintegrationsincreasesexponentiallyasthenumberoftheconnectionsincreases
#ofconnections
$
gain
value
cost
loss
Enterprise IntegrationPatterns*
*A.K.A. MessageExchangePatterns(MEP)
Publish Subscribe Domain (Topics)
Point-to-Point Domain (Queues)
Enterprise IntegrationPatterns
Channel Message FIlterSquareTriangle ChannelSquare SquareSquare
GuaranteedMessageDelivery
FilterMessages
AsynchronousRequestReply
Message OrientedArchitecture(MOA)
MessageSender Receiver
Broker
TopicQueue
createsprocesse
s
Delivers
Connects/ to Connects/ to
Hosted//by
AMQPMessageMessage
Properties{Standard or Broker Defined}
Headers{Custom}
Payload{Stream of bytes}
Property Purpose
ContentType ThecontenttypeofthemessageasaMediaType.
ContentEncoding Theencodingto andfrombytes(UTF-8)
RoutingKey UsedbyanExchangeformessagerouting
DeliveryMode Persistent(2)ornon-persistent(1)
Reply To FormessagingforwardingorRequest-ReplyPattern
Expiration Time-to-Live (TTL)inMilliseconds
Priority MovemessageupQueue(defaultis0)
CustomHeaderExample:TrackingID
AMQPVirtualHosts, Exchanges&RoutingKeys§ VirtualHostsprovidelogicalcontainersforExchanges&Queues
§ ReferredtoasVHost§ DefaultVirtualHost(/)– Donotdelete!§ SuggestedNamingScheme:<owner>.<domain || purpose>.vhost
§ ExchangesroutemessagesbasedonaRoutingKey§ DirectExchange:Point-to-Point(P2P)§ TopicExchange: Publish-Subscribe§ Fanout Exchange: Broadcast
§DefaultExchanges(AMQP)– Donotdelete!§ SuggestedNamingScheme:<owner>.<domain || purpose>.<Exchange Type>
§ RoutingKeysrepresentahierarchicalnamespace§ ws.cogito.magic.sales.orders§ ws.cogito.magic.finance.bills
Message Broker
Virtual Host
Publisher ExchangeMessage
AMQPQueues andBindingKeys§ QueuesareassociatedwithExchangesviaBindingKeys
§ Durable orTransient§ SuggestedNamingScheme:<owner>.<hierarchy>.<entities>
§ acme.magic.finance.invoices
§ acme.magic.sales.orders
§ BindingKeysoftenmatchQueueNamesexactlyforDirectExchanges§ acme.magic.finance.invoices
§ BindingKeysoftenusespecialcharactersforTopicExchanges§ * (star)cansubstituteforexactlyoneword§ # (hash)cansubstituteforzeroormorewords
§ acme.magic.finance.*§ acme.magic.#
AMQP Connections,ChannelsandAcknowledgements§ Connections arelonglivedTCPconnectionswhileChannelsmultiplexedinteractionswithinaConnection.
§MessageAcknowledgementsareautomaticbydefaultbutcanbemanual
§Messagescanalsoberejected orre-queued orsenttoDeadLetterQueue(DLQ)
AMQPExercisesSetup1. StartRabbitMQ
2. CreateUser(client/client)– AdminTag
3. CreateVirtualHost(VHost)– cogito
4. AssociateUserclient toVHost cogito&/
5. Buildandbootamqp-client
6. ViewdeclaredExchanges
http://localhost:15672/#/users
http://localhost:15672/#/vhosts
http://localhost:15672/#/vhosts/cogito
mvn clean installmvn spring-boot:run -Drun.arguments="-Xmx256m,-Xms128m"
http://localhost:15672/#/exchanges/cogito/magic.directhttp://localhost:15672/#/exchanges/cogito/magic.topic
Code:amqp-client/src/main/java/ws/cogito/magic/utilities/AmqpDeclarationsConfiguration.java
AMQPExercisesSetup1. CreateQueuemagic.tests *
2. CreateQueueDirect&TopicBindings**
3. CreateUsers)– AdminTag• consumer/consumer• producer/producer• evaluator/evaluator
4. AssociateUserstoVHost cogito&/
http://localhost:15672/#/queues
Exchange: magic.direct, Queue & Routing Key: magic.tests
Exchange: magic.topicTopic: magic.testing.eventsRouting Key: magic.testing.#
Knuffle-Bunny-Cautionary*Shortenedfortesting,shouldbews.cogito.magic….etc**Normallythereshouldbeonequeueperconsumer
AMQPSender
Note:normallythereshouldbeonequeueperconsumer;thisisjustfortesting
http://tryrabbitmq.com/
AMQPSender1. IssueHTTPRequestviacurlorRESTclient(e.g.Postman)- Direct
2. ViewmessageinQueueandthenpurgeQueue
3. IssueHTTPRequestviacurlorRESTclient(e.g.Postman)- Topic
4. ViewmessageinQueueandthenpurgeQueue
5. ViewCode
curl -i -H "Accept: application/json" -H "Content-Type: application/json" -X POST -d '{"order":1125, "customer":"Houdini", "item":"Cards", "amount":6}' http://localhost:9999/amqp/magic.direct/magic.tests
http://localhost:15672/#/queues/cogito/magic.tests
curl -i -H "Accept: application/json" -H "Content-Type: application/json" -X POST -d '{"order":1126, "customer":"Teller", "item":"Dice", "amount":4}' http://localhost:9999/amqp/magic.topic/magic.testing.foo
http://localhost:15672/#/queues/cogito/magic.tests
/amqp-client/src/main/java/ws/cogito/magic/utilities/web/AmqpController.java
1. StartRabbitMQ
2. Buildandbootamqp-consumer
3. ViewQueuesandBindings
4. ViewDeclarationCode
5. Bootamqp-client
AMQP Listener
mvn clean installmvn spring-boot:run -Drun.arguments="-Xmx256m,-Xms128m"
http://localhost:15672/#/queues/cogito/magic.eventshttp://localhost:15672/#/queues/cogito/magic.sales.orders
/amqp-consumer/src/main/java/ws/cogito/magic/AmqpDeclarationsConfiguration.java
mvn clean installmvn spring-boot:run -Drun.arguments="-Xmx256m,-Xms128m"
6. SendOrderMessageviaamqp-client
7. Viewamqp-client andamqp-consumer consolelogs
6. ViewOrderListenerCode
7. Stopamqp-consumerandsendanotherOrderMessage
8. ViewmessageviaManagementConsole
9. Bootamqp-consumerandviewconsolelogs
AMQP Listener
curl -i -H "Accept: application/json" -H "Content-Type: application/json" -X POST -d '{"order":1127, "customer":"Blain", "item":"Marbles", "amount":2}' http://localhost:9999/amqp/magic.direct/magic.sales.orders
/amqp-consumer/src/main/java/ws/cogito/magic/messaging/OrdersMessageListener.java/amqp-consumer/src/main/java/ws/cogito/magic/MessageLogging.java
curl -i -H "Accept: application/json" -H "Content-Type: application/json" -X POST -d '{"order":1128, "customer":"Blackstone", "item":"Scarf", "amount":4}' http://localhost:9999/amqp/magic.direct/magic.sales.orders
1. SendEventMessageviaamqp-client
2. Viewamqp-client andamqp-consumer consolelogs
6. ViewEventListenerCode
7. SendanotherEventMessageviaamqp-client
8. Viewamqp-client andamqp-consumer consolelogs
AMQP Subscriber
curl -i -H "Accept: application/json" -H "Content-Type: application/json" -X POST -d '{"event":21, "location":"Las Vegas", "title":"Penn and Teller", "date":"2016-12-31"}' http://localhost:9999/amqp/magic.topic/magic.events.shows
/amqp-consumer/src/main/java/ws/cogito/magic/messaging/EventsMessageListener.java
curl -i -H "Accept: application/json" -H "Content-Type: application/json" -X POST -d '{"event":22, "location":"Clearwater", "title":"ArchConf", "date":"2016-12-12"}' http://localhost:9999/amqp/magic.topic/magic.events.conferences
RabbitMQ HighAvailabilityTopologyRabbitMQ Server
Message BrokerVirtual Host
RabbitMQ Server
Message BrokerVirtual Host
Mirrored Queues
Exchange Exchange
Queue A
Queue B
Queue A
Queue B
§ RabbitMQ ServersareClustered – SharedSecret– Erlang Cookie
§ RabbitMQ HAPolicy:rabbitmqctl set_policy ha-all "" '{"ha-mode":"all"}'
§ ClientsconnecttosetofaddressesorLoadBalancercanproxyClients(e.g.HAProxy)
RabbitMQ Federated Topologies
RabbitMQ Server RabbitMQ Server
Finance Cluster
RabbitMQ Server
RabbitMQ Server RabbitMQ Server
Sales Cluster
RabbitMQ Server
Federation Plugin
Shovel Plugin
§ FederationPlugin– Both“upstream”and“downstream”Consumersreceivemessage
§ ShovelPlugin– ForwardsMessagefrom“upstream”Queuetoremote“downstream”Exchange
RabbitMQ Security§ UsershaveVHost Read,Write,andConfigurepermissions
§ LDAPPlugin – Authentication-Authorization
§ TransportLevelSecurity(TLS)– Port5671§ TLSConnectionFactoryGist
§EncryptionatRest(Mnesia DB&Logs)– e.g.VormetricRabbitMQ Server
Message BrokerVirtual Host
DB
User Security
Logs
Transport Level Security
Encryption at Rest
RabbitMQ Operations§WebConsole
§ HTTPRESTAPI (guest/guest)– ListoutVirtualHosts
§ NewRelic &Zabbix Plugins
§ CommandLineInterface(CLI)&Tool
1. DownloadCommandLineToolin$RABBITMQ_HOME/sbin
2. Mayneedtoupdatepermissionsonfile
3. ListoutVirtualHosts
http://localhost:15672/cli/
chmod 777 rabbitmqadmin
rabbitmqadmin list vhosts
http://localhost:15672/api/vhosts
AMQP– ComplexEventProcessing1. StartRabbitMQ
2. Buildandbootamqp-cep
3. ViewQueuesandBindings
4. ViewDeclarationCode
5. Bootamqp-consumer &amqp-client
mvn clean installmvn spring-boot:run -Drun.arguments="-Xmx256m,-Xms128m"
http://localhost:15672/#/queues/cogito/magic.evaluationshttp://localhost:15672/#/queues/cogito/magic.alerts
/amqp-cep/src/main/java/ws/cogito/magic/AmqpDeclarationsConfiguration.java
mvn clean installmvn spring-boot:run -Drun.arguments="-Xmx256m,-Xms128m"
AMQP– ComplexEventProcessing6. PublishEventviacurlorRESTclient(e.g.Postman)
7. Viewamqp-client, amqp-consumer, amqp-cep consolelogs
8. ViewtheEvaluationMessageListener
9. PublishEventviacurlorRESTclient(e.g.Postman)
10. Viewamqp-client, amqp-consumer, amqp-cep consolelogs• Notehowanalertisnotraisedbecauseofthelocation
curl -i -H "Accept: application/json" -H "Content-Type: application/json" -X POST -d '{"event":21, "location":"Las Vegas", "title":"Penn and Teller", "date":"2016-12-31"}' http://localhost:9999/amqp/magic.topic/magic.events.shows
/amqp-cep/src/main/java/ws/cogito/magic/messaging/EventsMessageListener.java
curl -i -H "Accept: application/json" -H "Content-Type: application/json" -X POST -d '{"event":22, "location":"Clearwater", "title":"ArchConf", "date":"2016-12-12"}' http://localhost:9999/amqp/magic.topic/magic.events.conferences
AMQP– WhynotSpringIntegrationsorApacheCamel?@Bean
public IntegrationFlow amqpFlow() {return IntegrationFlows
.from(Amqp.inboundGateway(this.rabbitConnectionFactory, queue())).transform("hello "::concat).transform(String.class, String::toUpperCase)
.get();}
@Beanpublic IntegrationFlow amqpOutboundFlow() {
return IntegrationFlows.from("amqpOutboundInput")
.handle(Amqp.outboundAdapter(this.amqpTemplate).routingKeyExpression("headers.routingKey"))
.get();
context.addRoutes(new RouteBuilder() {public void configure() {
from("rabbitmq://localhost?exchange=direct_logs&exchangeType=direct&bindingKey=ERROR")
.setBody().simple("ERROR: ${in.body}")
.to("stream:out");
from("rabbitmq://localhost?exchange=direct_logs&exchangeType=direct&bindingKey=WARN").setBody().simple("WARN: ${in.body}")
.to("stream:out");
}
});
SpringIntegrations– Immature- poorlydocumentedDSL
ApacheCamel– ImmatureAMQP&RabbitMQ Components
AMQPAsynchronousRequest-Reply– TempQueue
http://tryrabbitmq.com/
1
2
3
StartswithPOSThttp://localhost:9000/priority/orders
4
AMQP– Asynchronous Request-Reply – TempQueue1. StartRabbitMQ
2. Buildandbootamqp-producer
3. ViewQueueandBinding
4. ViewDeclarationCode
5. Bootamqp-consumer
mvn clean installmvn spring-boot:run -Drun.arguments="-Xmx256m,-Xms128m"
http://localhost:15672/#/queues/cogito/magic.priority.orders
/amqp-producer/src/main/java/ws/cogito/magic/AmqpDeclarationsConfiguration.java
mvn clean installmvn spring-boot:run -Drun.arguments="-Xmx256m,-Xms128m"
AMQP– Asynchronous Request-Reply – TempQueue6. IssuePriorityOrderRequestviacurlorRESTclient(e.g.Postman)
7. ViewthePriorityOrderController:processOrder
8. Viewamqp-consumer andamqp-producer consolelogs• NotetheTrackingIDandReplyTo Queue
curl -i -H "Accept: application/json" -H "Content-Type: application/json" -X POST -d '{"order":1129, "customer":"Penn", "item":"Rabbit", "amount":6}' http://localhost:9000/priority/orders
/amqp-consumer/src/main/java/ws/cogito/magic/web/PriorityController.java
“Beware of time-outs & downtime”
AMQPAsynchronousRequest-Reply– CorrelationID
http://tryrabbitmq.com/StartswithPOSThttp://localhost:9000/priority/vip/orders
12
3
4
AMQP– Asynchronous Request-Reply – CorrelationID1. StartRabbitMQ
2. Buildandbootamqp-producer
3. ViewQueueandBinding
4. ViewDeclarationCode
5. Bootamqp-consumer
mvn clean installmvn spring-boot:run -Drun.arguments="-Xmx256m,-Xms128m"
http://localhost:15672/#/queues/cogito/magic.priority.orders
/amqp-producer/src/main/java/ws/cogito/magic/AmqpDeclarationsConfiguration.java
mvn clean installmvn spring-boot:run -Drun.arguments="-Xmx256m,-Xms128m"
AMQP– Asynchronous Request-Reply– CorrelationID1. StartRabbitMQ
2. Buildandbootamqp-producer and amqp-consumer
3. IssueVIPOrderRequestviacurlorRESTclient(e.g.Postman)
4. ViewthePriorityOrderController:processOrderVip
5. View amqp-consumer andconsolelogsamqp-producer• NotetheTrackingID,CorrelationID,andReplyTo Queue
6. ViewthePriorityMessageListener
curl -i -H "Accept: application/json" -H "Content-Type: application/json" -X POST -d '{"order":1130, "customer":"Angel", "item":"quarters", "amount":6}' http://localhost:9000/priority/vip/orders
/amqp-consumer/src/main/java/ws/cogito/magic/web/PriorityController.java
/amqp-producer/src/main/java/ws/cogito/magic/messaging/PriorityMessageListener.java
Questions&FeedbackQuestions&Feedback
MyContactinformation:
[email protected]://jeremydeane.net
https://github.com/jtdeane/amqp-client
https://github.com/jtdeane/amqp-consumer
https://github.com/jtdeane/amqp-producer
https://github.com/jtdeane/amqp-cep