Click here to load reader

Distributed messaging with AMQP

  • View
    702

  • Download
    1

Embed Size (px)

DESCRIPTION

A presentation on the AMQP protocol with a brief live demo of RabbitMQ to illustrate the protocol.

Text of Distributed messaging with AMQP

  • 1.Distributed Messaging AMQP, RabbitMQ (Quick demo) Tuesday, 10 September 13

2. Event Logging Current architecture & problems Tuesday, 10 September 13 3. Event Logging Server Client Client Client MongoDB Tuesday, 10 September 13 4. Problem Server Client Client Client 80 110 65 MongoDB Tuesday, 10 September 13 5. Problem Server Client Client Client 80 110 65 255 MongoDB Tuesday, 10 September 13 6. Problem Server Client Client Client 0 0 0 0 MongoDB Tuesday, 10 September 13 7. Problem Server Client Client Client 0 0 0 0 MongoDB Tuesday, 10 September 13 8. Possible Solutions Server tweaks:TCP conns, ulimit, etc Multi-master model Localised queue ... and quite a few more Tuesday, 10 September 13 9. Possible Solutions Server tweaks:TCP conns, ulimit, etc Multi-master model Localised queue ... and quite a few more Tuesday, 10 September 13 10. Localised Queues Fast (no TCP latency) Network partition tolerant Modular & scalable Reliable & Durable Tuesday, 10 September 13 11. Why Localise? Server Client Client Client 80 110 65 MongoDB Q Q Q < 10 Tuesday, 10 September 13 12. Why Localise? Server Client Client Client 80 110 65 MongoDB Q Q Q 0 Tuesday, 10 September 13 13. Why Localise? Server Client Client Client 80 110 65 MongoDB Q Q Q 0 Tuesday, 10 September 13 14. Why Localise? Server Client Client Client 80 110 65 MongoDB Q Q Q < 10 Tuesday, 10 September 13 15. Why Localise? Server Client Client Client 80 110 65 MongoDB Q Q Q < 10 Client Q Tuesday, 10 September 13 16. Queue Criteria Reliable Durable Scalable Tuesday, 10 September 13 17. AMQP Advanced Message Queueing Protocol Tuesday, 10 September 13 18. Why AMQP? Stable and mature Designed for SX markets Widely supported (good interop) Modular, Fast & Flexible Tuesday, 10 September 13 19. Service ModelWire AMQP Overview AMQP Advanced Message Queueing Protocol Tuesday, 10 September 13 20. Service Model Wire AMQP Overview AMQP Tuesday, 10 September 13 21. Model/component spec Exchange Message Queue Binding AMQ Model Wire AMQP Overview AMQP Tuesday, 10 September 13 22. Model/component spec Exchange Message Queue Binding AMQ Model Wire AMQP Overview AMQP Function layer Transactions, Exchange, Queues, etc Transport layer Protocol format Data rep/framing Multiplexing, heart-beating, etc Tuesday, 10 September 13 23. AMQ Model Wire AMQP Overview AMQP Function layer Transactions, Exchange, Queues, etc Transport layer Protocol format Data rep/framing Multiplexing, heart-beating, etc Model/component spec Exchange Message Queue Binding Tuesday, 10 September 13 24. ExchangeQueuesBindingAMQ Model AMQ Model Tuesday, 10 September 13 25. QueuesExchange Binding AMQ Model AMQ Model Tuesday, 10 September 13 26. Virtual Host QueuesExchange Binding AMQ Model AMQ Model Tuesday, 10 September 13 27. Virtual Host QueuesExchange Binding AMQ Model AMQ Model Publisher Consumer Tuesday, 10 September 13 28. Virtual Host QueuesExchange Binding AMQ Model Publisher Consumer Tuesday, 10 September 13 29. Virtual Host QueuesExchange Binding AMQ Model Router Message Publisher Consumer Tuesday, 10 September 13 30. Virtual Host QueuesExchange Binding AMQ Model Router Message Publisher Consumer Tuesday, 10 September 13 31. Virtual Host QueuesExchange Binding AMQ Model Router Message Publisher Consumer Tuesday, 10 September 13 32. Virtual Host QueuesExchange Binding AMQ Model Router ?Message Publisher Consumer Tuesday, 10 September 13 33. Virtual Host QueuesExchange Binding AMQ Model Message Router ? Publisher Consumer Tuesday, 10 September 13 34. Virtual Host QueuesExchange Binding AMQ Model Message Router Publisher Queue: critical_error_queue Exchange: errors_exchange Filter: *.error.critical Consumer Tuesday, 10 September 13 35. Virtual Host QueuesExchange Binding AMQ Model Queue: critical_error_queue Exchange: errors_exchange Filter: *.error.critical Message Router Publisher Consumer Tuesday, 10 September 13 36. QueuesExchange Binding AMQP Service Model Tuesday, 10 September 13 37. QueuesExchange Binding AMQP Service Model routes messages based on criteria doesnt store messages can inspect message content can be created at runtime consumers Tuesday, 10 September 13 38. QueuesExchange Binding AMQP Service Model store messages named bound-able to exchange criteria can be created at runtime by consumers routes messages based on criteria doesnt store messages can inspect message content can be created at runtime consumers Tuesday, 10 September 13 39. QueuesExchange Binding AMQP Service Model creates a relationship between queues and exchanges contains criteria and properties can be created at runtime by consumers store messages named bound-able to exchange criteria can be created at runtime by consumers routes messages based on criteria doesnt store messages can inspect message content can be created at runtime consumers Tuesday, 10 September 13 40. QueuesExchange Binding AMQP Service Model creates a relationship between queues and exchanges contains criteria and properties can be created at runtime by consumers store messages named bound-able to exchange criteria can be created at runtime by consumers routes messages based on criteria doesnt store messages can inspect message content can be created at runtime consumers Tuesday, 10 September 13 41. QueuesExchange Binding AMQP Service Model Publisher Consumer creates a relationship between queues and exchanges contains criteria and properties can be created at runtime by consumers store messages named bound-able to exchange criteria can be created at runtime by consumers routes messages based on criteria doesnt store messages can inspect message content can be created at runtime consumers Tuesday, 10 September 13 42. QueuesExchange Binding AMQP Service Model Publisher Consumer creates a relationship between queues and exchanges contains criteria and properties can be created at runtime by consumers store messages named bound-able to exchange criteria can be created at runtime by consumers routes messages based on criteria doesnt store messages can inspect message content can be created at runtime consumers can create exchanges and queues can create exchanges and queues Tuesday, 10 September 13 43. Exchange AMQP Service Model Tuesday, 10 September 13 44. FanoutTopicHeadersSystem Exchange AMQP Service Model Types (routing algo) Tuesday, 10 September 13 45. Fanout Topic Headers System Exchange AMQP Service Model Types (routing algo) Direct Tuesday, 10 September 13 46. Fanout Topic Headers System Exchange AMQP Service Model Types (routing algo) Direct Tuesday, 10 September 13 47. Exchange AMQP Service Model Direct Type Tuesday, 10 September 13 48. Exchange events AMQP Service Model Direct Type Tuesday, 10 September 13 49. Exchange events AMQP Service Model Direct Type Message Message Message event.user.click event.user.view event.user.share Tuesday, 10 September 13 50. Exchange events AMQP Service Model Direct Type Message Message Message event.user.click event.user.view event.user.share user_click_queue Tuesday, 10 September 13 51. Exchange events Binding AMQP Service Model Queue: user_click_queue Direct Type Message Message Message event.user.click event.user.view event.user.share user_click_queue Tuesday, 10 September 13 52. Exchange events Binding AMQP Service Model Queue: user_click_queue Exchange: events Direct Type Message Message Message event.user.click event.user.view event.user.share user_click_queue Tuesday, 10 September 13 53. Exchange events Binding AMQP Service Model Queue: user_click_queue Exchange: events Routing Key: event.user.click Direct Type Message Message Message event.user.click event.user.view event.user.share user_click_queue Tuesday, 10 September 13 54. Exchange events Binding AMQP Service Model Queue: user_click_queue Exchange: events Routing Key: event.user.click Direct Type Message Message Message event.user.click event.user.view event.user.share PHP Script PHP Script PHP Script user_click_queue Tuesday, 10 September 13 55. Exchange events AMQP Service Model Direct Type Message Message Message event.user.click event.user.view event.user.share PHP Script PHP Script PHP Script user_click_queue ConsumerPublisher/ Producer Tuesday, 10 September 13 56. Exchange events AMQP Service Model Direct Type Message Message Message event.user.click event.user.view event.user.share user_click_queue user_view_queue user_share_queue Tuesday, 10 September 13 57. Exchange events AMQP Service Model Direct Type Message Message Message event.user.click event.user.view event.user.share user_click_queue user_view_queue user_share_queue Message Message Message Tuesday, 10 September 13 58. Exchange events AMQP Service Model What if... Message Message Message event.user.click event.user.view event.user.share user_interaction_queue user_shares_queue Tuesday, 10 September 13 59. Exchange events AMQP Service Model What if... user_interaction_queue user_shares_queue Message Message Message event.user.click event.user.view event.user.share Tuesday, 10 September 13 60. Exchange events AMQP Service Model What if... Message Message Message event.user.click event.user.view event.user.share user_interaction_queue user_shares_queue Tuesday, 10 September 13 61. Exchange events AMQP Service Model Topic Exchange user_interaction_queue Binding Message Message Message event.user.click event.user.view event.user.share Tuesday, 10 September 13 62. Exchange events AMQP Service Model Topic Exchange user_interaction_queue Binding Queue: user_interaction_queue Exchange: events Routing Key: event.user.* Message Message Message event.user.click event.user.view event.user.share Tuesday, 10 September 13 63. Exchange events AMQP Service Model Topic Exchange user_interaction_queue Binding Queue: user_interaction_queue Exchange: events Routing Key: event.

Search related