Embed Size (px)
DESCRIPTION
Developing highly scalable applications with Symfony and RabbitMQ
Citation preview
Developing highly scalable
applications with
Symfony and RabbitMQ
Alexey Petrov
Symfony and RabbitMQ
Agenda
• Story 1: For those who not used queueing before
• Story 2: For those who started using queueing
• Story 3: For those who have to scale and distribute
• Story 4: Symfony2 Integration
Symfony and RabbitMQ
Story 1
For those who not used queueing before
Symfony and RabbitMQ
Request
ResponseTime
P
R
O
C
E
S
S
I
N
G
Symfony and RabbitMQ
Time min
Muscules
(better hardware)
Brains
(better software)
Prepare
Response
Symfony and RabbitMQ
Request
ResponseTime
Parsing
Request
P
R
O
C
E
S
S
I
N
G
Symfony and RabbitMQ
Request
Response
Time
Parsing
Request
Prepare
Response
P
R
O
C
E
S
S
I
N
G
Symfony and RabbitMQ
Request
Response
Time
Parsing
Request
Prepare
Response Task 1 Task 2
Symfony and RabbitMQ
What is asynchronous?
Symfony and RabbitMQ
Symfony and RabbitMQ
How to run asynchronous
process from PHP
Symfony and RabbitMQ
• popen
• proc_open
• shell_exec
• system
• exec
Symfony and RabbitMQ
Symfony Process
Symfony and RabbitMQ
Symfony and RabbitMQ
Symfony and RabbitMQ
Symfony and RabbitMQ
Problems?
Symfony and RabbitMQ
1 Request = X tasks
N Requests = N*X tasks
Symfony and RabbitMQ
T
A
S
K
T
A
S
K
T
A
S
K
T
A
S
K
T
A
S
K
T
A
S
K
T
A
S
K
T
A
S
K
Worker
Symfony and RabbitMQ
T
A
S
K
T
A
S
K
T
A
S
K
T
A
S
K
T
A
S
K
T
A
S
K
T
A
S
K
T
A
S
K
Worker
Database Cron Job
Symfony and RabbitMQ
Problems?
Symfony and RabbitMQ
T
A
S
K
T
A
S
K
T
A
S
K
T
A
S
K
T
A
S
K
T
A
S
K
T
A
S
K
T
A
S
K
Worker
Producer
Producer
Producer
Symfony and RabbitMQ
T
A
S
K
T
A
S
K
T
A
S
K
T
A
S
K
T
A
S
K
T
A
S
K
T
A
S
K
T
A
S
K
WorkerProducer
Producer
Producer
Worker
Worker
Symfony and RabbitMQ
Solution:
USE RIGHT TOOLS!
Symfony and RabbitMQ
Use RabbitMQ
Symfony and RabbitMQ
Symfony and RabbitMQ
Story 2
For those who started using queueing
Symfony and RabbitMQ
Basic Examples
http://www.rabbitmq.com/getstarted.html
Symfony and RabbitMQ
Connection and Channels
Symfony and RabbitMQ
COOL STORY BRO
But we’re in PHP :(
Symfony and RabbitMQ
Exchanges To Exchanges
Symfony and RabbitMQ
Exchanges and Queues
FanoutDB1
DB2
Msg Topic Stats
SMSTopic
Symfony and RabbitMQ
Libraries
• PhpAmqpLibhttps://github.com/videlalvaro/php-amqplib
• PECL amqphttp://php.net/manual/pl/book.amqp.php
https://github.com/alanxz/rabbitmq-c
https://github.com/bkw/pecl-amqp-official/tree/master/stubs
Symfony and RabbitMQ
PECL amqp
https://bugs.php.net/bug.php?id=66661
Symfony and RabbitMQ
Durability
Symfony and RabbitMQ
Durability
• Exchanges
• Queues
• Messages
Symfony and RabbitMQ
Durability
Symfony and RabbitMQ
Transactions
Symfony and RabbitMQ
Transactions
Symfony and RabbitMQ
4 minutes to publish
10 000 messages
Symfony and RabbitMQ
Publisher Confirms
Symfony and RabbitMQ
Publisher Confirms
Symfony and RabbitMQ
Publisher Confirms
Symfony and RabbitMQ
Dead Letter Exchanges
Symfony and RabbitMQ
Dead Letter Exchanges
• Delayed Messages (Events)
• Retries
• Errors processing
• etc.
Symfony and RabbitMQ
Dead Letter Exchanges
• The message is rejected
(basic.reject or basic.nack)
with requeue=false
• The TTL for the message expires;
• The queue length limit is exceeded.
Symfony and RabbitMQ
Dead Letter Exchanges
Symfony and RabbitMQ
Dead Letter Messages
x-death header:
queue – name of the queue the message was in before
reason – rejected | expired | maxlen
time – timestamp the message was dead lettered
exchange – the exchange the message was published to
routing-keys – the routing keys the message was published with.
original-expiration – the original expiration message property
Symfony and RabbitMQ
Delayed Pattern
MSG ExchangeQ1
TTL = 60000
DLX Q2
Symfony and RabbitMQ
More Queues = Better
Queue lives in 1 Eralng process
Symfony and RabbitMQ
Per-Message TTL
vs
Per-Queue TTL
Symfony and RabbitMQ
Symfony and RabbitMQ
Story 3
For those who have to scale and distribute
Symfony and RabbitMQ
Clustering
Symfony and RabbitMQ
Clustering
Symfony and RabbitMQ
1 client = 1 socket
Cluster helps to scale!
Symfony and RabbitMQ
Mirrored queues
Symfony and RabbitMQ
Mirrored queues
Symfony and RabbitMQ
Network Partitions
Symfony and RabbitMQ
Network Partitions
• Ignore mode
• Pause-minority mode
• Autoheal mode
Symfony and RabbitMQ
Federation
Symfony and RabbitMQ
Federation
Symfony and RabbitMQ
Federated Exchanges
Symfony and RabbitMQ
Federated Queues
Symfony and RabbitMQ
Shovel
Symfony and RabbitMQ
The Symfony
Symfony and RabbitMQ
Story 4
Symfony2 Integration
Symfony and RabbitMQ
Installing Bundle
Symfony and RabbitMQ
Configuration
Symfony and RabbitMQ
Producer
Symfony and RabbitMQ
Consumer
Symfony and RabbitMQ
Questions?
Alexey Petrov
Senior PHP Developer / System Architect
Email: [email protected]
Twitter: @alexhelkar
