Low Latency Loggingwith RabbitMQ
BrnoPHP Conference 2014
James Titcumbwww.jamestitcumb.comwww.protected.co.ukwww.phphants.co.uk@asgrim
Who is this guy?
Let’s go back to basics...
Errors
source: http://www.dpaddbags.com/blog/episode-119-technology-sucks/
error_reporting(0);
They look rubbish!
@
Log to file only
Ways Around// Will raise E_WARNING
fopen($somefile, 'r');
Ways Around// No error! :)
if (file_exists($somefile)) {
fopen($somefile, 'r');
} else {
// nice handling...
// maybe throw exception...
}
Exceptions
Jargon Buster
● throwTriggering
● tryTry to run
● catchHandle it
● finallyRun code after try/catch
source: http://www.dpaddbags.com/blog/episode-119-technology-sucks/
C♯/C++
Catchable
Turn into fatal errorsif not caught
Classy
Descriptive exceptions
Example (exception class)class DivisionByZeroException
extends LogicException
{
}
Example (throw)class Mathematics {
public function divide($a, $b) {
if ($b == 0) {
$msg = sprintf(“Divide %s by zero”, $a);
throw new DivisionByZeroException($msg);
}
return ($a / $b);
}
}
Example (catch)$maths = new Mathematics();
try
{
$result = $maths->divide(5, 0);
}
catch (DivisionByZeroException $exception)
{
$logger->warning($exception->getMessage());
}
Throw Descriptively
Logging
What is “logging”?
What is “logging”?Keeping a record of all events...
What is “logging”?Keeping a record of all events...
exceptions, errors, warnings, info, debug
Paper Trail
Log like you just don’t care
Log like you just don’t care
Log wherever you like
What about Apache’s error_log?
source: http://up-ship.com/blog/?p=20903
Doin’ it rightwrong… // Magic file that makes your entire project work perfectly
@ini_set('display_errors', 0);
@error_reporting(0);
function __globalErrorHandler() {
return true;
}
@set_error_handler('__globalErrorHandler');
@set_exception_handler('__globalErrorHandler');
@register_shutdown_function(function() {
if(error_get_last())
echo "Script executed successfully!";
}); https://github.com/webarto/boostrap.php
Fire & Forget
Low Latency
Highly Available
PSR-3
● monolog (PSR-3)● Drupal - PSR-3 Watchdog● phpconsole● log4php● RavenPHP + Sentry● FirePHP (dev environment)● Roll your own
Logging Options
source: http://mirror-sg-wv1.gallery.hd.org/_c/natural-science/cogs-with-meshed-teeth-AJHD.jpg.html
Capturing Messages
Capturing Messagesset_error_handler()
Capturing Messagesset_exception_handler()
Capturing Messagesregister_shutdown_function()
Capturing MessagesUsing the Logger Directly
Adapter / Handler
Capture Method
Logger (PSR-3)
Handler / Adapter
Data Storage
Typical PSR-3 Compatible Design
Monolog\ErrorHandler->handleException()
Monolog\Logger->log()
Monolog\Handler->handle()
Monolog
Low Latency Logging
Make it easy
Fire & Forget
Minimum Impact
Slow Logging
ApplicationBrowser Log Server
HTTP request
Send log message to log server
Error!
Acknowledge message
HTTP response to client
Zero Latency Logging (ideal)
ApplicationBrowser Log Server
HTTP request
Send log message to log server
Error!
HTTP response to client
UDP?
We need a balance.
Low Latency Logging (balance)
ApplicationBrowser Log Server
HTTP request
Send log message to log server
Error!
HTTP response to client
…so how?
Disclaimer...
Say hello to RabbitMQ
RabbitMQ - Basic
source: http://www.rabbitmq.com/tutorials/tutorial-one-php.html
Producer Consumer
test_queue
1 2 3 4 5
RabbitMQ - Exchanges
source: http://www.rabbitmq.com/tutorials/tutorial-three-php.html
Exchange
Consumertest_queue
1 2 3 4 5
Consumer
Producer
Producer
Producer
Exchanges are flexible
RabbitMQ === Fast!
Ed\Log\Handler\ErrorHandler->handleException()
Ed\Log\Logger->log()
Ed\Log\Publisher\AmqpPublisher->publish()
Logging Server
Low Latency (using AMQP)
RabbitMQ JSON payload
Fetch message
Low Latency Logging (with AMQP)
ApplicationBrowser Log Server
HTTP request
JSON via AMQP
Error!
HTTP response
RabbitMQ
Why bother?● Scalability
RabbitMQ
Application A
Application B
Application C
Log Worker
Log Worker
Log Worker
Log Worker
Single Point of Failure...
RabbitMQNode 1
RabbitMQNode 3
RabbitMQNode 2
RabbitMQNode 4
RabbitMQNode 5
RabbitMQNode 6
To recap...
Scalable.
Easy.
Fast.
Future proof.
Questions?
James Titcumb@asgrim
Thanks for watching!
Recommended
