Scaling :rubywith Evented I/O

Omer Gazit[github.com/omerisimo]

<< "Machines"

<< "Processes"

<< "Threads"

<< "Reactor pattern"

Scaling Strategies[]

Blocking I/O

start

wait for I/O [e.g. file,network]

complete

Non-Blocking I/O

start

Do something else

complete

Kernel I/O

callback()

call I/O operation

:sync

def sync_io(file_name)file = File.open(file_name)puts "File #{file_name} opened"

end

:async

def async_io(file_name)file = File.async_open(file_name)file.callback do |result|

puts "File #{file_name} opened"end

end

“The reactor design pattern is an event handling pattern for handling service requests delivered concurrently…

The service handler then demultiplexes the incoming requests and dispatches them synchronously to the associated request handlers.”

-wikipedia

Reactor pattern

“The reactor design pattern is an event handling pattern for handling service requests delivered concurrently…

The service handler then demultiplexes the incoming requests and dispatches them synchronously to the associated request handlers.”

-wikipedia

Reactor pattern

Reactor pattern

IO Stream

Demultiplexer

Event Handler A

Event Handler B

Event Handler C

Event Dispatcher

A reactor is a single thread running an endless loop that reacts to incoming events

Reactor pattern

A reactor is a single thread running an endless loop that reacts to incoming events

Reactor pattern

event_loop do

while reactor_running?

expired_timers.each { |timer| timer.process }

new_network_io.each { |io| io.process }

end

Reactor.when?

● Proxies● Real time data delivery

(Websockets)● Streaming● Background processing (MQ listener)● High throughput and mostly I/O

bound

{

javascript: "Node.js",

ruby: "EventMachine",

perl: "AnyEvent",

python: "Twisted",

c: ["libev", "libevent"]

}

implementations=

EventMachine

A toolkit for writing evented applications in Ruby

Is it really faster?

But node.js is so much faster...

Simple HTTP Server

var http = require('http');

http.createServer(function (request, response) {

response.writeHead(200,

{'Content-Type': 'text/plain'}

);

response.send('Hello World\n');

}).listen(8080, '0.0.0.0');

console.log('Server running on port 8080');

Node.js

Simple HTTP Server

EM.run do

EM.start_server "0.0.0.0", 8080 do |server|

def server.receive_data(data)

response = EM::DelegatedHttpResponse .new(self)

response.status = 200;

response.content_type 'text/plain'

response.content = "Hello World\n"

response.send_response

end

end

end

EventMachine

Showdown

results = {

node: { req_per_sec: 2898,

req_time_ms: 35 },

em: { req_per_sec: 6751,

req_time_ms: 15 }

}

ab -n 1000 -c 100 "http://localhost:8080/"

* executed on my MacBook Air

var amqp = require('amqp');

var connection = amqp.createConnection();

connection.on('ready', function() {

connection.queue( 'my_queue', function(queue) {

queue.subscribe(function(payload) {

console.log("Received message: " + payload.body);

}

});

var exchange = connection.exchange();

exchange.publish( 'my_queue', { body: 'Hello World!'});

});

RabbitMQ ProcessorNode.js

RabbitMQ Processor

require 'amqp'

EM.run do

connection = AMQP.connect(host: '0.0.0.0')

channel = AMQP::Channel.new(connection)

queue = channel.queue("my_queue")

queue.subscribe do |metadata, payload|

puts "Received message: #{payload}."

end

exchange = channel.default_exchange

exchange.publish "Hello world!",routing_key: "my_queue"

end

EventMachine

Showdown

average_time_ms = {

node: 4285,

em: 3488

}

send and receive 10k messages

* executed on my MacBook Air

If used correctly!

EM can be really fast

Never Block the Event Loop

Never Block

EM.run do

puts "Started EM"

sleep 2.0

puts "Shutting down EM"

EM.stop

end

EM.run do

puts "Started EM"

EM.add_periodic_timer( 1.0) do

puts "Tick"

end

EM.add_timer(2.0) do

puts "Shutting down EM"

EM.stop

end

end

Blocking Non-Blocking

Never Block

require 'net/http'

EM.run do

response = Net::HTTP.get(URL)

puts "Completed HTTP request"

EM.stop

end

Blocking Non-Blocking

require 'em-http'

EM.run do

http = EM::HttpRequest.new(URL).get

http.callback do |response|

puts "Completed HTTP request"

EM.stop

end

end

<< "igrigorik/em-http-request" # Asynchronous HTTP Client<< "eventmachine/evma_httpserver" # HTTP Server<< "igrigorik/em-websocket" # WebSockets server<< "igrigorik/em-proxy" # High-performance transparent proxies << "brianmario/mysql2" # Make sure to use with :async => true<< "royaltm/ruby-em-pg-client" # PostgreSQL EM client<< "bcg/em-mongo" # EM MongoDB driver (based off of RMongo)<< "simulacre/em-ssh" # EM compatible Net::SSH<< "pressly/uber-s3" # S3 client with asynchronous I/O adapters<< "tmm1/amqp" # AMQP client for EM

module NonBlock

* See full list of protocols at: github.com/eventmachine/eventmachine/wiki/Protocol-Implementations

<< "igrigorik/em-http-request"<< "eventmachine/evma_httpserver"<< "igrigorik/em-websocket"<< "igrigorik/em-proxy"<< "brianmario/mysql2"<< "royaltm/ruby-em-pg-client"<< "bcg/em-mongo"<< "simulacre/em-ssh"<< "pressly/uber-s3"<< "tmm1/amqp"

module NonBlock

* See full list of protocols at: github.com/eventmachine/eventmachine/wiki/Protocol-Implementations

Never Block the Event Loop

EM.defer

EM.run do

long_operation = proc {

sleep(1.0)

"result"

}

callback = proc {|result|

puts "Received #{result}"

EM.stop

}

EM.defer(long_operation, callback)

end

Defer blocking code to a thread

EM.run do

(1..10000).each do |index|

puts "Processing #{index}"

end

EM.stop

end

Blocking Non-Blocking

EM.run do index = 0 process_index = proc { if index < 10000

puts "Processing #{index}" index += 1

EM.next_tick &process_index else EM.stop end } EM.next_tick &process_indexend

Postpone execution to the next iteration

EM.next_tick

EM::Primitives

class DeferrableTimer include EM::Deferrable def wait

EM.add_timer(1.0) do succeed "result"

end self endend

EM.run do timer = DeferrableTimer.new.wait timer.callback do |result|

puts "1 second has passed!" EM.stop endend

EM::DeferrableEM::Deferrable != EM.defer

class EchoServer < EM::Connection def post_init

puts "Client connecting" end

def receive_data(data)puts "Client sending data #{data}"send_data ">> #{data}"

end def unbind

puts "Client disconnecting" end end

EM.run do EM.start_server("0.0.0.0", 9000, EchoServer) # Listen on TCP socketend

EM::Connection

EM::Queue

EM.run do queue = EM::Queue.new queue_handler = proc { |message|

puts "Handling message #{message}"EM.next_tick{ queue.pop( &queue_handler) }

} EM.next_tick{ queue.pop( &queue_handler) }

EM.add_periodic_timer( 1.0) domessage = Time.now.to_sputs "Pushing message ' #{message}' to queue"

queue.push(message) endend

A cross thread, reactor scheduled, linear queue

EM.run do channel = EM::Channel.new handler_1 = proc { | message| puts "Handler 1 message #{message}" } handler_2 = proc { | message| puts "Handler 2 message #{message}" } channel.subscribe &handler_1 channel.subscribe &handler_2

EM.add_periodic_timer( 1.0) domessage = Time.now.to_sputs "Sending message ' #{message}' to channel"

channel << message endend

EM::ChannelProvides a simple thread-safe way to transfer data

between (typically)long running tasks

<< EM::Queue # A cross thread, reactor scheduled, linear queue<< EM::Channel # Simple thread-safe way to transfer data between (typically)long running tasks<< EM::Iterator # A simple iterator for concurrent asynchronous work<< EM.System() # Run external commands without blocking

EM::Primitives

EM.run do EM.add_timer(1) do

json = nilbegin

data = JSON.parse(json) puts "Parsed Json data: #{data}"

rescue StandardException => e puts "Error: #{e.message}"

endEM.stop

endend

Error Handling

Error Handling

EM.run do EM.add_timer(1) do http = EM::HttpRequest.new(BAD_URI).get http.callback do puts "Completed HTTP request" EM.stop end

http.errback do |error| puts "Error: #{error.error}"

EM.stopend

endend

EM::SynchronyFiber aware EventMachine clients and

convenience classes

github.com/igrigorik/em-synchrony

require 'em-synchrony'

require 'em-synchrony/em-http'

EM.synchrony do

res = EM::HttpRequest.new(URL).get

puts "Response: #{res.response}"

EM.stop

end

EM::Synchronyem-synchrony/em-http

require 'eventmachine'

require 'em-http'

EM.run do

http = EM::HttpRequest.new(URL).get

http.callback do

puts "Completed HTTP request"

EM.stop

end

end

em-http-request

Testing

require 'em-spec/rspec'describe LazyCalculator do include EM::SpecHelper default_timeout( 2.0)

it "divides x by y" doem do

calc = LazyCalculator.new.divide(6,3) calc.callback do |result| expect(result).to eq 2 done end

end endend

EM::Spec

class LazyCalculator include EM::Deferrable

def divide(x, y)EM.add_timer(1.0) do

if(y == 0) fail ZeroDivisionError .new else result = x/y succeed result end end

self endend

require 'em-spec/rspec'describe LazyCalculator do include EM::SpecHelper default_timeout( 2.0)

it "fails when dividing by zero" doem do

calc = LazyCalculator.new.divide(6,0) calc.errback do |error| expect(error).to be_a ZeroDivisionError done end

end endend

EM::Spec

class LazyCalculator include EM::Deferrable

def divide(x, y)EM.add_timer(1.0) do

if(y == 0) fail ZeroDivisionError .new else result = x/y succeed result end end

self endend

require 'rspec/em'describe LazyCalculator do include RSpec::EM::FakeClock before { clock.stub } after { clock.reset }

it "divides x by y" do calc = LazyCalculator.new.divide(6,3) expect(calc).to receive(: succeed).with 2 clock.tick(1) end

it "fails when dividing by zero" do calc = LazyCalculator.new.divide(6,0) expect(calc).to receive(: fail).with(kind_of(ZeroDivisionError )) clock.tick(1) endend

RSpec::EM::FakeClock

Project DemoA proxy for Google Geocode API

Limitations

● Can only use async libraries or have to defer to threads.

● Hard to debug (no stack trace)● Harder to test● Difficult to build full blown

websites.

Caveats

● Low community support● The last release is almost two

years old

Summary

● Evented I/O offers a cost effective way to scale applications

● EventMachine is a fast, scalable and production ready toolbox

● Write elegant event-driven code ● It is not the right tool for every

problem

EM.next?

<< ['em_synchrony' + Fiber]

<< ['async_sinatra' + Thin] # Sinatra on EM

<< ['goliath'] # Non-blocking web framework

<< [EM.epoll + EM.kqueue] # maximize demultiplexer polling limits

<< [Celluloid + Celluloid.IO] # Actor pattern + Evented I/O

Thank you

EM.stop

Code examples: github.com/omerisimo/em_underground