Understanding the Rails web model and scalability options

Genius is the gold in the mine; talent is the miner who works and brings it out.Lady Marguerite Blessington

“”

Donets BasinCoal Mine-Ukraine

Genius is the gold in the mine; talent is the miner who works and brings it out.Lady Marguerite Blessington

“”

Donets BasinCoal Mine-Ukraine

Презентация начнется в течение нескольких минут ...

доброе утро

Fabio Akita, Co-Founder @CodeMiner42

“Off-centered software for off-centered people”

Fabio Akita, Co-Founder @CodeMiner42

Understandingthe Rails Web Modeland Scalability Options

?

rails new app

somethings don’t change

CGI

1 request blocks 1 process

require 'rubygems'require 'rack'

class Test def call(env)

sleep 1 # on purpose, simulating a blocking operation

[200, {"Content-Type" => "text/plain"}, ["Hello World!"]] endend

Rack::Handler::Thin.run Test.new






$ rackup config.ru

>> Thin web server (v1.3.1 codename Triple Espresso)>> Maximum connections set to 1024>> Listening on 0.0.0.0:8080, CTRL+C to stop

ab -n 10 -c 1 http://127.0.0.1:8080/

This is ApacheBench, Version 2.3 <$Revision: 1178079 $>...Concurrency Level: 1Time taken for tests: 10.020 seconds...Requests per second: 1.00 [#/sec] (mean)Time per request: 1002.015 [ms] (mean)Time per request: 1002.015 [ms] (mean, across all concurrent requests)Transfer rate: 0.12 [Kbytes/sec] received...

ab -n 10 -c 1 http://127.0.0.1:8080/


ab -n 10 -c 1 http://127.0.0.1:8080/


ab -n 10 -c 1 http://127.0.0.1:8080/


ab -n 10 -c 1 http://127.0.0.1:8080/


Passenger, Unicorn

NginX, HAProxy

Browser Browser Browser Browser Browser Browser

Optimize

Passenger, Unicorn (CoW GC - REE or MRI 2)

NginX, HAProxy


Ruby Enterprise Edition 1.8.7(about to be deprecated)

Copy on Write patchedmark and sweep GC

Ruby 1.9.2 (current) mark and sweep GC

Ruby 1.9.3 (transitioning) Lazy Sweep GC

Ruby 2.0 (future)Copy on Write compatible

Bitmap Marking GC






Bitmap Marking GC






Bitmap Marking GC






Bitmap Marking GC

Passenger, Unicorn

NginX, HAProxy

Browser Browser Browser Browser

Passenger, Unicorn

NginX, HAProxy


Memcached

Passenger, Unicorn

NginX, HAProxy


Memcached

Varnish

Passenger, Unicorn

NginX, HAProxy


Memcached

Varnish

Resque

Passenger, Unicorn

NginX, HAProxy


Memcached

Varnish

Resque

Too Long or Periodic Polling

Notifications and Events in general

Online Gaming

Chatting and Presence

Collaborative Applications

Advanced and more Interactive UI

(function poll(){

$.ajax({ url: "server", success: function(data){ // do something with the received ‘data’

//Setup the next poll recursively poll(); }, dataType: "json"});

})();

(function poll(){ setTimeout(function(){ $.ajax({ url: "server", success: function(data){ // do something with the received ‘data’

//Setup the next poll recursively poll(); }, dataType: "json"}); }, 5000);})();

Cross Frame communication

HTTP Polling (Ajax)

Liveconnect (Java applets, Flash socket)

Long Polling

Comet & HTTP Streaming(ex. forever frame/chunked encode)

W3C/IETF Standard - RFC 6455(12/2011)

Full duplex persistent communication channel

Less overhead than HTTP(down to 2 bytes per frame)

No latency (no reconnections)No polling overhead (on demand)

“Upgrades” HTTP, uses 80/443(kind of friendly to existing proxies)

<script type="text/javascript"> var ws = new WebSocket("ws://example.com:10081/");

</script>

<script type="text/javascript"> var ws = new WebSocket("ws://example.com:10081/");

ws.onopen = function() { ws.send("Hello"); // Sends a message. };

ws.onmessage = function(e) { // Receives a message. alert(e.data); };

ws.onclose = function() { alert("closed"); }; </script>

Protocol IE Firefox Chrome Safari Opera

hixie-75 4 5

hixie-76hybi-00

4(disabled)

6 5.0.1 11(disabled)

hybi-06 HTML5 Labs dev

hybi-07 6

hybi-09 HTML5 Labs

hybi-10 IE 10 7 14

RFC 6455 11 16

POLYFILL/FALLBACK(web-socket-js)

<script type="text/javascript" src="swfobject.js"></script><script type="text/javascript" src="web_socket.js"></script><script type="text/javascript"> // Let the library know where WebSocketMain.swf is: WEB_SOCKET_SWF_LOCATION = "WebSocketMain.swf"; var ws = new WebSocket("ws://example.com:10081/"); ws.onopen = function() { ws.send("Hello"); // Sends a message. }; ws.onmessage = function(e) { // Receives a message. alert(e.data); }; ws.onclose = function() { alert("closed"); }; </script>





Client: kind of OKServer: what do do?

Long running requests

Reactor Pattern(Eventmachine)

synchronous I/O

non-blocking synchronous I/O

non-blocking asynchronous I/0

Main Loop(main thread)

Wait Events(I/O, timers)

Notifies Event Handlers

select poll

select poll epoll kqueue IOCP

libevent libev


libevent libev


Twisted

Tornado

libevent libev


Twisted

Tornado

Node.js

libevent libev


Twisted

Tornado

Node.js

Cool.io

libevent libev


EventMachine

Twisted

Tornado

Node.js

Cool.io

libevent libev


EventMachine

Twisted

Tornado

Node.js

Cool.io

Thin

Goliath







class Test def call(env) EM.defer do sleep 1 # CPU bound, throw to thread-pool (cheating) end [200, {"Content-Type" => "text/plain"}, ["Hello World!"]] endend








ab -n 10 -c 1 http://127.0.0.1:8080/This is ApacheBench, Version 2.3 <$Revision: 1178079 $>...Concurrency Level: 1Time taken for tests: 0.003 seconds...Requests per second: 3219.58 [#/sec] (mean)Time per request: 0.311 [ms] (mean)Time per request: 0.311 [ms] (mean, across all concurrent requests)Transfer rate: 380.44 [Kbytes/sec] received...






#ZOMG!


Block Non-Block

Total Time 10 sec 0.0001 sec

Requests per Second 1 + 8000

Block Non-Block



Block Non-Block



Reactor Loop(Eventmachine)

Timers#add_timer

#add_periodic_timer

Blocking Tasks#defer

#next_tick

“Tick”

require 'rubygems' # or use Bundler.setuprequire 'eventmachine'

class EchoServer < EM::Connection def receive_data(data) if data.strip =~ /[exit|quit]$/i EM.stop else send_data("Repeating: #{data}") end endend

require 'rubygems' # or use Bundler.setuprequire 'eventmachine'

class EchoServer < EM::Connection def receive_data(data) if data.strip =~ /[exit|quit]$/i EM.stop else send_data("Repeating: #{data}") end endend

EventMachine.run do # hit Control + C to stop Signal.trap("INT") { EM.stop } Signal.trap("TERM") { EM.stop }

EM.start_server("0.0.0.0", 10000, EchoServer)end

$ telnet localhost 10000Trying 127.0.0.1...Connected to localhost.Escape character is '^]'.Hello World

$ telnet localhost 10000Trying 127.0.0.1...Connected to localhost.Escape character is '^]'.Hello WorldRepeating: Hello World

$ telnet localhost 10000Trying 127.0.0.1...Connected to localhost.Escape character is '^]'.Hello WorldRepeating: Hello WorldPlay again, Sam

$ telnet localhost 10000Trying 127.0.0.1...Connected to localhost.Escape character is '^]'.Hello WorldRepeating: Hello WorldPlay again, SamRepeating: Play again, Sam

$ telnet localhost 10000Trying 127.0.0.1...Connected to localhost.Escape character is '^]'.Hello WorldRepeating: Hello WorldPlay again, SamRepeating: Play again, Samquit.

$ telnet localhost 10000Trying 127.0.0.1...Connected to localhost.Escape character is '^]'.Hello WorldRepeating: Hello WorldPlay again, SamRepeating: Play again, SamquitConnection closed by foreign host.

require 'rubygems'require 'eventmachine'

EM.run do # ... main reactor loop EM.stop # stop the main loop and exitend


EM.run do puts 1 puts 2 puts 3end

$ ruby em.rb 123


EM.run do EM.defer do sleep 3 puts 1 end

EM.add_timer(1) do puts 2

EM.add_timer(1) do puts 3 end end puts 4end

$ ruby em.rb 4231

var http = require('http');

http.createServer(function (req, res) { res.writeHead(200, {'Content-Type': 'text/plain'}); res.end('Hello World');}).listen(9876);



Rack::Handler::Thin.run Proc.new { [200, {"Content-Type" => "text/plain"}, ["Hello World!"]]}

ab -n 1000 -c 5 http://127.0.0.1:8080/This is ApacheBench, Version 2.3 <$Revision: 1178079 $>...Concurrency Level: 5Time taken for tests: 0.121 seconds..Requests per second: 8239.06 [#/sec] (mean)Time per request: 0.607 [ms] (mean)Time per request: 0.121 [ms] (mean, across all concurrent requests)Transfer rate: 973.56 [Kbytes/sec] received...

Node.js Ruby

libev Eventmachine, Cool.io

Google V8 MRI, JRuby, Rubinius

Less Resources, Faster Processing

More Resources, Slightly slower processing

callbacks only Threads, Fibers, Callbacks

Node.js Ruby






Node.js Ruby






Node.js Ruby






Node.js Ruby






EM-Websocket

require "rubygems"require 'eventmachine'require 'em-websocket'

config = {:host => "0.0.0.0", :port => 8080, :debug => true}EM::WebSocket.start(config) do |ws| ws.onopen { ws.send "Welcome!"} ws.onmessage { |msg| ws.send "Sending Pong: #{msg}" } ws.onclose { puts "connection closed" } ws.onerror { |e| puts "Error: #{e.message}" }end

Caveats in Callback Driven Development

fs.rename('/tmp/hello', '/tmp/world', function (err) { if (err) throw err; console.log('renamed complete');});

fs.stat('/tmp/world', function (err, stats) { if (err) throw err; console.log('stats: ' + JSON.stringify(stats));});

fs.rename('/tmp/hello', '/tmp/world', function (err) { if (err) throw err; fs.stat('/tmp/world', function (err, stats) { if (err) throw err; console.log('stats: ' + JSON.stringify(stats)); });});

EM.run do page = EM::HttpRequest.new(@url1).get page.errback do puts "Site is down! terminate?" end page.callback do about = EM::HttpRequest.new(@url2).get about.callback do # callback nesting, ad infinitum end about.errback do # error-handling code end endend

“Callback Spaghetti”

Ruby 1.9 Fibersto the rescue!

number_generator = Fiber.new do start = 0 loop do start += 1 Fiber.yield(start) endend


> number_generator.resume => 1


> number_generator.resume => 1 > number_generator.resume => 2


> number_generator.resume => 1 > number_generator.resume => 2 > number_generator.resume => 3

Less expensive than Threads

cooperative vs preemptive multitasking

developer controls scheduling

no need to have mutexes, no shared data

coroutines / can be implemented with continuations





















EM.run do page = EM::HttpRequest.new(@url1).get page.errback do puts "Site is down! terminate?" end page.callback do about = EM::HttpRequest.new(@url2).get about.callback do # callback nesting, ad infinitum end about.errback do # error-handling code end endend

page = http_get(@url1) puts "Fetched page: #{page.response_header.status}" if page page = http_get(@url2) puts "Fetched page 2: #{page.response_header.status}" end

def http_get(url) f = Fiber.current http = EM::HttpRequest.new(url).get http.callback{ f.resume(http) } http.errback { f.resume(http) } return Fiber.yieldend

EM.run do Fiber.new do page = http_get(@url1) puts "Fetched page: #{page.response_header.status}" if page page = http_get(@url2) puts "Fetched page 2: #{page.response_header.status}" end do.resumeend













EM-Synchrony

EM.synchrony do page = EM::HttpRequest.new("http://www.google.com").get puts "Look Ma! No callbacks! Fetched page: #{page}" EM.stopend

# old wayEM.run do page = EM::HttpRequest.new("http://www.google.com").get page.callback do puts "Lame ... Fetched page: #{page}" EM.stop endend

Goliath

require 'goliath'require 'yajl'

G = Goliath::Rack # don’t to this, just to fit in this slide :-)class Echo < Goliath::API use G::Render, 'json' # auto-negotiate response format use G::Params # parse & merge query and body parameters use G::Validation::RequiredParam, {:key => 'echo'}

def process_request logger.info "Processing request" {response: env.params['echo']} end

def response(env) [200, {}, process_request] endend

$ ruby echo.rb -p 9001


$ curl http://localhost:9001/{"error":"Echo identifier missing"}


$ curl http://localhost:9001/{"error":"Echo identifier missing"}

$ curl http://localhost:9001?echo=Hello%20World{"response":"Hello World"}

Web App Server and App Framework

Fully asynchronous using Eventmachine

Lightweight and High Performance (+ 3k req/s in 1 process)

Rack aware (but not 100% Rack compatible)

Fibers with EM-Synchrony for easier development

Recap so far ...

Ruby can using Reactors to massively scale

Good contender to Node.js-style (getting better)

One single Ruby process can handle thousands ofconcurrent requests

Web browsers got smarter with Websockets

Ruby implements Websocket support

Pusher

Rails App

Browser

Resque Queues

Persistentconnection!

async

Eventmachine App

sync

Rails App

Browser

Resque Queues

Persistentconnection!

async

Eventmachine App

sync

<script src="http://ajax.googleapis.com/ajax/libs/jquery/1.7.1/jquery.min.js"></script><script src="http://js.pusher.com/1.11/pusher.min.js"></script><script> var pusher = new Pusher('7114e...c318e'); var channel = pusher.subscribe('demo-channel');

</script>

<script src="http://ajax.googleapis.com/ajax/libs/jquery/1.7.1/jquery.min.js"></script><script src="http://js.pusher.com/1.11/pusher.min.js"></script><script> var pusher = new Pusher('7114e...c318e'); var channel = pusher.subscribe('demo-channel');

channel.bind('create', function(message) { var elem = $("#" + message.elem_id); if (elem) elem.remove(); var html = "<li id='" + message.elem_id + "'>" + message.elem_id + " - " + message.value + "</li>"; $("#list").append(html); });

channel.bind('delete', function(message) { var elem = $("#" + message.elem_id); if (elem) elem.remove(); });</script>

require 'rubygems'require 'pusher'

Pusher.app_id = '14909'Pusher.key = '7114e...c318e'Pusher.secret = '25aa7...3d49c'

('a'..'z').each do |letter| doc = { :elem_id => letter, :value => "Letter: #{letter}"} Pusher['demo-channel'].trigger('create', doc)end

time ruby pusher_sync.rb

real 0m14.585suser 0m0.583ssys 0m0.105s

require 'rubygems'require 'pusher'require 'eventmachine'

Pusher.app_id = '14909'Pusher.key = '7114e...c318e'Pusher.secret = '25aa7...d49c'

EM.run do EM::Iterator.new("a".."z").each do |letter, iter| doc = { :elem_id => elem_id, :value => "Letter: #{letter}" } pusher = Pusher['demo-channel'].trigger_async('create', doc) pusher.callback { EM.stop if letter == 'z' } pusher.errback { |error| EM.stop if letter == 'z' } iter.next endend







time ruby pusher_async.rb

real 0m1.129suser 0m0.649ssys 0m0.063s

pusher.connection.bind('connected', function() { alert("You're up!")});

pusher.connection.bind('connecting_in', function(delay) { alert("Retrying in " + delay + " seconds.")});

pusher.connection.bind('failed', function() { document.write("Not able to connect.")});

Private Channels

Encryption

Authentication

Presence Events

Total API requests (11/02/2011)

(day has 86.400 seconds)

13.969.264Total API requests (11/02/2011)

(day has 86.400 seconds)

Amount of messages sentto clients since launch (11/02/2011)

35.552.810.379Amount of messages sent

to clients since launch (11/02/2011)

average latency (excluding internet - 11/02/2011)

< 10msaverage latency (excluding internet - 11/02/2011)

Wrapping up ...

CGI model is difficult to scale

Multi-processes vs multi-threads vs Reactors

HTTP 5 WebSockets is hot!

Eventmachine is great!

Fibers vs “callback spaghetti”

Try out Pusher!






Try out Pusher!






Try out Pusher!






Try out Pusher!






Try out Pusher!






Try out Pusher!

Ilya Grigorik(igvita)

PostRank (acquired by Google!)

EM-Synchrony, Goliath

Google SPDY research

Tony Arcieri(bascule)

Rev (later Cool.io), Revactor

Reia(Ruby syntax over Erlang, replaced by Elixir)

Celluloid (Threads abstraction to Actors)(Mark Perham used to create Sidekiq)

Ruby is very flexible

one more thing ...

Passenger, Unicorn

NginX, HAProxy


JRuby - TorqueBox, Trinidad(multiple concurrent native threads)

NginX, HAProxy


Ruby 1.9.xNative threads

(extensions can release the GIL)

JRubyNative threads

(no GIL)

RubiniusNative threads

(no GILcheck out the Puma webserver)



JRubyNative threads

(no GIL)





JRubyNative threads

(no GIL)



www.codeminer.com.brwww.akitaonrails.com@akitaonrails

Большое спасибо

Technology

Understanding the Rails web model and scalability options