Upload
trinhkhue
View
268
Download
0
Embed Size (px)
Citation preview
No Callbacks, No Threads & Ruby 1.9 @igrigorik h;p://bit.ly/ruby-‐stack
No Callbacks, No Threads & Ruby 1.9
Ilya Grigorik @igrigorik
async & co-‐opera-ve web-‐servers
No Callbacks, No Threads & Ruby 1.9 @igrigorik h;p://bit.ly/ruby-‐stack
The state of art is not good enough. (we’ve been stuck in the same local minima for several years)
No Callbacks, No Threads & Ruby 1.9 @igrigorik h;p://bit.ly/ruby-‐stack
require "active_record”
ActiveRecord::Base.establish_connection( :adapter => "mysql", :username => "root", :database => "database", :pool => 5)
threads = []10.times do |n| threads << Thread.new { ActiveRecord::Base.connection_pool.with_connection do |conn| res = conn.execute("select sleep(1)") end }end
threads.each { |t| t.join }
The “Experiment” vanilla everything…
No Callbacks, No Threads & Ruby 1.9 @igrigorik h;p://bit.ly/ruby-‐stack
require "active_record”
ActiveRecord::Base.establish_connection( :adapter => "mysql", :username => "root", :database => "database", :pool => 5)
threads = []10.times do |n| threads << Thread.new { ActiveRecord::Base.connection_pool.with_connection do |conn| res = conn.execute("select sleep(1)") end }end
threads.each { |t| t.join }
# time ruby activerecord-pool.rb## real 0m10.663s# user 0m0.405s# sys 0m0.201s
5 shared connecJons
No Callbacks, No Threads & Ruby 1.9 @igrigorik h;p://bit.ly/ruby-‐stack
Drivers
Ruby VM
Network
MySQL Mongo
PSQL Couch …
Mongrel Passenger
Unicorn
…
Threads
Fibers GIL …
No Callbacks, No Threads & Ruby 1.9 @igrigorik h;p://bit.ly/ruby-‐stack
Drivers
Ruby VM
Network
MySQL Mongo
PSQL Couch …
Mongrel Passenger
Unicorn
…
Threads
Fibers GIL …
We’re as fast as the slowest component
1
2
3
4
No Callbacks, No Threads & Ruby 1.9 @igrigorik h;p://bit.ly/ruby-‐stack
Concurrency is a myth in Ruby (with a few caveats, of course)
Global Interpreter Lock is a mutual exclusion lock held by a programming language interpreter thread to avoid sharing code that is not thread-‐safe with other threads.
There is always one GIL for one interpreter process.
hMp://bit.ly/ruby-‐gil
No Callbacks, No Threads & Ruby 1.9 @igrigorik h;p://bit.ly/ruby-‐stack
Concurrency is a myth in Ruby s-ll no concurrency in Ruby 1.9
N-‐M thread pool in Ruby 1.9… BeMer but s-ll the same problem!
hMp://bit.ly/ruby-‐gil
No Callbacks, No Threads & Ruby 1.9 @igrigorik h;p://bit.ly/ruby-‐stack
Concurrency is a myth in Ruby s-ll no concurrency in Ruby 1.9
Nick – tomorrow @ 11:45am
hMp://bit.ly/ruby-‐gil
No Callbacks, No Threads & Ruby 1.9 @igrigorik h;p://bit.ly/ruby-‐stack
Avoid locking interpreter threads at all costs let’s say you’re wri-ng an extension…
Blocks enJre Ruby VM
Not as bad, but avoid it sJll..
No Callbacks, No Threads & Ruby 1.9 @igrigorik h;p://bit.ly/ruby-‐stack
Will fetch_xyz() block the VM? when was the last -me you asked yourself this ques-on?
No Callbacks, No Threads & Ruby 1.9 @igrigorik h;p://bit.ly/ruby-‐stack
mysql.gem under the hood
require 'rubygems’require 'sequel'
DB = Sequel.connect('mysql://root@localhost/test')
while true DB['select sleep(1)'].select.first end
22:10:00.218438 mysql_real_query(0x02740000, "select sleep(1)", 15) = 0 <1.001100> 22:10:01.241679 mysql_real_query(0x02740000, "select sleep(1)", 15) = 0 <1.000812>
ltrace –MTg -‐x mysql_real_query –p ./example.rb
Blocking 1s call!
hMp://bit.ly/c3Pt3f
No Callbacks, No Threads & Ruby 1.9 @igrigorik h;p://bit.ly/ruby-‐stack
gem install mysql what you didn’t know…
1. Blocking calls to mysql_real_query 2. mysql_real_query requires an OS thread 3. Blocking on mysql_real_query blocks the Ruby VM 4. Aka, “select sleep(1)” blocks the enJre Ruby runJme for 1s
(ouch)
No Callbacks, No Threads & Ruby 1.9 @igrigorik h;p://bit.ly/ruby-‐stack
mysqlplus.gem under the hood
static VALUE async_query(int argc, VALUE* argv, VALUE obj) { ... send_query( obj, sql ); ... schedule_query( obj, timeout); ... return get_result(obj); }
static void schedule_query(VALUE obj, VALUE timeout) { ... struct timeval tv = { tv_sec: timeout, tv_usec: 0 };
for(;;){ FD_ZERO(&read); FD_SET(m->net.fd, &read);
ret = rb_thread_select(m->net.fd + 1, &read, NULL, NULL, &tv); ...
if (m->status == MYSQL_STATUS_READY) break; }}
send query and block
Ruby: select() = C: rb_thread_select()
No Callbacks, No Threads & Ruby 1.9 @igrigorik h;p://bit.ly/ruby-‐stack
mysqlplus.gem + ruby select
spinning in select
No Callbacks, No Threads & Ruby 1.9 @igrigorik h;p://bit.ly/ruby-‐stack
Step 1: Fix the drivers Many of our DB drivers don’t respect the underlying Ruby VM. Don’t blame the VM.
No Callbacks, No Threads & Ruby 1.9 @igrigorik h;p://bit.ly/ruby-‐stack
WEBRick
Drivers
Ruby VM
Network
Mongrel made Rails viable s-ll powers a lot of Rails apps today
No Callbacks, No Threads & Ruby 1.9 @igrigorik h;p://bit.ly/ruby-‐stack
Rails rediscovers Apache the worker/forker model…
No Callbacks, No Threads & Ruby 1.9 @igrigorik h;p://bit.ly/ruby-‐stack
An exclusive Ruby VM for EACH request am I the only one who thinks this is terrible?
…
*nix IPC is fast! Woo!
Full Ruby VM
No Callbacks, No Threads & Ruby 1.9 @igrigorik h;p://bit.ly/ruby-‐stack
An exclusive Ruby VM for EACH request am I the only one who thinks this is terrible?
Robustness? That sounds like a bug.
“Does not care if your applica-on is thread-‐safe or not, workers all run within their own isolated address space and only serve one client at a -me for maximum robustness.”
No Callbacks, No Threads & Ruby 1.9 @igrigorik h;p://bit.ly/ruby-‐stack
Step 2: consider enJre stack The driver, the web-‐server, and the network must all work together.
No Callbacks, No Threads & Ruby 1.9 @igrigorik h;p://bit.ly/ruby-‐stack
1. Node imposes the full-‐stack requirements 2. Node imposes async drivers 3. Node imposes async frameworks
Surprise: Node is “fast”
No Callbacks, No Threads & Ruby 1.9 @igrigorik h;p://bit.ly/ruby-‐stack
We can ignore the performance
issues at our own peril or, we can just fix the problem
No Callbacks, No Threads & Ruby 1.9 @igrigorik h;p://bit.ly/ruby-‐stack
I’ll take Ruby over JS gem install eventmachine
>
No Callbacks, No Threads & Ruby 1.9 @igrigorik h;p://bit.ly/ruby-‐stack
EventMachine Reactor concurrency without thread
EventMachine: The Speed Demon Wednesday @ 11:45am – Aman Gupta
while true do Jmers
network_io other_io
end
p "Starting"
EM.run do p "Running in EM reactor" end
p ”won’t get here"
No Callbacks, No Threads & Ruby 1.9 @igrigorik h;p://bit.ly/ruby-‐stack
Non-‐blocking IO requires non-‐blocking drivers:
AMQP h+p://github.com/tmm1/amqp MySQLPlus h;p://github.com/igrigorik/em-‐mysqlplus Memcached h+p://github.com/astro/remcached DNS h+p://github.com/astro/em-‐dns Redis h+p://github.com/madsimian/em-‐redis MongoDB h+p://github.com/tmm1/rmongo HTTPRequest h+p://github.com/igrigorik/em-‐h+p-‐request WebSocket h+p://github.com/igrigorik/em-‐websocket Amazon S3 h+p://github.com/peritor/happening
And many others: h+p://wiki.github.com/eventmachine/eventmachine/protocol-‐implementaJons
No Callbacks, No Threads & Ruby 1.9 @igrigorik h;p://bit.ly/ruby-‐stack
em-‐mysqlplus: example async MySQL driver
EventMachine.run do conn = EventMachine::MySQL.new(:host => 'localhost') query = conn.query("select sleep(1)")
query.callback { |res| p res.all_hashes } query.errback { |res| p res.all_hashes }
puts ”executing…”end
# > ruby em-mysql-test.rb## executing…# [{"sleep(1)"=>"0"}]
gem install em-‐mysqlplus
callback fired 1s aner “execuJng”
No Callbacks, No Threads & Ruby 1.9 @igrigorik h;p://bit.ly/ruby-‐stack
em-‐mysqlplus: under the hood mysqlplus + reactor loop
non-‐blocking driver require 'mysqlplus'
def connect(opts) conn = connect_socket(opts) EM.watch(conn.socket, EventMachine::MySQLConnection, conn, opts, self)end
def connect_socket(opts) conn = Mysql.init
conn.real_connect(host, user, pass, db, port, socket, ...) conn.reconnect = false connend
EM.watch: reactor will poll & noJfy
No Callbacks, No Threads & Ruby 1.9 @igrigorik h;p://bit.ly/ruby-‐stack
em-‐mysqlplus mysqlplus + reactor loop
Features:
-‐ Maintains C-‐based mysql gem API -‐ Deferrables for every query with callback & errback -‐ ConnecJon query queue -‐ pile 'em up! -‐ Auto-‐reconnect on disconnects -‐ Auto-‐retry on deadlocks
h+p://github.com/igrigorik/em-‐mysqlplus
No Callbacks, No Threads & Ruby 1.9 @igrigorik h;p://bit.ly/ruby-‐stack
We can do be;er than node.js all the benefits of evented code without the drawbacks
No Callbacks, No Threads & Ruby 1.9 @igrigorik h;p://bit.ly/ruby-‐stack
Ruby 1.9 Fibers and coopera-ve scheduling
Ruby 1.9 Fibers are a means of crea-ng code blocks which can be paused and resumed by our applica-on (think lightweight threads, minus the thread scheduler and less overhead).
f = Fiber.new { while true do Fiber.yield "Hi” end}
p f.resume # => Hip f.resume # => Hip f.resume # => Hi
Manual / cooperaJve scheduling!
h+p://bit.ly/d2hYw0
No Callbacks, No Threads & Ruby 1.9 @igrigorik h;p://bit.ly/ruby-‐stack
Ruby 1.9 Fibers and coopera-ve scheduling
h+p://bit.ly/aesXy5
Fibers vs Threads: creaJon Jme much lower
Fibers vs Threads: memory usage is much lower
No Callbacks, No Threads & Ruby 1.9 @igrigorik h;p://bit.ly/ruby-‐stack
Untangling Evented Code with Fibers h+p://bit.ly/d2hYw0
def query(sql) f = Fiber.current conn = EventMachine::MySQL.new(:host => 'localhost')
q = conn.query(sql)
c.callback { f.resume(conn) } c.errback { f.resume(conn) }
return Fiber.yieldend
EventMachine.run do Fiber.new {
res = query('select sleep(1)') puts "Results: #{res.fetch_row.first}"
}.resumeend
ExcepJon, async!
No Callbacks, No Threads & Ruby 1.9 @igrigorik h;p://bit.ly/ruby-‐stack
Untangling Evented Code with Fibers h+p://bit.ly/d2hYw0
def query(sql) f = Fiber.current conn = EventMachine::MySQL.new(:host => 'localhost')
q = conn.query(sql)
c.callback { f.resume(conn) } c.errback { f.resume(conn) }
return Fiber.yieldend
EventMachine.run do Fiber.new {
res = query('select sleep(1)') puts "Results: #{res.fetch_row.first}"
}.resumeend
1. Wrap into a conJnuaJon
No Callbacks, No Threads & Ruby 1.9 @igrigorik h;p://bit.ly/ruby-‐stack
Untangling Evented Code with Fibers h+p://bit.ly/d2hYw0
def query(sql) f = Fiber.current conn = EventMachine::MySQL.new(:host => 'localhost')
q = conn.query(sql)
c.callback { f.resume(conn) } c.errback { f.resume(conn) }
return Fiber.yieldend
EventMachine.run do Fiber.new {
res = query('select sleep(1)') puts "Results: #{res.fetch_row.first}"
}.resumeend
2. Pause the conJnuaJon
No Callbacks, No Threads & Ruby 1.9 @igrigorik h;p://bit.ly/ruby-‐stack
Untangling Evented Code with Fibers h+p://bit.ly/d2hYw0
def query(sql) f = Fiber.current conn = EventMachine::MySQL.new(:host => 'localhost')
q = conn.query(sql)
c.callback { f.resume(conn) } c.errback { f.resume(conn) }
return Fiber.yieldend
EventMachine.run do Fiber.new {
res = query('select sleep(1)') puts "Results: #{res.fetch_row.first}"
}.resumeend
3. Resume the conJnuaJon
No Callbacks, No Threads & Ruby 1.9 @igrigorik h;p://bit.ly/ruby-‐stack
em-‐synchrony: simple evented programming best of both worlds…
Good news, you don’t even have to muck around with Fibers!
gem install em-‐synchrony
-‐ Fiber aware connecJon pool with sync/async query support -‐ MulJ request interface which accepts any callback enabled client -‐ Fibered iterator to allow concurrency control & mixing of sync / async -‐ em-‐h+p-‐request: .get, etc are synchronous, while .aget, etc are async -‐ em-‐mysqlplus: .query is synchronous, while .aquery is async -‐ remcached: .get, etc, and .mulJ_* methods are synchronous
h+p://github.com/igrigorik/em-‐synchrony
No Callbacks, No Threads & Ruby 1.9 @igrigorik h;p://bit.ly/ruby-‐stack
Untangling Evented Code with Fibers h+p://bit.ly/d2hYw0
Async under the hood
require "em-synchrony/em-mysqlplus"
EventMachine.synchrony do db = EventMachine::MySQL.new(host: "localhost")
res = db.query("select sleep(1)") puts res
EventMachine.stopend
No Callbacks, No Threads & Ruby 1.9 @igrigorik h;p://bit.ly/ruby-‐stack
Untangling Evented Code with Fibers h+p://bit.ly/d2hYw0
require "em-synchrony/em-mysqlplus"
EventMachine.synchrony do db = EventMachine::MySQL.new(host: "localhost")
res = db.query("select sleep(1)") puts res
EventMachine.stopend
No Callbacks, No Threads & Ruby 1.9 @igrigorik h;p://bit.ly/ruby-‐stack
Drivers
Ruby VM
Network
Fibers
EM-‐HTTP, EM-‐MySQL, EM-‐Jack, etc.
Async-‐rack
Thin
One VM, full concurrency, network-‐bound Ruby 1.9, Fibers, Thin: in producJon!
Goliath
No Callbacks, No Threads & Ruby 1.9 @igrigorik h;p://bit.ly/ruby-‐stack
Async Rails with EventMachine & MySQL
development: adapter: em_mysqlplus database: widgets pool: 5 timeout: 5000
git clone git://github.com/igrigorik/em-‐mysqlplus.git git checkout ac-verecord rake install
database.yml
require 'em-activerecord’require 'rack/fiber_pool'
# Run each request in a Fiberconfig.middleware.use Rack::FiberPoolconfig.threadsafe!
environment.rb
No Callbacks, No Threads & Ruby 1.9 @igrigorik h;p://bit.ly/ruby-‐stack
Async Rails with EventMachine & MySQL
class WidgetsController < ApplicationController def index Widget.find_by_sql("select sleep(1)") render :text => "Oh hai” endend
ab –c 5 –n 10 hMp://127.0.0.1:3000/widgets
Server Soiware: thin Server Hostname: 127.0.0.1 Server Port: 3000
Document Path: /widgets/ Document Length: 6 bytes
Concurrency Level: 5 Time taken for tests: 2.210 seconds Complete requests: 10 Failed requests: 0 Requests per second: 4.53 [#/sec] (mean)
woot! Fiber DB pool at work.
No Callbacks, No Threads & Ruby 1.9 @igrigorik h;p://bit.ly/ruby-‐stack
Not only is it doable… it already works.
git clone git://…./igrigorik/mysqlplus git checkout ac-verecord rake install
No Callbacks, No Threads & Ruby 1.9 @igrigorik h;p://bit.ly/ruby-‐stack
Ruby 1.9 + Rails 3 + new stack
=
Order of magnitude be;er performance
(aka, enough of a reason to actually switch)
No Callbacks, No Threads & Ruby 1.9 @igrigorik h;p://bit.ly/ruby-‐stack
What do you think?
The state of art is not good enough, in fact, it’s terrible!
hMp://bit.ly/aHzdZw
Untangling Evented Code with Ruby Fibers: h+p://www.igvita.com/2010/03/22/untangling-‐evented-‐code-‐with-‐ruby-‐fibers/
Fibers & CooperaJve Scheduling in Ruby: h+p://www.igvita.com/2009/05/13/fibers-‐cooperaJve-‐scheduling-‐in-‐ruby/
EM-‐Synchrony: h+p://github.com/igrigorik/em-‐synchrony