Coroutines @ PyCon 2011 · Concurrency Models: Coroutines 3 Cooperative switching, no locks 3 Works for both C and Python stacks 3 No limits on call depths 7 Switching can make it

Coroutines @ PyCon 2011

Martin Blais

[email protected]

April 2011

Coroutine Buzzwords

geventeventlet

concurrencediesel

monoclegunicorn

tornadouwsgi

Talks about Coroutines

• Using Coroutines to Create Efficient, High-ConcurrencyWeb Applications – Matt Spitz

• An outsider’s look at co-routines – Peter Portante• Large Scale Data Conditioning & Processing with

Stackless Python and Pypes – Eric Gaumer• Prototyping Go’s Select with stackless.py for Stackless

Python – Andrew Francis

Also related:Using Python 3 to Build a Cloud Computing Service for mySuperboard II – David Beazley

Concurrency Models: CPU Registers

Concurrency Models: Stack/Execution





Concurrency Models: Processes






3 Linear function context3 Memory isolation3 Process management is convenient, many tools7 More difficult to share objects between processes

Concurrency Models: Threads




3 Linear function context7 You need to protect memory access to shared objects7 The number of threads of limited7 No thread management (e.g. you cannot kill a thread)7 Latency response depends on scheduler(up to 1ms on untuned kernel)7 (In Python) The interpreter code is not thread-safeGlobal Interpreter Lock (GIL) protects all C PythonfunctionsÑ No real concurrency!

Concurrency Models: Active Objects



3 No need for locking (if you don’t share objects)3 Limited thread management can be implemented asmessages7 Function invocations have to be wrapped in messages7 Contention on queues is possible

Concurrency Models: Reactor/Non-Blocking I/O





Uses non-blocking I/O + select()/epoll()

3 Very cheap, can create many tasks3 Only one thread, no locks!7 You have to break down long tasks in many functions

(No more linear function context)7 Cooperative: one task could hog all the execution7 select()/epoll() may typically cause context switch

e.g. Python Twisted, asyncore, Java SelectMultiplexer

Concurrency Models: Proactor/Asynchronous I/O






3 I/O can process concurrently with code3 Threading is controlled, no locks!7 You have to manage buffers7 Linux does not support this model well

(Problems with aio_*, see U.Drepper paper.)(Windows has “IO completion ports”)

e.g. C ACE, Python pyasynchio (inactive, win-only)

Non-Blocking I/O � Asynchronous I/O

Generators

Generators

Generators

Generators

Generators

Generators

3 Cooperative switching, no locks3 Linear function context7 Does not work for C functions, Python only7 Switching is limited to a single depth of call

This is very limited, not sophisticated enoughfor a general concurrency abstraction

e.g. Python monocle

Concurrency Models: Coroutines










3 Cooperative switching, no locks3 Works for both C and Python stacks3 No limits on call depths7 Switching can make it difficult to follow execution flow7 Potentially expensive stack manipulations7 Does not take advantage of multiple cores(e.g. gunicorn uses a combination of workers andcoroutines)

e.g. Python greenlet, Stackless, C Boost.Coroutine,libtask, . . .

Coroutine Switching can be Confusing

Any greenlet can “switch” to any other greenletThis could get nasty!

Interruptibility / Multi-Core

Greenlet-based Libraries

gevent eventlet concurrence

These libraries provide a “supervisor” / ”hub” which overseesswitches at points of I/O, driven by a multiplexer/reactor(usually a libevent wrapper)

• Task/greenlet switching is implicit• Hide the explicit coroutine switching• Provides alternatives to the standard I/O modules, or• Monkey-patches the standard modules

Simple Code Example: eventlet Echo Server

import eventlet

def handle(fd):while True:

x = fd.readline()if not x:

breakfd.write(x)fd.flush()

server = eventlet.listen((’0.0.0.0’, 6000))pool = eventlet.GreenPool()while True:

new_sock, address = server.accept()pool.spawn_n(handle, new_sock.makefile(’rw’))

Simple Code Example: gevent Downloader

import geventfrom gevent import monkey; monkey.patch_all()import urllib2

urls = [’http://www.google.com’,’http://www.python.org’,...]

def print_head(url):data = urllib2.urlopen(url).read()print ’%s: %s bytes: %r’ % (url, len(data), data[:50])

jobs = [gevent.spawn(print_head, url) for url in urls]

Comments

• Greenlet is just a C extension module(relevant bits were originally pulled from Stackless)

• Check out D. Beazley’s excellent course:“A Curious Course on Coroutines and Concurrency”http://www.dabeaz.com/coroutines/

• More advanced/general: “Continuations”

Fina

Questions

Documents

Coroutines @ PyCon 2011 · Concurrency Models: Coroutines 3 Cooperative switching, no locks 3 Works for both C and Python stacks 3 No limits on call depths 7 Switching can make it