PyConIT6 - MAKING SESSIONS AND CACHING ROOMMATES

Embed Size (px)

Text of PyConIT6 - MAKING SESSIONS AND CACHING ROOMMATES

  • BEAKER - MAKING SESSIONS AND CACHING ROOMMATES

    Alessandro Molina@__amol__

    amol@turbogears.org

  • Who am I

    CTO @ AXANT.it, mostly Python company

    (with some iOS and Android)

    TurboGears2 dev team member

    Took over Beaker maintenace in 2015

    Mostly contributed to web python libraries: Formencode, MING MongoDB ODM, ToscaWidgets2

  • Whats Beaker Framework for handling Caching and

    Sessions in web applications

    Used by many different frameworks:

    TurboGears, Bottle, Pyramid, etc...

    Created by Pylons Author to solve the

    dogpile effect.

  • DogPile!

  • The Good Handles the DogPile effect

    Provides many backends: Memcache, File,

    MongoDB, Redis, SQLAlchemy

    A single tool to handle both caching and

    sessions with common backends.

  • The Bad File based synchronization, what about

    distributed solutions?

    Session as a big BLOB of data doesnt

    perform well in case of big BLOBs of data

    Updated for too long without breaking

    backward compatibility: Too many APIs

  • Getting on project: PY3 Support Beaker supported Python3 using 2to3

    This lead to some bugs (not everything got

    properly converted)

    Made really hard to run the test suite and

    maintain the project.

  • So my first action as new maintainer...

  • Rewriting for Py3 a project that involves serializing/deserializing

  • Porting to PY3: #1 Snakes speak ?Beaker uses BASE64 to encode pickled data

    in sessions.

    >>> import base64, pickle

    >>> data = dict(key='value', otherkey='othervalue')

    >>> base64.b64encode(pickle.dumps(data))

    'KGRwMApTJ290aGVya2V5JwpwMQpTJ290aGVydmFsdWUnCnAyCnN

    TJ2tleScKcDMKUyd2YWx1ZScKcDQKcy4='

  • BASE64: rfc4648Here are a few requirements that

    determine which alphabet should be

    used:

    o Handled by humans.

  • Human handle?

    TEXT!

  • Human handle?

    TEXT!

  • Human handle?

    BYTES!>>> import base64, pickle

    >>> base64.b64encode(pickle.dumps(HELLO))

    b'gANYBQAAAEhFTExPcQAu'

  • Porting to PY3: #2 Session Cookies? Session ID is stored in cookies

    CookieSession stores even the whole

    session in a cookie Great idea, makes really simple to scale

    HTTP Headers are plain text

    We are pickling data and base64 so already ASCII

    Just need to encrypt it to avoid people from

    messing with them.

  • HTTP CookiesHTTP has allowed field content with text

    in the ISO-8859-1 charset [ISO-8859-1],

    supporting other charsets only through

    use of [RFC2047] encoding. In practice,

    most HTTP header field values use only a

    subset of the US-ASCII charset [USASCII].

    Newly defined header fields SHOULD limit

    their field values to US-ASCII octets

  • Encoding/Encripting data on Py3 Beaker expected to be able to perform

    text operations on the result of base64

    Relied on AES to encrypt cookies, pbkdf2

    to generates keys and base64/binascii

    to make it text.

    Reads the encrypt_key from config file

  • 4 lines of code == 10 doubts Even Python has no clear idea of what is

    text and what are bytes: Both BASE64 and BINASCII work with ASCII

    BINASCII accepts unicode (text)

    BASE64 accepts bytes

    Between Python3.2 and Python3.3 the binascii

    changed behaviour of accepting unicode vs bytes

    ConfigParser returns text on py3, bytes on py2

  • Now crying in a corner...

  • Lets go for something simpler... Review patch to caching decorator

    To cache a function, just apply a decorator

    that calls function and caches the result

    Caching decorator generates cache key Cache Key from the function name

    convert parameters to strings

    add parameters to cache key

  • Getting the Cache Key def cache(self, *args, **kwargs):

    """Decorate a function to cache itself with supplied parameters

    :param args: Used to make the key unique for this function, as in region()

    above.

    :param kwargs: Parameters to be passed to get_cache(), will override defaults

    Example::

    # Assuming a cache object is available like:

    cache = CacheManager(dict_of_config_options)

    def populate_things():

    @cache.cache('mycache', expire=15)

    def load(search_term, limit, offset):

    return load_the_data(search_term, limit, offset)

    return load('rabbits', 20, 0)

    """

    return _cache_decorate(args, self, kwargs, None)

  • Well, maybe...

  • Unknown parameters Decorator gets *args and **kwargs

    Makes sense... doesnt know the real

    function arguments

    Not so much in fact func(1) and func

    (a=1) end up with two different cache

    keys

  • Avoid the doubt Beaker solved this by accepting only

    positional arguments on cached functions .. note::

    The function being decorated must only be called with

    positional arguments.

    Users not so happy, breaks people code

    when they introduce caching: https://github.com/bbangert/beaker/issues/62

  • What to do? Leave it as is As been like that for years

    Add **kwargs and just document the issue

    Add **kwargs and try to be smart using

    inspect.getargspec / inspect.

    getcallargs but that would be slow

  • Leave it as is at least isnt broken

  • Has been a great trip! Making sessions and caching in a single

    solution seemed simple, but has actually a

    lot of corner case

    Beaker Co-Author thrown in the towel and

    created dopgile.cache which is based on

    beaker code but is much simpler and

    doesnt provide sessions

  • Still is incredibily convenient There are more shared features than

    differences (especially in cookie based

    marshalled sessions).

    Its really convenient, write backends once

    and have support for both sessions and

    caching on them.

  • Questions?