Meta-Classes in Python

Dynamic OOP and Meta-Classes in Python

Guy Wiener

Dept. of Applied Math and Computer ScienceWeizmann Institute for Science

24/5/2011

Guy Wiener (WIS) Dynamic OOP and Meta-Classes in Python 24/5/2011 1 / 57

Introduction

Outline

1 Introduction

2 OOP Hacking

3 Meta-Classes

4 Examples

5 Summary

Introduction

What is OOP?

The OOP Manifest (abridged)Alan Kay, 1993

Everything is an object

Computation is performed by objects communicating with eachother by sending and receiving messages

Each object has its own memory (state), which consists of otherobjects.

Every object is an instance of a class.

The class is the repository for behavior. All instances of thesame class can perform the same actions.

Classes are organized into a singly rooted tree, called theinheritance hierarchy.

Introduction

What is an object?

An object, according to the OOP manifest:State Memory

Behavior Receiving and sending messages

Identity Each object is unique

Introduction

Objects Can Change!

Changing objectsState Read, write, add and remove field

Behavior Create, add and replace methods

Identity Remains the same

Introduction

Static Objects and Classes

Objects and Classes

Field descriptors

Method desc. and code

Hierarchy information

Fieldscontent

ObjectRun-timetype infor-mation

Example

x: int

y: String

foo() {. . . }

goo() {. . . }

“hello”

Instance of A

Introduction

Dynamic Objects and Classes

Objects and Classes

A dictionaryof methodsand staticfields

A dictionary offields and boundmethods

Object

Classpointer

Example

“foo” ⇒ {. . . }

“goo” ⇒ {. . . }

“x” ⇒ 5

“y” ⇒ “hello”

Instance of A

OOP Hacking

Outline

1 Introduction

2 OOP Hacking

3 Meta-Classes

4 Examples

5 Summary

OOP Hacking

Everything is a Dictionary

Python’s modules, classes and objects = Dictionaries.

The dictionary holds:Classes Methods and class variables

Objects Bound methods and fields

The “.” operator = Access the inner dictionary

The dir command = Lists keys of the inner dictionary

OOP Hacking

Manipulating the Inner Dictionary

The inner dictionary is accessible and manipulable by string keys!

Inner dictionary manipulationgetattr(o, ’a’) Get the attribute a from the dictionary of o

setattr(o, ’a’, v) Set the attribute a of o to be v

obj.foo is same as getattr(obj, ’foo’)

obj.foo = val is same as setattr(obj, ’foo’, val)

OOP Hacking

Manipulating the Inner Dictionary (cont’d)

Example>>> a = A()

>>> a.x = 5

>>> dir(a)

[..., ’x’]

>>> getattr(a, ’x’)

>>> setattr(a, ’x’, 6)

>>> a.x

OOP Hacking

Adding Fields

Adding fields to an objectclass A: pass

>>> a = A()

>>> a.x = 5

>>> a.x

>>> setattr(a, ’y’, 6)

>>> a.y

Gain & LossGained Flexible object content

Lost Type safety

OOP Hacking

Adding Fields

Adding fields to an objectclass A: pass

>>> a = A()

>>> a.x = 5

>>> a.x

>>> setattr(a, ’y’, 6)

>>> a.y

Gain & LossGained Flexible object content

Lost Type safety

OOP Hacking

Replacing Methods

Swapping methodsclass A:

def f(self):

print "f"

def g(self):

print "g"

>>> a = A()

>>> a.f, a.g = a.g, a.f

>>> a.f()

>>> a.g()

Gain & LossGained Change an object

behavior at runtime

Lost Control flow analysis

OOP Hacking

Replacing Methods

Swapping methodsclass A:

def f(self):

print "f"

def g(self):

print "g"

>>> a = A()

>>> a.f, a.g = a.g, a.f

>>> a.f()

>>> a.g()

Gain & LossGained Change an object

behavior at runtime

Lost Control flow analysis

OOP Hacking

Replacing Methods for a Class

Swapping class methodsclass A:

def f(self):

print "f"

def g(self):

print "g"

>>> a = A()

>>> A.f, A.g = A.g, A.f

>>> a.f()

>>> a.g()

Gain & LossGained Change a class at

runtime

Lost Sanity

OOP Hacking

Replacing Methods for a Class

Swapping class methodsclass A:

def f(self):

print "f"

def g(self):

print "g"

>>> a = A()

>>> A.f, A.g = A.g, A.f

>>> a.f()

>>> a.g()

Gain & LossGained Change a class at

runtime

Lost Sanity

OOP Hacking

Classes are Return Values Too

A function that returns a classdef PersonClass(salut):

class Person(object):

def __init__(self, name):

self.name = name

def greet(self):

print salut, self.name

return Person

OOP Hacking

Classes are Return Values Too (cont’d)

Making classesMan = PersonClass("Mr.")

Woman = PersonClass("Ms.")

Doctor = PersonClass("Dr.")

a = [Woman("Rose Tylar"),

Woman("Amy Pond"),

Man("Mickey Smith"),

Man("Rory Williams"),

Doctor("Who")]

Outputfor c in a: c.greet()

>>> Ms. Rose Tylar

Ms. Amy Pond

Mr. Mickey Smith

Mr. Rory Williams

Dr. Who

Meta-Classes

Outline

1 Introduction

2 OOP Hacking

3 Meta-Classes

4 Examples

5 Summary

Meta-Classes

What is Meta-Programming?

Meta-ProgramA program that:

One of its inputs is a program (possibly itself)

Its output is a program

Meta-Classes

What is Meta-OOP?

Meta-ClassA class that creates classes

Objects are instances of classes

Classes are objects

Classes are instances of meta-classes

Meta-Classes

Object Systems

3-levels

Object

A Class Type

super: Type[]

An Object

Legend

instance ofextends

4-levels

Object

super: Type[]

A MetaClass

A Class

An Object

Meta-Classes Meta-Classes in Python

Outline

1 Introduction

2 OOP Hacking

3 Meta-ClassesMeta-Classes in Python

4 ExamplesLoggingCountingDelegationAssociations

5 Summary

Meta-Classes in Python

The type classBase class for meta-classes

Creates new classes

Dynamic members become static members of the instance class

type. init (cls, name, bases, dict)

cls The class object itself (i.e., “self”)

name The name of the class

bases A list of the base classes

dict A dictionary of the methods and static fields of the class

Creating Classes from Meta-Classes

Declaring a new class programatically# Meta-Class

class Printable(type):

def whoami(self): print "I am ", self.__name__

>>> Foo = Printable(’Foo’,(),{}) # Empty new class

>>> Foo.whoami()

I am Foo

>>> Foo.__class__

>>> f = Foo() # Object

>>> f.__class__

The metaclass field

Class C is an instance of a meta-class M if:1 C has a static field metaclass

2 One of the ancestors classes of C is an instance of M

3 There is a global variable metaclass

4 Otherwise, the default meta-class type is used

metaclass Example

Declaring a meta-class with metaclassclass Bar:

__metaclass__ = Printable

def foo(self): print ’foo’

>>> Bar.whoami()

I am a Bar

>>> b = Bar()

>>> b.foo()

Examples

Outline

1 Introduction

2 OOP Hacking

3 Meta-Classes

4 Examples

5 Summary

Examples Logging

Outline

1 Introduction

2 OOP Hacking

5 Summary

Examples Logging

Logging with Meta-Classes

A logger decoratordef log(name, f):

def ret(*args):

print "enter", name

f(*args)

print "exit", name

return ret

Examples Logging

Logging with Meta-Classes (cont’d)

A logger meta-classclass Logged(type):

def init (cls, name, bases, dict):

type. init (cls, name, bases, dict)

p = re.compile(cls. logmatch )

for attr, item in dict.items():

if callable(item) and p.match(attr):

setattr(cls, attr, log(attr, item))

Examples Logging

Logging with Meta-Classes (cont’d)

A logged classclass Test:

metaclass = Logged

logmatch = ’.*’

def foo(self):

print ’foo’

>>> t = Test()

>>> t.foo()

enter foo

exit foo

Examples Counting

Outline

1 Introduction

2 OOP Hacking

5 Summary

Examples Counting

Counting Instances

Countable meta-classclass Countable(type):

def new_init(cls, init):

def ret(self, *args):

init(self, *args)

cls.count = cls.count + 1

return ret

def __init__(cls, name, bases, dct):

type.__init__(cls, name, bases, dct)

cls.count = 0

cls.__init__ = cls.new_init(cls.__init__)

Examples Counting

Counting Example

Some classesclass Person:

__metaclass__ = Countable

def __init__(self, name):

self.name = name

class Book:

__metaclass__ = Countable

def __init__(self, name, author):

self.name = name

self.author = author

Examples Counting

Counting Example (cont’d)

Testingrah = Person("Robert Anson Heinlin")

b1 = Book("Stranger in a Strange Land", rah)

b2 = Book("Starship Troopers", rah)

print Person.count, Book.count

>>> 1 2

Examples Delegation

Outline

1 Introduction

2 OOP Hacking

5 Summary

Examples Delegation

Automatic Delegation

Delegate

An object of class A thatdispatches all message ofclass B to a target field oftype B.

Writing a delegate class is a monotonous work

But it can be done automatically

Examples Delegation

Delegation Using Meta-Classes

A delegation function decoratordef dlgt(cls, method):

method(self.__tgt__, *args)

return instancemethod(ret, None, cls)

instancemethod takes a function, an object/None and a class and

returns a method of the class

Auxiliary – Print class namedef clsname(self):

return self.__class__.__name__

Examples Delegation

Delegation Using Meta-Classes (cont’d)

The Delegate meta-classclass Delegate(type):

def __init__(cls, name, bases, dict):

type.__init__(cls, name, bases, dict)

tgtclass = cls.__tgtclass__

for name in dir(tgtclass):

val = getattr(tgtclass, name)

if callable(val):

setattr(cls, name, dlgt(cls, val))

Examples Delegation

The delegated classclass A:

def bar(self):

print clsname(self), ’bar’

def baz(self):

print clsname(self), ’baz’

Examples Delegation

The delegating classclass B:

metaclass = Delegate

tgtclass = A

def init (self, tgt):

self. tgt = tgt

def boo(self):

print clsname(self), ’boo’

Examples Delegation

Delegation test>>> b = B(A())

>>> b.bar()

>>> b.baz()

>>> b.boo()

Examples Delegation

Simplified Syntax for with Meta-Classes with Args.Thanks to Yoav Goldberg

Use a function to create the meta-classdef delegate_of(tgtclass):

class Delegate(type):

def __init__(cls, name, bases, dict):

type.__init__(cls, name, bases, dict)

for name in dir(tgtclass):

val = getattr(tgtclass, name)

if callable(val):

setattr(cls, name, dlgt(cls,val))

Examples Delegation

Simplified Syntax for Meta-Classes with Args.Thanks to Yoav Goldberg

Use the function to generate a meta-classclass B:

metaclass = delegate_of(A)

def init (self, tgt):

self. tgt = tgt

def boo(self):

print clsname(self), ’boo’

Examples Associations

Outline

1 Introduction

2 OOP Hacking

5 Summary

Associations

GoalsAggregation operations, like

a.getMyB().foreach().g()r = b.getMyA().filter().f().map().h()

Optimizations: Multithreading, caching

No textual code generation

Basic Aggregation

Replace all public methods with an aggregator wrapperclass Aggr(type):

def __init__(cls, name, bases, dct):

p = re.compile(’__.*__’)

type.__init__(cls, name, bases, {})

for key, item in dct.iteritems():

if callable(item) and not p.match(key):

m = instancemethod(cls.wrap(item), None, self)

setattr(self, key, m)

def wrap(cls, func):

return self.__aggr__(func, *args)

return ret

Basic Aggregation (cont’d)

Make aggrs. that extend impl and wrap ifce

class ListAggr(Aggr):

@staticmethod

def make(name, impl, ifce):

dct = {}

for key in dir(ifce):

dct[key] = getattr(ifce, key)

return ListAggr(name, (impl,), dct)

def __init__(self, name, bases, dct):

Aggr.__init__(self, name, bases, dct)

base = bases[0]

ctr = base.__init__ % Use impl’s constructor

self.__init__ = instancemethod(ctr, None, self)

List-Based Sequential Aggrs.

Store list in private fieldclass ListFunc(object):

def __init__(self, data):

self.__data__ = data

def onAdd(self, other): pass

def onRemove(self, other): pass

List-Based Sequential Aggrs. (cont’d)

Sequential Foreachclass ListSeqForeach(ListFunc):

ListFunc.__init__(self, data)

def __aggr__(self, func, *args):

for inst in self.__data__:

func(inst, *args)

List-Based Sequential Aggrs. (cont’d)

Sequential Mapclass ListSeqMap(ListFunc):

ret = []

for inst in self.__data__:

ret.append(func(inst, *args))

return ret

Binding It Together

Making an aggr. that extends seq. foreachdef makeForeachClass(cls):

return ListAggr.make(’...’, ListSeqForeach, cls)

ListFunc

ListSeqForeach

ListAggr

ForeachClasscls

Optimizations

Multithreaded Foreachclass ListSpawnForeach(ListFunc):

n = max(len(self.__data__)/K, 1)

ts = []

for c in chunks(self.__data__, n):

t = ForeachThread(c, func, args)

ts.append(t)

t.start()

for tr in ts: tr.join()

Binding It Together, Multithreaded Version

Making an aggr. that extends parallel foreachdef makeForeachClass(cls):

return ListAggr.make(’...’, ListSpawnForeach, cls)

ListFunc

ListSpawnForeach

ListAggr

ForeachClasscls

Download complete codehttp://www.cs.bgu.ac.il/~gwiener/software/associations/

Read full paper

Proceedings of ENASE’2010 / By request

Summary

Outline

1 Introduction

2 OOP Hacking

3 Meta-Classes

4 Examples

5 Summary

Summary

Advantages of Meta-Classes

Turn this. . .

ka-HakalTru-Saka-Uka

s Phaarazu-Urha-Ytahi

Into this

Summary

Thank You!

wiener.guy@gmail.com

Meta-Classes in Python

Documents

Algorithmic Meta Theorems for Circuit Classes of Constant and

18-Dec-15 Abstract Classes and Interfaces. Java is “safer” than Python Python is very dynamic—classes and methods can be added, modified, and deleted

Python: Classes

Creating Shapefiles of Feature Classes with FME & Python

Python Part4 Sets, Dictionaries, Classes

CS229 Python Tutorial...Python basics demo Python OOP Why Classes? • Logical grouping of data and functions (which are called methods) • We try to create classes with logical connections

Python Utils Documentation - Read the DocsUseful Python Utils Python Utils is a collection of small Python functions and classes which make common patterns shorter and easier. It is

Objects and classes in Python Documentation

Python Multiprocessing - Computer Sciencekena/classes/5448/s11/presentations/... · Python Multiprocessing Ali Alzabarah . 4 3 2 1 Introduction Python is OOP language Python and concurrency

Python - Marquette Universitytschwarz.mscs.mu.edu/Classes/Python_Big_Data/Module13/presentation.pdf• Python has made several attempts before settling on an efﬁcient syntax. •

Python classes: new and old. New and classic classes With Python 2.2, classes and instances come in two flavors: old and new New classes cleaned up

Python Meta Classes and how django uses them

Python Programming - IV. Program Components (Functions, Classes, Modules, Packages)

djangocms-page-meta Documentation€¦ · CHAPTER 1 djangocms-page-meta Meta tag information for django CMS 3 pages Python: 2.7, 3.5. 3.6, 3.7 Django: 1.11 to 2.2 django CMS: 3.4,

Object-based and class-based composition of transitive mixinsnm.wu-wien.ac.at/research/publications/b613.pdf[25,36], meta-object protocols (MOP) [19], meta-classes [12], dynamic classes,

Python classes: new and old

F21SC Industrial Programming: Python Classes & …F21SC Industrial Programming: Python Classes & Exceptions Hans-Wolfgang Loidl School of Mathematical and Computer Sciences, Heriot-Watt

Python Classes and Objects - sms.wgtn.ac.nz€¦ · Classes and their objects partake of the dynamic nature of Python: they are created at runtime, and can be modiﬁed further after

Objects and classes in Python Documentation · 2019-04-02 · Objects and classes in Python Documentation, Release 0.1 Bound methods Unless you tell it not to, Python will create

Python DCPS API Guidedownload.ist.adlinktech.com/docs/Vortex/pdfs/OpenSplice...2 Introduction The Python DCPS API provides users with Python classes to model DDS communication using