Introduction Scripting The JavaScript days Python Bindings Future Examples

Python usage in Samba and OpenChange

Jelmer Vernooij

Samba OpenChange

PUN December 7, 2008

Introduction Scripting The JavaScript days Python Bindings Future Examples

Agenda

1 Introduction

2 Scripting

3 The JavaScript days

4 Python

5 Bindings

6 Future

7 Examples

Introduction Scripting The JavaScript days Python Bindings Future Examples

What is Samba ?

Free (GPLv3) implementation of the SMB protocol andothers

DCE/RPC, NetBIOS, LDAP, CLDAP, Kerberos, . . .

“Network Neighborhood” (and more) for POSIXTraditionally written in CExtremely portableAbout the same age as Python (’91)Developed by a team of 25 peopleOriginally developed thru network analysis

Introduction Scripting The JavaScript days Python Bindings Future Examples

What is Samba ? - Branches

Samba 3Proven, stable, codebase

Samba 4Our very own “Duke Nukem Forever”Originally started in 2003, as an effort to improve the fileserverStrong focus on the right infrastructure

Introduction Scripting The JavaScript days Python Bindings Future Examples

What is OpenChange ?

Free implementation of the MAPI protocolAs used by Microsoft Outlook/ExchangeWorks on top of the DCE/RPC protocol

Being build on top of Samba 4French project, started in 2003Evolution OpenChange plugin will ship with GNOME 2.26

Introduction Scripting The JavaScript days Python Bindings Future Examples

Who am I?

CS StudentFOSS developer, working on:

Samba, mostly Samba 4OpenChangeBazaar

Introduction Scripting The JavaScript days Python Bindings Future Examples

Samba and scripting

We are all C programmersScripting: awk, shell or perl

Samba had Python bindings for a whileRemoved after several years because they wereunmaintained

Introduction Scripting The JavaScript days Python Bindings Future Examples

Why scripting?

Within SambaQuicker developmentEasier to debugEasier to understand

Lower barrier for contributions?

For usersEasier to customizeEasier to use (administrator scripts, etc)

Introduction Scripting The JavaScript days Python Bindings Future Examples

And the winner is...

Original choice: JavaScriptSmall, fast engine (small enough to include with Samba)Familiar to a lot of developers out thereA lot like C (familiar to developers)

Introduction Scripting The JavaScript days Python Bindings Future Examples

JavaScript: Problems

A lot like CSucks as a scripting language

No exceptionsPoor string manipulation functionsNo keyword arguments

No bindings for standard librariesThe library we were using was different from the standardlibrary

Contributors had to learn yet another language (dialect)

No development toolsHard to write unit tests

Introduction Scripting The JavaScript days Python Bindings Future Examples

Why Python?

Comes “with batteries included”No need to reimplement utility functions and bindings forSamba

Easy to create bindingsMost existing libraries already have Python bindings

GTK+, Qt, HTTP, .ini-parsers...Large existing developer base

Potential contributorsBetter scripting language

Nested functionsModularity

More development tools availableDebugger, profiler, code coverage analyser, ...

Introduction Scripting The JavaScript days Python Bindings Future Examples

Why not something else?

Several developers already knew (and liked) PythonAlternatives

Perl: Hard to use C API, silly syntaxRuby: Not well known enoughLua: Not really well suited for application development, justsmall snippetsScheme: well, . . .

Introduction Scripting The JavaScript days Python Bindings Future Examples

Concerns

Our first ever mandatory build-dependency other thanlibc. . .A lot of code to migrate, requiring effort that could beuseful elsewhereMaintainability

Unit tests mandatoryShould be used by core code

Introduction Scripting The JavaScript days Python Bindings Future Examples

SWIG

Doesn’t require a lot of effort to generate simple bindingsCan generate bindings for multiple languages at the sametimeGenerates portable code

However...

Tends to create very C-like Python bindingsCustomization language is hard to graspUnreadable generated C code

Introduction Scripting The JavaScript days Python Bindings Future Examples

SWIG

Doesn’t require a lot of effort to generate simple bindingsCan generate bindings for multiple languages at the sametimeGenerates portable code

However...

Tends to create very C-like Python bindings

Customization language is hard to graspUnreadable generated C code

Introduction Scripting The JavaScript days Python Bindings Future Examples

SWIG

Doesn’t require a lot of effort to generate simple bindingsCan generate bindings for multiple languages at the sametimeGenerates portable code

However...

Tends to create very C-like Python bindingsCustomization language is hard to grasp

Unreadable generated C code

Introduction Scripting The JavaScript days Python Bindings Future Examples

SWIG

Doesn’t require a lot of effort to generate simple bindingsCan generate bindings for multiple languages at the sametimeGenerates portable code

However...

Tends to create very C-like Python bindingsCustomization language is hard to graspUnreadable generated C code

Introduction Scripting The JavaScript days Python Bindings Future Examples

Pyrex/Cython

Very Python-like

But...

Needs to be run on the developer machine (extrabuild-dependency)Doesn’t support certainUnreadable generated C code

Introduction Scripting The JavaScript days Python Bindings Future Examples

Pyrex/Cython

Very Python-like

But...

Needs to be run on the developer machine (extrabuild-dependency)

Doesn’t support certainUnreadable generated C code

Introduction Scripting The JavaScript days Python Bindings Future Examples

Pyrex/Cython

Very Python-like

But...

Needs to be run on the developer machine (extrabuild-dependency)Doesn’t support certain

Unreadable generated C code

Introduction Scripting The JavaScript days Python Bindings Future Examples

Pyrex/Cython

Very Python-like

But...

Needs to be run on the developer machine (extrabuild-dependency)Doesn’t support certainUnreadable generated C code

Introduction Scripting The JavaScript days Python Bindings Future Examples

Manual bindings

The Python C API really isn’t that badAllows close integration between our memory managerand PythonsMuch more flexible than autogenerated Python bindings

Introduction Scripting The JavaScript days Python Bindings Future Examples

Partially generated from IDL

IDL CodeNTSTATUS unixinfo GetPWUid ([in,out,ref,range(0,1023)] uint32 *count,[in,size is(*count)] hyper uids[],[out,size is(*count)] unixinfo GetPWUidInfo infos[*]);

Python APIS.GetPWUid(uids) − > infos

Introduction Scripting The JavaScript days Python Bindings Future Examples

Current state of affairs

Mostly used for administrative tools:provisioning the databases after installationweb service? (wsgi compatible)Server functionality and performance-dependentSome nifty GUI tools based on GTK+

Popular for writing testsPerformance-dependent code is still all in C

And most importantly:Developers seem reasonably happy

Introduction Scripting The JavaScript days Python Bindings Future Examples

More crazy Samba Python hacks

Full Python coverage of our librariesMore GNOME integration in Pythonwin32com on Linux?Port to Samba 3?Server partially in Python?

Introduction Scripting The JavaScript days Python Bindings Future Examples

OpenChange

Provisioning already uses PythonMost client tools will be in PythonBindings still to be done

Introduction Scripting The JavaScript days Python Bindings Future Examples

Remaining and new concerns

No good standard mechanism for asynchronous functions(yet?)Some users are running Python older than 2.4Python3000 will drop support for some of our platformsSupporting all combinations of platforms with Pythoninstalled turned out to be quite a challenge

Introduction Scripting The JavaScript days Python Bindings Future Examples

Reading TDB files

1 import tdb , sys23 db = tdb . Tdb ( sys . argv [ 1 ] )4 for ( k , v ) in db . i tems ( ) :5 pr in t ” { ”6 pr in t ” key(%d ) = %r ” % ( len ( k ) , k )7 pr in t ” data(%d ) = %r ” % ( len ( v ) , v )8 pr in t ” } ”

Introduction Scripting The JavaScript days Python Bindings Future Examples

Using LDB

1 # ! / usr / b in / python23 import l db45 conn = ldb . Ldb ( ”msg . tdb ” )67 conn . add ({ ” dn ” : ” dc=samba , dc=org ” , ” a t t r 1 ” : ” foo ” } )89 for msg in conn . search ( ” dc=samba , dc=org ” ) :

10 pr in t s t r (msg . dn )

Introduction Scripting The JavaScript days Python Bindings Future Examples

Connecting to LDAP using LDB

1 # ! / usr / b in / python23 import l db45 # Connect to the LDAP server6 conn = ldb . Ldb ( ” ldap : / / ldap . abmas . org / ” )78 for msg in conn . search ( ” dc=samba , dc=org ” ) :9 pr in t s t r (msg . dn )

Introduction Scripting The JavaScript days Python Bindings Future Examples

Adding users

1 # ! / usr / b in / python2 import samr , l sa34 # Connect to the l o c a l SAM5 conn = samr . samr ( ” nca l rpc : ” , ” s t / dc / e tc / smb . conf ” )67 # Get SAMR connect handle8 samr handle = conn . Connect (0 , 0 x f f f f f f f )9

10 domainname = lsa . S t r i n g ( )11 domainname . s t r i n g = u ”SAMBADOMAIN”1213 s id = conn . LookupDomain ( samr handle , domainname )14 pr in t ” Found s id %s f o r SAMBADOMAIN” % s id1516 conn . Close ( samr handle )

Introduction Scripting The JavaScript days Python Bindings Future Examples

Unit tests

1 import winreg2 from samba . t e s t s import RpcInterfaceTestCase34 class WinregTests ( RpcInterfaceTestCase ) :5 def setUp ( s e l f ) :6 s e l f . conn = winreg . winreg ( ” nca l rpc : ” , s e l f . get loadparm ( ) , s e l f . g e t c r e d e n t i a l s ( ) )78 def t es t hk lm ( s e l f ) :9 handle = s e l f . conn .OpenHKLM(None ,

10 winreg .KEY QUERY VALUE | winreg .KEY ENUMERATE SUB KEYS)11 s e l f . conn . CloseKey ( handle )

Introduction Scripting The JavaScript days Python Bindings Future Examples

More information

http://www.samba.org/

http://www.openchange.org/

IRC: #samba-technical / #openchange on Freenode

If you have ideas about asynchronous function usage, pleaselet me know.