Install BuildBot with Perforce on Windows

Patrick Tsaihttp://baby.homeip.net/patrick/

BuildBot

• The BuildBot is a system to automate the compile/test cycle required by most software projects to validate code changes.

• buildbot-0.7.5 (http://buildbot.sourceforge.net)

Two Options

• Win32– Python win32 (http://www.python.org/)– Pywin32 (http://www.python.org/)– Twisted 2.5 Win32 Installer for Python2.5

(http://twistedmatrix.com)

• Cygwin– Cygwin Installation (http://www.cygwin.com)– Python 2.5– Twisted 2.5 tarball

Win32 Installation

• Run the installers of Python, pywin32, Twisted in order.

• Add c:\python25;c:\python25\scripts to %PATH.

• buildbot-0.7.5> python setup.py install

Cygwin Installation

• Install python and gcc in Cygwin installer.

• Install Twisted– Twisted-2.5.0/zope.interface-3.3.0$ python

setup.py install

– Twisted-2.5.0$ python setup.py install

• buildbot-0.7.5$ python setup.py install

BuildBot Setup

• Create one master and one slave (check BuildBot README for details):– buildbot create-master <basedir>

– buildbot create-slave <basedir> <master> <name> <passwd>

• Edit master.cfg

Master.cfg #1:Version Control System

• I am using Perforce (http://www.perforce.com) as version control system.– Create a P4 user for buildbot.

– Add the P4 common settings to the beginning of master.cfg:

p4env = {'port': 'localhost:1666', 'user': 'buildbot', 'passwd': ‘your password here',

}

Master.cfg #2:P4 Source

• Remember to specify p4bin otherwise the master will fail to start in Native Win32 installation.

• You don’t need p4bin for Cygwin installation. Just ensure p4.exe is in the PATH.

from buildbot.changes.p4poller import P4Sourcec['sources'].append(

P4Source(p4base='//depot/',p4port = p4env['port'], p4user = p4env['user], p4passwd = p4env['passwd'],pollinterval = 60,p4bin = 'd:/p4d/p4.exe',split_file = lambda branchfile: branchfile.split('/', 1)))

Master.cfg #3: Schedulers

• I’ve three folders: main, development, and release under Perforce depot.

• Therefore, I created three schedulers for them.

from buildbot.scheduler import Schedulerc['schedulers'] = []c['schedulers'].append(

Scheduler(name = "main", branch = 'main',treeStableTimer = 2*60,builderNames=["main"]))

c['schedulers'].append(Scheduler(

name = "development", branch = 'development',treeStableTimer = 2*60,builderNames=["development"]))

c['schedulers'].append(Scheduler(

name = "release", branch = 'release',treeStableTimer = 2*60,builderNames=["release"]))

Master.cfg #4: Builders

• Create three builders for each schedulers.

• Create one P4 client for each builder.

f1 = factory.BuildFactory()f1.addStep(

P4,p4port = p4env['port'], p4user = p4env['user'], p4passwd = p4env['passwd'], p4client = 'buildbot-main',p4base = '//depot/',mode = "copy")

b1 = {'name': "main",'slavename': "bot-blackbox",'builddir': "main",'factory': f1

}…c['builders'] = [b1, b2, b3]

Final Step?

• Run build master:– buildbot start <master_dir>

• Run build slave– buildbot start <slave_dir>

• If you are as unlucky as me, you may encounter the following problems…

Problem #1

• Win32: “buildbot start …” throws exception

• Remedy:

Problem #2

• Cygwin & Win32: the slave cannot run because of some exception related to ‘LOGNAME’.

• Remedy: patch buildbot\slave\commands.py– Native Win32:

replace LOGNAME to USERNAME

– Cygwin: replace LOGNAME to USER

Problem #3

• Win32: The web page shows garbage characters in UTF-8 encoding

• Remedy: patch buildbot\status\html.py

Problem #4

• Cygwin: P4poller cannot detect P4 changes• Remedy: patch twisted\internet\posixbase.py (It

works but I am not sure whether it’s correct)

Problem #5

• Cygwin: the slave will hang when executing ‘p4 …client –i’

• Remedy: replace the action with dummy one. However, you need to configure the root for each P4 client manually.

Finally BuildBot Can Run

Conclusion

• Both installations cannot run perfectly with patches.

• Cygwin installation is more problematic than Win32 one

• BuildBot is good and still worths the effort.