Upload
guestdd2966
View
683
Download
8
Tags:
Embed Size (px)
DESCRIPTION
Project-Builder.org is a new GPL v2 tool designed to help projects developers producing easily packages for multiple OS and architectures, on a regular basis, from a single source repository.
Citation preview
© Bruno Cornec / HP - Document under/sous CC 3.0 by SA Date : 02/06/2010 Page 1
Project-Builder.org
Project Builder:
A GPL continuous packaging solution
Bruno Cornec
Open Source and Linux Profession EMEA Lead
© Bruno Cornec / HP - Document under/sous CC 3.0 by SA Date : 02/06/2010 Page 2
Project-Builder.orgIntroducting myself
Software engineering and Unices since 1988
Mostly Configuration Management Systems (CMS), Build systems, quality tools, on multiple commercial Unix systems
Discover Open Source & Linux (OSL) & first contributions in 1993
Full time on OSL since 1995, first as HP reseller then @HP
Currently:
Technology Architect on OSL for the HP/Intel Solution Center, Grenoble
OSL HP Ambassador
EMEA OSL HP Profession Lead
Solutions Linux Conference board member
MondoRescue, Dploy.org, Project-Builder.org project lead
LinuxCOE, mrepo, tellico, rinse, fossology contributor
Mandriva, Fedora packager
© Bruno Cornec / HP - Document under/sous CC 3.0 by SA Date : 02/06/2010 Page 3
Project-Builder.orgWhat is the problem ?
Users / Sysadmins environment:
< 1%
< 10%
>= 90%
Users: easyness (GUI, CLI)Admins: distribution compliance, smooth integration – reduce admin load
© Bruno Cornec / HP - Document under/sous CC 3.0 by SA Date : 02/06/2010 Page 4
Project-Builder.org
Project
Flat Files
CMS
CVSSVNGIT, ...
What is the problem ?
Developers environment:
autotools ?Install proc ?Soft. Eng. ?Fear of distro ?
Sourceforgeftp, web
© Bruno Cornec / HP - Document under/sous CC 3.0 by SA Date : 02/06/2010 Page 5
Project-Builder.org
Package
Flat Files
CMSCVSSVNGIT, ...
What is the problem ?
Packagers environment:
rpmlint ?lintian ?
Distroftp, web
Does s/he only exists ?On time ? Lag behind ?Distro coverage (ver, #)Beta avail ?
© Bruno Cornec / HP - Document under/sous CC 3.0 by SA Date : 02/06/2010 Page 6
Project-Builder.org
Packaging should be a project concern as well as coding, testing, installing, .... especially for smaller projects
Package early, package always
Use packaging as your only way of delivery (not a dream)Overhead minimal (not much longer than providing a tgz), maximum benefit (consistancy, reproduceability, team work improvement, distribution integration, improved deployment)
Packaging is in fact a marketing activity based on a technical content. It's an easy way to extend your user base, and improve your community. Makes tests easy.
THE SOLUTION IS CONTINUOUS PACKAGING
What is the solution ?
© Bruno Cornec / HP - Document under/sous CC 3.0 by SA Date : 02/06/2010 Page 7
Project-Builder.orgMulti platform life cycle
User Requirements
FunctionalSpecification
DesignSpecification
Coding
Unit tests
Integration tests
Validation
© Bruno Cornec / HP - Document under/sous CC 3.0 by SA Date : 02/06/2010 Page 8
Project-Builder.org
I'm a sysadmin so I want to deal with packages only (deployment server integration)
I'm a developer so I want to ease my tests without screwing up my system and testing as a std user/admin
I'm a a packager so I want to streamline that work for new projects I'm interested in packaging
I'm the marketing department of my own project and use packages as a “competitive advantage”
I'm a software engineer and want no duplication of code nor information (metadata)
Founding no tool existing to help me doing all that, so I started writing one to scratch my own itch
My answer
© Bruno Cornec / HP - Document under/sous CC 3.0 by SA Date : 02/06/2010 Page 9
Project-Builder.org
Project-builder main goal is to help you package continuously being agnostic:
CMS agnostic (no CMS but guys it's 21st century now, SVN, CVS, Mercurial, GIT, SVK....)
OS agnostic: Linux: RPM, deb, ebuild, slack based, ... 60+ distros at the moment – repositories for yum, urpmi, apt. Solaris pkg.
Build environment agnostic: local, VM (QEMU, KVM...), VE (rinse, mock, debootstrap...)
No project impact (preserves the md5sum of the delivered upstream sources). Can be completely external.
Goals
© Bruno Cornec / HP - Document under/sous CC 3.0 by SA Date : 02/06/2010 Page 10
Project-Builder.orgMulti platform development cycle
.a
.o
.o.o.o .o.o.o
.o.o.o .o.o.o
.o.o.o .o.o.o
.o
SSPK
SLPK
.o
.a< 5%
90+%
< 10%
© Bruno Cornec / HP - Document under/sous CC 3.0 by SA Date : 02/06/2010 Page 11
Project-Builder.org
Project-builder provides additional goodies:
Set of existing VMs/VEs or easy to recreate
Macro system with perl variables to avoid duplication
Skeleton generation to help starting
Manages delivery up to your ftp server (ssh based)
Manages announces on mailing lists
May use snapshot features of VMs/VEs
Integration of tests in the process
Manages patches when not upstream
Checks validity of packages built (lintian, rpmlint)
Easy creation of new versions for upstream management
Goodies
© Bruno Cornec / HP - Document under/sous CC 3.0 by SA Date : 02/06/2010 Page 12
Project-Builder.org
upstream CMS repository
pbconf CMS repository
project-builder.org
local packaging& test system
1
2
upstream Web/ftp server
2
1 pbconf Web/ftp server
pbconf file/dirpbconf file/dir
upstream file/dir
2
2
3
4
5
ftp/ssh/nfs server
mailserver
CHROOT
ebuild
Architecture
1
© Bruno Cornec / HP - Document under/sous CC 3.0 by SA Date : 02/06/2010 Page 13
Project-Builder.org
ProjectPackages
cms2build
build2vm
build2pkgpkg2ssh
newvm / setupvm
newver / newproj
The Big picture
build2sshDevelopers
Packagers
© Bruno Cornec / HP - Document under/sous CC 3.0 by SA Date : 02/06/2010 Page 14
Project-Builder.org
cms2build: Create tar files for the project under your CMS. CMS supported are SVN and CVS parameters are packages to build if not using default listbuild2pkg: Create packages for your running distributionbuild2ssh: Send the tar files to a SSH hostpkg2ssh: Send the packages built to a SSH hostbuild2vm, build2ve: Create packages in VMs/VEs, launching them if needed and send those packages to a SSH host once builtlaunchvm: Launch one virtual machinescript2vm, script2ve: Launch a VM/VE if needed and executes a script on ittest2vm, test2ve: Test inside a VM/VEnewvm, newve: Create a new virtual machinesetupvm, setupve: Setup a virtual machine for pb usagenewver: Create a new version of the project derived from the current onenewproj: Create a new project and a template set of configuration files under pbconf
Commands
© Bruno Cornec / HP - Document under/sous CC 3.0 by SA Date : 02/06/2010 Page 15
Project-Builder.org
Packages
Multi platform packaging cycle
Developers
Packagers
VMs/VEs
VM/VE
© Bruno Cornec / HP - Document under/sous CC 3.0 by SA Date : 02/06/2010 Page 16
Project-Builder.org
More Tests on VEs (mock, debootstrap to be done, rinse OK + patches) LSB chroot support.
Support for libvirt, virsh
Other CMS (Bazaar, …) only when/if needed
Other VMs (VMWare, Xen, ...) only when/if needed
Multiple delivery means
Look at interactions with Buildbot
Add signature support
Create a reference documentation
Config-Model for configuration file management
To be done
© Bruno Cornec / HP - Document under/sous CC 3.0 by SA Date : 02/06/2010 Page 17
Project-Builder.org
Web Resources
Project-Builder Trac / Wiki
http://trac.project-builder.org
Open Source at HP
http://opensource.hp.com
Other tools: SuSE/Fedora/Mandriva Build Systems, Buildbot, vcs-pkg.org
“The evolution of FLOSSand the Internet are
tightly coupled”
© Bruno Cornec / HP - Document under/sous CC 3.0 by SA Date : 02/06/2010 Page 18
Project-Builder.org
”Changes are never easy to make. There is comfort and safety in tradition, but change must come, no matter how painful or expensive it may be.”
Bill Hewlett
Contact
Thanks
(Linux Solution Consultant in the HP/Intel Solution Center)
http://www.hp.com/linux
Linus Torvalds, Richard Stallman, Eric Raymond, Nat Makarevitch, René Cougnenc, Eric Dumas, Rémy Card, Bdale Garbee, Hugo Rabson, Gallig Renaud, Andree Leidenfrost among others, for their work and devotion to the Open Source Software cause... and my family for his patience :-)
© Bruno Cornec / HP - Document under/sous CC 3.0 by SA Date : 02/06/2010 Page 19
Project-Builder.org
V0.1-v0.4: script shells – linked to mondorescue – Usage of QEMU to generate cross distros packages
V0.5 (08/2007): rewritten in perl, keeping the principles, independent of the project – pbconf in the project – filtering support with macros – PoC state
V0.6: support multiple versions, perl conventions
V0.7: perl env var substitution in conf files for macros – first usage for mondorescue and itself including VM.
V0.8: functionaly similar to the original scripts (except ML, announces, ...). Multi CMS support. Trac BR. Removes AppConfig dep
V0.9: re-architectured for no project impact (pbconf outside of project, multi-conf file support, CMS independance, VM/VE, separate tar files for build info...)
History