View
438
Download
0
Tags:
Embed Size (px)
DESCRIPTION
Free and Open Source Software distributions raise difficult problems both for distribution editors and system administrators. Distributions evolve rapidly by integrating new versions of software packages that are independently developed. System upgrades may proceed on different paths depending on the current state of a system and the available software packages, and system administrators are faced with choices of upgrade paths, and possibly with failing upgrades.Mancoosi develops mechanisms that provide for rollbacks of failed upgrade attempts, allowing the system administrator to revert the system to the state before the upgrade, and better algorithms and tools to plan upgrade paths based on various information sources about software packages and on optimization criteria.The consortium as a whole is entirely committed to the free software movement, and the project sets up virtuous cycles associating users, industry and researchers that will outlive the project itself.Mancoosi is a European research project in the 7th Research Framework Programme (FP7) of the European Commission, which has started February 1st, 2008, and has a duration of 3 years.
Citation preview
Mancoosi: bridging communities to improvepackage-based systems
Roberto Di CosmoUniversity Paris Diderot Paris 7 and INRIA
Fossa 2009
November 18th, 2009
The promise of GNU/Linux distributions
GNU/Linux distributions are intermediaries between FOSS projectsand their users
They factor the work needed to mainain, configure, test, packagetogether tens of thousands of different software components.
Package-based distributions: a very successful idea . . .
Central notion : package, together with package managementsoftwareA main innovations of GNU/Linux distributions: it opens the doorto flexible configuration and management of complex systems viareusability and modularity.
Package management: a key industrial issue
A few facts:
In 2011, 80% of commercial software will contain OpenSource code. . . 49,7% of critical application come fromthe Open Source world...
Mark Driver (Gartner) sept.07
A key industrial issue is to build faster, stronger, harder, better,more user oriented packaging technology, to master the challengeof the Open Source success.And yet, surprisingly little has been done in terms of R&Dspecifically related to package technology (as opposed tocomponents or web services).
The Challenge of Distributions
Mandriva, Debian, Ubuntu, and all other distribution editors areresponsible for maintaining a free software distribution. It is achallenging task! (smooth upgrades, automatic dependencysolving, up to date software . . . )
Help is needed to build: better infrastructure for packagemaintainers and better package managers for final users.
Two projects to the rescue:
EDOS [2004–2007] aim: provide FOSS distribution editorswith better QA tools
Mancoosi [2008–2011] aim: provide with better packagemanagers to improve their “platform management”experiences (upgrade, downgrade)
Packages, metadata, installation
Package =
{some filessome scriptsmetadata
Identification
Inter-package rel.
DependenciesConflicts
Feature declarations
Other
Package maintainerTextual descriptions...
Example
Package: atermVersion: 0.4.2-11Section: x11Installed-Size: 280Maintainer: Goran Weinholt ...Architecture: i386Depends: libc6 (>= 2.3.2.ds1-4),libice6 | xlibs (>> 4.1.0), ...
Conflicts: suidmanager (<< 0.50)Provides: x-terminal-emulator...
a package is the elemental component of modern distributionsystems (not GNU/Linux specific)
a working system is deployed by installing a package set(≈ 1000/2000 for GNU/Linux distro)
.
Sample installation
Phase TraceUser request # apt-get install aterm
Constraint resolution
8>>>>>>>>>>>>>><>>>>>>>>>>>>>>:
Reading package lists... Done
Building dependency tree... Done
The following extra packages will be installed:
libafterimage0
The following NEW packages will be installed
aterm libafterimage0
0 upgraded, 2 newly installed, 0 to remove and 1786 not upgraded.
Need to get 386kB of archives.
After unpacking 807kB of additional disk space will be used.
Do you want to continue [Y/n]? Y
Package retrieval
8><>:Get: 1 http://debian.ens-cachan.fr testing/main libafterimage0 2.2.8-2 [301kB]
Get: 2 http://debian.ens-cachan.fr testing/main aterm 1.0.1-4 [84.4kB]
Fetched 386kB in 0s (410kB/s)
Pre-configuration {
Unpacking
8>>>><>>>>:
Selecting previously deselected package libafterimage0.
(Reading database ... 294774 files and directories currently installed.)
Unpacking libafterimage0 (from .../libafterimage0_2.2.8-2_i386.deb) ...
Selecting previously deselected package aterm.
Unpacking aterm (from .../aterm_1.0.1-4_i386.deb) ...
Configuration
(Setting up libafterimage0 (2.2.8-2) ...
Setting up aterm (1.0.1-4) ...
each phase can fail (it actually happens quite often . . . )
efforts are needed to identify errors as early as possible
One of EDOS outcomes: edos-debcheck
EDOS has brought some major results for distribution editors:
theory : checking if the dependencies of a package can besatisfied is NP-complete
practice : several tools have been developed during EDOS (≈110’000 OCaml LOCs), some examples:
edos-debcheck command line checker for packageinstallability
pkglab interactive, console-based environmentfor repository inspection
ceve parser/converter between package listformats
tart optimised algorithm to cut a repositoryinto slices (e.g. media), so thatpackages available on the i-th slice areinstallable using only slices up to i
Focus on edos-debcheck
Written by Jerome Vouillon, edos-debcheck takes as input anAPT package list (e.g. /var/lib/apt/lists/*) and checkswhether one, several, or all packages in it are installable wrt thatrepository.Customized SAT solver, quite fast: checking installability of allpackage in main testing/amd64 takes 5 seconds on an entry-levelmachine.
Example
edos-debcheck </var/lib/apt/lists/... main binary-amd64 Packages
Parsing package file... 1.2 seconds 21617 packages
Generating constraints... 2.3 seconds
Checking packages... 1.5 seconds
acx100-source (= 20070101-3): FAILED
alien-arena (= 7.0-1): FAILED
alien-arena-browser (= 7.0-1): FAILED
alien-arena-server (= 7.0-1): FAILED
alsa-firmware-loaders (= 1.0.16-1): FAILED
amoeba (= 1.1-19): FAILED
...
# explanation can be required as well
The Mancoosi project [ http://www.mancoosi.org ]
project strep FP7 (2008–2011)
focus coherence and maintenance of a FOSS distributioninstallation (user point of view)
Tools for user-side package management (package managers /meta-installer)
installationremovalupgradedowngrade. . .
An upgrade problem example
# sudo apt-get install debhelperReading Package Lists... Done
Building Dependency Tree... Done
The following extra packages will be installed:
armagetron armagetron-common autoconf bonobo-activation codebreaker debconf
debconf-i18n debconf-utils dialog esound-common fb-music-high fontconfig
frozen-bubble-data grepmail gv intltool-debian libaiksaurus-data
libaiksaurus0c102 libatk1.0-0 libatk1.0-dev libbonobo-activation4 libbonobo2-0
libbonobo2-common libdb3 libdbd-mysql-perl libdbi-perl libeel2-data libesd0
...
The following packages will be REMOVED:
autoconf2.13 frozen-bubble frozen-bubble-lib gconf2 gnomemeeting itk3.1-dev
libbonoboui2-0 libbonoboui2-common libdigest-md5-perl libforms0.89 libgconf2-4
libgnome2-0 libgnome2-common libgnomeui-0 libgnomevfs2-0 libgnomevfs2-common
libgtk1.2-dev libgtk2.0-0png3 libgtk2.0-dev libmime-base64-perl
libpango1.0-dev libsdl-mixer1.2-dev libsdl-perl libsdl-ttf1.2-dev
libsdl1.2-dev libsmpeg-dev libstorable-perl nautilus tk8.3-dev tktable-dev
x-window-system x-window-system-core xaw3dg-dev xlib6g xlib6g-dev xlibmesa-dev
xlibmesa3 xlibosmesa3 xlibs-dev xlibs-pic xpdf xpdf-reader
The following NEW packages will be installed:
armagetron-common debconf-i18n fb-music-high fontconfig intltool-debian
libaiksaurus-data libaiksaurus0c102 libeel2-data libfilehandle-unget-perl
libfontconfig1 libforms1 libgdbm3 libgnutls7 libgsf-1 libice-dev libice6
libidl0 liblzo1 libmagick5.5.7 libmail-mbox-messageparser-perl
libmysqlclient12 libncursesw5 libnet-daemon-perl libnewt0.51 libpaper1
libplrpc-perl libsdl-console ...
75 packages upgraded, 80 newly installed, 42 to remove and 858 not upgraded.
Need to get 67.1MB of archives. After unpacking 26.9MB will be used.
Do you want to continue? [Y/n] Abort.
Goals, approach and strategy
Mancoosi’s goal:
enable safe, efficient maintainability of the Open Sourcesoftware infrastructure built out of software packages
A twofold approach to reach this objective:
design algorithms and tools to tackle the upgradeabilityproblem;this allows to find good upgrade paths w.r.t. the user’s needs.
conceive and implement a transactional update process;this allows to roll-back an unsatisfactory upgrade.
A long term, ecosystem strategy to reach this objective:
bring users, distributions, developers and researchers together!
set up virtous cycles
All in a pictureMancoosi architecture
In Mancoosi, there are really two main activities, corresponding tothe architecture of our design, and they are loosely coupled
User request WP4,WP5↓
Platform management interface
Optimal upgrade path computation WP4, WP5
Transactional component manager WP2, WP3
↓Software Platform
Upgrade resolution
Abstract view of packages through the metadata lens.
Transactional updates
Concrete view of packages through the maintainer scripts lens.
Mancoosi technology preview: strong dependencies
GNU/Linux software distributions: probably the most complexsystem of systems on the planet
HOW can you visualize such a system of systems?
Draw a graph, no deep knowledge
kde-amusements and kde-toys in Debian 3.1 (aka sarge, 2005)were unnecessary components, overlapping kde-games andkde-edu.They are waste, and are gone today (2009).Should be easy to spot, only 244 components are concerned!Can you see this fact here?
Let’s try a more systemic approach
Here is a graph of strong dominators for the KDE subsystem:
See the problem, now?
More on this at http://www.mancoosi.org
Concluding remark
Everything developed in Mancoosi is already relevant for some, andwill be increasingly relevant to all component based technologies
Linux : packages
Eclipse : plugins
Firefox : extensions
Java : beans
Web services : predictable service assembly
. . . . . .
Project Partners
Family pictures. . .
the Mancoosi team, Feb 2008
Questions
Thank you for your attention.