Multi-platform Automated Software

Building and PackagingAndres ABAD RODRIGUEZ (IT-GT / CERN)

V.E. GOMES GOUVEIA (IT-GT/CERN), D. MENESES (IT-GT/CERN), F. CAPANNINI (INFN), A. AIMAR (IT-GT/CERN), A. DI MEGLIO (IT-DI/CERN)

v

Products & Languages

BDII

AMGA

Logging and Bookkeeping

APEL

FTS

UNICORE Container

EMIR

MPI

UNICORE security SAGA

VOMS

StoRM

ARC Infosys

gLite Security

CANL

Infosys

Argus

dCache

CREAM CE

LFC

WNoDeS

DPM

ARC Clients

ARC CE

EMI Messaging

WMS

The EMI (European Middleware Initiative) project is the integration of the main European pre-existing middlewares (gLite, UNICORE, ARC and dCache) into a single consistent set of packages with uniform distribution and repositories.

The products that became part of EMI were built using different tools and dedicated services. The software, millions of lines of code, is written in several programming languages and platforms.

Despite this heterogeneous initial situation, a single common solution was needed for building, testing, packaging and QA.

The ETICS service, already used by the gLite components, was selected for the EMI project.

Build/TestArtifacts

ConfigurationWeb Service

Report/MetricsDB

ConfigurationDB

Condor Execution Engine

RepositoryWeb Service

Virtual WorkerNodes

EMI build, test and packaging infrastructure

ETICS allows the specification of the check out, build and test commands, dependencies specification and environment definition of the whole software stack.

The ETICS jobs are run on demand created Virtual Worker Node. The platforms supported in EMI are Scientific Linux 5 and 6 (32 and 64 bits) and Debian 6 (64 bits). This solution reduces the number of physical machines to the minimum (only for the main servers).

Multiple metrics can be collected during the build execution. There are many out of the box metrics plugins for static source code analysis such as Findbugs, pUnit, etc. The system can be easily extended to support other QA tools. The reports and artifacts are stored in the repository and the worker node is reused.

Other interesting features are multi node testing and the repository graphical interface from where the artifacts can be easily accessed and browsed.

EMI repository

RPM packages DEB packages Source andBinary TAR.GZ Reports & metrics

ETICS also builds software using standard tools such as Mock and Pbuilder. The artifacts produced by the system are compliant with the EPEL and Debian guidelines. The compliance level is automatically monitored running the RPMLint and Lintian plugins.

The packages created, reports and metrics generated are stored in the EMI repository. These packages are ready to be included in the Debian, Fedora EPEL and EGI UMD repositories.

For more information: http://cern.ch/emi-tools-chep2012

EPEL Repository EGI UMD Repository Squeeze Repository

YAIM

glExec

LCAS

Scientific Linux