Nightly Releases and Testing

Alexander Undrus

Atlas SW week, May 24 2004

Atlas SW week, May 24, 04 – Alex Undrus – Nightly Builds 2

Outline Status of Nightly Builds New NICOS version 0.3 Testing in Nightly Builds

Atlas SW week, May 24, 04 – Alex Undrus – Nightly Builds 3

Status of Nightlies Three nightlies: opt, dbg, “8.0.x” 2x2GHz machines: 6 hrs gain:

Make is faster by factor 1.5 – 2.0 Builds are finished by ~ noon (CERN time)

More efforts for acceleration Parallel checkout: saves 1 hour but

connection problems with CVS server Multithreaded builds: new tests with CMT

v1r16 start soon Incremental builds (Simon George)

Atlas SW week, May 24, 04 – Alex Undrus – Nightly Builds 4

Plan for better robustness of Nightlies

Builds on local disk followed by copy to AFS Better stability, decreasing AFS load

Planned setting: Release is checked out to local disk Opt & dbg builds start simultaneously Interlock mechanism for building NOVA Copy to AFS (script copy_release available)

Waiting Database team for implementation of NOVA interlocks

Atlas SW week, May 24, 04 – Alex Undrus – Nightly Builds 5

Purpose of NICOS NIghtly COntrol System Provide flexible nightly build system for

coordination of distributed development Test new software submissions Send immediate feedback to developers Facilitate multi-person, multi-platform

development of code Portable System (perl based)

NICOS Organization

NICOS Controller

NICOS Job

Error Analysis

Testing

Project Make

ProjectSetup

Code Checkout

PackagesDatabase

PackagesDatabase

Configur. CacheConfigur. Cache

NICOS Webmaster

Version Admin.

Atlas SW week, Dec.03 - Alex Undrus - NICOS

7

Atlas SW week, May 24, 04 – Alex Undrus – Nightly Builds 7

History of NICOS Fall 2001: start of ATLAS nightly builds with set

of shell scripts Jan 2003: start of development of NICOS, perl

based nightly control system Feb 2003: version 0_1 of NICOS is released and

applied for POOL nightly builds (LCG project) July 2003: NICOS is accepted as LCG nightly

build tool, used in POOL and SEAL builds Nov 2004: NICOS version 0_2 applied for ATLAS

builds

Atlas SW week, May 24, 04 – Alex Undrus – Nightly Builds 8

NICOS version 0.3 Improved pages design:

Tag list emphasizes new packages and versions Three sorting options for list of packages Added category of QA tests (to previously used

integrated and unit tests) Test results on separate web page, while page

with build results contain test summary Added “warning” level for minor problems Builds “on demand” (use NICOS in users

work environment), tested by S. George

Atlas SW week, May 24, 04 – Alex Undrus – Nightly Builds 9

NICOS version 0.3 (cont.) Build results available immediately

after make loop Integration with QMTest, Python

based testing tool, supported by LCG

NICOS Project

Web Page

Atlas SW week, Dec.03 - Alex Undrus - NICOS

12

new

NICOS Tag List

Atlas SW week, Dec. 2003 - Alex Undrus - NICOS

13

NEW

NEW

NICOS Build Result

s

Atlas SW week, Dec.03 - Alex Undrus - NICOS 14

NEW

NEW

NICOS Test

Results

CHEP’03 - Alexander Undrus - NICOS system 15

NEW

Atlas SW week, May 24, 04 – Alex Undrus – Nightly Builds 14

Nightly Testing Principles Automation Smoke tests Support different granularity levels

QA tests Unit (structural) tests Integrated (behavioral) tests

Use of test tools (CppUNIT, QMTest) Convenience for developers Fast feedback to users

Atlas SW week, May 24, 04 – Alex Undrus – Nightly Builds 15

QMTEST Open-source, cross-platform testing tool Python based, extensions possible Tests are described in xml format Organizes tests hierarchically (in suites) Provides a graphical interface for

creating and running tests Also can be used from command line

Atlas SW week, May 24, 04 – Alex Undrus – Nightly Builds 16

QMTEST & NICOS integration

Integration is provided by Python script (qmtest_config): Searches for test scripts

NICOS test database Release tree (in directories test)

Determines test suites NICOS test database Pattern “TEST_SUITE” in script body Tries to guess

Creates XML descriptions for QMTest

Atlas SW week, May 24, 04 – Alex Undrus – Nightly Builds 17

QMTEST & NICOS integration (cont.)

QMTest runs tests, determines results (regression is possible), handles log info

Results are reflected on NICOS pages and also can be browsed with QMTest graphical interface

Test can be rerun from QMTest graphical interface

NICOS & QMTEST integration

NICOS Job

TestDatabaseTestDatabase

NICOS Webmaster

Atlas SW week, Dec.03 - Alex Undrus - NICOS

7

qmtest_config.py QMTEST

XML test descriptions

XML test descriptions

QMTEST page

CHEP’03 - Alexander Undrus - NICOS system 15

QMTEST suite

CHEP’03 - Alexander Undrus - NICOS system 15

Atlas SW week, May 24, 04 – Alex Undrus – Nightly Builds 21

CppUnit Tool for unit testing of C++ code/classes allows to run tests automatically and gives a

report about successes and failures For developers that write software and test

simultaneously ATLAS TestPolicy provides access to LCG

installation of CppUnit and integrates it in CMT environment (since January 2004).

Details: http://atlas.web.cern.ch/Atlas/GROUPS/SOFTWARE/OO/dist/nightlies/nicoswww/cppunit_testing.html

Atlas SW week, May 24, 04 – Alex Undrus – Nightly Builds 22

Status of NICOS 0.3 Trial 8.1.0 builds on lxbuild005 local disk:

Since ~ May 10 atlas.web.cern.ch/Atlas/GROUPS/SOFTWARE/OO/dist/

nightlies/testww Only one significant problem revealed: QMTest

does not work with PyXML used by ATLAS Temporary solution: exclusion of PyXML from PATHs

Test in real nightlies is scheduled on the next weekend

Update of manual to be completed by June 1

Atlas SW week, May 24, 04 – Alex Undrus – Nightly Builds 23

To Do list for NICOS Improve convenience features for

“builds on demand” Create scripts for administrative

tasks for stable releases (rebuilds, tests, documentation)

Atlas SW week, May 24, 04 – Alex Undrus – Nightly Builds 24

Further Information ATLAS nightly builds web page

atlas.web.cern.ch/Atlas/GROUPS/SOFTWARE/OO/dist/nightlies/atlaswww/nightly_builds.html

NICOS web page www.usatlas.bnl.gov/computing/software/nicos/index.html

QMTest integration page http://atlas.web.cern.ch/Atlas/GROUPS/SOFTWARE/OO/dist/nightlies/nicoswww/qmtest_atlas.html