Upload
bk422
View
232
Download
2
Tags:
Embed Size (px)
DESCRIPTION
cwg11
Citation preview
Tools and software process for the FLP prototype
B. von Haller
9. June 2015
CERN
Overview
▶Scope ▶FLP prototype▶O2
▶Software development ▶Software quality ▶Software deployment ▶Continuous Integration
▶Specific to FLP proto :
B. von Haller | CWG11 | 09.06.2015 2
▶Operating Systems▶SLC6, CC7, Ubuntu, Mac
▶Compilers▶Gcc, llvm, icc
▶ Languages▶Low level : C▶Middle level : C++11, stl, boost libraries, ROOT6▶Open to higher level languages (e.g. python)
▶Expose interfaces using generators such as SWIG
▶Web : PHP or Python with Django. To be decided
Ecosystem
B. von Haller | CWG11 | 09.06.2015 3
OnlyLimited to these 2
Software development
▶Tools▶Version control system: Git with GitLab/GitHub▶Software build system: CMake ▶Issue tracking system: JIRA▶Code formatter: clang-format
▶Procedures▶Workflows A la GitHub (pull request, see next
slides)
B. von Haller | CWG11 | 09.06.2015 4
GitLab to be gradually
introduced
Software development GitHub development workflow
B. von Haller | CWG11 | 09.06.2015 5
▶ Collaborators have a full remote clone of the main repo, called a fork▶ Full rights on their fork, none on the main repo▶ Anything in the master branch is always deployable.▶ Owners of the main repo (librarians) comment and accept/reject the
pull requests▶ Open source clone of GitHub: GitLabs
Software developmentGitHub/Gitlab
B. von Haller | CWG11 | 09.06.2015 6
▶Advantages of GitHub/GitLab and the related development workflow▶Collaborative and intuitive web interface▶Code always reviewed and commented before
being accepted▶Code can be checked automatically by tools before
human reviews▶Feature-rich api
Software development
▶ Procedures (continued)▶ Workflows A la GitHub (pull request, see next slides)▶ Coding conventions, naming, formatting -> CWG2▶ Code access policy Open (license still under discussion) -> CWG2
▶ Repositories ▶ Neither monolithic nor excessively fragmented▶ Group components that share same dependencies and/or similar functionalities
▶ Packages▶ Does not always match with repositories (also more flexible)▶ Release pace or common dependent packages to take into account
▶ Dependency handling for developers▶ External : original + patch -> recompile (tool to be decided) ; packages▶ Internal : packages (avoid recompiling what does not need to) ▶ CernVM-FS
B. von Haller | CWG11 | 09.06.2015 7
Software quality
▶Tools▶Static analysis -> Clang Static Analyzer▶Dynamic analysis -> Valgrind▶Tool for code reviews -> GitLab▶Code conventions and formatting enforcer ->
clang-format
B. von Haller | CWG11 | 09.06.2015 8
Not initially
Software quality
▶Testing▶Unit tests, usually at the function level
▶CTest and boost.test
▶Functional tests ▶Integration tests, software modules are combined
and tested as a group. Full chain. ▶Requires the code to be designed for it and to provide
mock systems at the interfaces▶Environment must be controlled
B. von Haller | CWG11 | 09.06.2015 9
Software deployment
▶Packaging▶CPack or fpm (to be decided, evaluation ongoing)
▶Dependency manager for users▶Packages using native system offered by OS▶CernVM-FS
B. von Haller | CWG11 | 09.06.2015 10
Continuous Integration (CI)
▶Actually wider than pure CI▶ Inter connects development, quality and deployment
▶ Jenkins▶Extensible, open source CI server▶Hundreds plugins, wide industry support▶Can do many type of tasks (e.g. build, test, release)▶On scheduled time or following a trigger (e.g. pull
request)▶API to build our own tools in front or make other tools
collaborate (e.g. JIRA)
B. von Haller | CWG11 | 09.06.2015 11
Continuous Integration
▶Master hosted in IT for ALICE on a VM▶Use VMs with Docker on Openstack▶Authentication using CERN SSO and egroups▶Mesos deployment containers to deal with
non-homogeneous tasks and load▶Integration with JIRA and GitHub/GitLab▶Time scale : infrastructure ready this summer
Jenkins for O2
B. von Haller | CWG11 | 09.06.2015 12
Conclusion
▶Tools have been chosen and procedures defined to ensure ▶Code quality, readability and maintainability▶Products delivery, stability and performance
▶The FLP prototype development can start in July directly within this frame
B. von Haller | CWG11 | 09.06.2015 13