Embed Size (px)
Citation preview
Leveraging version control and continuous integration toimprove a Linux HPC eco system.
Yorick PoelsKU LeuvenJune 15 2017
ToC INTRO TOOLS EVOLUTION OF SETUP WORKFLOW CONCLUSION
CONTENTINTROTOOLSEVOLUTION OF SETUPWORKFLOWCONCLUSION
ToC INTRO TOOLS EVOLUTION OF SETUP WORKFLOW CONCLUSION
Tier1: BrENIACI 580 nodesI CentOS 7.xI Puppet
Tier2: ThinKing - CerebroI 352 nodes - shared memory systemI RedHat 6.xI Bash
Service nodes (tier1 & 2)I 25+I Foreman + Puppet
ToC INTRO TOOLS EVOLUTION OF SETUP WORKFLOW CONCLUSION
ToC INTRO TOOLS EVOLUTION OF SETUP WORKFLOW CONCLUSION
GIT
ToC INTRO TOOLS EVOLUTION OF SETUP WORKFLOW CONCLUSION
REDMINE
ToC INTRO TOOLS EVOLUTION OF SETUP WORKFLOW CONCLUSION
JENKINS
ToC INTRO TOOLS EVOLUTION OF SETUP WORKFLOW CONCLUSION
DOCKER
I CentOS 6I CentOS 7
ToC INTRO TOOLS EVOLUTION OF SETUP WORKFLOW CONCLUSION
PUPPET
ToC INTRO TOOLS EVOLUTION OF SETUP WORKFLOW CONCLUSION
FOREMAN
ToC INTRO TOOLS EVOLUTION OF SETUP WORKFLOW CONCLUSION
MATTERMOST
ToC INTRO TOOLS EVOLUTION OF SETUP WORKFLOW CONCLUSION
EVOLUTION OF SETUP
1. Bash scripts2. Puppet code + version control3. Puppet code rpm in Jenkins4. Puppet code rpm in Jenkins, multiple git branches5. Lots of tests (lint,rspec, catalog,...)
ToC INTRO TOOLS EVOLUTION OF SETUP WORKFLOW CONCLUSION
ToC INTRO TOOLS EVOLUTION OF SETUP WORKFLOW CONCLUSION
ToC INTRO TOOLS EVOLUTION OF SETUP WORKFLOW CONCLUSION
ToC INTRO TOOLS EVOLUTION OF SETUP WORKFLOW CONCLUSION
GIT HOOKS
ToC INTRO TOOLS EVOLUTION OF SETUP WORKFLOW CONCLUSION
JENKINS TESTS
ToC INTRO TOOLS EVOLUTION OF SETUP WORKFLOW CONCLUSION
PUPPET-LINT
ToC INTRO TOOLS EVOLUTION OF SETUP WORKFLOW CONCLUSION
JENKINS - PUPPET-CATALOG-TEST
ToC INTRO TOOLS EVOLUTION OF SETUP WORKFLOW CONCLUSION
JENKINS BUILD
ToC INTRO TOOLS EVOLUTION OF SETUP WORKFLOW CONCLUSION
JENKINS - MATTERMOST
ToC INTRO TOOLS EVOLUTION OF SETUP WORKFLOW CONCLUSION
DEPLOY
ToC INTRO TOOLS EVOLUTION OF SETUP WORKFLOW CONCLUSION
PUPPET-HPC SCRIPT
ToC INTRO TOOLS EVOLUTION OF SETUP WORKFLOW CONCLUSION
DEPLOY MULTIPLE
ToC INTRO TOOLS EVOLUTION OF SETUP WORKFLOW CONCLUSION
CONCLUSION
DisadvantagesI Slower than file based setupI Logical mistakes still happenI Learning curve
ToC INTRO TOOLS EVOLUTION OF SETUP WORKFLOW CONCLUSION
AdvantagesI Lot of errors caught during testingI Versioning (rpms)I Reproducible builds / codeI (More) predictable outcomeI Puppet code as documentation
ToC INTRO TOOLS EVOLUTION OF SETUP WORKFLOW CONCLUSION
FutureI Spec tests (serverspec,...)I Production branchI Code reviewI ...
ToC INTRO TOOLS EVOLUTION OF SETUP WORKFLOW CONCLUSION
I Step by stepI Fail early & often (before breaking too much ;) )I Small improvementsI Version controlI LintI Suitable solutionI ...
ToC INTRO TOOLS EVOLUTION OF SETUP WORKFLOW CONCLUSION
REFSI http://puppet-lint.comI http://serverspec.orgI http://www.kuwata-lab.com/kwalifyI https://about.mattermost.comI https://git-scm.comI https://github.com/42wim/matterstuffI https://github.com/cea-hpc/clustershell:I https://github.com/drwahl/puppet-git-hooksI https://github.com/github/octocatalog-diffI https://github.com/grondo/pdshI https://github.com/invadersmustdie/puppet-catalog-testI https://github.com/voxpupuli/librarian-puppetI https://jenkins.ioI https://puppet.comI https://www.theforeman.org
