Embed Size (px)
Citation preview
Python tooling for continuousPython tooling for continuousdeploymentdeployment
PyParis Novembre 2018PyParis Novembre 2018Arthur Lutz - LogilabArthur Lutz - Logilab
1
Introduction /meIntroduction /meArthur Lutz
/
@arthurlutz
Logilab
Python Nantes Paris Salt Meetup
2
IntroductionIntroductionHow we benefited from a rich
Python ecosystem to move fromsprint-based delivery to continuous
delivery.
3
Continous deliveryContinous delivery
Check in
Check in
Check in
Trigger
Trigger
Trigger
Trigger
Trigger
Feedback
Feedback
Feedback
Feedback
Feedback
Feedback
Approval
Approval
Delivery team Version control Build & unittests
Automatedacceptance tests
User acceptancetests
Release
Source: wikipedia4
Python StackPython Stack(ignore the order)
View it on StackShare5
Reproducible environmentReproducible environment using volumes for dev parts (equivalent
to pip install -e) to set up postgres, redis,
celery, etc. using docker-compose.override.yml and then .envsame docker images for acceptancetesting, pre-production and productionwith configuration via environment variables
docker
docker-compose
6
Test often, break, test againTest often, break, test again
- reproducible test envs - test runner / - functionnal testing (launch daemons in
pytest fixtures)
Tools used in dev, CI/test andsupervision.
toxpytestflake8 isortpifpaf
7
Ramping up CIRamping up CI1. draft-based2. parametric (manual)3. public changeset (hg phases) + user-centric4. component-centric5. feature-centric jenkins (not python) with phabricator + jenkins = and JenkinsFile
jenkins-job-builderDifferential jenkins job
8
Acceptance test often, break,Acceptance test often, break,test againtest again
BDD with / behave robberselenium python bindings
9
Push, decrease quality, pushPush, decrease quality, pushagainagain
Jenkins test metrics and coverageSonarQube (not python) with Peer review with + (amend) +
(phabsend)
Continuous Quality Control
SonarPythonmercurial evolve
phabricator mercurial extension
10
Ship often, break, ship againShip often, break, ship againrundeck (not python) & for shareddeploy logsgenerate unique version informationtag for validation the shipped Jira tickets
& for continuous collectionof errors
rundeckrun
sentry raven-python
11
Ship often, measure, ship againShip often, measure, ship againcollect metrics with and serve with
collecting custom metrics see
carbongraphite-api
cfmgmtcamp:Use Saltstack to deploy a full monitoring and supervision stack
12
Agile infrastructureAgile infrastructure infrastructure as code, deploy multiple
environments portable cloud scaling for in-house cloud hosting
used for example to restore productiondatabase snapshots in pre-production
saltstack
salt-cloudopenstackawscli
13
Agile infrastructure -Agile infrastructure -supervision and metricssupervision and metrics
sensu (not python) supervision using , ease of deploy of new checks
netdata supervision and metrics with
to test out complex scenarios
test infra
sensu-formula
python.d.plugintestinfra
14
Dashboards everywhereDashboards everywhere to extract data from Jira
to extract data from RunDeckBadges everywhere! (notpython)grafana (not python) serving data from
Jira python bindingsrequests
https://shields.io
graphite
15
Project impactsProject impactsshorter tickets, split them into tiny chunksunstable environments can be OK - work withthe usersfeature flags to encourage early integration ofcodemore peer review, collective appropriationno more semantic versionning
16
FutureFutureMerge request / feature stack centric generatedenvironmentsAutomation of data migration betweenenvironmentsUse of version control in sentry and regressionidentificationOpenShift / OKD / Kubernetes type continuousdeploymentDev environments could be less necessary
17
Logilab is hiring !Logilab is hiring !
18
Conclusion / QuestionsConclusion / QuestionsThanks for your attentionAny questions ?Slides (full of links !) : , html pdf
19
