Embed Size (px)
Citation preview
Scaling Your Jenkins Master with Docker
London, UK 23-24 June
#jenkinsconf
Contents
● WHO: Wyplay
● WHAT: Docker, Docker in Jenkins, Jenkins in Docker
● WHY: Situation Before, Issues
● HOW: Setup, Usage
● WHERE: Future
2
#jenkinsconf
Contents
● WHO: Wyplay
● WHAT: Docker, Docker in Jenkins, Jenkins in Docker
● WHY: Situation Before, Issues
● HOW: Setup, Usage
● WHERE: Future
3
#jenkinsconf
WHO am I
I Worked in Queueing Networks, Simulation, Smartcards, now TV Software for Operators
– Develop Set-top-box OS (Canal+, Proximus, SkyIt,..)
– 80% of Developers
4
#jenkinsconf
Contents
● WHO: Wyplay
● WHAT: Docker, Docker in Jenkins, Jenkins in Docker
● WHY: Situation Before, Issues
● HOW: Setup, Usage
● WHERE: Future
5
#jenkinsconf
WHAT: Docker
● Company (Docker Inc)
● Open Source (Apache license) platform for building, shipping, running distributed applications
● Technology: manage and run lightweight Linux containers
– Isolation
– Portability, Shipping
6
#jenkinsconf
WHAT: Docker in Jenkins7
● Docker Plugin
– Dynamically provisionslaves
– Run build
– Tear-down slave
● Slaves in the Cloud
#jenkinsconf
WHAT: Jenkins in Docker
● Official Jenkins LTSdocker image
– From Cloudbees, sponsorof DockerCon
– Open Source
https://github.com/jenkinsci/docker
– Easy to usedocker pull jenkins
docker run -p 8080:8080 jenkins
8
#jenkinsconf
Contents
● WHO: Wyplay
● WHAT: Docker, Docker in Jenkins, Jenkins in Docker
● WHY: Situation Before, Issues
● HOW: Setup, Usage
● WHERE: Future
9
#jenkinsconf
WHY: Situation Before10
● One Master
– 34 attached slaves
– 527 jobs
– 78 plugins
– admin users
● Fast-paced environment. No real control
#jenkinsconf
WHY: Issues11
● Performance
● Reliability
● Inability to upgrade
● Downtime
=> Projects spin-off masters
● Security
#jenkinsconf
Contents
● WHO: Wyplay
● WHAT: Docker, Docker in Jenkins, Jenkins in Docker
● WHY: Situation Before, Issues
● HOW: Setup, Usage
● WHERE: Future
12
#jenkinsconf
HOW: Setup for one master (1)
● Three possibilities for /var/jenkins_home
– inside the container:
docker run -p 8080:8080 jenkins
– mounted from a host directory:
docker run -v <host-path>:/var/jenkins_home jenkins
– mounted from a data volume container:
docker run -v /var/jenkins_home --name=data busybox true
docker run --volumes-from=data jenkins
13
#jenkinsconf
HOW: Setup for one master (2)
● Summary for one master:
– pull the official Jenkins image:
$ docker pull jenkins
– create the data volume and master containers:
$ docker run -v /var/jenkins_home --name=frog-data busybox true
– run the master container:
$ docker run -t -i -u root -p 8084:8080 -p 50004:50004 --volumes-from=frog-data --name=frog jenkins
14
#jenkinsconf
HOW: Setup n masters
● Launch an httpd dispatcher (e.g., Apache)
– 443 (https) → vhosts(name) → 808i
● Launch docker containers:
– the Data Volume containers, with names
– the Jenkins masters containers linked with --volumes-from and 808i / 5000i ports
– map each container port 8080 to host ports 8080+i
15
#jenkinsconf
HOW: Usage16
PressedJson
DRAFT
Apache
cert.crtprivat.key
frog-data
polka-data
frog
PressedJson
DRAFT
pressed.json
LIVE
PressedJson
DRAFT
PressedJson
DRAFT
polka
PressedJson
DRAFT
PressedJson
DRAFT
pressed.json
LIVE
PressedJson
DRAFT
PressedJson
DRAFT
Frog.confServerNamefrog-ci.comProxyPasslocalh:8084X-F-Port443
g6-data
tools-data
443
8084
8080
8080
808i
--volumes -from
--volumes -from
50004
Slaves
#jenkinsconf
HOW: Admin Usage (1)
● Install plugins (from /tmp/plugins)
$ docker run -t -i --rm --volumes-from=frog-data -v /tmp:/tmp ubuntu bashroot@1a210c0a1846:/# cd /var/jenkins_home/pluginsroot@1a210c0a1846:/# cp /tmp/plugins/* .root@1a210c0a1846:/# exit
● Similar technique for:
– full backup
– debug data (SCM Sync git repos, Plots csv, etc.)
17
#jenkinsconf
HOW: Admin Usage (2)
● Change image
– stop container, destroy, create a new one
$ docker stop frog$ docker rm frog$ docker run … --volumes-from=frog-data --name frog
$ docker logs -f frog
18
#jenkinsconf
HOW: Admin Usage (3)
● Three levels of backup
– Vmware: dump of entire docker server
– Docker: copy all Data Volume
– Jenkins: SCM Sync plugin
● Three levels of monitoring
– Vmware, Docker, Jenkins
● Security: Role Strategy plugin
19
#jenkinsconf
Contents
● WHO: Wyplay
● WHAT: Docker, Docker in Jenkins, Jenkins in Docker
● WHY: Situation Before, Issues
● HOW: Setup, Usage
● WHERE: Future
20
#jenkinsconf
WHERE: do we go from here?
● Orchestration
– manual setup ok for a few
– more => orchestration tool (e.g., Docker Compose)
21
#jenkinsconf
WHERE: do we go from here?22
● Docker vs Config/Automation tools (chef, puppet, ansible, saltstack)
Before: use ansible to setup hardware/VM, install packages, deploy code,run app.
After: use ansible to setup hardware/VM, install Docker, run containers.
use Dockerfiles to install packages, deploy code, run app.
#jenkinsconf
WHERE: do we go from here?
● Monitoring (docker level)
● Find a way to “Configure System” programmatically
– LDAP
– Plugins upgrades
– TCP port for slaves
– Markup formatter
– SCM Sync, Roles, IRC, etc.
23
#jenkinsconf
Thank You@cmuller13 on twitter
24
