Embed Size (px)
Citation preview
1 / 61
Docker:automation forthe rest of us
2 / 61
OutlineWho I am
What I do
How Docker is helping
3 / 61
Who am I?
(And why am I here?)
4 / 61
Jérôme Petazzoni@jpetazzo
Tamer of Unicorns and Tinkerer Extraordinaire*
Grumpy French DevOps
Dislikes: repetitive tasks
Likes: shell scripts
"Go Away Or I Will Replace You Wiz Le Very Small Shell Script!"
* At least one of those is actually on my business card
5 / 61
What do I do?
6 / 61
I am tech support
7 / 61
I am tech support
8 / 61
I am tech support ... for a PaaS!Every day a new kind of fun!
Monday: Node.js
Tuesday: Python
Wednesday: Ruby
Thursday: Java
Friday: PHP
9 / 61
Lessons learnedLots of support requests are: "How do I do X with your product?"
Good documentation → fewer tech support requests
"Give a man a fish and you feed him for a day; teach a man to fish and you feed him for a lifetime."
"Give someone good support and you help them today; write some docs and you help everybody forever."
Unless your business model relies on a product that youcan't install unless you have a PhD or a support contract
10 / 61
I am a technical writerWrote 60-80% of dotCloud documentation
Wrote 30-40% of Docker training materials
I ♥ to explain things
11 / 61
I am a technical writerWrote 60-80% of dotCloud documentation
Wrote 30-40% of Docker training materials
I ♥ to explain things
Documentation is part of your product
It's NOT the least important part
We're talking about technical products here. Don't brag because youwere able to place a phone call without reading the user manual.
12 / 61
I am a developer
13 / 61
I am a developer
14 / 61
I am a sysadmin
16 / 61
I am a sysadmin opsServer whisperer
Fixes other people's ʇᴉɥs
On-call
17 / 61
Public Service AdvisoryNever, ever be proud of being on-call
Never, ever be on-call for free
18 / 61
Public Service AdvisoryNever, ever be proud of being on-call
Never, ever be on-call for free
You're harming yourself
19 / 61
Public Service AdvisoryNever, ever be proud of being on-call
Never, ever be on-call for free
You're harming yourself
You're harming all of us
20 / 61
Public Service AdvisoryNever, ever be proud of being on-call
Never, ever be on-call for free
You're harming yourself
You're harming all of us
Make sure people are aware of the issues
21 / 61
I am an ImpostorImpostor syndrome, anyone?
Am I really qualified to do this?
What the F am I doing here?
Affects women more than men
This can lead to burn-out and sleep deprivation
This is more important than you think
22 / 61
What's Docker?
23 / 61
an open platform
to build, ship, and run
any app, anywhere
24 / 61
What can Dockerdo for me?
25 / 61
Some challenges with tech support"Works For Me"
"Can't reproduce"
"I would appreciate if you could test between 3 and 4am"
"So to trigger the bug you have to install X and Y thenconfigure A, B, and C, then download the extra file, put it inthis directory (which doesn't exists?!?) and then if yourestart three times in approximatively 5 minutes butsometimes it takes longer you will see that the images areshifted by a few pixels but if it doesn't work try to upgradeY to version Z and try all over again..."
26 / 61
Docker for tech supportGet a well-defined, reproducible environment
Define this environment in a Dockerfile
Build this Dockerfile into a container image
Run this container image anywhere
Same behavior, guaranteed (or your Open Source money back)
27 / 61
Dockerfile
FROM ubuntu:14.04RUN apt-get updateRUN apt-get install -y libx liby git wgetRUN git clone git://github.com/a/b/cRUN mkdir -p /extra/dirRUN wget -O /extra/dir http://extra/fileCMD start-service & sleep 60; \ stop-service; start-service; sleep 60; \ stop-service; start-service; sleep 60; \ start-service
$ docker build -t pixelbug ....$ docker run pixelbug...
Automate those repetitive, unreliable tasks yourself
28 / 61
Some challenges for tech writersWe use sphinx, showoff, LaTeX ...
"To see what the final documentation will look like, you need Ruby, 45 libraries, a custom fork of this project, and a philosopher's stone."
Insert scary anecdote about API documentations
Result:
tech writers can't see what they're doing
other team members won't/can't contribute to the docs
29 / 61
Docker for tech writersFROM ubuntu:14.04MAINTAINER Education Team at Docker <[email protected]>
RUN apt-get updateRUN apt-get install -y curl wget git ruby ruby-dev libxml2-dev libxslt-dev \ build-essential zlib1g-dev
RUN git clone https://github.com/puppetlabs/showoff.gitWORKDIR /showoffRUN gem build showoff.gemspecRUN gem install --no-rdoc --no-ri ./showoff-*.gem
# Let's install prince. The first dpkg will fail because of missing dependencies,# so we'll install the dependencies with "install -f" then try again.WORKDIR /usr/srcRUN wget http://www.princexml.com/download/prince_9.0-5_ubuntu14.04_amd64.debRUN dpkg -i prince_9.0-5_ubuntu14.04_amd64.deb || trueRUN apt-get install -fyRUN dpkg -i prince_9.0-5_ubuntu14.04_amd64.deb
COPY /slides/ /slides/WORKDIR /slides
CMD [ "showoff", "serve" ]
EXPOSE 9090
30 / 61
Docker for tech writersWith this Dockerfile, anyone can run our "doc pipelines"
Results are consistent, no more: "you forgot to install 字形 so the output is different!"
Less wasted time for onboarding, upgrades, reinstalls...
Automate those repetitive, difficult tasks yourself
31 / 61
Some challenges for developersSet up this Ruby + Postgres + Mongo + Cassandra stack
Make sure that all team members have the same env
Have consistent library versions between dev and prod
Basically, the same challenges as before, but worse
32 / 61
Docker for developersWrite Dockerfiles for each component
Put components together with Fig/Compose
Profit!
33 / 61
Describing a complex stackweb: build: src/front links: - redis - postgres:db - api - zookeeper:zk
redis: image: redis
postgres: image: postgres
api: build: src/backend links: - redis - postgres:db - cassandra - zookeeper:zk
zookeeper: image: jplock/zookeeper
cassandra: image: spotify/cassandra
34 / 61
OnboardingHire developer*
Give them a computer
Install Docker
git clone ...
docker-compose up ...
Your stack is up and running
*Actually the most difficult part.
35 / 61
Cold, hard dataHow long does it take for a developer to join a new project?
Before Docker: 2 days
After Docker: 2 hours
(Source: Worldline)
36 / 61
Docker vs Configuration ManagementQuick poll:
37 / 61
Docker vs Configuration ManagementQuick poll:
who is a dev?
38 / 61
Docker vs Configuration ManagementQuick poll:
who is a dev?
who uses Chef/Puppet/Salt/Ansible/...?
39 / 61
Docker vs Configuration ManagementQuick poll:
who is a dev?
who uses Chef/Puppet/Salt/Ansible/...?
who is happy with it and found it easy to learn?
40 / 61
Docker vs Configuration ManagementQuick poll:
who is a dev?
who uses Chef/Puppet/Salt/Ansible/...?
who is happy with it and found it easy to learn?
CM is good, but learning curve is steep
Docker lets you automate deployment yourself
41 / 61
ProductionPeople keep asking all the time:
Is Docker ready for production?
Can I run Docker in production?
Who runs Docker in production?
42 / 61
ProductionPeople keep asking all the time:
Is Docker ready for production?
Can I run Docker in production?
Who runs Docker in production?
Why don't you ask the same questions for:
Eclipse
SublimeText
Vagrant
43 / 61
Repeat after me:
It is OK to use Docker in dev, even without going to prod.
44 / 61
— But can I...
— No! Later. Thanks.
45 / 61
Some challenges for opsHow do I reduce the pain of deployment?
How do I scale up and down?
How do I move apps from colo to cloud and vice versa?
How do I use my resources efficiently? (i.e. without wasting CPU, RAM, disk)
46 / 61
Deployment painDevelopers send a tarball* Friday evening
Deployment deadline is Monday morning
Installation instructions are not up to date
Dependencies are incompletely specified
External services are hard-coded all over the place
They use Debian, we use CentOS
* Or a git hash, in more sophisticated places.
47 / 61
Deployment with DockerDevelopers ship a container image (or a repo with a Dockerfile)
Configuration is done through environment variables*
External dependencies are expressed with ambassadors (e.g. the redis server address is redis, and Docker takescare of setting up a proper DNS entry in the container)
The dev-to-prod pipeline was already tested by the devs, when they did the dev-to-test deployment
* See also: twelve-factor app principles.
48 / 61
Scaling upwww7 needs to be exactly like www[1-6]
Possible solutions:
Configuration Management (steep learning curve, not a silver bullet)
Golden Images (AMI...) (slow, especially for small changes)
Server Cloning (can affect existing server; requires manual touch-ups)
Docker
49 / 61
Scaling up with DockerCreate new Docker host (with e.g. docker-machine)
Deploy application (SSH + docker run, or with remote API)
Done!
(You still need to update load balancers etc., but you have todo that even without Docker anyway.)
50 / 61
From colo to cloud to coloMust make sure that servers are absolutely identical
Possible solutions:
Configuration Management
Tedious, manual work
It would be nice if we had a self-contained meta-packageholding our application and all its dependencies, all theway down to the OS!
51 / 61
From colo to cloud to coloMust make sure that servers are absolutely identical
Possible solutions:
Configuration Management
Tedious, manual work
It would be nice if we had a self-contained meta-packageholding our application and all its dependencies, all theway down to the OS!
Guess what, that's exactly what Docker does. Cool!
(You can see Docker images as "super-debs" or "super-rpms".)
52 / 61
Optimize resource usage (problem 1)You have:
5 hypervisors (physical machines)
Each server has:
16 GB RAM, 8 cores, 1 TB disk
Each week, your team asks:
one VM with X RAM, Y CPU, Z disk
Difficulty: easy
53 / 61
Optimize resource usage (problem 2)You have:
1000+ hypervisors (and counting!)
Each server has different resources:
8-500 GB of RAM, 4-64 cores, 1-100 TB disk
Multiple times a day, a different team asks for:
up to 50 VMs with different characteristics
Difficulty: ???
54 / 61
MesosGeneric solution to resource usage problem
Open Source project (Apache Foundation)
In production use at Twitter, AirBNB, eBay, ...
Runs your code but doesn't deploy or distribute it
How can we distribute our code on our nodes?
55 / 61
Mesos + DockerPrepare your workload in a container image
Test it locally or on a smaller cluster
Submit it to Mesos
Mesos picks the right nodes to run your workload
Docker downloads the container images and runs them
56 / 61
Mesos + Docker + SwarmMesos is complicated-ish to deploy
Mesos is complicated-ish to manage
Mesos is complicated-ish to use
Docker Swarm can expose a cluster* with the Docker API
Swarm lets you use a cluster without learning a new tool (Deployment and management are still a challenge) (Damn! Where is my free lunch!)
* Currently a vanilla Docker cluster, soon a Mesos cluster
57 / 61
SummaryI am a tech support engineer: Docker helps me to reproduce issues
I am a technical writer: Docker helps me to render my docs
I am a developer: Docker helps me to abstract environments, expressdependencies, be operational faster
I am a sysadmin: Docker helps me to deploy, scale, orchestrate
58 / 61
How much does it cost?
59 / 61
How much does it cost?The Docker Engine is Open Source
The Docker Registry is Open Source
I'm not here to sell anything
(But if you insist, we have commercial products, support, and all that stuff!)
60 / 61
