Embed Size (px)
Citation preview
STEVE SMITH • DEVOPS ADVOCATE • ATLASSIAN • @TARKASTEVE
A practical introduction to DockerCode: bitbucket.org/ssmith/devweek15-code
Warning:bad analogies coming up…
VMs vs ContainersHuffington Post
Virtual Machine
Virtual HW
BIOS/EFI
Guest FS / Kernel
Applications
Physical Hardware
Host FS / Kernel
VM Hypervisor
BIOS/EFI
Virtual HW
BIOS/EFI
Guest FS / Kernel
Applications
Containers / Docker
Physical Hardware
Host FS / Kernel
DockerEngine
BIOS/EFI
Guest FS
Application
Guest FS
Application Virtual HW
BIOS/EFI
Guest FS / Kernel
Applications
Physical Hardware
Host FS / Kernel
VM Hypervisor
BIOS/EFI
Virtual HW
BIOS/EFI
Guest FS / Kernel
Applications
Refresher: chroot
/
/etc/var /usr
/var/nginx -> /
/usr
Chroot: Process only sees this
/var /etc
IO and process accounting / limits
2
3
Containers take this further…
Network abstraction
1 Process isolation
Container lifecycle management
2
3
Docker takes this to the next level…
Packaged/sharable containers (via Hub)
1 Container build tools
Cross-container communication4
Cross-host dev/test support (boot2docker)5
Dev/Test: Boot2Docker
Virtual Hardware
Host FS / Kernel
BIOS/EFI
Guest FS
Application
Guest FS
Application
Quick Demo
Peter Kastner
Docker Concepts
Carlsberg
Compiled to images
2
3
Docker containers are like OO classes…
Single inheritance of other images
1 Source for an image is in ‘Dockerfile'
Linking is (manual) dependency injection4
Tools for declarative injection (like Spring)5
Dockerfile ~= SourcefileFROM ubuntu:trusty
RUN apt-get update && \ apt-get install -y nginx && \ apt-get clean
EXPOSE 80 EXPOSE 443
CMD nginx -g daemon off;
Single inheritance
Add member
Override parent behaviour
Exposed linking points
‘build’ compiles to binary[ssmith] $ docker build -q --tag=mynginx . Sending build context to Docker daemon 12.72 MB Sending build context to Docker daemon Step 0 : FROM ubuntu:trusty ---> 8eaa4ff06b53 Step 1 : RUN apt-get update && apt-get install -y nginx && apt-get clea ---> Running in f05a4a034fe3 ---> 1e84ac75dd9a Removing intermediate container f05a4a034fe3 Step 2 : EXPOSE 80 ---> Running in 77602b8eb9ec ---> 4076f56233ff Removing intermediate container 77602b8eb9ec Step 3 : EXPOSE 443 ---> Running in b627b8c1d328 ---> f5a3ff140964 Removing intermediate container b627b8c1d328 Step 4 : CMD nginx -g daemon off; ---> Running in 132573fe4fc9 ---> c1df0408cf70 Removing intermediate container 132573fe4fc9
Inheritance resolved at runtime
Ubuntu
Add nginx
Add run command
(sorta like JVM class assembly)
Union Filesystem - AUFS - BTRFS - Devicemapper
Inspecting the stack
[ssmith] $ docker history mynginx IMAGE CREATED CREATED BY 606f0f611cb2 6 days ago /bin/sh -c #(nop) CMD [nginx -g daemon off;] 27d0fad8620d 6 days ago /bin/sh -c #(nop) EXPOSE map[443/tcp:{}] dc26b50e9806 6 days ago /bin/sh -c #(nop) EXPOSE map[80/tcp:{}] 7a04b014a0a7 6 days ago /bin/sh -c apt-get update && apt-get install b39b81afc8ca 9 days ago /bin/sh -c #(nop) CMD [/bin/bash] 615c102e2290 9 days ago /bin/sh -c sed -i 's/^#\s*\(deb.*universe\)$/ 837339b91538 9 days ago /bin/sh -c echo '#!/bin/sh' > /usr/sbin/polic 53f858aaaf03 9 days ago /bin/sh -c #(nop) ADD file:ca5b63647c6b7a419e 511136ea3c5a 19 months ago
‘run’ instantiates a new container
Ubuntu
Add nginx
Add run command
Writable
Immutable
(kinda like ‘new’)
Runtime layer
Linked containers(dependency injection; sort-of…)
Docker host
postgresql client
--link postgresql:pg
can refer to hostname pg in commands
Volumes
Docker host
/var/volume1
DATA
/var/volume2
/var/volume1
client1
/var/volume2
--volumes-from DATA
(haven’t got a class analogy for this one)
Develop and test with Docker
The example project
?
(See http://bit.do/postgres-es for details)
The example project
Transfer
Trigger / Async
Data
Data
(See http://bit.do/postgres-es for details)
Docker?
2
3
How to test…
Install Postgres/Clojure/ES everywhere?
1 Virtualbox/Vagrant?
Idempotent; create/destroy containers
2
3
Docker advantages
The tested container can be deployed
1 Reuse existing images from hub/registry
Container startup is faster than VM4
Dev/CI test environments are the same5
Creating our images
Elasticsearch DockerfileFROM java:openjdk-7-jre
ENV ES_VER 1.3.4 ENV ES_URL https://download.elasticsearch.org/elasticsearch/elasticsearch/elasticsearch-${ES_VER}.tar.gz
WORKDIR /opt/
RUN adduser --system elasticsearch
RUN curl ${ES_URL} -o elasticsearch.tgz && \ tar xzf elasticsearch.tgz && \ ln -s elasticsearch-${ES_VER} elasticsearch && \ mkdir /opt/elasticsearch/data/ && chown elasticsearch /opt/elasticsearch/data/ && \ mkdir /opt/elasticsearch/logs/ && chown elasticsearch /opt/elasticsearch/logs/ && \ rm elasticsearch.tgz
EXPOSE 9200 EXPOSE 9300
USER elasticsearch CMD /opt/elasticsearch/bin/elasticsearch -f
Elasticsearch build[ssmith] docker build -q --tag=elasticsearch docker/elasticsearch/ Sending build context to Docker daemon 2.56 kB Sending build context to Docker daemon Step 0 : FROM java:openjdk-7-jre ---> b797b3ba124d Step 1 : ENV ES_VER 1.3.4 ---> Running in 4842001c48d0 ---> 2717f23208b7 Removing intermediate container 4842001c48d0 Step 2 : ENV ES_URL https://download.elasticsearch.org/elasticsearch/elasticsearch/elasticsearch-${ES_VER}.tar.gz ---> Running in b2f574153a18 ---> a292b608b854 Removing intermediate container b2f574153a18 Step 3 : WORKDIR /opt/ ---> Running in 0f138382e071 ---> aa2ebf2e8061 Removing intermediate container 0f138382e071 Step 4 : RUN adduser --system elasticsearch ---> Running in 0800878ae8c0 ---> 2b0c2619f328 Removing intermediate container 0800878ae8c0 Step 5 : RUN curl ${ES_URL} -o elasticsearch.tgz && tar xzf elasticsearch.tgz && ln -s elasticsearch-${ES_VER} elasticsearch && mkdir /opt/elasticsearch/data/ && chown elasticsearch /opt/elasticsearch/data/ && mkdir /opt/elasticsearch/logs/ && chown elasticsearch /opt/elasticsearch/logs/ && rm elasticsearch.tgz
Postgresql DockerfileFROM postgres:9.2
ENV POSTGRES_USER customer
COPY testschema.sql / COPY init-test-db.sh /docker-entrypoint-initdb.d/
Postgresql init file#!/bin/bash
echo "#### Create test DB ####"
gosu postgres postgres --single <<EOF CREATE USER customer PASSWORD 'customer'; CREATE DATABASE customer OWNER customer; GRANT ALL PRIVILEGES ON DATABASE customer TO customer; EOF
echo "#### Initialise test DB ####"
gosu postgres postgres --single customer -j < /testschema.sql
echo echo "#### Test DB initialised ####"
Postgresql build[ssmith] docker build -q --tag=postgres docker/postgres/ Sending build context to Docker daemon 5.12 kB Sending build context to Docker daemon Step 0 : FROM postgres:9.2 ---> 898ba6de4072 Step 1 : ENV POSTGRES_USER customer ---> Running in 932957e34d5e ---> 91b679d67fd9 Removing intermediate container 932957e34d5e Step 2 : COPY testschema.sql / ---> 65b67419fe54 Removing intermediate container 54298baaa088 Step 3 : COPY init-test-db.sh /docker-entrypoint-initdb.d/ ---> b6d84236c1f7 Removing intermediate container f9000db9da92 Successfully built b6d84236c1f7
Transfer DockerfileFROM java:openjdk-7-jre
# Install netcat for waitport RUN apt-get update && \ apt-get install -y netcat-openbsd && \ apt-get clean
COPY docker/waitport /usr/local/bin/
RUN curl https://raw.githubusercontent.com/technomancy/leiningen/stable/bin/lein -o /usr/local/bin/lein && \ chmod a+rx /usr/local/bin/lein
RUN adduser --disabled-password --gecos '' transfer ADD . /code RUN chown -R transfer.transfer /code
USER transfer WORKDIR /code
RUN lein deps
CMD waitport elasticsearch 9200 && \ waitport postgres 5432 && \ lein test-out junit
Running tests manually
Manual run[ssmith] docker run -d —name=postgres postgres 181470ed78836c9285664c18f0d5b4c6a9069d28c4f71b9d621d4c24762ad938
[ssmith] docker run -d --name=elasticsearch elasticsearch 5718e1ba9b7caf0e25f53d803a128d2a3d4d518ae8d747b843cc2cbdd78620ce
[smith] docker run --link postgres:postgres --link elasticsearch:elasticsearch transfer Waiting for TCP connection to elasticsearch:9200...OK Waiting for TCP connection to postgres:5432...OK Received notification for accountupdate for 55d3290b-5baa-4ca6-b353-cacf5b104c60 Performing update of 55d3290b-5baa-4ca6-b353-cacf5b104c60 Indexing row 55d3290b-5baa-4ca6-b353-cacf5b104c60 Mr. 55d3290b 55d3290b Esq. Dispatch complete
Manual cleanup[ssmith] docker ps CONTAINER ID IMAGE COMMAND CREATED … 5718e1ba9b7c elasticsearch:latest "/bin/sh -c '/opt/el 5 minutes ago … 181470ed7883 postgres:latest "/docker-entrypoint. 6 minutes ago …
[ssmith] docker kill elasticsearch postgres elasticsearch postgres
[ssmith] docker ps CONTAINER ID IMAGE COMMAND CREATED …
[ssmith] docker ps -a CONTAINER ID IMAGE COMMAND CREATED ff6e5409d8ba transfer:latest "/bin/sh -c 'waitpor 11 minutes ago 5718e1ba9b7c elasticsearch:latest "/bin/sh -c '/opt/el 11 minutes ago 181470ed7883 postgres:latest "/docker-entrypoint. 11 minutes ago
[smith] docker rm transfer postgres elasticsearch transfer postgres elasticsearch
Simplifying with Fig
Fig automates Docker
Can build images locally
2
3
Fig
Can fetch existing images
1 Declarative image orchestration
Configures images for run4
Link images together during run5
Our fig.ymltransfer: build: . volumes: - .:/code links: - postgres - elasticsearch
postgres: build: docker/postgres
elasticsearch: build: docker/elasticsearch
Fig Run[ssmith] fig up Creating devweek15code_postgres_1... Creating devweek15code_elasticsearch_1... Creating devweek15code_transfer_1... Attaching to devweek15code_postgres_1, devweek15code_elasticsearch_1, devweek15code_transfer_1 postgres_1 | The files belonging to this database system will be owned by user "postgres". postgres_1 | This user must also own the server process.
<SNIP> <SNIP> <SNIP> <SNIP> <SNIP>
transfer_1 | Waiting for TCP connection to postgres:5432...OK transfer_1 | Received notification for accountupdate for dcf286bf-7011-40f5-abb0-153d94acde transfer_1 | Performing update of dcf286bf-7011-40f5-abb0-153d94acde69 transfer_1 | Indexing row dcf286bf-7011-40f5-abb0-153d94acde69 Mr. dcf286bf dcf286bf Esq. elasticsearch_1 | creating index, cause [auto(index api)], shards [5]/[1], mappings [] elasticsearch_1 | update_mapping [account] (dynamic) transfer_1 | Dispatch complete devweek15code_transfer_1 exited with code 0 Gracefully stopping... (press Ctrl+C again to force) Stopping devweek15code_elasticsearch_1... Stopping devweek15code_postgres_1...
Fig Cleanup[ssmith] fig ps Name Command State Ports -------------------------------------------------------------------------------- devweek15code_elasticsearch_1 /bin/sh -c /opt/elasticsea ... Exit -1 devweek15code_postgres_1 /docker-entrypoint.sh postgres Exit 0 devweek15code_transfer_1 /bin/sh -c waitport elasti ... Exit 0
[ssmith] fig rm —force Going to remove devweek15code_transfer_1, devweek15code_elasticsearch_1, devweek15code_postgres_1 Removing devweek15code_postgres_1... Removing devweek15code_elasticsearch_1... Removing devweek15code_transfer_1…
[ssmith] fig ps Name Command State Ports -------------------------------------------------------------------------------- [ssmith]
Continuous Integration
Continuous IntegrationServer Agent
Continuous Deployment
Continuous DeploymentAgentServer
Caveats
There is no silver bullet
Docker is changing… rapidly
http://fig.sh
2
3
Useful Links
https://www.docker.com/subscribe_newsletter/
1 https://www.docker.com/
http://boot2docker.io/4
http://www.nkode.io/2014/08/24/valuable-docker-links.html5
STEVE SMITH • DEVOPS ADVOCATE • ATLASSIAN • @TARKASTEVE
Thank you!
