Upload
lucas-jellema
View
1.163
Download
1
Embed Size (px)
Citation preview
Lucas Jellema
AMIS Platform SIG - 1 October 2015
Introduction to automated environment management with Docker Containers - for platform specialists and developers
2
Overview
3
Overview
• Docker – what is behind the hype?• A grasp on terminology such as Container, Build, Ship and Run
– Images, registry, micro services, • Enough to participate in the discussion• How to get started and going with Docker
– On Windows (or Mac OS)• What can Docker mean to me
– In my personal R&D environment– In my daily work situation
• What’s Oracle’s position regarding Docker?• What is Docker’s current status and near by future• Handson resulting in
– A Docker enabled environment– Several containers running (Ubuntu, Wordpress, Oracle 11g XE DB, JDeveloper)– A stepping stone for creating additionals containers and interdependencies
4
What can Docker give us (developers/platform specialists)
• An efficient way – in terms of work and physical machine resources – to rapidly create, easily manage isolated environments– That can be created and deleted easily– That start and stop rapidly– That can be exchanged simply– That consume minimal system resources
• Use cases:– Quickly try out some software (release) – without upsetting your laptop– Easily create and distribute a complete training environment– Prepare an environment for deployment to a cloud environment– Locally construct and test a potentially complex environment that is then transfered to
a shared infrastructure– Implement a micro services infrastructure– Emulate/Test complex clusters on a single host
5
Docker - Run
6
Demo – Run our first container
dockerhost
port 80
IP 172.17.0.6
7
Demo – Run a second container based on same image
8
Demo – two running NGINX containers; port forwarding for one
port 80
port 80
port 90
IP 172.17.0.7
IP 172.17.0.6
9
Demo: run container in interactive mode using shell
dockerhost
/tmp/mynewfile.txt
10
Demo: restart container and attach to locate my file
dockerhost
/tmp/mynewfile.txt
11
Final status demo: Running Multiple Containers in a Single Linux Host
port 90 port 80
port 80
/tmp/mynewfile.txt
/var/lib/docker/aufs/diff/… 6af69dd33af4…/tmp/mynewfile.txt
12
Run
• Docker Container runs Linux – as does the host• Container is isolated - feels as
stand alone run time environment– Directory structure, IP address, users and groups
• Shared resources with underlying host (and therefore other containers)– memory, CPU, host
• Light weight:– Quick starting up and stopping– Leverages underlying Linux, only adds what is different/additional
• Clusters of containers– Dynamic adding/removing containers from clusters can be done very quickly (Google)– Especially when containers are stateless – (no shared session state in containers; possibly in joint cache, shared file system or
NoSQL database)• Management tools – to monitor and manage individual containers and
clusters of containers (dynamically scale up/scale down)
Docker HostDocker Container
Docker Container• ip address• directories & files• users & groups• process table
13
When it walks and talks like a Virtual Machine…
• Virtual Machine vs Container
14
Containers vs Virtual Machines
• Containers do not run through a hypervisor– They are logically isolated from the host and each other– They run the same kernel as the host (i.e. Linux!)
• Containers share resources (the kernel) with host– the rest of the OS can be unique per container (namespaces on host in mounts,
process table, network resources)• Containers are based on images defined in a layered file system; all
containers based on the same image share the image definition– Adding a copy-on-write file system layer– 50 containers based on the same image require same file system resources as 1
container – until they start changing or creating files• Containers start, stop, restart in seconds• Containers do not have physical resources (memory, CPU) pre-allocated
– Resources are allocated at run time
15
Docker is a high level API on core Linux Kernel features
• Solaris Zones and Windows Server offer similar isolation capabilities
16
Micro Services
• Architect the application into a set of collaborating services. • Each service implements a set of narrowly, related functions. • The services are elastic, resilient, composable, minimal, and complete.• Services communicate using standard protocols such as HTTP/REST• Services are developed
and deployed independently of one another.
• Each service manages its own state
17
Micro Services
• With Docker, each Micro Service is implemented with a single container– The micro service is not just encapsulated functionality that needs to be deployed
onto some platform (such as an ESB or BPEL engine) – instead it ships complete with the fully configured engine that runs in the
standardized container platform• All you need to run is:
– Start container. Period.
Linux Host + Docker Engine
18
A more complex network of microservices…
19
Micro Services and containers
• Containers can run on the same host– Or on different hosts
• Containers can be run as a single node– Or scale up to run as many nodes
• Containers can easily be replaced by other containers– That can be connected to in the same way
dockerhost
my-little-container
my-tiny-container
my-data-container
http
Simple Docker UI
GUIs for monitoring & managing a Docker runtime environment
21
GUIs for monitoring & managing a Docker runtime environment
• Dockland - Yet another docker web UI– https://github.com/dynport/dockland
• Shipyard – a web UI for Docker– https://github.com/ehazlett/shipyard
• DockerUI – Web interface to interact with the Docker Remote API– https://github.com/crosbymichael/dockerui
• Simple Docker UI– https://github.com/felixgborrego/docker-ui-chrome-app
• Panamax– https://github.com/CenturyLinkLabs/panamax-ui
22
Docker – Ship and Run
23
Demo ShipRun MySQL and Wordpress
24
Running Wordpress instance by pulling two public images
port 8080 port 80
dockerhostvmIP: 10.10.10.29
Docker Hub
25
Commit container as image and push to registry
dockerhostvm
Docker Hub
/tmp/mynewfile.txt
Dockersig-trial:1.0
26
Image published on public Docker Hub registry
27
Run container based on my published image
dockerhostvm
Docker Hub
/tmp/mynewfile.txt
Dockersig-trial:1.0
Dockersig-trial:1.0
28
Containers in the world of physical logistics
29Docker Containers Ship microservice – app, libs, configuration, platformready to run on any Linux Host+Docker
30
Docker Registry
• Images can be published to Public and Private Registries – Docker Hub is the default registry– Docker Hub contains official repositories from many projects and vendors– Private Registries can be created in the cloud and on premises
• Containers can be started from such images
31
Public Docker Registry
Docker Hub
Registry
push
Private Docker Registry
Docker Hub
pull &run
push
pull &run
32
Docker &Cloud
Public Docker Registry
Docker Hub
pull &run
33
How to Ship a Stand-Alone product?
• Create Installers + Configuration Instructions?• Make your product success dependent on platform configuration and OS
settings?
• Ship as a container – everything set up and ready to run!
• For example:
– RubiconRed – Preferred way to deliver their tool MyST: as Docker Container (image)
34
Docker – Build, Ship and Run
35
Demo Build
36
Demo run container after build
37
Demo Build – slightly more advanced Dockerfile
38
Demo – run image created in build process
39
Building a Docker Container
• The Build Steps are prescribed in Dockerfile– Could be called the container recipe
• Docker commands specify: – Start from Base Image– Perform commands such as ADD, COPY, RUN
that modify the starting situation– The Build Context contains the contents of the folder on the Docker host that
contains the Dockerfile (including sub folders)• This can become quite large• During build – files can be downloaded over HTTP using standard wget (but not copied from the host outside
the build context)• All files added to the container become part of its history (and increase the size)
– Specify the default CMD to run when the container is started
– Optionally: tag the container as image, push it to registry and start one or more containers from the new image
40
Building a Docker Container
• Docker files specifies all build steps– With fairly low level commands
• Start from base image - each step in the Docker Script adds a layer
• A layer is– A logical ‘savepoint’ in the container history
• That marks an intermediate ‘image’
– A physical directory somewhere on the Docker Host
Base ImageUbuntu 14.04
FROM Ubuntu:14.04COPYRUNWORKDIRRUNEXPOSECMD
COPY
RUN
RUN
Final Image
41
Run a Docker Container
• When a Docker Container is run based on an image then all these “physical” layers are combined into a “logical” read-only file system– Union file system– Copy-on-write– Like a stack: higher layers may override lower layers
• A writable layer is added for the running container
Base ImageUbuntu 14.04
COPY
RUN
RUN
Base ImageUbuntu 14.04
COPY
RUN
RUN
Writable Layer
run
Base ImageUbuntu 14.04
COPY
RUN
RUN
Writable Layer
run
42
StandardImage,
locally built
Build
• In addition to 10Ks of reusable images to start containers from• There are a zilion Dockerfiles to leverage for building images
– Download script– Add software packages and installers (because of license reasons)
– Tweak the script to fit your own needsOR (preferably)
– Run the script, create a local image and then create your own Docker File that takes this image as its starting point
Your OwnDockerfile
Your TweakedImage
43
“Docker” Search results on GitHub
44
For example: build container for Oracle WebLogic
• Clone from GitHub to Docker host– Dockerfile– Shell scripts– Supporting files
• Download RPMs for– JDK 8– WebLogic 12.1.3
• Docker Build• Optionally use second
Docker file on top ofWebLogic image tocreate a WLS Domain
download and
add to build contextStandard
Oracle WebLogicImage,
locally built
45
Build File for WebLogic
Base ImageOraclelinux:7
RUN
COPY
RUN
COPY
COPY
COPY
46
Build File for WebLogic (2)
Base ImageOraclelinux:7
RUN
COPY
RUN
COPY
COPY
COPY
RUN
RUN
RUN
RUN
RUN
47
Base ImageOraclelinux:7
RUN
COPY
RUN
COPY
COPY
COPY
RUN
RUN
RUN
RUN
RUN
Turn container into image
Base ImageOraclelinux:7
RUN
COPY
RUN
COPY
COPY
COPY
RUN
RUN
RUN
RUN
RUN
docker commit <container-id> weblogic:12.1.3-dev
48
Build File for WebLogic Domain
weblogic:12.1.3-dev
Base ImageOraclelinux:
7
RUNCOPY
RUN
COPYCOPYCOPY
RUNRUNRUNRUNRUN
Base ImageWeblogic:12.1.3-
dev
COPY
RUN
Base ImageOraclelinux:7
RUN
COPY
RUN
COPY
COPY
COPY
RUN
RUN
RUN
RUN
RUN
49
Build File for WebLogic Domain (2)
Base ImageWeblogic:12.1.3-
dev
COPY
RUN
RUN
RUN
Base ImageOraclelinux:7
RUN
COPY
RUN
COPY
COPY
COPY
RUN
RUN
RUN
RUN
RUN
50
Tag container as reusable image
Base ImageWeblogic:12.1.3-
dev
COPY
RUN
RUN
RUN
Base ImageOraclelinux:7
RUN
COPY
RUN
COPY
COPY
COPY
RUN
RUN
RUN
RUN
RUN
docker commit <container-id> weblogic-basedomain:12.1.3
oraclelinux:7
weblogic:12.1.3-dev
weblogic-basedomain:12.1.3
51
Build Microservice based on WebLogic plus Custom App
download custom app EAR
add to build context
Microservice: custom app
oraclelinux:7
weblogic:12.1.3-dev
weblogic-basedomain:12.1.3
WLST deploy EARWLST config JMSWLST config data sources Microservice:
database
52
Automated Configuration Management
• Use of Shell Scripting to create | compose | configure environments is not exactly the latest fashion
• Declarative, automated configuration management tools have us specify what we need and then make that happen– No scripting– Cross platform– Parametrized
– Leveraging public catalogs of environment definitions
Container Build process• Regular Docker Build
– From base image– Add Puppet support– Add Puppet Manifests &
Modules
• Start Container– Optional: Map Volume from
host with large files – Run Puppet to apply Manifests– Stop Container, Tag as Image
dockerhostvm
Dockerfile
my-base-container
/files
/puppet
/files
volu
me
1
Very big files
Advanced Docker Container Build using Puppet
2
3
45 4
21
3/puppet Manifests/Modules
5
54
Notes on Using Puppet with Docker
• After applying Puppet – the container can be stopped, tagged and used as base image for next Docker Build– That could EXPOSE, ENV, CMD or ENTRYPOINT
• With some workarounds, Puppet apply can be made to run during Docker Build (with RUN in Dockerfile)– Less control over build context– No Volume mapping from host
• There are Puppet Modules to use for automating the build pipeline of Docker (leveraging the Docker API)– To install Docker, build container, create and ship an image, run container
• What applies to Puppet by and large applies to similar tools such as Chef, Salt and Ansible
• Puppet Modules are available for many Oracle Database & Fusion Middleware configuration management tasks– Oracle Database (EE, SE, XE)– WebLogic, SOA Suite, OSB, BPM Suite, WLST– JDK, Opatch, VirtualBox, GlassFish, Hudson, Maven
55
Continuous Delivery in the Container Era
• What if development team delivers the complete, runnable environment – instead of fine grained artifacts that need to be deployed to an environment?
56
The Image is Delivered
• Source code is built into testable and deployable units• When build and unit test is complete, a Docker Image is built• This image is the unit of delivery
– It is independent of / unimpeeded by the run time platform
57
Automated Workflow
• Source code commit triggers build, test, ship and deploy of Docker Container– For example: Docker Hub can be hooked into GitHub with build trigger
58
Docker – Concepts in Perspective
Dockerfile Image Docker HubRegistry
Internet
Container
buildpush
pull
run commit
Docker Private
Registrypush
Text fileSize = ~ KBInstallation & config instructions
Composed of many read-only layersTypical size = ~ hundred(s) MBCan have multiple versions (akin Git tags)
Shares the image’s read-only layers1 private writeable layer (copy-on-write)Initial size = 0 bytesCan be stopped, started, paused, etc.
Free public hosting
Source: Sébastien Bonami, IT Engineering Student and David Lauzon, Researcher (DevOps 2015)
59
Docker – internal structure
Source: Sébastien Bonami, IT Engineering Student and David Lauzon, Researcher (DevOps 2015)
Docker Daemon Container 1
Host OS Kernel
Docker Storage Backend Container 2 ...
Hardware
Setups & manage the Linux containers.
Stores the image and container’s data layers locally.
60
Quick Docker Recap
61
Image and Container Specifics
Container
Base ImageUbuntu 14.04
COPY
RUN
RUN
Writable Layerrun
commit
Container “state”
diff
history
tagremoveinspect
create save tar load …
pullregistry
62
Image and Container Specifics
Container
Base ImageUbuntu 14.04
COPY
RUN
RUN
Writable Layerrun
startattach(un)pausekillstoprestartremoveinspect
logs
commit
Container “state”
diff
export tarimport
…
Flattened, no image details
63
Container Details & Operations
Container
web
Containerdblink
docker run –d –it
<image-id> /bin/bash
Containerxxx
link
link
808080
/tmp/filesShared Files
/data
/host_files
Docker storage
/softwareShared Files
/repos/repos
1
2
3
4
1
2
3
4
2
4
–p 8080:80
-v /data -v /tmp/files:/host_files
-volumes-from xxx
--name web --link db:db1 –link xxx:web_xxx
64
Run GUI applications in Docker Container
Container
docker run –d –it
-v /tmp/.X11-unix:/tmp/.X11-unix
-e DISPLAY=$DISPLAY
<image-id> /bin/bash
/tmp/.X11-unix
/tmp/.X11-unix
dockerhost
GUI applications
65
Docker and Windows
• Docker sits on Linux Containers– Windows Server 2016 will have containers too – and Docker will sit on those as well– However, today, Docker does not run on Windows
66
Docker cannot run on Windows- directly, without Linux VM
dockerhost
Container
Container
Container
67
Vagrant to the rescue
dockerhost
Container
Container
Container
68
Vagrant to the rescue
• Based on simple declarative definitions…• Vagrant provisions environments through various providers
– VirtualBox, VMware, AWS• Subsequently, provisioning (‘configuration management’) using shell
scripts, Chef, Anisble, Salt or Puppet• Vagrant supports Docker
– Create Docker Host VM, Build | Run | Manage Container• Vagrant makes host-container folder mapping and networking quite easy
dockerhost
Container
69
Vagrant Docker Provisioning
• Vagrantfile defines the Container to run – including name and initial command and also synched folders (i.e. host <=> container mapping)
• Dockerfile contains build recipe for the Container we want to build• DockerHostVagrantfile describes the VM to be used as Docker Host
Vagrantfile
DockerHostVagrantfile
Dockerfile
70
Vagrant Docker Provisioning
dockerhostvm
Vagrantfile
DockerHostVagrantfile
Dockerfilemy-little-container
build process
Docker Hub
ubuntu:14.04
/u01/readme.txt
Vagrant Boxes
ubuntu/trusty64
71
Vagrant with Docker Folder Mapping
dockerhostvm
Vagrantfile
DockerHostVagrantfile
my-little-container
/vagrant
/vagrant
/host_temp
/host_data
/var/lib/docker/docker_generatedId
72
Vagrant on Windows – other options
• Boot2Docker– a lightweight Linux distribution based on Tiny Core Linux made
specifically to run Docker containers. It runs completely from RAM, weighs ~27MB and boots in ~5s
Windows or Mac host
73
Vagrant on Windows – other options
• Docker Toolbox (since August 2015) replaces Boot2Docker– Contains Docker Client for Windows, Kitematic (Docker GUI, alpha release), Docker
Machine, Docker Engine and leverages Oracle VirtualBox– Still uses Boot2Docker Linux Distribution to run containers– Currently no support for Windows host-container folder mapping – No support for GUI in containers
74
Oracle and Docker
• Oracle Linux 6 and 7 Images • WebLogic certified on Docker
– Official “Docker Build-scripts in GitHub to create images”• Solaris Zones as Docker backend• Participate in OCI• Docker on Oracle Cloud??
75
Open Container Initiative
76
Summary
• Build• Ship• Run• Docker – started in 2013• Open Container Initiative
77
Hands On
• Copy workshop resources from USB stick to c:\docker_sig• Install VirtualBox 5.0 and Vagrant 1.7.4• Use vagrant up to build and run a simple Docker Container
– This will make Vagrant create the Linux Docker Host VM• Use vagrant ssh to open a terminal in the Docker Host and start your
Docker adventures– Manage containers and images– Use link, port forwarding and volume configurations
• Use vagrant up to build a Docker Container with Puppet support; then start the container and use Puppet to perform advanced configuration
• Run a container with an Oracle Database inside• Run a container with a WebLogic Domain inside• Bonus:
– Create Linux Desktop Docker Host VM (to run GUI applications in Docker Container)