OTech – Fall 2015

Author: Lucas Jellema (AMIS)

Lean Environment Management with Docker Containers

You really must have been living under that proverbial rock to not have heard anything about Docker

[Containers]. So much buzz has surrounded Docker, you could hardly fail to notice it. And the buzz is still

getting louder as well, with companies such as Oracle gradually, and initially a little grudgingly, becoming

part of the conversation.

This article discusses Docker with a specific target audience in mind. I am assuming that like me you are a

developer or platform administrator with an interest in trying out new stuff. You want to run beta

software, try out new tools and frameworks, do some R&D, prepare for demonstrations or training

sessions – and you want to do so quickly, smoothly, and without messing up your environment. Besides,

you have limited physical computer resources (such as CPU, Memory, Disk Space) and you need a lean

approach that optimizes the use of these resources.

You may want to create things you want to share with friends, colleagues or even the world at large –

such as a running environment with various components installed and configured to work together or a

running application prototype. And you would like to share your stuff without creating elaborate

instructions for installing and configuring and certainly without support conversations around ‘it works

on my machine’ ‘okay, big sigh, send me your config files and I will look into them’. You may want to run

some of your stuff on a cloud environment – without too much hassle of getting it installed, deployed

and configured on the cloud (“just lift, shift and run sounds nice, thank you very much!”). From all the

buzz you have a certain interest in learning more about ‘that Docker thing’. And finally: even though the

host operating system of your laptop could be Mac OS or Windows, you do not suffer from a severe

Linux allergy.

If you match the above description of the target audience, then you definitely should read on. This article

will then provide you with a number of things:

insight into what this Docker thing is and why it has become such a hype?

understanding of what Containers are and how they compare to Virtual Machines?

feel for how you can build, ship | share | distribute and run containers, also through cloud

infrastructure?

a way to more efficiently leverage the physical resources in your computer than through juggling

VMs

a structured and fast way to try out new software without messing up your local environment.

a set of tools to get started with Docker on a non-Linux laptop and pointers to going with Docker

on your own

a feel for the status of Docker and current developments such as the Open Container Initiative as

well as the relation between Oracle and Docker

Build, Ship and Run The tag line for Docker is ‘build, ship and run any app, anywhere’. The initial focus for Docker – and for us

in this article - was Run. Subsequently, we will discuss Ship and Build. This section will provide a high

level overview of build, ship and run. The next section will provide more details and concrete examples.

Run

Docker offers the ability to run isolated environments (called containers) in a Linux host that to all

intents and purposes feel like standalone Virtual Machines. These containers have their own file system,

users and groups, network configuration and IP address and process table. However, they do not contain

a full blown operating system and they do not require a hypervisor layer in between. The containers

share the Linux Kernel and the physical resources are allocated at runtime, not pre allocated in a VM

definition. As a result, even though containers walk and talk like VMs, they are far leaner than VMs: they

consume less runtime resources and start and stop much faster. They are also much smaller in terms of

disk space, because of the snapshot or layer based way of defining containers. Because containers run

within the same Linux (host) environment and are therefore ‘closer together’ than true VMs are, there

are some shortcuts available to have containers interact and share resources.

Docker can be seen as a great way to expose core Linux Kernel container namespace isolation features in

a way that made them accessible to the masses. Docker sits on top of what the Linux Kernel offers at

least since release 3.10 (2013). The Docker project was born in 2013 too. Docker is free and open source

under Apache 2 license. All code is available on GitHub (https://github.com/docker/docker).

Docker consists of several parts. The most relevant for our discussion are the Docker host – the engine

that runs Docker containers inside a Linux host – and the Docker client – the command line interface

through which we interact with the Docker host. The Docker host exposes an API that can be accessed

over HTTP. Through this APIs, containers can be started and stopped, built, published and monitored. A

plethora of tools has come into existence that leverage this API from pretty user interfaces that allow

easy Docker container administration.

The fact that containers can be so quickly started and stopped on demand, with relatively small physical

overhead, makes them a great vehicle for implementing dynamic clusters. On top of Docker Host and

this API, there are also various solutions for container orchestration that support running such dynamic

clusters of Docker Containers. Docker Swarm and Google’s Kubernetes are prominent examples of this,

along with native facilities offered by several container-enabled cloud vendors.

Ship

A container can be committed as an image. This means that the state of the container is persisted, to be

used locally as a starting point for new containers. This state consists of the image from which the

container was started originally, complemented with all installation, configurations and other

manipulation that has taken place during the lifetime of the container. The image also has a default

command that is executed when a container is run from the image. Usually this command starts

whatever engine is the heart of the micro service offered by the container, such as the database, the

web server or the application server.

The container image – the frozen state of the container - can be pushed to an image registry, such as

Docker Hub. Image registries can be personal, private (within an enterprise) or public – cloud based.

Anyone with the appropriate access privileges to an image registry can pull images from the registry and

use them locally run containers from.

The public Docker Hub registry contains 1000s of Docker container images. These range from very

specific, hardly reusable configurations to official distributions from software vendors and open source

products. Among the most popular images on Docker Hub are NGINX, Tomcat, Jenkins, MongoDB, Redis,

Ubuntu, Java, Wordpress, Node[JS], PostgreSQL, MySQL, Debian, CentOS, Ruby, Rails, and the Docker

Registry image. Oracle has also – besides MySQL - published images for Oracle Enterprise Linux – though

not for any other software.

Docker Hub – and various other cloud based registries – provide private image repositories where access

is limited to parties you have explicitly authorized. These private registries are typically not free. A

private, local – for example enterprise wide- image registry can be created by running the Docker

Registry container image, which results in a running container that provides the image registry micro

service.

In addition to exchanging images through a registry, local containers and images can also be delivered as

a tarball and distributed using traditional means. This tarball can be loaded into Docker environments to

produce a local image from which containers can be run.

Portability The portability provided by Docker is one of its most prominent features. Portable container images –

either through registries or in tarballs – can very easily be handed around. And because Linux servers

with Docker support are omnipresent, the Docker container image can almost anywhere be loaded and

run. Remember that the image contains not just the software specific to whatever service the container

provides but also the necessary setup of the underlying operating system and platform engine. The

container is a ready to run service specific IaaS + PaaS + SaaS package – that needs little additional

generic IaaS + PaaS to actually run.

Not surprisingly, this portability has led many software vendors to leveraging the Docker Container

image as an important or even their main means of distributing software. Instead of tiresome installation

procedures and complex configuration instructions, they make the container available that has it all. The

number of private enterprise data centers ready to manage Docker containers is rapidly growing and

there is a large set of cloud providers that are also Docker container enabled. The latter means that

when you package any workload you want to run on the cloud can very easily be shipped to the cloud as

a Docker container image. If you can make your cloud bound application run in a local container – fine

tuning all installation, setup, configuration and deployment – then it is a piece of cake to turn that

container in an image that you can then hand off to the cloud.

Some use cases for the shipping power provided by Docker include:

using images as the distribution mechanism for beta-software (software for which no installers

are available yet and whose test results should not depend on environment issues)

using images as the vehicle to hand out training environments (and containers as the means to

run these environments)

using container images as the artifact delivered from the Continuous Delivery process: instead of

application artifacts like WAR and EAR files, CD brings a container that is ready to run with

software and environment to the testing and production environment; especially when Docker

Containers are adopted as a production infrastructure, this delivery approach seems desirable

Implicit Image Interface Container images can have implicit interfaces. These interfaces describe the dependencies of an image –

that need to be satisfied through injection when you run a container based on the image. The Wordpress

image for example states a dependency on a MySQL container into which it can create its content store.

The MySQL image defines an interface through which we can specify a directory on the Docker host file

system to use for storing data files (instead of having them inside the container).

Docker makes it easy to inject dependencies when a container is run. It works with three kinds of

injection:

link – under a certain alias - to another container; this is used to inject the MySQL container into

the Wordpress container

volume – a mapping from a local directory on the Docker host to a mount point inside the

container, replacing a local directory in the container; the MySQL image uses a volume to

optionally store data files in what amounts to a directory in the host rather than in the container

parameter - used to inject a value to be assigned to a predefined environment variable inside the

container; the MySQL image for example allows us to set the name of a database to be created

on image startup

Note that there is no formalized format for specifying the image dependencies. The image description in

the Docker Hub typically provides hints for whatever can be configured and injected into containers run

from the image – but not in a structured, standardized way.

Layers and the Union File System Before discussing the container build process, we need to look a little bit into the internal structure of

images. Images capture the state of a container. They do so through gathering all the files that make up

the file system of the container. Now for the smart parts: when an image – say Tomcat - is created from

a container that was run from the Ubuntu image – Docker is aware of the fact that in the Tomcat image

is hidden away the base Ubuntu image. And when you create an image from a container that was based

on the Tomcat image, Docker will know that too. Because every image is uniquely identified – by a very

long number – Docker can leverage the same base image multiple times in any environment. That means

that only a single copy of a base image needs to be present in any Docker host. In the figure we have five

containers running that are based – directly or indirectly – on the Ubuntu image. This 188 MB sized

image exists only once on the local environment.

An image can add files to the contents of the base image, and also change or delete files from the base

image. Docker makes use of so called union file systems to combine the layers from various images into a

single image. Changes take precedence top to bottom: changes in Tomcat override files in Ubuntu. Union

file systems allow files and directories of separate file systems, known as branches, to be transparently

overlaid, forming a single coherent file system.

Another smart element in how Docker operates: when you run a container, based on an image

definition, a writable layer is created – which is initially empty. When a new file is created in that

container – or an existing file inherited from one of the underlying images is changed or removed – then

the change is recorded in that writable layer. For many containers, this layer will be fairly small in

physical size on disk.

The figure visualizes how container My Web App was started from the image Tomcat that is based on the

Ubuntu image. In the container, some changes were made to the server.xml file that is inherited from

the Tomcat image. This file was copied-on-write to the writable layer created for My Web App. When the

container is stopped – the writable layer is preserved. When the container is started again, it runs with

the same state it had when it was stopped.

At some point, a container can be committed – saved as an image. The image in this example consists

logically of the changed server.xml and a reference to the Tomcat image (that in turn contains a

reference to Ubuntu image).

One or many new containers can be started from the My Web App image. Each new running container

adds exactly nothing in terms of disk space usage – and only consumes memory for its isolated Tomcat

process. This explains why running many containers can be done quickly and with a relatively small

overhead.

Build

Building a Docker container is the least interesting of the threesome build, ship and run. Docker provides

a build mechanism that processes a recipe – the Dockerfile – and executes the commands in that recipe,

starting from the specified base image.

Most commands boil down to the execution of Shell-scripts to create, install or configure items on the

Linux file system. During the build process, any file available in the build context – the directory that

contains the Dockerfile including all subdirectories – is available, to copy, execute or otherwise leverage

from the commands.

After this standard Docker build process is complete and the container is running, any additional change

wrought in a container adds to the definition of the container – and can therefore be seen as part of an

extended build process. For the final image, it does not matter whether the Shell-script based standard

Docker build process modifies the state of the container or whether a subsequent operation takes over

for more complex configuration management activities.

In recent years, we have collectively moved away from Shell-scripting based environment provisioning

and configuration management. These scripts are hard to create, hard to maintain, hard to reuse, very

fragile, not easily created as idem potent (multi-time runnable) and typically not portable. For these and

other reasons, we have embraced tools such as Ansible, Salt, Chef and Puppet to help us perform

provisioning and configuration management in a better way: more declarative (focus on what to achieve

instead of what to do), easier to executed in automated workflows, more portable, easier to configure

through parameters and with a large degree of reuse. The Puppet Forge – one case of a repository of

reusable configuration management recipes - contains many ready to use modules that can perform

provisioning of for example a complete SOA Suite 12c environment.

Because of this, I recommend the workflow illustrated in the next figure for building Docker container

(images).

First, the foundation is laid through the Dockerfile – the container based on a designated image (step 1 )

with some groundwork in place, such as the installation of a configuration management tool of choice

(step 2, using Puppet in this case). Next, the manifests that describe the result to be achieved through

Puppet are copied into the container (step 3). When the classic Docker build is complete and the

container is running, the execution of the Puppet manifests takes place (step 4). In this step, the complex

installation and configuration activities take place that may include running installers and post-

installation steps, cloning projects from GitHub or other source code repositories, deploying custom

applications on top of standard platform components (for example your web application on top of

Tomcat). Finally, when Puppet has done its thing, we can decide to commit the container as a new image

(step 5). This image can be our starting point for running many containers.

Note that some steps in the Dockerfile contribute to the runtime attributes of the container image –

such as the exposed ports and the command executed during start up. These commands cannot be

replaced by configuration management tools.

Docker on Windows or Mac OS Docker exposes and leverages Linux Kernel features. It is therefore obvious that Docker does not run

natively on Windows or Mac OS (or indeed Solaris, AIX or any other operating systems besides a

relatively recent distribution of Linux). In order to work with Docker on a non-Linux machine, we need to

work with a Virtual Machine that does run Linux. This can be a very tiny, trimmed down VM – entirely

focused on running the Docker engine.

Several tools are available to help work with Docker on Mac OS or Windows. Boot2docker was

the most widely used tool; it was recently succeeded by the Docker Toolbox. A third option – one

that I prefer myself – is Vagrant. All three tools basically offer a similar way of working: through a

hypervisor – commonly Oracle VirtualBox – a Virtual Machine is created and run, based on Linux.

This dockerhost VM is Docker enabled (the latest version of Docker is installed and configured).

Subsequently, Docker commands can be handed to the tool of choice – boot2docker, Vagrant of

the toolbox – and are sent to the Docker API running inside the VM. Commands include

instructions to run (first time), build, start and stop containers, to commit or export containers,

to pull, push or save images and to execute commands in a running container. All provided in the

native non-Linux host environment – all taking effect in the Docker environment.

Microsoft has announced native container support in Windows Server 2016. Additionally, Docker and

Microsoft have stated that the Docker client will become available on top of those container features.

Note that this does not mean of course that Docker images based on Linux will be portable to the

Windows host running the Docker client. Oracle has announced the same thing for Solaris Zones (with

the large difference being the fact that Solaris Zones has been around since 2005 and support for

containers in Windows is still being invented).

Oracle and Docker Oracle has not been at forefront of Docker supporters. Only in the last few quarters did Oracle make any

noise around and overtures in the general direction of Docker. For example, there have been no

announcement around support for running Docker containers on the Oracle Public Cloud. Oracle

provides a few Docker images – for MySQL and for Oracle Enterprise Linux – but not for any other

software. The main reason given is that a copyright license agreement has to be explicitly accepted

before Oracle software can be downloaded and installed. Perhaps Docker container images can be

published similar to the prebuilt developer VMs on OTN

(http://www.oracle.com/technetwork/community/developer-vm/index.html ) – but for the moment the

only images available are complete, stand-alone VirtualBox VMs. Note that for some of the beta-

programs, Oracle is starting to use Docker container images as a way to distribute the beta-software. In

the absence of the finalized installer and in order to prevent beta-issues from environment problems,

this sounds like a good way of working.

Oracle has published the Dockerfile and instructions to build a Docker container image with WebLogic

12c. You have to download a few pieces of software – JDK 8, WebLogic installer – and then you can run

the provided Dockerfile to build the container. The final result is a WebLogic image that – if it weren’t for

the license situation – Oracle could have published in its entirety. WebLogic is also officially certified by

Oracle to run on Docker – as was announced in the Spring of 2015:

https://blogs.oracle.com/WebLogicServer/entry/oracle_weblogic_server_now_running.

Important steps were taken in the direction of Docker in early Summer 2015: the announcement of

Docker integration into Solaris Zones and the fact that Oracle joined the Open Container Initiative (OCI).

The latter is an industry wide collaboration to develop a standard for container definitions – as open

source, based on the Docker container definition – to ensure portability and prevent emergence of

competing container definitions. Again, Oracle joined OCI a bit later – lagging by just a few weeks - than

the initial launch of the initiative.

More clarity around Oracle’s relationship with Docker is expected to emerge during Oracle OpenWorld

2015, later this month.

Getting started with Docker Getting started with Docker is pretty easy. Especially when you are already on a Linux machine. The all it

takes is install Docker, run Docker and start your first container with a single command, such as:

docker run –d –p 80:80 nginx

which will start a new container, based on the NGINX image, run it in the background and expose the

webserver at port 80 of the local host. With

curl localhost:80

you can access the NGINX engine in the container and see its HTML response.

To get started with Docker on a Windows or Mac OS machine, one approach is to install VirtualBox

(preferably 5.0) and Vagrant. Then have Vagrant generate the VM with Linux and Docker and

subsequently create and run the container inside that VM.

I have created a small workshop to quickly get started with Docker – especially on a Windows or Mac OS

machine. In a few hours, you will see the essential bits and pieces that allow you to add Docker to your

personal tool belt. The workshop has you run containers with NGINX, MySQL and Wordpress, Java,

JDeveloper (yes, a container with a GUI) and Oracle XE database. You will find the workshop in GitHub:

https://github.com/lucasjellema/introduction-docker-workshop.

Conclusion Docker provides a widely accepted infrastructure for running containers – light weight, isolated compute

cells that are similar to VMs but far outperform them. The Docker container image format is portable

across all environments that can run Docker containers – of which there are many, from laptop to public

cloud. Creating and sharing such images is easy – and ready to run images are available for most popular

frameworks, technologies and products.

Docker provides a great mechanism for any developer or platform administrator who wants to try out

new software, quickly get started with a product, share an application or environment configuration with

colleagues, peers or for example students in a training session. Or with the entire world for that matter.

Through Docker, isolated, stand-alone environments can be created, linked together, started and

stopped and published – all while using minimal physical computer resources.

The build process that produces the container images can be automated. A combination of the out of the

box Docker container build with an advanced configuration management tool such as Chef or Puppet

seems to provide the optimal workflow.

Docker leverages Linux kernel features. To run Docker containers on a non-Linux host, an intermediate

Linux-layer is required: a virtual machine that runs Docker on top of a Linux distribution. Various tools

are available to create and manage this intermediate layer – such as the Docker Toolbox and Vagrant.

The industry is embracing containerization in general and Docker specifically- through the publication of

Docker images, support for containerization and for running containers on public cloud infrastructures.

Oracle has made some moves – and is expected to make some more in the near future.

Now is a very good time for you to get your hands dirty and try out Docker for yourself. In 30 minutes

you should be able to have your first containers up and running. What are you waiting for?