Docker containers & the Future of Drupal testing

Docker Containers: Testing automation and development in Drupal.

Amsterdam - DevOps meetup, September 2014

A new strategy for automating Drupal testing@ricardoamaro

Free/Opensource software loverSenior Cloud Engineer @AcquiaDrupal.org infrastructure/devopsDrupalist & Linux enthusiast

Father, artist, community facilitator

@ricardoamarohttps://drupal.org/user/666176

About me

Story of an investigation to improve cloud

1. The sad VirtualMachine story

2. Containers and non-containers

3. DEMO - Drupal Docker

4. Drupal Testbots story in a Glance

5. Docker as a testing automation factor

6. DEMO - Docker Tesbot

7. Integration path

today’s meetup

Hardware virtualization or platform virtualization refers to the creation of a virtual machine that acts like a real computer with an operating system.

Software executed on these virtual machines is separated from the underlying hardware resources.

What is virtualization?

Cloud infrastructure providers like Amazon Web Service sell virtual machines. EC2 revenue is expected to surpass $1B in revenue this year. That's a lot of VMs…

Why should i care?Increase

+ efficiency+ availability+ security

Reduce

- costs- hardware- energy

Virtual Machine platforms

➢ We are also paying for lot of avoidable overhead.

➢ The Virtual Machine is a full-blown operating system image.

➢ This is a heavyweight solution to run applications in the cloud.

The sad Virtual Machine story...

What is the solution?

Containers used to be terrible, but not anymoreContainers used to be terrible, but not anymore

A new concept, a new hope

Because Docker has arrived!

On any recent Linux Kernel near you!

Source : http://www.linuxjournal.com/content/containers%E2%80%94not-virtual-machines%E2%80%94are-future-cloud

Virtual Machines vs Containers

Virtualization and paravirtualization require a full operating system image for each instance.

Source : http://www.linuxjournal.com/content/containers%E2%80%94not-virtual-machines%E2%80%94are-future-cloud

Virtual Machines vs Containers

Containers can share a single Linux Kernel and, optionally, other binary and library resources.

Source : http://www.linuxjournal.com/content/containers%E2%80%94not-virtual-machines%E2%80%94are-future-cloud

Virtual Machines vs Containers

Virtualization and paravirtualization require a full operating system image for each instance.

Containers can share a single operating system and, optionally, other binary and library resources.

The time to provision

Source : http://www.linuxjournal.com/content/containers%E2%80%94not-virtual-machines%E2%80%94are-future-cloud

mount /dev/sda /targetchroot /target

but that had no resource and security isolation goals for multi-tenant designs...

From the simple concept of “chroot”

source: http://openvz.org

CpuDevicesProcessesMemoryDisk spaceNetwork

Wha

t if y

ou co

uld

cont

rol..

.

Docker

Needcontrol over specifichost resources

cgroupsControl Groups provide a mechanism for aggregating/partitioning sets of tasks, and all their future children, into hierarchical groups with specialized behaviour.

~$ ls /sys/fs/cgroupblkio cpu cpuacct cpuset devices freezer hugetlb memory perf_event

Containers & Cgroups

https://www.kernel.org/doc/Documentation/cgroups/cgroups.txt

use Docker

Docker Who??

this Docker

and ship them as containers

Ship containers? Build Once, Run Anywhere

https://github.com/ricardoamaro/docker-drupal

https://github.com/ricardoamaro/docker-drupal-nginx

Build and ship Drupal the Docker way:

the Commands:

attach Attach to a running container

commit Create a new image from a container's changes

diff Inspect changes on a container's filesystem

export Stream the contents of a container as a tar archive

history Show the history of an image

images List images

import Create a new filesystem image from the contents of a tarball

info Display system-wide information

inspect Return low-level information on a container

kill Kill a running container

login Register or Login to the docker registry server

logs Fetch the logs of a container

port Lookup the public-facing port which is NAT-ed to PRIVATE_PORT

ps List containers

pull Pull an image or a repository to the docker registry server

push Push an image or a repository to the docker registry server

restart Restart a running container

rm Remove a container

rmi Remove an image

run Run a command in a new container

start Start a stopped container

stop Stop a running container

tag Tag an image into a repository

version Show the docker version information

wait Block until a container stops, then print its exit code

The docker is awesome!the Apihttp://docs.docker.io/en/latest/api/registry_index_spec/

the Builderhttp://docs.docker.com/reference/builder/

the Registryhttp://docs.docker.io/en/latest/api/index_api/

Docker on Docker > v0.6

Changes to the container can be committed to the central index or rolled back

Continuous Deployments, Development & Testing... Keep only the good apples

How does this picture help the Drupal testing and the jobrunner?

TESTBOTS

Testbot is `Sad`

Actual running testbots were created in 2007 and revamped in 2009, need modernizing in order to overcome several pending issues that make them difficult to debug, go faster and be less prone to error.

Since they are full virtual servers, they consume much cpu, memory and IO. A part from that developers have ever increasing demands. Multiple environments and databases need to be tested and contained.

Testbots(jobrunner)

Runs the actual test

drupal.org

Issue queue starts processand displays

results

qa.drupal.org

keeps track of jobs and detailed

info

Current Drupal testing workflow (simplified):

Integration round table:

Testbot1

drupal.org

Issue queue starts processand displays

results

resultsdrupal.org

keeps track of jobs and detailed

info

New proposed Drupal testing workflow:

Jenkins

Dispatcher

Testbot2

Testbot3

Testbot4

Testbotn

Docker Container types:

- PHP5.3 / PHP5.4 / PHP5.5 / PHP5.6- MariaDB - MySQL- Sqlite - Postgresand others...

API

@ricardoamaro

Docker Containers: Testing automation

and development in Drupal.Amsterdam - DevOps meetup, September 2014

@ricardoamaro

Thank you!