MySQL on Docker: Sep 2017 - Percona on Docker... · Sep 2017 MySQL on Docker: Containerizing the Dolphin Ashraf Sharif, Support Engineer Presenter [email protected]

Sep 2017

MySQL on Docker: Containerizing the Dolphin

Ashraf Sharif, Support EngineerPresenter

[email protected]

Copyright 2017 Severalnines AB


About Severalnines and ClusterControl

Agenda


● Introduction● Image● Container● Volume● Network● Orchestration● MySQL Container Management



Introduction

What is Docker?


● Lightweight application container platform.● Each container has its own:

○ Filesystem○ Process space○ Network stack

● Use pre-built image.● Advantages:

○ Rapid deployment○ Greater density per host○ Isolation

Docker Architecture

Docker Components


Docker Components



Docker Image

Docker Image - Introduction


● A template for Docker container.● Create an image:

○ Commit the container's changes○ Build from Dockerfile

● Share the image:○ Save as tarball.○ Push to registry:

■ Public - Docker Hub, Docker Store■ Private, hosted - Quay, ECR, GCR, Bintray,

Artifactory■ Private, self-hosted - Your own registry

# Save image to tarball$ docker save -o mysql.tar abc

# Start your own Docker registry service$ docker run -d -p 5000:5000 --name registry registry:2

# Push image to Docker Hub$ docker push myimage/mysql:5.7

Docker Image - Commit a Container


Host

# Save image to tarball$ docker run -d --name=abc ubuntu:16.04

# Commit the changes (in another window)$ docker commit abc myimage/mysql:5.7

# Push image to Docker Hub$ docker exec -it abc /bin/bashroot@abc:/# apt-get updateroot@abc:/# DEBIAN_FRONTEND=noninteractive apt-get install -y mysql-server root@abc:/# vi /etc/mysql/my.cnf

# Ubuntu 16.04 host$ apt-get update$ DEBIAN_FRONTEND=noninteractive apt-get install -y mysql-server$ vi /etc/mysql/my.cnf

# Run MySQL server as a process$ mysqld --socket=/var/lib/mysql/mysqld.sock --pid-file=/var/lib/mysql/mysqld.pid

Docker

# Run MySQL server as a container$ docker run -d myimage/mysql:5.7 mysqld --socket=/var/lib/mysql/mysqld.sock --pid-file=/var/lib/mysql/mysqld.pid

Docker Image - Using Dockerfile


Host

# Build as image$ docker build -t myimage/mysql:5.7 .

# vi DockerfileFROM ubuntu:16.04RUN apt-get updateRUN DEBIAN_FRONTEND=noninteractive apt-get install -y mysql-serverCOPY my.cnf /etc/mysql/my.cnfEXPOSE 3306

# Ubuntu 16.04 host$ apt-get update$ DEBIAN_FRONTEND=noninteractive apt-get install -y mysql-server$ vi /etc/mysql/my.cnf

# Run MySQL server as a process$ mysqld --socket=/var/lib/mysql/mysqld.sock --pid-file=/var/lib/mysql/mysqld.pid

Docker

# Run MySQL server as a container$ docker run -d myimage/mysql:5.7 mysqld --socket=/var/lib/mysql/mysqld.sock --pid-file=/var/lib/mysql/mysqld.pid

# Create MySQL config file$ vi my.cnf

Docker Image - Build an Image


# Get image size$ docker imagesREPOSITORY TAG SIZEmyimage/mysql 5.7 529MB

# Show created layers$ docker history myimage/mysql:5.7CREATED BY SIZE/bin/sh -c #(nop) COPY file:e8163ef656b6da... 101B/bin/sh -c DEBIAN_FRONTEND=noninteractive ... 370MB/bin/sh -c apt-get update 39.1MB/bin/sh -c #(nop) CMD ["/bin/bash"] 0B/bin/sh -c mkdir -p /run/systemd && echo '... 7B/bin/sh -c sed -i 's/^#\s*\(deb.*universe\... 2.76kB/bin/sh -c rm -rf /var/lib/apt/lists/* 0B/bin/sh -c set -xe && echo '#!/bin/sh' >... 745B/bin/sh -c #(nop) ADD file:39d3593ea220e68... 120MB

de70fa77eb2b

f6016aab25f1

0639788facc8

35369f9634e1

937bbdd4305a

724179e94999

f6016aab25f1

d51f459239fb

Image: myimage/mysql:5.7

ubuntu:16.04

ad74af05f5a2

Dockerfile

Docker Image - Best Practice for MySQL


# Get image size$ docker imagesREPOSITORY TAG SIZEmysql 5.6 299MBmysql 5.7 412MB

# tail entrypoint.sh......exec mysqld "$@"

● Extend an existing image available in the registry, instead of building a new one from scratch.

● The smaller the image size, the faster the deployment. (Hint: alpine).

● If using an entrypoint script, ensure mysqld command runs with "exec", indicating it's the primary process that holds PID 1 inside container.

● General guidelines: https://docs.docker.com/engine/userguide/eng-image/dockerfile_best-practices/

# vi DockerfileFROM mysql:5.7

# vi DockerfileFROM ubuntu:16.04RUN apt-get install {all MySQL stuff}

https://docs.docker.com/engine/userguide/eng-image/dockerfile_best-practices/

https://docs.docker.com/engine/userguide/eng-image/dockerfile_best-practices/



Docker Container

Docker Container - Run a Single Container


# Run a MySQL container$ docker run -d \--name my-test \--env MYSQL_ROOT_PASSWORD=mypassword \mysql

# Get the container layer size$ docker ps -sCONTAINER ID IMAGE NAMES SIZEd51f459239fb mysql my-test 4B (virtual 412MB)

Image: docker.io/mysql:latest

de70fa77eb2b

f6016aab25f1

0639788facc8

35369f9634e1

937bbdd4305a

ad74af05f5a2

d51f459239fb

0639788facc8

35369f9634e1

937bbdd4305a

my-test

Image layers (RO)

412 MB

Container layers (RW)4 B

Docker Container - Run another Container


# Run a MySQL container$ docker run -d \--name my-test2 \--env MYSQL_ROOT_PASSWORD=mypassword \mysql

# Get the container layer size$ docker ps -sCONTAINER ID IMAGE NAMES SIZEd51f459239fb mysql my-test 4B (virtual 412MB)e9de9ed50ced mysql my-test2 4B (virtual 412MB)

Image: docker.io/mysql

de70fa77eb2b

f6016aab25f1

0639788facc8

35369f9634e1

937bbdd4305a

ad74af05f5a2

d51f459239fb

0639788facc8

35369f9634e1

937bbdd4305a

my-test

Image layers (RO)

412 MB

Container layers (RW)

4 B + 4 B

e9de9ed50ced

my-test2

Docker Container - Container Layer Changes


# Operation 2 - Create a schema and table$ docker exec -it my-test \mysql -uroot -pmypassword -e \'CREATE SCHEMA testdb; CREATE TABLE t1 (id INT PRIMARY KEY AUTO_INCREMENT, data TEXT);'


de70fa77eb2b

f6016aab25f1

0639788facc8

35369f9634e1

937bbdd4305a

ad74af05f5a2

d51f459239fb

0639788facc8

35369f9634e1

937bbdd4305a

my-test

c33a2b88395d

Copy-on-Write (CoW)

416f2a97355f

Supported CoW file system & devices: AUFS, Btrfs, Device Mapper, OverlayFS/Overlay2, ZFS, VFS

# Operation 1 - Update /etc/mysql/my.cnf$ docker exec -it my-test vi /etc/mysql/my.cnf

1

2New Branch

Docker Container - Application vs Data


Application Data

Stateless Stateful

Ephemeral Persistent

Independant Dependant

Frontend Backend

Web, API Database, File Storage



Docker Volume

Docker Volume - Type


# Mount a named volume into container-v mysql1-data:/var/lib/mysql

# Mount tmpfs into container--tmpfs /tmp

● Bind mount:○ A file or directory on the host machine is

mounted into a container.○ You manage the directory’s contents.

● Named volume:○ A new directory is created within Docker’s

storage directory.○ Docker manages the directory’s contents.○ Extensible via volume plugin.

● tmpfs:○ Stored on host machine’s memory (or swap).○ Linux only.

# Bind mount a directory into container-v /data/mysql1/data:/var/lib/mysql

# Bind mount a file into container-v /data/mysql1/my.cnf:/etc/mysql/my.cnf

Docker Volume - Persistent Volumes


# Run with persistent named volume, using -v or --volume$ docker run -d \--name mysql-local \--env MYSQL_ROOT_PASSWORD=mypassword \--volume local-datadir:/var/lib/mysql \mysql:5.7

Image: docker.io/mysql:5.7

de70fa77eb2b

f6016aab25f1

0639788facc8

35369f9634e1

937bbdd4305a

ad74af05f5a2

0639788facc8

35369f9634e1

937bbdd4305a

mysql-local

416f2a92355f

/var/lib/mysql

local-datadir

# Run with persistent named volume, using --mount$ docker run -d \--name mysql-local \--env MYSQL_ROOT_PASSWORD=mypassword \--mount source=local-datadir,target=/var/lib/mysql \mysql:5.7

Docker Volume - Changes on Volume


# Run a container with port 3308$ docker run -d \--name mysql-local \-p 3308:3306 \-e MYSQL_ROOT_PASSWORD=mypassword \-v local-datadir:/var/lib/mysql \mysql


de70fa77eb2b

f6016aab25f1

0639788facc8

35369f9634e1

937bbdd4305a

ad74af05f5a2

0639788facc8

35369f9634e1

937bbdd4305a

mysql-local

416f2a92355f

/var/lib/mysql

local-datadir

# Run with persistent named volume, using --mount$ mysql -uroot -p -h127.0.0.1 -P3308 < dump.sql

dump.sql

3308

Docker Volume - Remote Volume


# Run a container with NFS bind mount$ docker run -d \--name mysql-nfs \-p 3308:3306 \-e MYSQL_ROOT_PASSWORD=mypassword \-v /nfs/mysql-nfs:/var/lib/mysql \mysql


de70fa77eb2b

f6016aab25f1

0639788facc8

35369f9634e1

937bbdd4305a

ad74af05f5a2

0639788facc8

35369f9634e1

937bbdd4305a

mysql-nfs

416f2a92355f

/var/lib/mysql

192.168.1.100/storage/docker

# Mount NFS with extra options for MySQL$ mount 192.168.1.100:/storage/docker /nfs -o noatime,nodiratime,hard,intr

3308

/nfs/mysql-nfs

Further Reading - Using NFS with MySQL:https://dev.mysql.com/doc/refman/5.7/en/disk-issues.html#disk-issues-nfs

https://dev.mysql.com/doc/refman/5.7/en/disk-issues.html#disk-issues-nfs

https://dev.mysql.com/doc/refman/5.7/en/disk-issues.html#disk-issues-nfs

Docker Volume - MySQL Persistency


Description Variable Name Default Value (5.7)

MySQL data directory datadir /var/lib/mysql

MySQL plugin directory plugin_dir {basedir}/lib/plugin

MySQL configuration directory config_dir /etc/my.cnf.d

MySQL binary log log_bin {datadir}/{hostname}-bin

Slow query log slow_query_log_file {datadir}/{hostname}-slow.log

Error log log_error {datadir}/{hostname}.err

General log general_log_file {datadir}/{hostname}.log

Docker Volume - Non-Persistent Volume


# Run a container with NFS bind mount and tmpfs volumes$ docker run -d \--name mysql-nfs-tmp \-p 3308:3306 \-e MYSQL_ROOT_PASSWORD=mypassword \-v /nfs/mysql-nfs:/var/lib/mysql \--tmpfs /tmp:rw,size=1g,mode=177 \mysql


de70fa77eb2b

f6016aab25f1

0639788facc8

35369f9634e1

937bbdd4305a

ad74af05f5a2

0639788facc8

35369f9634e1

937bbdd4305a

mysql-nfs-tmp

416f2a92355f

/var/lib/mysql

192.168.1.100/storage/docker

3308

/nfs/mysql-nfs

/tmp

Docker Volume - Drivers


● Default is local.● You can use other different volume driver (or plugin).

○ Install the plugin.○ Use --volume-driver option.

● Volume drivers allow you to extend Docker's volume capabilities:

○ Snapshot○ Backup○ Encryption

# Install Rexray EBS docker plugin$ docker plugin install rexray/ebs EBS_ACCESSKEY=XXXX EBS_SECRETKEY=YYYY

# Verify if the plugin is loaded$ docker info -f '{{json .Plugins.Volume}}' | jq[ "local", "rexray"]

# Run a container with volume plugin$ docker run -d \--name=mysql-ebs \--volume-driver=rexray/ebs \-v ebs-datadir:/var/lib/mysql \mysql:5.7

# List volumes$ docker volume lsDRIVER VOLUME NAMElocal local-datadir1local local-datadir2rexray ebs-datadir

Docker Volume - Drivers and Storage Platforms


Further Reading - Volume Plugins List: https://docs.docker.com/engine/extend/legacy_plugins/#volume-plugins

Contiv

Fuxi Netshare

REX-ray

Horcrux

Convoy

OpenStorage

Amazon S3Azure FS

Google PD

Amazon EBS DigitalOcean BS

Flocker

EMC

FUSE

vSphere

iSCSI

CIFS NetApp OpenStack Cinder Minio CephFS

BeeGFS

NFS

SMB

Infinit

https://docs.docker.com/engine/extend/legacy_plugins/#volume-plugins

Docker Volume - Simple Benchmark


Local vs NFS vs NFS-tweaked vs EBS-standard



Docker Network

Docker Network - Types


● Single-host:○ Host○ None○ Bridge

■ Default bridge (docker0)■ User-defined bridge

● Multi-host:○ Default overlay○ User-defined overlay

● Extensible via Docker network plugins.

# List all networks$ docker network lsNAME DRIVER SCOPEbridge bridge localdb_multi overlay swarmdb_single bridge localdocker_gwbridge bridge localhost host localingress overlay swarmnone null local

Docker Network - Host Network


# Run a container on host network$ docker run -d \--name mysql-host \--net host \-e MYSQL_ROOT_PASSWORD=mypassword \mysql:5.7

Container 1

eth0

eth0

Container 2

eth0

Docker

Host

# Display active TCP connections$ netstat -tulpn | grep mysqltcp6 0 0 :::3306 :::* LISTEN 24601/mysqld

● Container’s network interfaces will be identical with the machine host.

● Only one host network per machine host. ● Container linking, --link mysql-test:mysql is not

supported.● Port mapping, --publish 3307:3306 is not

supported.

Docker Network - Default Bridge Network


# Run a container on docker0 bridge network$ docker run -d \--name mysql-bridge \-p 3308:3306 \-e MYSQL_ROOT_PASSWORD=mypassword \mysql:5.7

Container 1

eth0

eth0

Container 2

eth0

Docker

Host

# Display active TCP connections$ netstat -tulpn | grep 3308tcp6 0 0 :::3308 :::* LISTEN 24601/docker-proxy

● Packet forwarding by iptables to the bridge network.● --publish is supported. Run multiple containers with

same image through different ports.● --link is supported. Link with other containers to

expose environment variables.● docker-proxy redirects connection to the correct

container through NAT.

docker0(virtual Ethernet bridge)

vethxxx vethyyy

Docker Network - User-Defined Bridge Network


# Run a container on user bridge network$ docker run -d \--name mysql1 \--net db_single \-p 3308:3306 \--ip 192.168.10.10 \--hostname mysql1 \-e MYSQL_ROOT_PASSWORD=mypassword \mysql:5.7

Container 1

eth0

eth0

Container 2

eth0

Docker

Host

# Create a new bridge network$ docker network create subnet=192.168.10.0/24 db_single

Similar to the default docker0 bridge, plus:● --hostname is supported for sticky hostname.● --ip is supported for sticky IP address.● --link is not supported. Use embedded DNS to

resolve container's name in the same network.

db_single(virtual Ethernet bridge)

vethxxx vethyyy

DNS

Docker Network - Plugins


● Similar to volume, Docker network can be extended through plugins:

○ Multi-host networking (must use outside of Swarm)

○ Name resolver and service discovery.○ Encryption.

● Popular network plugins:○ Contiv○ Calico○ Weave○ Flannel

Further Reading - Multi-host Networking with Calico:https://severalnines.com/blog/mysql-docker-multi-host-networking-mysql-containers-part-2-calico

https://severalnines.com/blog/mysql-docker-multi-host-networking-mysql-containers-part-2-calico

https://severalnines.com/blog/mysql-docker-multi-host-networking-mysql-containers-part-2-calico

Docker Network - Default Overlay Network


# Run a service on default overlay network$ docker service create \--name mysql-service \--replicas 1 \-p 3308:3306 \-e MYSQL_ROOT_PASSWORD=mypassword \-v nfs-datadir:/var/lib/mysql \mysql:5.7

Container 1

eth0

eth0 eth1

Docker

Host

# Initialize Docker Swarm$ docker swarm init

● Docker Swarm must be enabled.● Each task (container) has:

○ eth0 - Interface to external network via docker_gwbridge.

○ eth1 - Interface to overlay network via VXLAN.

docker_gwbridge

vethxxx vethyyy

br0

External

Container 2

eth0 eth1

Docker

docker_gwbridge

vethxxx vethyyy

br0

eth0Host

VXLANudp/4789

Docker Network - User-Defined Overlay Network


# Run a service on default overlay network$ docker service create \--name mysql-service \--replicas 1 \--net db_multi \-p 3308:3306 \-e MYSQL_ROOT_PASSWORD=mypassword \-v nfs-datadir:/var/lib/mysql \mysql:5.7

Container 1

eth0

eth0 eth1

Docker

Host

# Create an overlay network$ docker network create -d overlay db_multi

Similar with the default overlay, plus:● Embedded DNS resolver:

○ {service_name} → Service VIP○ tasks.{service_name} → Container's IP

docker_gwbridge

vethxxx vethyyy

br0

External

Container 2

eth0 eth1

Docker

docker_gwbridge

vethxxx vethyyy

br0

eth0Host

VXLANudp/4789

DNS



Docker Orchestration

Docker Orchestration - Container Orchestration


● Simplify container's management at scale.● Concept:

○ Multiple containers as one entity.○ Structural YAML formatting.

● Official Docker comes with:○ Docker Compose:

■ single-host, small scale, separate package.○ Docker Swarm:

■ multi-host, big scale, out-of-the-box.■ support compose formatting through

docker stack.

Docker Orchestration - Docker Compose


Docker CLI

version: '2'services: mysql-prod: image: mysql:5.7.16 command: - --innodb_buffer_pool_size=2G - --max_connections=50 ports: - "3307:3306" environment: MYSQL_ROOT_PASSWORD: r00tP4ssw0rD MYSQL_USER: myshop MYSQL_PASSWORD: MyS3creTP4s5 MYSQL_DATABASE: myshop hostname: mysql-prod networks: db-prod: ipv4_address: 192.168.10.101 volumes: - mysql-datadir:/var/lib/mysql - mysql-tmp:/tmp ulimits: nofile: soft: 16824 hard: 16824 mem_limit: 4g memswap_limit: 4gvolumes: mysql-datadir: mysql-tmp:networks: db-prod: driver: bridge ipam: driver: default config: - subnet: 192.168.10.0/24

# Production MySQL container - full command$ docker network create --subnet=192.168.10.0/24 db-prod$ docker run -d \--name mysql-prod \--net db-production \--publish 3307:3306 \--ip 192.168.10.101 \--hostname mysql-prod \--memory 4g \--memory-swap 4g \--ulimit nofile=16824:16824 \--volume mysql-datadir:/var/lib/mysql \--volume mysql-tmp:/tmp \--env MYSQL_ROOT_PASSWORD=r00tP4ssw0rD \--env MYSQL_USER=myshop \--env MYSQL_PASSWORD=MyS3creTP4s5 \--env MYSQL_DATABASE=myshop \mysql:5.7.16 \--innodb_buffer_pool_size=2G \--max_connections=50

Docker Orchestration - Run with Docker Compose


# Bring up the production MySQL container in background$ docker-compose up -dCreating network "compose_db-prod" with driver "bridge"Creating compose_mysql-prod_1

Image: docker.io/mysql:5.7.18

de70fa77eb2b

f6016aab25f1

0639788facc8

35369f9634e1

937bbdd4305a

ad74af05f5a2

0639788facc8

35369f9634e1

937bbdd4305a

mysql-prod

416f2a92355f

/var/lib/mysql

mysql-datadir

/tmp

mysql-tmp

3307

# Stop the container$ docker-compose stop

# Destroy everything$ docker-compose down

# Start the container$ docker-compose start

Scaling is supported via docker-compose scale, however conflicting options must be omitted:hostname, ports, ipv4_address, volume

Docker Orchestration - Docker Swarm


● Multi-host deployment, through overlay network.● Concepts:

○ Node■ Manager - Dispatches tasks■ Worker - Executes tasks by manager

○ Service■ Replicated - Number of replicas■ Global - One task per available node

○ Task■ Container

○ Mesh routing■ All Swarm nodes route to the running tasks■ Ingress load balancing (round-robin only)

○ Scheduler■ Resources availability■ Label and constraints

Docker Orchestration - Simple Auto Failover using Swarm


# Run a replica of MySQL service$ docker service create \--name mysql-service \--replicas 1 \--net db_multi \-p 3307:3306 \-e MYSQL_ROOT_PASSWORD=mypassword \--mount type=volume,source=ebs-datadir,destination=/var/lib/mysql,volume-driver=rexray \--mount type=volume,source=ebs-conf,destination=/etc/my.cnf.d,volume-driver=rexray \mysql:5.7

mysql-service

Docker

Host 1

ebs-datadir

mysql-service

Docker

Host 2

3307

ebs-conf

Docker Orchestration - Galera Cluster on Swarm


# Bootstrap-only container$ docker service create \--name pxc-bs \--replicas 1 \--net db-multi-hosts \-e MYSQL_ROOT_PASSWORD=mypassword123 \-e CLUSTER_NAME=my_galera \-e CLUSTER_JOIN= \percona/percona-xtradb-cluster

DockerHost 1

tasks.pxc1

# 3-node Galera$ for i in {1..3}; do docker service create \--name pxc${i} \--replicas 1 \--net db-multi-hosts \-p 330${i}:3306 \-v MYSQL_ROOT_PASSWORD=mypassword123 \-e CLUSTER_NAME=my_galera \-e CLUSTER_JOIN=tasks.pxc-bs,tasks.pxc1,tasks.pxc2,tasks.pxc3 \--volume pxc{i}:/var/lib/mysql \percona/percona-xtradb-cluster; \sleep 1m; \done

tasks.pxc-b

s

# Remove bootstrap-only container$ docker service rm pxc-bs

DockerHost 2

tasks.pxc2

DockerHost 3

tasks.pxc3

HAproxy

330133023303

330133023303

330133023303

3306

Docker Orchestration - Other Orchestration Tools


Docker Orchestration - Best Practice for MySQL


● Add label to node and use placement constraints:○ Avoid overcommit resources.○ Manageability and predictability.

● Use Service's name and virtual IP address to distinguish container's role.

● Understand the scheduling strategy:○ Spread○ Binpack○ Random○ Custom

● Rolling update:○ Only perform update one container at a time.○ Add delay before move to the next container.



MySQL Container Management

MySQL Container - Service Control


# Get container's log$ docker logs -f mysql-staging...[Note] mysqld: Shutdown complete

# List processes inside container$ docker exec -it mysql-staging ps -e PID TTY TIME CMD 1 ? 00:00:00 mysqld 62 pts/0 00:00:00 bash 86 pts/0 00:00:00 ps

● mysqld must hold PID 1 inside container to receive the signal.

● docker stop might be dangerous. Default is 10s grace period. If still running, SIGKILL.

● Use docker kill with proper signal (SIGTERM). No grace period.

● Verify the shutdown process through docker log.● Stopping won’t delete the container.

# Stop MySQL container using kill$ docker kill --signal=TERM mysql-staging

# Stop MySQL container using stop$ docker stop mysql-staging

MySQL Container - Resource Control


# Limit memory and open file descriptors$ docker run -d \--name mysql-staging \--memory 4g \--memory-swap 4g \--ulimit nofile=16824:16824 \mysql:5.7.6

● By default, no resource constraints, use as much of a given resource as the host’s kernel will allow.

● Important options!○ --memory & --memory-swap

■ Must be equal to disable swap.■ If --memory-swap is not set, container

swap is default to --memory * 2○ --ulimit nofile={soft:hard}

■ Maximum number of files MySQL server can open simultaneously.

● Some of the container resources can be controlled dynamically using docker update:

○ Tune my.cnf accordingly.

# Increase container's memory.$ docker update \--memory 6g \--memory-swap 6g \mysql-staging

MySQL Container - Resource Monitoring


# Monitor containers$ docker stats --format "table {{.Name}}\t{{.CPUPerc}}\t{{.MemUsage}}"NAME CPU % MEM USAGE / LIMITminio1 0.00% 6.078MiB / 7.78GiBmysql2 0.12% 194.1MiB / 7.78GiBmysql-local 0.14% 220MiB / 7.78GiB

● Use docker stats.● Inside container, free and top commands are not

accurate:○ Memory - /sys/fs/cgroup/memory/memory.*○ CPU - /sys/fs/cgroup/cpu/cpu.*○ IO - /sys/fs/cgroup/blkio/blkio.*

● External open-source monitoring tools:○ CLI: sysdig, dockviz○ UI: Portainer, Shipyard, Rancher, Prometheus

Portainer

MySQL Container - Configuration Management


# Bind mount my.cnf$ docker run -d \--name mysql1 \-e MYSQL_ROOT_PASSWORD=mypassword \-v /storage/mysql1/my.cnf:/etc/mysql/my.cnf \mysql:5.7.6

● Most MySQL configuration parameters can be changed during runtime. (Hint: SET GLOBAL)

● You can either:○ Use persistent storage:

■ Bind mount your my.cnf file.■ Create a named volume for /etc/my.cnf.d

○ Append the configuration as a "CMD" (right after image name), passes as flags to mysqld inside container.

● Standard MySQL and InnoDB optimization applies.

# Named volume for MySQL configuration directory$ docker run -d \--name mysql2 \-e MYSQL_ROOT_PASSWORD=mypassword \-v mysql2-conf:/etc/my.cnf.d \mysql:5.7.6

# Append configuration options$ docker run -d \--name mysql3 \-e MYSQL_ROOT_PASSWORD=mypassword \mysql:5.7.6 \--innodb_buffer_pool_size=1G \--max_connections=100

MySQL Container - Security


# Create a secret$ echo 'MyP2s$w0rD' | docker secret mysql_root_passwd -

● Docker Secrets○ Only for Swarm service.○ Accessible under /run/secrets/ inside

container.● Runtime privileges:

○ Unprivileged is default and recommended. Container is not allowed to access other devices.

○ Use --privileged only if you really need to modify kernel parameters (sysctl, /proc,/sys) or you want to run container inside container.

# Create a service and use the secret as root password$ docker service create \--name mysql-container \--publish 4406:3306 \--secret mysql_root_passwd \-e MYSQL_ROOT_PASSWORD_FILE=/run/secrets/mysql_root_passwd \-v local-datadir:/var/lib/mysql \mysql:5.7

MySQL Container - Backup


# mysqldump and store inside physical host$ docker exec -it mysql-prod mysqldump -uroot -p --single-transaction > /path/in/physical/host/dump.sql

● Logical backup:○ Mysqldump○ mysqlpump

● Xtrabackup (hot-backup):○ Install inside machine host.○ Install inside the MySQL container.○ Use perconalab/percona-xtrabackup image.

● Snapshot:○ MyISAM only - FTWRL + copy over○ Stop container + copy over

# FTWRL$ docker exec -it mysql-prod /bin/bashroot@mysql-prod:# mysql -uroot -pmysql> FLUSH TABLE WITH READ LOCK;

# Copy volume to another location (another terminal)$ cp /var/lib/docker/volumes/mysql-datadir/_data /destination/in/physical/host

# Release lock (same terminal as FTWRL)mysql> 'UNLOCK TABLES'

# Use perconalab/percona-xtrabackup image to backup$ docker run --rm -it \--net db-prod \-v mysql-datadir:/var/lib/mysql \-v /tmp/backup:/xtrabackup_backupfiles \ perconalab/percona-xtrabackup \--backup --host=mysql-prod --user=root --password=mypassword

MySQL Container - Restore


# Restore a MySQL dump$ docker exec -it mysql-staging mysql -uroot -p < /path/in/physical/host/dump.sql

● Logical backup:○ Fairly straightforward. Use standard procedure.

● Xtrabackup (hot-backup):○ Have to be prepared first○ Once prepared, simply starts a new container by

mounting the prepared directory as datadir.● Snapshot & Copy:

○ Bind mount:i. Direct mount the directory as a volume.

○ Named volume:i. Create a volume.ii. Copy the content to the volume's

directory.iii. Start a new container by mounting the

volume.

# Run a test container from a copied datadir (snapshot) or from a prepared xtrabackup directory.$ docker run -d \--name mysql-restored \-v /tmp/backup:/var/lib/mysql \mysql:5.7.16

# Prepare the backup created by xtrabackup$ docker run --rm -it \-v mysql-datadir:/var/lib/mysql \-v /tmp/backup:/xtrabackup_backupfiles \ perconalab/percona-xtrabackup \--prepare --target-dir /xtrabackup_backupfiles

MySQL Container - Logical Upgrade


1. Pull the new image, M.2. Start new container, B with new image, M.3. Export MySQL on A.4. Stop A.5. Import into B.6. Run mysql_upgrade inside B.7. OK? Remove A.8. Fallback? Stop B, start A.

# 1$ docker pull mysql:5.7.18# 2$ docker run -d \--name=mysql-new \-e MYSQL_ROOT_PASSWORD=mypassword \-v local-datadir:/var/lib/mysql \mysql:5.7.18# 3$ docker exec -it mysql-old mysqldump -uroot -p > dump.sql# 4$ docker kill --signal=TERM mysql-old# 5$ docker exec -it mysql-new mysql -uroot -p < dump.sql# 6$ docker exec -it mysql-new mysql_upgrade -uroot -p# 7$ docker rm -f mysqld-old# 8$ docker stop mysqld-new$ docker start mysqld-old

MySQL Container - In-Place Upgrade


# 1$ docker pull mysql:5.7.18# 2$ docker exec -it mysql-old mysql -uroot -p -e 'SET GLOBAL innodb_fast_shutdown = 0'# 3$ docker kill --signal=TERM mysqld-old# 4$ docker run -d \--name mysql-new \--publish 3307:3306 \-e MYSQL_ROOT_PASSWORD=mypassword \-v local-datadir:/var/lib/mysql \mysql:5.7.18# 5$ docker exec -it mysql-new mysql_upgrade -uroot -p# 6$ docker rm -f mysqld-old# 7$ docker stop mysqld-new$ docker start mysqld-old

1. Pull the new image, M.2. Set innodb_fast_shutdown=0 inside container.3. Stop the container, A.4. Start a new container, B with new image, M.5. Run mysql_upgrade inside B.6. OK? Remove A.7. Fallback? Stop B, start A.



What's Next?


● Visit our booth for a free T-shirt and other goodies. While stock lasts!

● Follow our MySQL on Docker blog series at severalnines.com/blog

● Attend the MySQL on Docker webinar, Wednesday Sept 27th, 2017:○ 09:00 BST / 10:00 CEST○ 09:00 PST (US) / 12:00 EST (US)

● Check out ClusterControl Docker image.

What's Next

https://severalnines.com/blog

https://severalnines.com/resources/webinars

https://hub.docker.com/r/severalnines/clustercontrol/



Q & A

Documents

MySQL on Docker: Sep 2017 - Percona on Docker... · Sep 2017 MySQL on Docker: Containerizing the Dolphin Ashraf Sharif, Support Engineer Presenter [email protected]