View
6
Download
0
Category
Preview:
Citation preview
Con$nuous Deployment with Docker Andrew Aslinger
Oct 9. 2014
Who is Andrew #1
So#ware / Systems Architect for OpenWhere Passion for UX, Big Data, and Cloud/DevOps
Previously Designed and Implemented automated DevOps processes for several organiza@ons Mul$ple environments, many user / developer groups, big data clusters, hundreds of machines
A Recovering “Chef”
Who is Andrew #2 (Andy)
Host of AWS DC Meet Up Started as a developer who could talk to opera@ons and infrastructure teams Lived the transi@on from data center, Co-‐Lo, internal cloud, to AWS To much fun to ever go back
4
Agenda
What is Docker ?
Con$nuous Delivery with AWS and Docker
Docker Advanced Prac$ces and Lessons Learned
Demo & Discussion
What is Docker?
“Docker is an open pla/orm for developers and sysadmins to build, ship, and run distributed applica:ons”
light weight container
Yet another DevOps tool
Latest hype?
Docker Benefits
• Portability Across Machines – Local Vagrant like deployment, Remote Deployment including Amazon EC2
– Independent of host OS – Lightweight vs VMs
• Supports DevOps Best Prac@ces & Re-‐use – Infrastructure as Code – Container Extensibility
• Fits into a Con@nuous Deployment Pipeline
• Isola@on and Single Responsibility – Reduce dependency conflicts, security
Virtual Machines
Docker
7
Docker Image
Dockerfile conf
docker build –t imagename .
Docker Registry registryname:
docker tag imagename registryname/imagename
Docker Image
Docker Container
docker run –p 9000:9000 –d --name containername imagename
Development Environment
Host Environment
docker stop/restart/start containername
https://registry.hub.docker.com/ or
Private registry
A Typical Docker Flow with six simple commands
OpenWhere DevOps Philosophy
1. Immutable Infrastructure – Focus on one process (build) vs. two (build & maintain)
2. Simplify the Tools – Minimize On-‐boarding & Training – Minimize Tools
3. Purpose Built Environments 4. DevOps code should co-‐exist with applica@on code
– How to build infrastructure for project inline with project – Jenkins scripts – Docker
5. Auto-‐scale from start
Why Docker with EC2?
• Build once, run many places – Dev Machines, different clouds, different VPCs,
share and build – Not $ed to box size or guest OS
• AMI’s are difficult to rebuild and more difficult to maintain – Snapshots retain baggage – There is no “git” version control for AMI – AMI’s are not cross region compa$ble – Other AMI tools like Amiator aren’t easy to use
• Easier to achieve immutable infrastructure and one click deploys – Maintaining machines sucks
• Treat your infrastructure as code with a Con@nuous Deployment Pipeline
Why Docker with EC2?
• Separate your data and state from your image • Machine layers evolve at different velocity and have different reuse (Example Later) – Core OS – Organiza$on Level Requirements – Applica$on Level Requirements – Applica$on Code
• Test and version control your deploy ar$facts and configura$on together – Same container for developers and EC2 – Java Example
• Stop maintaining AMIs
+
Docker Con@nuous Deployment Flow: Set up AWS infrastructure
Create AWS Ec2 Instances and install Docker
1
yum install -y docker-io service docker start ….
Commit New Code w/ Docker file
SCM Poll Trigger build
Push Artifacts (optional)
Push new Image to public or private repo
2
3
4
6
Build new Docker Image from Docker File 5
Docker Con@nuous Deployment Flow: Build and push docker image with latest code
Docker Con@nuous Deployment Flow: Trigger EC2 to pull latest Docker image
Trigger Docker Pull on target machines
7
Docker Pull & start container
8
Advanced #1: Docker Image Layers using FROM and OnBuild commands in the Docker File
Core OS
Organiza$on Base OS
Organiza$on Applica$on Image (Example NodeJs Web App Image)
Project Code Specific Image Project SCM
Common Dependencies & Config: Security, LDAP, Logging, etc.
Base OS: CentOs, Ubuntu, etc.
Application Dependencies & Config: NodeJS, express, supervisord etc.
ONBUILD
Doc
ker I
mag
e La
yers
Jenkins Built
Reusable
Machine layers can evolve at different velocity and have different levels of reuse
Advanced #2: Linked Containers
Host OS
Container 1 DB
Container 2 Web
$DB_PORT = 6379, etc.
--link name:alias docker run -d -P --name web --link db:db training/webapp python app.py
/etc/hosts db 172.0.0.2
Run multiple containers on a machine. Each with a responsibility linked by ports / address https://docs.docker.com/userguide/dockerlinks/
Advanced #3: Data Containers
Host OS
Data Container (non running)
Container 2 Web
sudo docker run -d --volumes-from dbdata --name db2 training/postgres
/volume
A container which allows sharing of data across containers https://docs.docker.com/userguide/dockervolumes/
Benefits: Change image independent of data (maintainability) Enables Data sharing patterns (single responsibility) Limitations: Data is Tied to Host OS
/volume -file1.txt -file2.txt
Physical files docker managed
Container 3 Web
/volume
Advanced #4: Customize on Init
You need to parameterize and modify items in a container based on the environment
Examples: Set a IP address or DNS name, Set a DB password Pattern: 1. Load a script onto the box as part of an building the image 2. Run script at container launch 3. Script uses environment variables to customize settings files etc 4. Environment variables are passed to the docker run Command
Data Container (non running)
start.sh config.conf sed docker run -e
DB=10.0.0.1
Service Discov
ery &
Registra$on
is an
emerging area
OpenWhere Docker Wins
1. Full con@nuous deployment and write once, deploy many @mes realized
2. Developer Replacement for vagrant – Less complexity – Test with what you deploy!
3. Deployment build process co-‐exists with the so#ware! – Reduced Maintenance, developer buy-‐in, testability
4. Awesome Re-‐use with minimal code – 1 line Dockerfile(s) for new MEAN Stack apps
5. Simplicity (Streamlined DevOps)
OpenWhere Docker Burns
• Learning Curve / Maturity – Learning a Docker way for each DevOps task (s$ll in progress) – Example: Need “tricks” to do things like have more than one process running in a container – Example: Learning layers are independent and don’t retain state between RUN commands
• A Docker Container is not the same as a typical virtual machine – Basic stuff like syslog off and tar, wget not installed – Base Image: hmp://phusion.github.io/baseimage-‐docker/
• Containers not restar@ng on EC2 Restart – We hacked a cloud-‐init work around
• Upda@ng to new images is dirty – Manually clean up images / containers or run out of disk!
• Debugging more complex: You can only a`ach to a Docker container if the program you are running is a shell – Install SSH or we have a workaround trick
Recommenda$ons
• Docker works great for a Con$nuous Deployment flow on EC2 – Especially for stateless applica$ons or a micro-‐service architecture
• Complex cluster deployments, stateful applica$ons or databases, service discovery etc. not quite there yet – You may need to roll your own solu$on or integrate with another technology
Ques$ons
22
Contact Informa$on
Andrew Heifetz Chief Cloud Officer OpenWhere aheifetz@openwhere.com @andyheifetz hmp://www.linkedin.com/pub/andrew-‐heifetz Cell: 240-‐481-‐7442
Andrew Aslinger Senior Systems Architect OpenWhere aaslinger@openwhere.com @aaa572 hmp://owaaa.github.io/
Recommended