Upload
stephen-wallace
View
104
Download
2
Embed Size (px)
Citation preview
The cloud is my laboratory Experimenting with Docker
Adam Friedman Dimension Data (ITaaS R&D) [email protected]
What this talk is about»
What this talk is about» Experimenting with Docker
What this talk is about» Experimenting with Docker (der)
What this talk is about» Experimenting with Docker
» Using virtualisation to reduce cycle time when trying out ideas
What this talk is about» Experimenting with Docker
» Using virtualisation to reduce cycle time when trying out ideas
» Using the cloud to achieve scale
What this talk is about» Experimenting with Docker
» Using virtualisation to reduce cycle time when trying out ideas
» Using the cloud to achieve scale
» Without breaking the bank
What I'd like to leave you with1.
What I'd like to leave you with1. We learn more from our failures than our successes
What I'd like to leave you with1. We learn more from our failures than our successes
2. Occam's razor
What I'd like to leave you with1. We learn more from our failures than our successes
2. Occam's razor (prefer to minimise variables)
What I'd like to leave you with1. We learn more from our failures than our successes
2. Occam's razor
» Aim for repeatability where possible
What I'd like to leave you with1. We learn more from our failures than our successes
2. Occam's razor
3. Virtualisation + automation make it cheap to experiment
What I'd like to leave you with1. We learn more from our failures than our successes
2. Occam's razor
3. Virtualisation + automation make it cheap to experiment
4. If experimentation is cheap enough, you can try out almost any idea you can come up with
What I'd like to leave you with1. We learn more from our failures than our successes
2. Occam's razor
3. Virtualisation + automation make it cheap to experiment
4. If experimentation is cheap enough, you can try out almost any idea you can come up with
» Think it's a silly idea? Prove it.
What I'd like to leave you with1. We learn more from our failures than our successes
2. Occam's razor
3. Virtualisation + automation make it cheap to experiment
4. If experimentation is cheap enough, you can try out almost any idea you can come up with
» Don’t be afraid to experiment!
Why I love Virtualisation Virtualisation is the bedrock on which most things cloud are built.
Why I love Virtualisation Virtualisation is the bedrock on which most things cloud are built.
It is the key to: »
Why I love Virtualisation Virtualisation is the bedrock on which most things cloud are built.
It is the key to: » Economies of scale
Why I love Virtualisation Virtualisation is the bedrock on which most things cloud are built.
It is the key to: » Economies of scale » Speed of provisioning
Why I love Virtualisation Virtualisation is the bedrock on which most things cloud are built.
It is the key to: » Economies of scale » Speed of provisioning » Repeatability
Why I love Virtualisation Virtualisation is the bedrock on which most things cloud are built.
It is the key to: » Economies of scale » Speed of provisioning » Repeatability
» The ability to readily and reliably duplicate an environment and its resources
Why I love the concept of CloudFor all its potential downsides, the cloud has vastly improved the process of experimentation:
Why I love the concept of CloudFor all its potential downsides, the cloud has vastly improved the process of experimentation:
» By selectively abstracting away levels of infrastructure, it enables us to focus on what we're trying to do
Why I love the concept of CloudFor all its potential downsides, the cloud has vastly improved the process of experimentation:
» By selectively abstracting away levels of infrastructure, it enables us to focus on what we're trying to do, rather than how to do it
Why I love the concept of CloudFor all its potential downsides, the cloud has vastly improved the process of experimentation:
» By selectively abstracting away levels of infrastructure, it enables us to focus on what we're trying to do, rather than how to do it
» Economies of scale also come into play:
Why I love the concept of CloudFor all its potential downsides, the cloud has vastly improved the process of experimentation:
» By selectively abstracting away levels of infrastructure, it enables us to focus on what we're trying to do, rather than how to do it
» Economies of scale also come into play:
» You can’t fit 20 servers under your desk
Why I love the concept of CloudFor all its potential downsides, the cloud has vastly improved the process of experimentation:
» By selectively abstracting away levels of infrastructure, it enables us to focus on what we're trying to do, rather than how to do it
» Economies of scale also come into play:
» You can’t fit 20 servers under your desk, but your cloud provider has plenty to spare (for as long as you need them)
Why I love the concept of CloudFor all its potential downsides, the cloud has vastly improved the process of experimentation:
» By selectively abstracting away levels of infrastructure, it enables us to focus on what we're trying to do, rather than how to do it
» Economies of scale also come into play:
» Most providers now charge only for what you actually use
Why I love the concept of CloudFor all its potential downsides, the cloud has vastly improved the process of experimentation:
» By selectively abstracting away levels of infrastructure, it enables us to focus on what we're trying to do, rather than how to do it
» Economies of scale also come into play
» Lower costs make it easier to explore a problem space, which helps to mitigate risk
Experimenting in the cloudThe trick to being productive when experimenting in the cloud:
» Get set up
Experimenting in the cloudThe trick to being productive when experimenting in the cloud:
» Get set up
» Do your experiment
Experimenting in the cloudThe trick to being productive when experimenting in the cloud:
» Get set up
» Do your experiment
» Clean up the evidence
Without breaking the bankThe trick to being productive when experimenting in the cloud:
» Get set up
» Do your experiment
» Clean up the evidence
» Before the mob arrives with pitchforks and flaming torches
Docker is great for experimenting
with new technologies
If you've heard of it, there's probably a Docker image of it.
If you've heard of it, there's probably a Docker image of it.
Example: What’s Neo4J?
If you've heard of it, there's probably a Docker image of it.
Example: What’s Neo4J?
docker run -p 7474:7474 neo4j
If you've heard of it, there's probably a Docker image of it.
Ok, how about RabbitMQ?
If you've heard of it, there's probably a Docker image of it.
Ok, how about RabbitMQ?
docker run -p 5672:5672 rabbitmq
If you've heard of it, there's probably a Docker image of it.
Ok, smartarse
If you've heard of it, there's probably a Docker image of it.
Ok, smartarse, but I bet there isn't one for cowsay
If you've heard of it, there's probably a Docker image of it.
Ok, smartarse, but I bet there isn't one for cowsay
docker run mwendler/cowsay "Sorry."
________< Sorry. > -------- \ ^__^ \ (oo)\_______ (__)\ )\/\ ||----w | || ||
That's super useful, but what if you're experimenting with docker itself?
That's super useful, but what if you're experimenting with docker itself?
docker-machine create docker1eval $(docker-machine env docker1)
That's super useful, but what if you're experimenting with docker itself?
docker-machine create docker1eval $(docker-machine env docker1)
Ok, but not very cloudy.
That's super useful, but what if you're experimenting with docker itself?
How about:
That's super useful, but what if you're experimenting with docker itself?
How about:
docker-machine create --driver amazonec2 docker1
docker-machine create --driver amazonec2 docker2
That's super useful, but what if you're experimenting with docker itself?
Or:
That's super useful, but what if you're experimenting with docker itself?
Or:
docker-machine create --driver digitalocean docker1
docker-machine create --driver digitalocean docker2
Docker Machine is nice
Docker Machine is nice, but it's opaque
Docker Machine is nice, but it's opaque
Sometimes that’s what you want.
Docker Machine is nice, but it's opaque
Sometimes that’s what you want (again, the focus may be on what you're trying to do).
Docker Machine is nice, but it's opaque
Sometimes that’s what you want (again, the focus may be on what you're trying to do, not how).
Docker Machine is nice, but it's opaque
Sometimes that’s what you want (again, the focus may be on what you're trying to do, not how).
Other times, not so much.
Docker Machine is nice, but it's opaque
Sometimes that’s what you want (again, the focus may be on what you're trying to do, not how).
Other times, not so much.
So what other options are there?
Do it by hand
You could do this at least once - a useful learning experience (especially for when things go wrong).
Automate it!
I'm probably preaching to the converted here :)
Infrastructure with Terraform
Declarative configuration for infrastructure
» Multi-cloud / multi-provider
» Repeatable
» Easy to consistently create / destroy / re-create infrastructure
But Terraform is only half the story. Once the infrastructure has been created, how do you get Docker itself deployed and configured?
Software with Ansible
» Hardly the only option out there, but if you're experimenting with throw-away systems then it's a good choice because it's just SSH (no master / agents to deploy)
» Chef Solo (or Chef Zero) is an alternative, but may still require more work to bootstrap each node
» Quick to get started (just run commands or modules on target machines)
» Scales up to Roles + Playbooks for repeatability
» Not great for managing large numbers of machines
Software with Ansible
» Can be run from Terraform (via a plug-in provisioner) but it's a lot easier to run it separately while you're experimenting.
» There are Ansible inventory plugins that can read a Terraform state file (so it knows server roles, host names, IP addresses, etc).
» Ansible Galaxy has modules for everything, including Docker (hint, hint)
» Beware if you are deploying an OS with only Python 3.x (Ansible needs 2.x)
A quick detour: Docker from simple to complex» Stand-alone
» Docker
» Orchestrated
» Local
» Docker Compose
» Clustered
» Docker Swarm
» Clustered with GUI
» Cattle / Rancher
» Kubernetes
» Mesos / Marathon
» Kitchen Sink
» Mantl
Note that simple and easy-to-use are not the same thing :)
Cisco Mantl
» Mantl is a microservices platform based on Docker, Mesos / Marathon, Consul, Traefik, Kubernetes, Calico, Contiv, etc (pretty much everything but the kitchen sink).
» Because it already includes several popular systems that extend / orchestrate Docker, it’s a useful starting point if you’re stuck trying to work out how to deploy or integrate a particular component
Cisco Mantl
» Mantl is deployed using Terraform with Ansible, and can therefore be deployed on a variety of clouds by swapping out Terraform modules as required.
» Their Ansible inventory plugin understands which cloud provider was used to create the infrastructure and so Ansible playbooks can adjust their behaviour to suit (if required).
Build your own lab
Build your own (throw-away) lab
Build your own (throw-away) lab
» Digital Ocean is fast Add Terraform and you can repeatedly create and destroy environments in seconds, rather than minutes.
» Obviously you can use AWS / Azure, too (or Dimension Data Cloud Control) but for quick-and-dirty experiments I prefer the simplest thing that works).
» Create an Ansible playbook to install Docker
» Start small - put everything in a single playbook and then move stuff out to separate reusable roles when needed
Build your own lab
4 servers (1 master, 3 workers).
lab.tf:
resource "digitalocean_droplet" "master" {
count = 1
image = "ubuntu-14-04-x64"
name = "master-${count.index + 1}"
region = "nyc2"
size = "1024mb"
}
resource "digitalocean_droplet" "worker" {
count = 4
image = "ubuntu-14-04-x64"
name = "worker-${count.index + 1}"
region = "nyc2"
size = "1024mb"
}
Build your own lab
» terraform apply
Build your own lab
» terraform apply
» …
Build your own lab
» terraform apply
» …
» Profit!
Build your own lab
» terraform apply
» …
» Profit!
» terraform destroy
Build your own lab
» terraform apply
» …
» Profit!
» terraform destroy
» No pitchforks
Build your own lab
» terraform apply
» …
» Profit!
» terraform destroy
» No pitchforks (probably)
Why not use Rancher / Docker Cloud?
» If all you want to do is deploy containers then these are a much better choice
Why not use Rancher / Docker Cloud?
» If all you want to do is deploy containers then these are a much better choice
» Rancher uses docker-machine to create nodes for you
Why not use Rancher / Docker Cloud?
» If all you want to do is deploy containers then these are a much better choice
» Rancher uses docker-machine to create nodes for you
» But this is a good choice if you want to learn about:
Why not use Rancher / Docker Cloud?
» If all you want to do is deploy containers then these are a much better choice
» Rancher uses docker-machine to create nodes for you
» But this is a good choice if you want to learn about:
» How to deploy docker
Why not use Rancher / Docker Cloud?
» If all you want to do is deploy containers then these are a much better choice
» Rancher uses docker-machine to create nodes for you
» But this is a good choice if you want to learn about:
» How to deploy docker
» Non-standard docker configurations
Why not use Rancher / Docker Cloud?
» If all you want to do is deploy containers then these are a much better choice
» Rancher uses docker-machine to create nodes for you
» But this is a good choice if you want to learn about:
» How to deploy docker
» Non-standard docker configurations
» It's useful to have a feeling for what's behind the UI
Links
» github.com/tintoy
» github.com/DimensionDataResearch
» mantl.io
» terraform.io
» ansible.com
» github.com/angstwad/docker.ubuntu
Questions?