Deploying Elasticsearch on Docker with Weave

Ilya Dmitrichenko @weaveworks

Docker is great!

• Highly portable delivery framework

• Fuelling CI/CD/DevOps with

• tight and transparent dependency packaging

• good isolation, without virtualisation overhead

• easy to setup, manage and automate

Ok, I want Docker!• I surely want to use Docker for next-gen

microservices architecture, it looks just right.

• I also see many other great tools  that are related, e.g.:

• Kubernetes/Mesos/Marathon/Swarm

• Terraform/Brooklyn/fleet/Machine

• etcd/Consul/Zookeeper

• OK, may be I should start with something basic

Can I run Elasticsearch on Docker out-of-the-box?

• Yes, there is an official image:

> docker run -d elasticsearch

…and what about clustering?

• I don’t see how to do it with the official image, but there seems to be another one by @itgz

> docker run -d \

-p 9200:9200 -p 9300:9300 itzg/elasticsearch

> docker run -d \

-p 9200:9201 -p 9300:9301 itzg/elasticsearch

> docker run -d \

-p 9200:9202 -p 9300:9302 itzg/elasticsearch

• That’s not “official”, right?

I am still not sure how clustering will work ;’(

• What are `-p 9200:9201 -p 9300:9301`?

• How do I run the instances on different hosts?

• Which port should my client use?

• What is the “ambassador patter”?

• May be I need a kind of orchestrator tool?

I found a few examples of what others did…

• Mesos + HAProxy + Marathon + Ansible

• This one uses Ubuntu in EC2 and I really should be using RedHat in Azure, I am also not sure about switching to Ansible from Puppet, and still need to learn more about all the Mesos stuff…

• CoreOS + fleet + etcd

• I like the idea of CoreOS, but not sure if my boss likes it and, it still looks a bit complicated…

• Kubernetes

• Looks also interesting, but I am not sure we can introduce Kubernetes either… I just want to demo proof-of-concept next week

Introducing Weave

• Easiest way to build portable Docker app clusters

• You won't have to change your app

• Lean service discovery which doesn't in your way

• Simple to use on its own or with an orchestrator

Install Weavesudo curl -L git.io/weave \ -o /usr/local/bin/weave

sudo chmod +x /usr/local/bin/weave

weave launch \ -iprange <CIDR> \ <PEERS>

weave launch-dns <CIDR>

Run Containers

weave run --with-dns \ —h myapp.weave.local \ [docker_args] \ <docker_image> \ [app_args]

WeaveDNS rocks!

> curl es-1.weave.local:9200/_cat/nodes es-1.weave.local 10.2.3.1 4 18 0.50 d * Shuma-Gorath es-3.weave.local 10.2.3.192 4 17 0.04 d m Frankie and Victoria es-2.weave.local 10.2.3.128 3 17 0.02 d m Lady Jacqueline Falsworth Crichton

WeaveDNS rocks!

var es = new elasticsearch.Client({ hosts: [ 'es-1.weave.local:9200' , 'es-2.weave.local:9200' , 'es-3.weave.local:9200' ] });

Demo

Let's go!

Based on previous blog post: http://blog.weave.works/2015/01/20/elasticsearch-and-weave/

Other benefits

• No host port mapping or NAT

• Encryption on network layer

• Multiple ES clusters on same hosts

• Cross-datacenter connectivity

• Multicast works anywhere

New features

• distributed IP address allocator (out today)

• native Docker plugin

• fast in-kernel data path

Container visibility

• Weave Scope automatically generates a map of your containers, enabling you to intuitively understand, monitor, and control your applications.

• weave.works/scope

Questions?