1 . 1

Simple and ScalableMicroservices

NATS and the Docker tooling

+

Waldemar Quevedo / @wallyqs

Docker SF Meetup, Oct 2016

1 . 1

2 . 1

ABOUTWaldemar Quevedo / So�ware Developer at in SF

Development of the Apcera PlatformPast: PaaS DevOps at Rakuten in TokyoNATS client maintainer (Ruby, Python)

@wallyqsApcera

3 . 1

ABOUT THIS TALKWhat is NATSFeatures from NATSNATS + Docker �

4 . 1

What is NATS?

4 . 2

4 . 3

NATSHigh Performance Messaging SystemCreated by First written in in 2010

Originally built for Cloud FoundryRewritten in in 2012

Better performanceOpen Source, MIT License

Used by Apcera, Ericsson, HTC, GE, Baidu

Derek CollisonRuby

Go

https://github.com/nats-io

4 . 4

NATSDesign constrained to keep it as operationally simple andreliable as possible while still being both performant andscalable.

4 . 5

Acts as an always available dial-tone

4 . 6

It's Fast

4 . 7

single byte message

Around 10M messages/second

4 . 7

4 . 8

and Simple

4 . 9

DESIGNConcise feature set (pure PubSub!)No built-in persistence of messagesNo exactly-once-delivery promises eitherThose concerns are simplified away from NATS

→ NATS Streaming

4 . 10

DESIGNTCP/IP basedPlain text protocolfire and forgetat most once

4 . 11

Very simple protocol

telnet demo.nats.io 4222

sub hello 1+OKpub hello 5world+OKMSG hello 1 5worldpingPONG

nats.io/documentation/nats-protocol

4 . 12

Demo

4 . 13

Clients

4 . 14

GOnc, err := nats.Connect()// ...nc.Subscribe("hello", func(m *nats.Msg){ fmt.Printf("[Received] %s", m.Data)})nc.Publish("hello", []byte("world"))

4 . 15

RUBYrequire 'nats/client'

NATS.start do |nc| nc.subscribe("hello") do |msg| puts "[Received] #{msg}" end

nc.publish("hello", "world")end

4 . 16

PYTHON (ASYNCIO)yield from nc.connect()

@asyncio.coroutinedef handler(msg): print("[Received] {data}".format( data=msg.data.decode()))

# Coroutine based subscriberyield from nc.subscribe("foo", cb=handler)yield from nc.publish("foo", "bar")

4 . 17

NODE.JSvar nats = require('nats').connect();

// Simple Publishernats.publish('foo', 'Hello World!');

// Simple Subscribernats.subscribe('foo', function(msg) { console.log('[Received] ' + msg);});

4 . 18

CnatsConnection_Publish(nc,"foo",data,5);natsConnection_Subscribe(&sub,nc,"foo",onMsg, NULL);

voidonMsg(natsConnection *nc, natsSubscription *sub, natsMsg *msg, void *closure){ printf("[Received] %.*s\n", natsMsg_GetData(msg)); // ...}

4 . 19

C#using (ISyncSubscription s = c.SubscribeSync("foo")) { for (int i = 0; i < 10; i++) { Msg m = s.NextMessage(); System.Console.WriteLine("[Received] " + m); } }

4 . 20

JAVA// Simple Publishernc.publish("foo", "Hello World".getBytes());

// Simple Async Subscribernc.subscribe("foo", m -> { System.out.println("[Received] %s\n", new String(m.getData()));});

4 . 21

Many more available!

C C# Java

Python NGINX Go

Node.js Elixir Ruby

PHP Erlang Rust

Haskell Scala Perl

( italics → community contributed)

4 . 21

4 . 22

Great community!

4 . 23

Growing ecosystem

nats.io/community

4 . 23

4 . 24

Connectors

4 . 24

4 . 25

Community contributed dashboards

github.com/cmfatih/natsboard

4 . 25

4 . 26

Docker Hub activity from NATS users

4 . 26

4 . 27

Docker Hub activity from NATS users

4 . 27

4 . 28

Docker Store (Beta)

4 . 28

5 . 1

NATS Features

5 . 2

FEATURESPure PubSub based Request/ReplySubject routing with wildcards

AuthorizationDistribution queue groups for balancingCluster mode for HA

Auto discovery of topology/varz style monitoringSecure TLS connections with certificates

5 . 3

REQUEST/REPLYIt is pure PubSub based using and bysignaling in the subject:

NATS unique identifierslimited interest

SUB _INBOX.y1JxglDi76shQQIhPbTDme 2UNSUB 2 1PUB help _INBOX.y1JxglDi76shQQIhPbTDme 6please

tells the server to unsubscribe from subscription with sid=2a�er getting 1 message before sending a request on thehelp subject.

5 . 4

REQUEST/REPLYIf another subscriber is connected and interested in thehelp subject, it will then receive a message with that inbox.

SUB help 90

# Message received from serverMSG help 90 _INBOX.y1JxglDi76shQQIhPbTDme 6please

# Use the inbox to reply backPUB _INBOX.y1JxglDi76shQQIhPbTDme 11I can help!

5 . 5

REQUEST/REPLYFinally, original requestor will be receiving the message:

SUB _INBOX.y1JxglDi76shQQIhPbTDme 2

UNSUB 2 1

PUB help _INBOX.y1JxglDi76shQQIhPbTDme 6

please

MSG _INBOX.y1JxglDi76shQQIhPbTDme 2 11

I can help!

5 . 6

REQUEST/REPLYNATS clients libraries have helpers for generating the uniqueinboxes which act as ephemeral subscriptions:

nats.NewInbox()// _INBOX.y1JxglDi76shQQIhPbTDme

Used internally when making a Request:

nc, _ := nats.Connect(nats.DefaultURL)t := 250*time.Millisecond// Request sets to AutoUnsubscribe after 1 responsemsg, err := nc.Request("help", []byte("please"), t)if err == nil { fmt.Println(string(msg.Data)) // => I can help!}

REQUEST/REPLY → LOWEST LATENCYResult of publishing a request to all nodes with limitedinterest means we are getting the fastest reply back:

5 . 7

REQUEST/REPLY → LOWEST LATENCYResult of publishing a request to all nodes with limitedinterest means we are getting the fastest reply back:

5 . 85 . 9

SUBJECTS ROUTINGWildcards: *

SUB foo.*.bar 90 PUB foo.hello.bar 2hiMSG foo.hello.bar 90 2hi

e.g. subscribe to all requests being made on the demo site:

telnet demo.nats.io 4222INFO {"auth_required":false,"version":"0.9.4",...}

SUB _INBOX.* 99MSG _INBOX.y1JxglDi76shQQIhPbTDme 99 11I can help!

5 . 10

SUBJECTS ROUTINGFull wildcard: >

SUB hello.> 90PUB hello.world.again 2hiMSG hello.world.again 90 2hi

e.g. subscribe to all subjects and see whole traffic goingthrough the server:

telnet demo.nats.io 4222INFO {"auth_required":false,"version":"0.9.4",...}

sub > 1+OK

5 . 11

DISTRIBUTION QUEUESBalance work among nodes randomly

5 . 12

DISTRIBUTION QUEUESBalance work among nodes randomly

5 . 13

DISTRIBUTION QUEUESBalance work among nodes randomly

5 . 14

DISTRIBUTION QUEUESService A workers subscribe to service.A and createworkers distribution queue group for balancing the work.

nc, _ := nats.Connect(nats.DefaultURL)// SUB service.A workers 1\r\nnc.QueueSubscribe("service.A", "workers", func(m *nats.Msg) { nc.Publish(m.Reply, []byte("hi!")) })

5 . 15

DISTRIBUTION QUEUESNote: NATS does not assume the audience!

5 . 16

DISTRIBUTION QUEUESAll interested subscribers receive the message!

5 . 17

CLUSTERINGAvoid SPOF on NATS by assembling a full mesh cluster

5 . 18

CLUSTERINGClients reconnect logic is triggered

5 . 19

CLUSTERINGConnecting to a NATS cluster of 2 nodes explicitly

srvs := "nats://10.240.0.1:4222,nats://10.240.0.2:4223"nc, _ := nats.Connect(srvs)

NOTE: NATS servers have a forwarding limit of one hop.

Each server will only forward a message that it hasreceived from a client to all connected servers thatexpressed interest in the message's published subject.A message received from a route will only be distributedto local clients.

5 . 20

CLUSTERINGReconnect and disconnect handlers can be useful to traceconnection failures.

nc, err := nats.Connect(uri, nats.DisconnectHandler(func(nc *nats.Conn) { fmt.Printf("Got disconnected!\n") }), nats.ReconnectHandler(func(nc *nats.Conn) { fmt.Printf("Got reconnected to %v!\n", nc.ConnectedUrl()) }), nats.ClosedHandler(func(nc *nats.Conn) { fmt.Printf("Connection closed. Reason: %q\n", nc.LastError()) }),)

5 . 21

CLUSTER AUTO DISCOVERYSince release, topology can be discovered

dynamically by clients!v0.9.2

5 . 22

We can start with a single node…

5 . 22

5 . 23

Then have new nodes join the cluster…

5 . 23

5 . 24

As new nodes join, server announces INFO to clients.

5 . 24

5 . 25

Clients auto reconfigure to be aware of new nodes.

5 . 25

5 . 26

Clients auto reconfigure to be aware of new nodes.

5 . 26

5 . 27

Now fully connected!

5 . 27

5 . 28

On failure, clients reconnect to an available node.

5 . 28

5 . 29

MONITORING style monitoring endpoint available for inspecting the

internal state of the server./varz

Other available endpoints:

- Info of clients connected to this server/connz

- Subscriptions metrics/subsz

- Cluster routes/routez

5 . 30

MONITORING: EXAMPLESGathering connections metrics from demo:

curl demo.nats.io:8222/varz | grep connections

Result:

"max_connections": 65536, "connections": 25, "total_connections": 12429,

5 . 31

MONITORING: EXAMPLESGathering metrics regarding languages used whenconnecting to demo

curl demo.nats.io:8222/connz?subsz=1 | grep lang | sort | uniq -c

Result:

10 "lang": "go", 7 "lang": "node", 8 "lang": "python2",

5 . 32

MONITORING polls from these endpoints providing a terminal UInats-top

5 . 33

TLSSupported for client and route connections, and formonitoring.

https_port: 6443

tls { cert_file: "./configs/certs/server-cert.pem" key_file: "./configs/certs/server-key.pem" ca_file: "./configs/certs/ca.pem" # Require client certificates verify: true}

5 . 34

CLUSTER TLSSecuring route connections with TLS

cluster { listen: 0.0.0.0:6222 tls { # Route cert cert_file: "./configs/certs/srva-cert.pem" # Private key key_file: "./configs/certs/srva-key.pem" # Optional certificate authority verifying routes # Required when we have self-signed CA, etc. ca_file: "./configs/certs/ca.pem" }}

5 . 35

SUBJECTS AUTHORIZATIONPubSub on certain subjects can be disallowed in the server'sconfiguration:

authorization { admin = { publish = ">", subscribe = ">" } requestor = { publish = ["req.foo", "req.bar"] subscribe = "_INBOX.*" }

users = [ {user: alice, password: foo, permissions: $admin} {user: bob, password: bar, permissions: $requestor} ]}

5 . 36

SUBJECTS AUTHORIZATIONClients are not allowed to publish on _SYS (reserved):

PUB _SYS.foo 2hi-ERR 'Permissions Violation for Publish to "_SYS.foo"'

6 . 1

NATS and the Docker tooling

6 . 2

THE NATS DOCKER IMAGESmall binary → Lightweight Docker image

No deployment dependencies

6 . 3

2-STEP BUILD PROCESS

6 . 4

FIRST STEP: COMPILE

github.com/nats-io/gnatsd/Dockerfile

FROM golang:1.6.3

MAINTAINER Derek Collison ＜derek@apcera.com＞

COPY . /go/src/github.com/nats-io/gnatsdWORKDIR /go/src/github.com/nats-io/gnatsd

RUN CGO_ENABLED=0 go install ...(elided)

EXPOSE 4222 8222ENTRYPOINT ["gnatsd"]CMD ["--help"]

6 . 5

SECOND STEP: FROM SCRATCH

github.com/nats-io/nats-docker/Dockerfile

FROM scratch

COPY gnatsd /gnatsdCOPY gnatsd.conf /gnatsd.conf

EXPOSE 4222 6222 8222

ENTRYPOINT ["/gnatsd", "-c", "/gnatsd.conf"]CMD []

6 . 6

THE NATS DOCKER IMAGEBy default it is exposing these ports:

# Clients, Cluster and Monitoring portsEXPOSE 4222 6222 8222

- Clients will be connecting against this port4222

- Port used for the cluster routes6222

- HTTP Monitoring endpoint8222

6 . 7

USING NATS + DOCKER

6 . 8

All examples can be found at

github.com/wallyqs/nats-docker-examples

6 . 8

EXAMPLE: SINGLE NODE

API Server which receives HTTP requests and communicatesinternally through NATS to a pool of workers.

6 . 96 . 10

EXAMPLE: CLUSTERED SETUP

API Server which receives HTTP requests and communicatesinternally through NATS to a pool of workers.

6 . 11

LOCAL NATS CLUSTER VIA DOCKER COMPOSE

Docker Compose tooling comes in handy here for doingdevelopment as it helps us in assembling NATS clusterslocally.version: "2"

networks: nats-network: {}services:

nats-server-A: networks: ["nats-network"] image: "nats:0.9.4" entrypoint: "/gnatsd --routes nats://nats-server-B:6222,nats://nats-server-C:6222 --cluster nats://0.0.0.0:6222"

nats-server-B: networks: ["nats-network"] image: "nats:0.9.4" entrypoint: "/gnatsd --routes nats://nats-server-A:6222,nats://nats-server-C:6222 --cluster nats://0.0.0.0:6222"

nats-server-C: networks: ["nats-network"] image: "nats:0.9.4" entrypoint: "/gnatsd --routes nats://nats-server-A:6222,nats://nats-server-B:6222 --cluster nats://0.0.0.0:6222"

6 . 12

NATS + Docker Compose Demo

6 . 12

6 . 13

DOCKER SWARM BASED NATS CLUSTER

Auto discovery becomes helpful in this context, we canassemble one with the following commands.docker network create --driver overlay nats-cluster-example

docker service create --network nats-cluster-example --name nats-A \ nats:0.9.4 -cluster nats://0.0.0.0:6222

docker service create --network nats-cluster-example --name nats-B \ nats:0.9.4 -routes nats://nats-A:6222 \ -cluster nats://0.0.0.0:6222

docker service create --network nats-cluster-example --name nats-C \ nats:0.9.4 -routes nats://nats-A:6222,nats://nats-B:6222 \ -cluster nats://0.0.0.0:6222

6 . 14

DOCKER SWARM BASED NATS CLUSTER

Even easier: just use same configuration in all nodes.

for node in A B C; do docker service create --network nats-cluster-example \ --name nats-$node nats:0.9.4 \ -routes nats://nats-A:6222 \ -cluster nats://0.0.0.0:6222done

Initial server becomes the seed server, ignoring route to self.

6 . 15

NATS + Docker Swarm mode Demo

6 . 15

7 . 1

Summary

7 . 2

NATS is a simple, fast and reliable solution for the internalcommunication of a distributed system.

Docker flexible tooling is a good complement for buildingNATS based applications.

7 . 2

7 . 3

BONUS: NATS STREAMING

NATS Streaming recently became an official image too!

It is a layer on top of NATS totalling in ~10MB.

Further info: github.com/nats-io/nats-streaming-server

8 . 1

THANKS! / github.com/nats-io @nats_io

Play with the demo site!

telnet demo.nats.io 4222