Open sourcing the IoT : EnMasse running on Kubernetes

Open sourcing the IoT

Running EnMasse on Kubernetes

Paolo PatiernoSenior Software Engineer @ Red Hat05/06/2017

INSERT DESIGNATOR, IF NEEDED2

Who am I ? @ppatierno

● Senior Software Engineer @ Red Hat○ Messaging & IoT team

● Lead/Committer @ Eclipse Foundation○ Hono, Paho and Vert.x projects

● Microsoft MVP● Technologies and protocols “globetrotter”● Hacking low constrained devices in spare time● Blogger and speaker about distributed systems, messaging, IoT

and embedded “world”

INSERT DESIGNATOR, IF NEEDED3

Agenda

● Messaging … what ?● Messaging … for IoT● Messaging & IoT … in the cloud● EnMasse : the open source MaaS !● Running EnMasse …

○ Kubernetes○ Azure Container Service○ OpenShift

INSERT DESIGNATOR, IF NEEDED4

● It’s about messages exchange○ Internally in distributed systems○ Externally between systems

● Communication at the application level● Messages go from sender/producer/publisher to receiver/consumer/subscriber

○ Asynchronously○ Time decoupling○ … or directly and synchronously

What is messaging?

INSERT DESIGNATOR, IF NEEDED5

Messaging patterns

C S P

S

S

S

msg X

msg X

msg X

P

C

C

C

msg X

msg Y

msg Z

Request/Response Publish/Subscribe Competing Consumers

msg req

msg resp

INSERT DESIGNATOR, IF NEEDED6

Quality of Service

P C

At Most Once At Least Once Exactly Once

msg X

P C

msg Y

P C

msg Z

P C

msg X

P C

msg Y

P C

msg Y

P C

msg X

P C

msg Y

P C

msg Z

INSERT DESIGNATOR, IF NEEDED7

IoT : messaging vengeance

● … maybe in the past … ● … messaging was not so cool for developers ...● … but today with IoT this is changed because … ● … IoT is all about messaging so …

“Messaging vengeance” !

INSERT DESIGNATOR, IF NEEDED8

“give me a scalable messaging platform, and I shall move the Internet of Things world” (Archimedes)

IoT : messaging as a “lever”

INSERT DESIGNATOR, IF NEEDED

IoT Core Services

9

What makes an IoT platform ?

Messaging infrastructure

Device Provisioning

Business Services

Monitoring Real time streaming

Machine Learning …

Authentication & Authorization

Device Registration

INSERT DESIGNATOR, IF NEEDED10

IoT : communication patterns

Telemetry Inquiries Commands Notifications

INSERT DESIGNATOR, IF NEEDED11

IoT : communication patternsMessaging patterns & protocols

● Telemetry & Notifications are about … ○ …. messaging publish/subscribe

● Commands & Inquiries are about … ○ … messaging request/response

● Different protocols (AMQP, MQTT, HTTP, …) implement them in different way○ As built-in support … ○ … or on top of it at application level○ Read more on “Strengths And Weaknesses Of IoT Communication Patterns” *

* DZone IoT Guide : https://dzone.com/guides/iot-applications-protocols-and-best-practices

INSERT DESIGNATOR, IF NEEDED12

IoT : interoperabilityOpen standards

AMQP 1.0

MQTT

HTTP

CoAP

XMPP

STOMP

INSERT DESIGNATOR, IF NEEDED13

● Microsoft Azure ○ Service Bus + Event Hub○ IoT Hub

● Amazon Web Services○ Simple Queue Service (SQS)○ AWS IoT

● Google○ FireBase Cloud Messaging○ IoT Core

Messaging & IoT in the cloud

INSERT DESIGNATOR, IF NEEDED14

● They are not open source !● Freedom of choice

○ On-premise or in the cloud ○ Ability to choose which cloud○ Open Standards protocols allows users to choose client freely

● Migrating from one to the other can be complex

Cloud provider limitations

INSERT DESIGNATOR, IF NEEDED15

● Open source cloud messaging running on Kubernetes and OpenShift● enmasse.io

EnMasseMessaging-as-a-Service

INSERT DESIGNATOR, IF NEEDED16

● Multiple communication patterns: request/response, publish/subscribe and competing consumers

● Support for “store and forward” and direct messaging mechanisms● Scale and elasticity of message brokers● AMQP 1.0 and MQTT support● Simple setup, management and monitoring● Multitenancy: manage multiple independent instances● Deploy “on premise” or in the cloud

EnMasseFeatures

INSERT DESIGNATOR, IF NEEDED17

● Authentication and authorization● Service broker API● HTTP(S)● Message grouping● Distributed transactions● Message ordering● Multiple flavors

○ Apache Kafka● ...

EnMasseComing features

INSERT DESIGNATOR, IF NEEDED18

Architecture

INSERT DESIGNATOR, IF NEEDED19

Routing vs “Broking”Broker

Producer Broker Consumer

Send message

Accepted

Send message

Accepted

INSERT DESIGNATOR, IF NEEDED20

Routing vs “Broking”Router

Producer Router Consumer

Send message

Accepted

Send message

Accepted

INSERT DESIGNATOR, IF NEEDED

MQTT over AMQP

21

● MQTT gateway○ Handles connections with remote MQTT clients○ Bridges MQTT - AMQP protocols

● MQTT lwt○ Provides the “will testament” feature○ In charge to recover & send the “will” if client dies

● It brings MQTT features over AMQP so …○ … “will testament” works for AMQP clients as well

INSERT DESIGNATOR, IF NEEDED22

Configuration distribution

INSERT DESIGNATOR, IF NEEDED23

Configuration interface

{ "apiVersion": "v3", "kind": "Address", "metadata": { "name": "myqueue" }, "spec": { "store_and_forward": true, "multicast": false, "flavor": "vanilla-queue" }}

INSERT DESIGNATOR, IF NEEDED

{ "apiVersion": "v3", "kind": "Flavor", "metadata": { "name": "vanilla-queue" }, "spec": { "type": "queue", "Description": "Simple in-memory queue", "templateName": "queue-inmemory", "templateParameters": {} }}

24

Configuration interface

INSERT DESIGNATOR, IF NEEDED25

Continuous integration

INSERT DESIGNATOR, IF NEEDED

Continuous integration

26

Component build pipeline

build test Build docker image & push success

failure

trigger systemtests

INSERT DESIGNATOR, IF NEEDED

AMQP 1.0

27

Eclipse HonoConnect. Command. Control

API Endpoints

Auth Service

Device Registry

Protocol Adapters Business ApplicationsDevices

DevicesAMQP 1.0

AMQP 1.0

AMQP 1.0

HTTP, MQTT

INSERT DESIGNATOR, IF NEEDED28

Eclipse HonoIoT API

● Telemetry○ used by devices to send data downstream○ leverages on “direct messaging”

● Device Registration○ used to make Hono aware of devices that will connect to the service○ register, deregister, get information …

● Event○ used by devices to send event downstream○ differ from Telemetry on using “store and forward” (with TTL)

● Command & Control○ used by applications to send commands to devices○ command execution can be “just in time” or “deferred”

INSERT DESIGNATOR, IF NEEDED29

IoT : how to deploy ?

● “On premise” … ○ … maybe for a not so big solution○ … ingesting few data and handling few devices

● “Cloud” … ○ … needs for more scalability○ … don’t want to manage the infrastructure

● “Hybrid” … ○ … needs for processing at the edge○ … needs for not making sensible data public

INSERT DESIGNATOR, IF NEEDED30

Azure Container Service

● A containers hosting solution● Scale and orchestrate using …

○ Kubernetes○ Docker Swarm○ DC/OS

● Deploying a cluster using Azure CLI / portal○ Resource group with VMs, load balancer, ...

● Managing directly your preferred “orchestrator”○ ACS provides you “only” the infrastructure

INSERT DESIGNATOR, IF NEEDED31

Azure & OpenShift

● OpenShift Origin○ the upstream open source project

● OpenShift Container Platform○ the Red Hat productized version○ enterprise grade container platform +

Demo : the deployment on Kubernetes

Qpid Dispatch Router

ActiveMQ Artemis

Spark Streaming driver

AMQP clients

temperature

max

24 → /temperature /temperature → … 23, 24, 23, …

/max ← … 23, 25, …

25 ← /max

Demo : the deployment on OpenShift

Qpid Dispatch Router

ActiveMQ Artemis

Spark Streaming driver

AMQP clients

temperature

max

24 → /temperature /temperature → … 23, 24, 23, …

/max ← … 23, 25, …

25 ← /max

INSERT DESIGNATOR, IF NEEDED34

DEMO

INSERT DESIGNATOR, IF NEEDED

Resources

35

● EnMasse : https://enmasseproject.github.io/● Qpid Dispatch Router : http://qpid.apache.org/components/dispatch-router/● ActiveMQ Artemis : https://activemq.apache.org/artemis/● Azure Container Service : https://azure.microsoft.com/en-us/services/container-service/● OpenShift on Azure : http://aka.ms/openshift● Eclipse Hono : https://www.eclipse.org/hono/● Demo : https://github.com/ppatierno/enmasse-spark-demo● My blog : https://paolopatierno.wordpress.com/

Thank you ! Questions ?

@ppatierno