@matteocollinamatteocollina.com

Node.js Conference - Brescia 2012/11/09

Building a multi-protocol broker for the Internet of Things using node.js

How many people will own a smartphone by 2014

1.000.000.000

Did you see it coming

http://www.flickr.com/photos/12738000@N00/360231193/

...They didn’t!

What's next(hint: it's a big number)

50.000.000.000

1.000.000.000

50.000.000.000

"Things"

Let's experiment

I want to chat with my house

Goal:

>: hi househi matteo

>: what's the temperature?20

>: 4 8 15 16 23 42

If you don’t understand this, you are not a geek!

Hubot © 2012 GitHub Inc. All rights

Enter Hubot

Hubot © 2012 GitHub Inc. All rights

A programmable robotthat is controlled through chat

We can supercharge our house with hubot

module.exports = (robot) ->

robot.respond /what’s the temperature?/i, (msg) -> msg.http("http://mchouse.it/temperature") .header("Accept", "application/json") .get() (err, res, body) -> msg.send JSON.parse(body)

We can supercharge our house with hubot

How

In order to ask our temp to hubot, we need to:

1. sense the temp2. get the temp on the web3. build a web API

We are building an API

In Italy,“API” means

“bees”http://www.flickr.com/photos/theseanster93/4056815767

What do we need to sense my house temperature

TMP36

TMP36

We add a sensor to an Arduino

TMP36

We add an ethernet shield to connect to the Internet

http://www.flickr.com/photos/snootlab/6052465980/

What protocol do we use to push our temperature to our API?

http://www.flickr.com/photos/clearlyambiguous/48185613/

HTTP is slow and safe

We need a fast, binary protocol

http://www.flickr.com/photos/grrphoto/305649629

M2M protocols are mostly ad-hoc, and researchers and businesses focus on low level problems.

“Things” should interact with our lives, and all the

technology should be built to make them easy

to use.

• “things” exposed with binary protocol

• publish/subscribe

• topics as the naming system

• “things” exposed to the web

• request/response

• URIs as the naming system

QESTDiscover

qest.me

• MQTT broker

• REST interface

• HTTP semantics

• no QoS

• built on node.js and redis

QESTREST Server

Redis

MQTT Server

QEST

Data Layer

HTTP Clients MQTT Clients

Web App

DeviceSC

LS

DA

AR

EF

GN

D

IOR

EF

RE

SE

T3V

3P

WM

PW

M

PW

M

L

TXRX ON

ICSP

PW

MP

WM

PW

M TX

RX

31

21

11

01

9 8DIGITAL

7 6 5 4 3 2 1 0

1

5V GndPOWER

www.arduino.cc

ANALOG INVin 0 1 2 3 4 5

Arduino UNO

IoTBroker

QEST

state-of-artapproach to IoT apps

QEST-basedsolution to IoT apps

Bridge

Web App

Device

Web App

Device

IoTBroker

QEST

state-of-artapproach to IoT apps

QEST-basedsolution to IoT apps

Bridge

Web App

DeviceSC

LS

DA

AR

EF

GN

D

IOR

EF

RE

SE

T3V

3P

WM

PW

M

PW

M

L

TXRX ON

ICSP

PW

MP

WM

PW

M TX

RX

31

21

11

01

9 8DIGITAL

7 6 5 4 3 2 1 0

1

5V GndPOWER

www.arduino.cc

ANALOG INVin 0 1 2 3 4 5

Arduino UNO

QEST• retains every message received

: MQTT to REST

curl -H "Accept: txt" http://qest.me/topics/temp

client = PubSubClient(server, 1883, callback);client.publish("temp", "30");

• every topic has its own URI: /topics/<NAME>

QEST• transform every HTTP PUT received to a MQTT message

: REST to MQTT

void callback(char* topic, byte* payload, int length) { ...}PubSubClient(server, 1883, callback);client.subscribe("temp");

curl -X PUT -d '{ "payload": 42 }' \ -H "Content-Type: application/json" \ http://qest.me/topics/temp

• devices can listen directly to MQTT topics

How to scale

Redis

HTTP/MQTT Clients

REST Server MQTT Server

QESTData Layer

...

Load Balancer

REST Server MQTT Server

QESTData Layer

How much time took me to write the first version of QEST?

A day.

Thanks

How much time took me to write the first version of QEST?

• Express

• Socket.io

• node_redis

• MQTT.js

• Coffeescript

A day.

Thanks

What happens if you write a broker in a day?

(Hint)

Spaghetti code!

How to untangle it?

Refactoring!!!

http://www.flickr.com/photos/adewale_oshineye/2933030620/

1.A single file can go really far!

Well, the first improvement were two files

Coffee-scriptis awesome for prototyping!

Coffee-scriptis a real pain to debug

2.Extracting a data layer!

REST Server

Redis

MQTT Server

QEST

Data Layer

HTTP Clients MQTT Clients

Discover my roots

Ruby Language

Ruby Language

Ruby on Rails

Ruby on Rails

The ActiveRecord

pattern is AWESOME!

ActiveRecord

1. Best pattern for storing data2. Its main responsibilities are

CRUD operations3. I used that pattern to build a

cross-process pub/sub system

Hubot © 2012 GitHub Inc. All rights

3.Extracting an inter process pub/sub library

AscoltatoriDiscover

github.com/mcollina/ascoltatori

Asco

ltator

i• Redis

• ZeroMQ

• RabbitMQ

• MQTT (Mosquitto)

• Memory (EventEmitter)

is a multi-process pub/sub library backed by

Asco

ltator

ivar  ascoltatori  =  require('ascoltatori');

//  you  can  use  RedisAscoltatore,  ZeromqAscoltatore,//  RabbitAscoltatore,  MQTTAscoltatorevar  ascoltatore  =  new  ascoltatori.MemoryAscoltatore();

ascoltatore.subscribe("hello/*",  function()  {    //  this  will  print  {  '0':  "hello/42",  '1':  "a  message"  }    console.log(arguments);      process.exit(0);});

ascoltatore.publish("hello/42",  "a  message",  function()  {    console.log("message  published");});

Asco

ltator

i

1 us

10 us

100 us

1.000 us

10.000 us

100.000 us

1 10 100 1000Listeners

Memory Redis ZeroMQ RabbitMQ Mosquitto

Asco

ltator

iallowed me to refactor QEST as just an MQTT wrapper!

I am se

eking

contr

ibutors

github.com/mcollina/qest

github.com/mcollina/ascoltatori

DEMO!

Source NASA

Matteo Collina

Software Engineer @ Mavigex

Ph.D. Student @ University of Bologna

matteocollina.com

Matteo Collina (matteo.collina2@unibo.it)

Thank You!

@matteocollina

http://www.flickr.com/photos/axel-d/479627824/