Vlad Trifa - Final PhD Thesis Defense at ETH Zurich

Building Blocks for a Participatory Web of Things: Devices, Infrastructures, and Programming Frameworks

Doctoral Defense – 26 August 2011 Vlad Trifa

Vlad TrifaDoctoral Defense, 26.8.2011

Context

2

■ Networked embedded devices (NEDs) omnipresent■ Increasingly powerful CPUs, various sensors/actuators■ Mobile Internet cheap and ubiquitous

■ Tremendous benefits when integrating their data and services ■ Social: energy-efficient building, smarts buildings and cities■ Business: real-time enterprise, optimized logistics


Distributed Sensing Applications (DSA) Today

■ Tedious process that requires many resources (skills, time, $$$)■ Various functionalities, sensors, requirements■ Incompatible protocols, standards, programming models, APIs, etc.■ “Wheel reinvention” is common (hard-wired applications)

3

Web Gateway

low-power radioprotocols

(ZigBee, etc.)

base-stationconnected via

serial linestorage

analysis&

processingWeb page


Problem Statement

4

How to create an infrastructure for facilitating the development of asynchronous DSAs on top of heterogeneous, mobile NEDs?


The Web of Things

5

■ Leverage Web architecture, standards and techniques■ REST, HTTP, HTML, JSON, MIME, caching, authentication, etc.■ TCP/IP & Web granted, Wi-Fi routers ubiquitous

■ HTTP/REST: many advantages for larger DSAs■ Flexible, loosely coupled, scalable, lightweight

■ Smooth integration with existing Web infrastructure■ Blend real-world services and devices with the Web■ Development of simple Web apps: cheaper & faster


Research Questions

6

Is the Web as is sufficient to support DSAs? What is required for most DSA?

How to build a framework for developing applications that are well integrated into the Web?

Part 1

Web-enabled Devices


Web-enabled Devices

■ Devices run Web servers ■ services RESTful resources

■ REST core architecture of Web■ HTTP implements REST style

■ URI for each component■ Uniform Interface■ HTTP verbs & codes

■ Representations■ HTML: humans■ JSON/XML/CSV: machines

8

D. Guinard, V. Trifa, F. Mattern, and E. Wilde. Architecting the Internet of Things, Chapter From the Internet of Things to the Web of Things: Resource Oriented Architecture and Best Practices. Number 5. Springer, May 2011.

GET fire/alerts.xml

PUTtv/channel/4

GET fridge/food.html

WebHTTP

proprietary

Bluetooth

X10

IEEE802.15.4

DLNA

HTTP

HTTP

Google APIs

Flickr API

GatewayAPI


Gateways

9

V. Trifa, S. Wieland, D. Guinard, and T. M. Bohnert. Design and implementation of a gateway for web-based interaction and management of embedded devices. In Proc. of the 2nd International Workshop on Sensor Network Engineering (IWSNE’09), Marina del Rey, CA, USA, 2009.


Gateway Internal Architecture

10

Sun

SPOT

TMoteDT1

DT2

device

services

device layer

control layer

presentation layer

REST

engine

HTTP

server

proxy application

user

services

application

services

templating

engine

devices

DB

device

drivers

device thread pool

../devices/1

../devices/2

direct

cached

0 100 200 300 4000

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Response time [ms]

Cum

ulat

ive

Den

sity

Fun

ctio

n

DirectCached


Response time for Consecutive HTTP GET Requests

11

D CMin. 159 2Max. 3075 36Avg. 203 3.9q99% 282 19

RTT [ms] of 10’000 local read requests for direct and cached approaches

Part 2

Distributed Infrastructures


Distributed, Location-aware Infrastructures for WoT

13

V. Trifa, D. Guinard, and S. Mayer. Leveraging the Web for a Distributed Location-aware Infrastructure for the Real World. Chapter 17 in REST: From Research to Practice, Edited by Wilde and Pautasso, Springer-Verlag, 2011.

/ethz

/ethz/CNB

/ethz/CNB/FloorD

../southWing/D48.1

/ethz/CNB/FloorEvirt

ual g

atew

ays

(can

run

anyw

here

)ph

ysic

al g

atew

ays

(mus

t be

embo

died

)

../southWing/D48.2

../floorD/southWing


Service 1: HTTP-based Devices and Resources Discovery

14



14

Device with machine-readable

embedded metadata (description,

API, etc.)

1LAN

1. Device connects to router and

gets an IP address via DHCP

RouterLAN (Ethernet)



14



API, etc.)

1LAN



RouterLAN (Ethernet) LAN

2. Gateway polls the DHCP

allocation table on the router Gateway

2



14



API, etc.)

1LAN



RouterLAN (Ethernet) LAN

2. Gateway polls the DHCP

allocation table on the router Gateway

2

3

3. For each new device it gets the root

page, parses metadata and adds it to

registry


Service 2: Infrastructure-aided Location-aware Search

15

/zurich/ethz

/zurich/ethz/CAB

/zurich/ethz/CAB

"GET all meters at

/zurich/ethz/CNB/"/zurich

../room103 ../room102

"Here you are:

../room102/meter,

../room103/meter"

../meter ../meter ../spot

query scope

Part 3

Web-based Messaging

Client Server Application

Event

Event

Event

ResponseRequest

ResponseRequest

ResponseRequest


Real-time Web

■ HTTP designed as a request-response protocol (scalability)■ “Eventing” done via polling or federation (feeds: RSS/ATOM)■ Growing community around Real-time Web techniques■ HTTP callbacks, Comet, XMPP, Web Sockets, etc.

17

Client Server Application

Request

Event

Event

Event

Response

Response

Response

Comet, Web socketsAJAX (Web polling)


Web Messaging System (WMS)

■ Gateways & devices need a minimal Web push mechanism■ Not a protocol (Google’s pubsubhubbub), only Web patterns

■ Notification implemented as HTTP push callbacks [1]■ RESTful, Simple, light (less options) to run on NEDs■ Generic pub/sub support (queues & subscriptions)

18

Subscribe to a channel:POST SUB http://.../channels/ethz/CNB/H/subscribers

Publish to a channel:POST MSG http://.../channels/ethz/CNB/H/

Receive notifications:foreach SUB in http://.../channels/ethz/CNB/H/subscribers POST MSG SUB

[1] V. Trifa, D. Guinard, V. Davidovski, A. Kamilaris, and I. Delchev. Web messaging for open and scalable distributed sensing applications. In Proc. of the 10th International Conference on Web Engineering (ICWE 2010), Vienna, Austria, June 2010.

http://wms.com/channels/ethz/CNB/H/subscribers


http://wms.com/channels/ethz/CNB/H/

http://wms.com/channels/ethz/CNB/H/




Web Sensor Streams

■ Sensor data represented as sequence of messages ■ More complex constructs on top of minimal pub/sub implementaiton■ Not bound to transport protocol (HTTP callbacks, WebSockets, etc.)

■ Parameterized■ devices■ sensors■ sampling frequency■ filter data

■ Messaging: bi-directional glue between components to support asynchronous communication

19

t4 t3 t2

Devices

t1 Client

D1

D2

D3

Web Sensor Streamdata=light,temperature&devices=D1,D2,D3

{"time":123223, "data": [{"D1":{"light":4, "temp":22}}, {"D2":{"light":21, "temp":22}}, {"D3":{"light":200, "temp":33}} ]}

Part 4

End-to-end Event-driven Applications


WISSPR - Web Infrastructure for Sensor Stream PRocessing

21

MCDCvarious data

acquisition protocols

1. raw

3. raw data

Client SC

QC

3'. processed

5. raw & processed

Client

2. raw/filtered 4. processed

Sensors 1. Device Connector

2. Messaging Connector

3. Query Connector

4. Storage Connector

Vlad TrifaDoctoral Defense, 26.8.2011 22

MC

Same-machine brokerlow latency

low scalability

high coupling

Same-LAN brokermedium scalability

medium latency

lower coupling

Cloud-based broker

Highest scalability and flexibility

lower latency

lowest coupling

Wisspr main nodehttp://wisspr.net

Uniform messaging

interface (Web API)

JAVA

AMQP [1] REST

REST

[1] Advanced message queuing

protocol amqp.org


Complete HTTP Request (& Application)

23

POST /streams/ HTTP/1.1Host: wisspr.netContent-Type: application/x-www-form-urlencoded

devices=D1,D2,D3&data=temperature,light&frequency=2Hz&filter=light<200&&temperature>19&wms.cbk=http://store.wisspr.net/stores/31

Note: content should be %-encoded.

http://example.com/client_callback_handler

http://example.com/client_callback_handler


0 500 1000 1500 200010

0

101

102

103

104

Load [msg/s]

Dela

y [m

s]

processingfetching

End-to-end Notification Delay

20 Devices simulated in DC■ Varied sampling frequency (SF)

■ Average notification delay [ms](DC-MC-Clients) delay:

■ Scalability limited by clients

SF Msg/s Proc. Fetch. Total25 500 2.5 17.3 2050 1000 3.4 98 101.575 1500 4.2 250.8 255.1

100 2000 5.2 1177.7 1183


Main Contributions

1. Set of reusable patterns & best practices for physical Web■ Eventing, streaming, querying constructs■ Pervasive middleware services (discovery, search)■ Location concepts directly integrated

2. Proof of concept implementations and evaluations■ Entirely Web-based infrastructure for heterogeneous DSAs■ Reasonable scalability for devices, users, and real-time data

3. End-to-end 1-line programming framework■ Collection, processing, sharing, and storage■ Seamless Web integration

25

Thanks for your attention!

Questions?

M. [email protected]. vladtrifa.comT. @vladounetB. webofthings.com

Part 5

Backup slides

fears?


Local network (Ethernet LAN)

Main

broker

Gateway(broker)

WMS

Subscriber

Subscriber

Subscriber

ATOM Pull

Comet

Web Hooks

POST

POST

Gateway(broker)

WMSPOST

POST

Private sensor network (radio, Wi-Fi, etc.)

Public network(Internet)


MCDCOSGi Framework

OSGi Framework

3. Streamoutput

2. Raw Data

QC

Client

1. Register query [Query, CBK-URI]

3'. Stream output

HTTP/AMQPWisspr main node

http://wisspr.net

External query processor http://query.wisspr.net

JAVA

OSGi

HTTP

HTTP

real-time (raw) data streams

this is you

Web API

LIVE Singapore!

http://www.faroutof.it

http://senseable.mit.edu/livesingapore

http://aqueduct.nus.edu.sg/waterwise

http://aqueduct.nus.edu.sg/waterwise

t4 t3 t2

Devices

t1 Client

D1

D2

D3

Data Streamdata=light,temperature&devices=D1,D2,D3

{"time":123223, "data": [{"D1":{"light":4, "temp":22}}, {"D2":{"light":21, "temp":22}}, {"D3":{"light":200, "temp":33}} ]}


Web Stream details

32


Web stream with frequency and filtering

33

L=25 L=18 L=28 L=22

Client

Raw Data Stream

L=25L=22

samplingperiod

L=22L=25

YES YESNO

Resulting Stream

data=light,temperature&filter=light>20&frequency=2

or this?http://liip.to/niwea

http://blog.liip.ch/archive/2010/06/08/niwea-native-interoperable-web-apps.html

http://blog.liip.ch/archive/2010/06/08/niwea-native-interoperable-web-apps.html

devices and services ecosystem


20 Background and Motivation

extensions that add many features from secure communication to discovery. Most applications

and programming languages support XML and HTTP, and one could easily build stubs to

interact with specific services that do not need to be upgraded as long as the WSDL file does

not change. Unfortunately, even though tools that help to automate the development of such

services became widely spread and used, SOAP-based Web services remain quite difficult to

understand and use, as in practice different implementations of the same WS-* standards are

rarely entirely compatible with each other.

Another problem persists with SOAP-based Web services: they are a legacy of RPC style

protocols, therefore are not flexible enough for highly dynamic environments where new, un-

known devices continuously appear and disappear. Besides, WS-* consider HTTP only as a

transport protocol to perform remote procedure calls instead of being directly integrated into

the Web [244], in which case there would be no need for any additional API or descriptions of

resource/function. This situation gave birth to numerous debates about ROA versus SOA5.

Attribute OOA SOA ROA

Granularity objects services data (resources)Main focus marshaling creation of payload addressing (URI)Addressing object instance unique endpoint individual resources

Replies cacheable No No YesAPI language-specific application-specific generic

Lightweightness +++ + ++Flexibility + ++ +++Scalability +++ ++ +++

Loose-coupling + ++ +++Simplicity +(+) + +++Standard + ++ +++

Tools available +++ +++ +Security +++ ++ ++Verbosity + +++ ++Ambiguity + ++ ++

Table 2.1: Overall feature comparison of different architectural styles (+ means low, ++ means

medium, +++ means high).

In Table 2.1, we summarize the differences and properties of the various styles we presented.

Based on the requirements we defined in Section 2.1.4, we suggest the ROA style is the most

suitable architectural style to implement a scalable participatory infrastructure for NED ap-

plications. In particular, as long as applications do not require custom protocols to meet

particular “hard” requirements (such as reliability, security or throughput), the loose coupling

of ROA architectures offers many advantages and features useful for NED applications. Also,

5An insightful discussion about the roots of this debate can be found here:http://www.prescod.net/rest/rest_vs_soap_overview/

Technology

Vlad Trifa - Final PhD Thesis Defense at ETH Zurich