Upload
antonio-pintus
View
1.595
Download
0
Tags:
Embed Size (px)
DESCRIPTION
My presentation about a scalable architecture for a social Web of Things (Paraimpu) @ WoT2011 Workshop in San Francisco, June 12, 2011
Citation preview
Antonio Pintus - [email protected] Davide Carboni - [email protected] Andrea Piras - [email protected]
The anatomy of a large scale Social Web for Internet enabled Object
Antonio Pintus Davide Carboni Andrea Piras
CRS4
WoT 2011 - June 2011 - San Francisco, CA, USA
1
Antonio Pintus, Davide Carboni, Andrea Piras WoT2011 - June 2011- San Francisco, CA, USA
Who, where...
• Antonio Pintus and Davide Carboni and Andrea Piras
• ICT - LBS Group @ CRS4, an interdisciplinary applied research center based in Pula (Cagliari), SARDINIA, Italy
2
Antonio Pintus, Davide Carboni, Andrea Piras WoT2011 - June 2011- San Francisco, CA, USA
The Web of Things
• In the Web of Things (WoT) vision, things are connected to a pervasive wireless/wired network and can communicate with other objects and services using HTTP-based protocols
• Everyday surrounding objects (phones, domestic appliances, advertisement billboards, musical instruments,... ) become nodes of the WoT
• Web-enabled objects are available on the market just now!
3
Antonio Pintus, Davide Carboni, Andrea Piras WoT2011 - June 2011- San Francisco, CA, USA 4
part #1
Our Vision
Ph. http://www.flickr.com/photos/salendron/
Antonio Pintus, Davide Carboni, Andrea Piras WoT2011 - June 2011- San Francisco, CA, USA
Vision: a Social Web of Things /1
5
#1: connecting Things to the Web must be simple
Ph. Nigel Marshall, http://www.flickr.com/photos/nige_mar/
- simple to “plug”- simple to con!gure- simple to manage- Web-based UX
Antonio Pintus, Davide Carboni, Andrea Piras WoT2011 - June 2011- San Francisco, CA, USA
Vision: a Social Web of Things /2
• #2: not only “physical” Things but also “virtual” ones–social networks, Web sites, feeds, APIs, ...
6
Antonio Pintus, Davide Carboni, Andrea Piras WoT2011 - June 2011- San Francisco, CA, USA
Vision: a Social Web of Things /3
7
#3: interconnecting Things in the Web must be simple
- including “physical” and “virtual” Things
- data adapting / transformation
Ph. John Thurm, http://www.flickr.com/photos/thurm
Antonio Pintus, Davide Carboni, Andrea Piras WoT2011 - June 2011- San Francisco, CA, USA
Vision: a Social Web of Things /4
8
Ph. Nigel Marshall, http://www.flickr.com/photos/nige_mar/
#4: Social /1: existing social networks become connectable Things
- Authentication facilities, friends/contacts import
Antonio Pintus, Davide Carboni, Andrea Piras WoT2011 - June 2011- San Francisco, CA, USA
Vision: a Social Web of Things /5
9
Ph. Ryan Roberts http://www.flickr.com/photos/ryanr
#5: Social /2: Share your Things with friends!
Antonio Pintus, Davide Carboni, Andrea Piras WoT2011 - June 2011- San Francisco, CA, USA
Things Sharing
– Things sharing:
• it should be possible to share a Thing with friends and let them use it in their connections (according to a de!ned policy: public, open, moderated)
• sharing objects goes toward a “collaborative consumption” as shaped by the authors of the book What’s mine is yours*
* What's Mine Is Yours: The Rise of Collaborative Consumption, Rachel Botsman and Roo Rogers . Harperbusiness, http://www.collaborativeconsumption.com/
10
Antonio Pintus, Davide Carboni, Andrea Piras WoT2011 - June 2011- San Francisco, CA, USA 11
Ph. Windell Oskay http://www.flickr.com/photos/oskay/265899865/
The Model: 4 basic abstractions
- Services:- Sensors- Actuators
- Connections
Antonio Pintus, Davide Carboni, Andrea Piras WoT2011 - June 2011- San Francisco, CA, USA 12
Services: Sensors
- every Thing capable to produce data
- using HTTP
- anemometers, Arduino boards, but also Foursquare, Pachube, other services in the Web...
Ph. Windell Oskay http://www.flickr.com/photos/oskay/
Antonio Pintus, Davide Carboni, Andrea Piras WoT2011 - June 2011- San Francisco, CA, USA 13
Services: Actuators
- every Thing capable to receive/consume data and to perfom an action
- using HTTP
- lighting systems, Arduino, Chumby, Karotz, but also Twitter, Facebook, other applications...
Ph. Dieter Drescher http://www.flickr.com/photos/cosmosfan
Antonio Pintus, Davide Carboni, Andrea Piras WoT2011 - June 2011- San Francisco, CA, USA
Connections
14
Ph. supermaryan http://www.flickr.com/photos/33975256@N06/
- between two Services:
- a Sensor as the source- an Actuator as the destination
- they are con!gurable:
- data !ltering- data mapping
Antonio Pintus, Davide Carboni, Andrea Piras WoT2011 - June 2011- San Francisco, CA, USA
problems: architecture & scalability
• Scalability
– What happens when the number of users grows?
– What happens when thousands of things are added?
– What happens when thousands of near real-time data have to be managed?
– System MUST be scalable by design!
15
Antonio Pintus, Davide Carboni, Andrea Piras WoT2011 - June 2011- San Francisco, CA, USA 16
Ph. MrFaber, http://www.flickr.com/photos/mrfaber
problems: architecture & scalability
C10K problem
Horizontal Scalability
Database
New Sensors/Actuators
Web server
Load balancing
Web-based Architecture
WS-* or REST
?
Antonio Pintus, Davide Carboni, Andrea Piras WoT2011 - June 2011- San Francisco, CA, USA 17
part #2
Architectural & Design Choices
Ph. Jaime Silva, http://www.flickr.com/photos/20792787@N00/
Antonio Pintus, Davide Carboni, Andrea Piras WoT2011 - June 2011- San Francisco, CA, USA
WS-* or REST?
• in many cases adopting a WS-* stack is too complex: WSDL, SOAP over HTTP, ...
• often, we’re talking about (limited) physical digital Things
• we’ve chosen REST as architectural style:
» Things, connections, data, users, ..., are REST resources
» simpler, "exible, “mashable”, APIs18
Antonio Pintus, Davide Carboni, Andrea Piras WoT2011 - June 2011- San Francisco, CA, USA
Data persistence: SQL or NoSQL?
• Considering:– high data heterogeneity: DB schema "exibility required– database scalability across distributed systems– data partitioning & data replication
• We’ve chosen a NoSQL DB:
» no !xed schemas, no tables, no relations» high concurrent read-write, scalability, auto-
sharding» JSON native format
19
Antonio Pintus, Davide Carboni, Andrea Piras WoT2011 - June 2011- San Francisco, CA, USA
Web server, Web front-end
• Full Web-based system
• C10K problem: 10,000 simultaneous open standing HTTP connections (clients)
• Horizontal scalability: add new server instances when required
• Load balancing/reverse proxy
20
Antonio Pintus, Davide Carboni, Andrea Piras WoT2011 - June 2011- San Francisco, CA, USA
Things Connections
21
• data transfers between DB-based queues• data adaptation performed by system processes• data adaptation rules de!ned by users for each owned
connection
Antonio Pintus, Davide Carboni, Andrea Piras WoT2011 - June 2011- San Francisco, CA, USA
Data adaptation: !lters
• a Filter is a boolean expression evaluated against data coming from a Sensor
–using Javascript as expression language
» sensor > 12» sensor.!eld == ‘mylabel’» sensor.data >20 && sensor.type == ‘temp’» ...
22
Antonio Pintus, Davide Carboni, Andrea Piras WoT2011 - June 2011- San Francisco, CA, USA
Data adaptation: mappings
• Mappings are couple of expressions: (cond, repl)
• cond : a boolean expression evaluated against Sensor data (same as for !lters)
• repl : an expression producing a data value to be
forwarded to the Actuator when cond is true
23
Antonio Pintus, Davide Carboni, Andrea Piras WoT2011 - June 2011- San Francisco, CA, USA
Mappings, examples
24
Sensor data type
CONDITION REPLACE Actuator data type
numeric sensor > 30 <% sensor * 3.14 / 100 %> numeric
numeric sensor > 25Summer is coming, temperature is:
<% sensor %> °Ctext
JSON sensor.data > 30Data is <% sensor.data %>
and type is <% sensor.type %> text
JSONsensor.A0 > 30 &&
sensor.A5 <= 10
<?xml version="1.0" encoding="UTF-8"?> <value>
<% sensor.A0 * 100 %>
</value>
XML
– Javascript as expression language
Antonio Pintus, Davide Carboni, Andrea Piras WoT2011 - June 2011- San Francisco, CA, USA
Paraimpu
• is a Web-based social tool
• allows people to:
– connect, use, share and compose Things, services and devices to create personalized WoT applications
25
Antonio Pintus, Davide Carboni, Andrea Piras WoT2011 - June 2011- San Francisco, CA, USA
General Architecture
26
DB scalability
Web front-end scalability
Load balancing
System processes (back-end) scalability
Antonio Pintus, Davide Carboni, Andrea Piras WoT2011 - June 2011- San Francisco, CA, USA
Paraimpu: the workspace
27
Antonio Pintus, Davide Carboni, Andrea Piras WoT2011 - June 2011- San Francisco, CA, USA
Current version
• ~ 50 active alpha users
• Palette: • 5 ready-to-use Sensors + 1 generic Sensor• 7 ready-to-use Actuators + 1 generic Actuator
• Sharing policy: (private), public, open, moderated
• Noti!cation system between users for Things sharing28
Antonio Pintus, Davide Carboni, Andrea Piras WoT2011 - June 2011- San Francisco, CA, USA
Technologies
• Architecture mainly written in Python
• Web stack based on Tornado Web server
• NoSQL DBMS (MongoDB)
• HTML+jQuery
29
Antonio Pintus, Davide Carboni, Andrea Piras WoT2011 - June 2011- San Francisco, CA, USA
Conclusion
• We believe that a social Web of Things can help in its diffusion...
• Results are encouraging: alpha users are helping us in testing and improving the system
• More and better test beds are under de!nition to prove system scalability in each of its critical aspects
• We’re working hard to add new features and new ready-to-use, easy-con!guration, components and Things
30
Antonio Pintus, Davide Carboni, Andrea Piras WoT2011 - June 2011- San Francisco, CA, USA
The prototype
http://paraimpu.crs4.it
twitter: @paraimpu
31
Antonio Pintus, Davide Carboni, Andrea Piras WoT2011 - June 2011- San Francisco, CA, USA
Contacts
• Antonio Pintus, [email protected], twitter: @apintux• Davide Carboni, [email protected]• Andrea Piras, [email protected]
• LBS group @ CRS4: http://geoweb.crs4.it/
• CRS4 Home Page: http://www.crs4.it
32
Antonio Pintus, Davide Carboni, Andrea Piras WoT2011 - June 2011- San Francisco, CA, USA
Thank you for your attention
33
Q & A