Upload
angelo-corsaro
View
904
Download
3
Embed Size (px)
Citation preview
Data Sharing in Industrial IoT
AngeloCorsaro,PhDChiefTechnologyOfficer
Vortex is a standard technology for efficient, ubiquitous, interoperable, secure and platform independent data sharing across network connected devices
in151 Characters
Vortex's Coordination Model
Applications can autonomously and asynchronously read and
write data enjoying spatial and temporal decoupling
DDS Global Data Space
...
Data Writer
Data Writer
Data Writer
Data Reader
Data Reader
Data Reader
Data Reader
Data Writer
TopicAQoS
TopicBQoS
TopicCQoS
TopicDQoS
Global Data Space
Built-in dynamic discovery isolates applications from
network topology and connectivity details
DDS Global Data Space
...
Data Writer
Data Writer
Data Writer
Data Reader
Data Reader
Data Reader
Data Reader
Data Writer
TopicAQoS
TopicBQoS
TopicCQoS
TopicDQoS
Dynamic Discovery
The relevant portion of the data space is projected on
the application address space. Each typed projection is commonly called a Cache
No single point of failure or bottleneck
Decentralised Data Space
Data Writer
Data Writer
Data Writer
Data Reader
Data Reader
Data Reader
Data Writer
TopicAQoS
TopicBQoS
TopicCQoS
TopicDQoS
TopicDQoS
TopicDQoS
TopicAQoS
Connectivity is dynamically adapted to chose the most
effective way of sharing data
Adaptive ConnectivityData
Writer
Data Writer
Data Writer
Data Reader
Data Reader
Data Reader
Data Writer
TopicAQoS
TopicBQoS
TopicCQoS
TopicDQoS
TopicDQoS
TopicDQoS
TopicAQoS
ThecommunicationbetweentheDataWriterandmatchingDataReaderscanbepeer-to-peerexploitingUDP/IP(UnicastandMulticast)orTCP/IP
ThecommunicationbetweentheDataWriterandmatchingDataReaderscanbe“brokered”butstillexploitingUDP/IP(UnicastandMulticast)orTCP/IP
Topic
A domain-wide information’s class A Topic defined by means
of a <name, type, qos>
TopicDDS Global Data Space
...
Data Writer
Data Writer
Data Writer
Data Reader
Data Reader
Data Reader
Data Reader
Data Writer
TopicAQoS
TopicBQoS
TopicCQoS
TopicDQoS
TopicTypeName
QoS
Topic Types
Topic Types: Language Independent
Definitions
Topic types can be expressed using different syntaxes,
including IDL and ProtoBuf
Topic Type struct CarDynamics { string cid; long x; long y; float dx; long dy; } #pragma keylist CarDynamics cid
IDL
Topic types can be expressed using different syntaxes,
including IDL and ProtoBuf
Topic Type message CarDynamics { option (.omg.dds.type) = {name: "CarDynamics"}; required string cid = 0 [(.omg.dds.member).key = true]; required long x = 1; required long y = 2; required float dx = 3; required long dy = 4; }
ProtoBuf
Topic Types: Language Specific
Definitions
Topic types can be expressed using different syntaxes,
including IDL and ProtoBuf
Topic Type class CarDynamics: constructor: (@cid, @x, @y, @dx, @dy) ->
CoffeeScript
Topic types can be expressed using different syntaxes,
including IDL and ProtoBuf
Topic Type public struct CaDynamics { public string cid { get; set; } public int x { get; set; } public int y { get; set; } public int dx { get; set; } public int dy { get; set; } public CaDynamics (string cid, int x, int y, int dx, int dy) { this.cid = cid; this.x = x; this.y = y; this.dx = dx; this.dy = dy; } }
C#
Topic types can be expressed using different syntaxes,
including IDL and ProtoBuf
Topic Type @KeyList ( topicType = "CarDynamics", keys = {"cid"})public class CarDynamics { public String cid; public int x; public int dx; public int y; public int dy; public CarDynamics(String s, int a, int b, int c,int d) { this.cid = s; this.x = a; this.dx = b; this.y = c; this.dy = d; } @Override public String toString() { … }}
Java
Topics, Instances and Samples
Cop
yrig
ht P
rism
Tech
, 201
5
Each unique key value identifies a stream of data
Vortex demultiplexes “streams” and provides lifecycle information
A DataWriter can write samples belonging to multiple instances
Topic Instances
CarDynamics
\
Instances
struct CarDynamics { @key string cid; long x; long y; float dx; long dy; }
Topic
Topic Type
Sample
QoS policies allow to express temporal and availability
constraints for data
DDS Global Data Space
...
Data Writer
Data Writer
Data Writer
Data Reader
Data Reader
Data Reader
Data Reader
Data Writer
TopicAQoS
TopicBQoS
TopicCQoS
TopicDQoS
QoS - Enabled
A collection of policies that control non-
functional properties such as reliability,
persistence, temporal constraints and priority
QoS
HISTORY
LIFESPAN
DURABILITY
DEADLINE
LATENCY BUDGET
TRANSPORT PRIO
TIME-BASED FILTER
RESOURCE LIMITS
USER DATA
TOPIC DATA
GROUP DATA
OWENERSHIP
OWN. STRENGTH
LIVELINESS
ENTITY FACTORY
DW LIFECYCLE
DR LIFECYCLE
PRESENTATION
RELIABILITY
PARTITION
DEST. ORDER
RxO QoS Local QoS
QoS Policies controlling end-to-end properties
follow a Request vs. Offered
QoS Domain
Participant
DURABILITY
OWENERSHIP
DEADLINE
LATENCY BUDGET
LIVELINESS
RELIABILITY
DEST. ORDER
Publisher
DataWriter
PARTITION
DataReader
Subscriber
DomainParticipant
offered QoS
Topicwrites reads
Domain Idjoins joins
produces-in consumes-from
RxO QoS Policies
requested QoS
Interacting with the Data Cache
Each Data Reader is associated with a Cache
The Cache stores the last n∊𝜨∞ samples for each
relevant instance
Data Cache
Data Writer
Data Writer
Data Writer
Data Reader
Data Reader
Data Reader
Data Writer
TopicAQoS
TopicBQoS
TopicCQoS
TopicDQoS
TopicDQoS
TopicDQoS
TopicAQoS
Each Data Reader is associated with a Cache
The Cache stores the last n∊𝜨∞ samples for each
relevant instance
Data Cache
DataReader Cache
DataReader
...
Samples
Instances
Cache
The action of reading samples for a Reader Cache
is non-destructive.
Samples are not removed from the cache
Reading Data
DataReader Cache
DataReader
...
DataReader Cache
DataReader
...read
The action of taking samples for a Reader Cache
is destructive.
Samples are removed from the cache
Taking Data
DataReader Cache
DataReader
...
DataReader Cache
DataReader
...take
Samples can be selected using composable content
and status predicates
Sample Selectors
DataReader Cache
DataReader
...
Filters allow to control what gets into a DataReader
cache
Filters are expressed as SQL where clauses or as
Java/C/JavaScript predicates
Content-Filtering
DataReader Cache
DataReader
...
Filter
Application
Network
Content Filters can be used to project on the
local cache only the Topic data
satisfying a given predicate
Content Filters structCarDynamics{
@keystringcid;longx;longy;floatdx;longdy;}
cid x y dx dyGR 33N GO 167 240 45 0LO 00V IN 65 26 65 0AN 637 OS 32 853 0 50AB 123 CD 325 235 80 0
“dx>50ORdy>50”
Type
CarDynamics
cid x y dx dyLO 00V IN 65 26 65 0AB 123 CD 325 235 80 0
Reader Cache
Queries allow to control what gets out of a
DataReader Cache
Queries are expressed as SQL where clauses or as
Java/C/JavaScript predicates
Content-Based Selection
DataReader Cache
DataReader
...
Query
DataReader Cache
DataReader
...
Application
Network
Reader Cache
Queries can be used to select out of the local cache
the data matching a given predicate
QueriesstructCarDynamics{@keystringcid;longx;longy;floatdx;longdy;}
cid x y dx dyGR 33N GO 167 240 45 0LO 00V IN 65 26 65 0AN 637 OS 32 853 0 50AB 123 CD 325 235 80 0
“dx>50ORdy>50”
Type
CarDynamics
cid x y dx dyGR 33N GO 167 240 45 0LO 00V IN 65 26 65 0AN 637 OS 32 853 0 50AB 123 CD 325 235 80 0
cid x y dx dyLO 00V IN 65 26 65 0AB 123 CD 325 235 80 0
query
State based selection allows to control what gets out of a DataReader Cache
State base selectors predicate on samples meta-
information
State-Based Selection
DataReader Cache
DataReader
...
State Selector
DataReader Cache
DataReader
...
Application
Network
Sample State (READ, NOT_READ) allows to distinguish between new
samples and samples that have already been read
State-Based Selection
DataReader Cache
DataReader
...
State Selector
DataReader Cache
DataReader
...
Application
Network
The View State (NEW, NOT_NEW) allows to distinguish a new instance
from an existing one
State-Based Selection
DataReader Cache
DataReader
...
State Selector
DataReader Cache
DataReader
...
Application
Network
The Instance State (ALIVE, NOT_ALIVE_DISPOSED,
NOT_ALIVE_NO_WRITERS) allows to track the life-cycle transitions of the instance to which a sample belongs
State-Based Selection
DataReader Cache
DataReader
...
State Selector
DataReader Cache
DataReader
...
Application
Network
Selector Example
// == ISO C++ DDS API ==
auto data = dr.select() .content(query) .state(data_state) .instance(handle) .read();
your first vortex app
Cop
yrig
ht P
rism
Tech
, 201
4
Anatomy of a DDS Application
Cop
yrig
ht P
rism
Tech
, 201
5
Writing Data in C++#include <dds.hpp>
int main(int, char**) {
DomainParticipant dp(0); Topic<Meter> topic(“SmartMeter”); auto pqos = dp.get_default_pubqos() << Partition (“FR:IdF:Essonne:Gif:Essonne:Mnts:123”); Publisher pub(dp, pqos); DataWriter<Meter> dw(pub, topic);
while (!done) { auto value = readMeter() dw.write(value); std::this_thread::sleep_for(SAMPLING_PERIOD); }
return 0; }
enumUtilityKind{ ELECTRICITY, GAS, WATER};structMeter{ stringsn; UtilityKindutility; floatreading; floaterror;};#pragmakeylistMetersn
Cop
yrig
ht P
rism
Tech
, 201
5
Reading Data in C++#include <dds.hpp>
int main(int, char**) {
DomainParticipant dp(0); Topic<Meter> topic(”SmartMeter”); Subscriber sub(dp); DataReader<Meter> dr(dp, topic);
LambdaDataReaderListener<DataReader<Meter>> lst; lst.data_available = [](DataReader<Meter>& dr) { auto samples = data.read(); std::for_each(samples.begin(), samples.end(), [](Sample<Meter>& sample) { std::cout << sample.data() << std::endl; } } dr.listener(lst); // Print incoming data up to when the user does a Ctrl-C std::this_thread::join(); return 0; }
enumUtilityKind{ ELECTRICITY, GAS, WATER};structMeter{ stringsn; UtilityKindutility; floatreading; floaterror;};#pragmakeylistMetersn
Cop
yrig
ht P
rism
Tech
, 201
4
Writing Data in Scalaimport dds._import dds.prelude._import dds.config.DefaultEntities._
object SmartMeter { def main(args: Array[String]): Unit = { val topic = Topic[Meter](“SmartMeter”) val dw = DataWriter[Meter](topic) while (!done) { val meter = readMeter() dw.write(meter) Thread.sleep(SAMPLING_PERIOD) } }}
enumUtilityKind{ ELECTRICITY, GAS, WATER};structMeter{ stringsn; UtilityKindutility; floatreading; floaterror;};#pragmakeylistMetersn
Cop
yrig
ht P
rism
Tech
, 201
4
Reading Data in Scala
import dds._import dds.prelude._import dds.config.DefaultEntities._
object SmartMeterLog { def main(args: Array[String]): Unit = { val topic = Topic[Meter](“SmartMeter”) val dr = DataReader[Meter](topic) dr listen { case DataAvailable(_) => dr.read.foreach(println) } }}
enumUtilityKind{ ELECTRICITY, GAS, WATER};structMeter{ stringsn; UtilityKindutility; floatreading; floaterror;};#pragmakeylistMetersn
Cop
yrig
ht P
rism
Tech
, 201
5
Writing Data in Python
import dds import timeif __name__ == '__main__': topic = dds.Topic("SmartMeter", "Meter") dw = dds.Writer(topic) while True: m = readMeter() dw.write(m) time.sleep(0.1)
enumUtilityKind{ ELECTRICITY, GAS, WATER};structMeter{ stringsn; UtilityKindutility; floatreading; floaterror;};#pragmakeylistMetersn
Cop
yrig
ht P
rism
Tech
, 201
5
Reading Data in Pythonimport ddsimport sys def readData(dr): samples = dds.range(dr.read()) for s in samples: sys.stdout.write(str(s.getData())) if __name__ == '__main__': t = dds.Topic("SmartMeter", "Meter") dr = dds.Reader(t) dr.onDataAvailable = readData
enumUtilityKind{ ELECTRICITY, GAS, WATER};structMeter{ stringsn; UtilityKindutility; floatreading; floaterror;};#pragmakeylistMetersn
Vortex Technology Stack
Device implementations optimised for OT, IT and
consumer platforms
Native support for Cloud and Fog Computing Architectures
Device-2-DeviceDevice-2-Cloud
Fog-2-Cloud
Device-2-Fog
Cloud-2-Cloud
Fog-2-Fog
infra
structure
sdk
Available across IT, Consumer and OT platforms Device-2-DeviceDevice-2-Cloud
Fog-2-Cloud
Device-2-Fog
Cloud-2-Cloud
Fog-2-Fog
infra
structure
sdk
Polyglot and Interoperable across Programming
Languages Device-2-DeviceDevice-2-Cloud
Fog-2-Cloud
Device-2-Fog
Cloud-2-Cloud
Fog-2-Fog
infra
structure
sdk
Fully Independent of the Cloud Infrastructure
Private Clouds
Device-2-DeviceDevice-2-Cloud
Fog-2-Cloud
Device-2-Fog
Cloud-2-Cloud
Fog-2-Fog
infra
structure
sdk
Native Integration with the hottest real-time analytics
platforms and CEP Device-2-DeviceDevice-2-Cloud
Fog-2-Cloud
Device-2-Fog
Cloud-2-Cloud
Fog-2-Fog
infra
structure
sdk
Integration with mainstream Dashboard Technologies
Device-2-DeviceDevice-2-Cloud
Fog-2-Cloud
Device-2-Fog
Cloud-2-Cloud
Fog-2-Fog
infra
structure
sdk
High Performance 30 μs peer-to-peer latency
4+ Mmsgs/sec p2p throughput
Device-2-DeviceDevice-2-Cloud
Fog-2-Cloud
Device-2-Fog
Cloud-2-Cloud
Fog-2-Fog
infra
structure
sdk
>10 μs fog/cloud routing latency
High Performance Device-2-DeviceDevice-2-Cloud
Fog-2-Cloud
Device-2-Fog
Cloud-2-Cloud
Fog-2-Fog
infra
structure
sdk
SecureData-Level security with Pluggable Authentication Access Control and Crypto
Device-2-DeviceDevice-2-Cloud
Fog-2-Cloud
Device-2-Fog
Cloud-2-Cloud
Fog-2-Fog
infra
structure
sdk
Default Plug-insX.509 Public Key
Infrastructure (PKI) based authentication
Device-2-DeviceDevice-2-Cloud
Fog-2-Cloud
Device-2-Fog
Cloud-2-Cloud
Fog-2-Fog
infra
structure
sdk
Default Plug-insAccess Control List file
signed by a shared Certificate Authority for
Device-2-DeviceDevice-2-Cloud
Fog-2-Cloud
Device-2-Fog
Cloud-2-Cloud
Fog-2-Fog
infra
structure
sdk
Default Plug-ins
Crypto based on TLS Cipher Suite
Device-2-DeviceDevice-2-Cloud
Fog-2-Cloud
Device-2-Fog
Cloud-2-Cloud
Fog-2-Fog
infra
structure
sdk
SecureData-Security as opposed to simply Transport-Level
security
Arthur Dent
Arthur Dent
Ford Prefect
Zaphod Beeblebrox
Marvin
Trillian
left/A(r,w), left/B(r)
left/A(r,w), left/B(r,w), left/X(r)
left/*(r,w)
left/*(r), right/(w)
left/A(r,w), left/B(r,w), right/C(r,w)
Ford Prefect
Zaphod Beeblebrox
Trillian
Marvin
A
B
A,BX
*
*
A,B,C
Identity Access RightsSessions are authenticated and communication is encrypted
Only the Topic included as part of the access rights are visible and accessible
SecureFine-grained access control
over Partition/Topic/Instance regular expressions
Arthur Dent
Arthur Dent
Ford Prefect
Zaphod Beeblebrox
Marvin
Trillian
left/A(r,w), left/B(r)
left/A(r,w), left/B(r,w), left/X(r)
left/*(r,w)
left/*(r), right/(w)
left/A(r,w), left/B(r,w), right/C(r,w)
Ford Prefect
Zaphod Beeblebrox
Trillian
Marvin
A
B
A,BX
*
*
A,B,C
Identity Access RightsSessions are authenticated and communication is encrypted
Only the Topic included as part of the access rights are visible and accessible
SecureSupport for Symmetric and Asymmetric Authentication
Arthur Dent
Arthur Dent
Ford Prefect
Zaphod Beeblebrox
Marvin
Trillian
left/A(r,w), left/B(r)
left/A(r,w), left/B(r,w), left/X(r)
left/*(r,w)
left/*(r), right/(w)
left/A(r,w), left/B(r,w), right/C(r,w)
Ford Prefect
Zaphod Beeblebrox
Trillian
Marvin
A
B
A,BX
*
*
A,B,C
Identity Access RightsSessions are authenticated and communication is encrypted
Only the Topic included as part of the access rights are visible and accessible
Boundary SecurityBoundary security support is
enabled by Vortex-Fog
Device-to-Cloud Communication
Peer-to-Peer (Broker-less)
Device-to-Device Communication
Fog Computing Fog ComputingFog Computing
TLS
TLS
Boundary SecuritySeparates security concerns at
different scales and controls what information is exposed
Device-to-Cloud Communication
Peer-to-Peer (Broker-less)
Device-to-Device Communication
Fog Computing Fog ComputingFog Computing
TLS
TLS
Typical Deployment
Vortex device, such as Lite, Café, and Web are used to share data between different kinds of applications within a car
Café and Web are typically used Android / HTML5 based infotainment
Lite in typically used in ECU, sensors and onboard analytics
Vortex Fog is used to transparently (for in car apps) decouple and control the data sharing within and across the car
Vortex Fog also helps defining security boundaries and policies
Vortex Fog efficiently and securely deals with car to car communication
Vortex Fog allows to decouple the transport used for in-car communication and that used for car-to-car communication
Vortex Fog efficiently and securely deals with cloud connectivity adapting traffic flows and protocols
Device-to-Cloud Communication
Cloud Analytics
Fog AnalyticsFog Analytics
Fog Analytics
Vortex Cloud efficiently and securely makes data available to any device at an Internet Scale
Vortex Cloud efficiently and securely makes data available to any device at an Internet Scale
Device-to-Cloud Communication
Cloud Analytics
Fog AnalyticsFog Analytics
Fog Analytics
Use Cases
Cop
yrig
ht P
rism
Tech
, 201
5
Connected Boulevard — Nice, FranceVortex enables is the data sharing platform
for the City Operating System
Tens of thousands of devices connected by Vortex
Autonomous Vehicles
coordination of fast moving autonomous vehicles
intermittent connectivity
dynamic pairing of devices
Cop
yrig
ht P
rism
Tech
, 201
4
Smart Factory0.5 TB of data
produced per day
Cop
yrig
ht P
rism
Tech
, 201
4
Smart-Grid
20ms deadline for phase alignment data
ESA Extremely Large & SMART Telescope (ELT)
1750 computing nodes
100.000 mirrors whose position is
adjusted 100 times per second!
ELT will allow astronomers to probe the earliest stages
of the formation of planetary systems and to detect water and organic
molecules in proto-planetary discs around
stars in the making
Launch System
80K+ data points with aggregate updates rate of ~400K msgs/sec
Connected Aircrafts
10 TB of data every 30m of flight
Oil Rig 30000 data pointsonly 1% of available data
used today
Smart Lightbulbs96Kbytes Memory
Connected Medical Devices
12 msec period for pleath data
Peer-to-Peer connectivity
Vortex enables Smart, Secure and Efficient Data Sharing in IoT
In Summary
Cop
yrig
ht P
rism
Tech
, 201
5