Upload
adam-leadbetter
View
254
Download
2
Embed Size (px)
Citation preview
ODIP II – Data Streams and FlowsSeptember 2015
Adam Leadbetter, Damian Smyth, Rob [email protected]
http://www.confluent.io/blog/apache-kafka-samza-and-the-unix-philosophy-of-distributed-data
20th Century 21st Century
Single machine Distributed processing
One-to-one communication Publish-subscribe pattern
No fault tolerance Replication, auto-recovery
Fixed schema, encoding Schema management, evolvable encoding
20th Century 21st CenturyByte stream Messages
In-memory Durable
Blocking / backpressure Buffering
One-to-one Multi-subscriber
https://github.com/peterataylor/om-json
Further Work
• Look at the architectures for reprocessing historic data
• Incorporate Sensor Web Enablement, OM-JSON
• Deploy on vessels / mobile, remote platforms
• Investigation of Apache NiFi