Fitman webinar 2015 06 Dynamic CEP

Welcome to theFITMAN Specific Enabler Webinar on DyCEP

16th June 2015

Nenad StojanovicAleksandar Stojadinovic

16/06/2015 FITMAN Webinars - June 2015

Introduction

• DyCEP - Dynamic CEP

– DyCEP enables very complex real-time processing of the shop-floor events and other events relevant for manufacturing process (e.g. from MES).

• Targeted at Smart Factories

• Based on GE Esper4FastData, i.e. it extends the functionalities of Esper4FastData with more complex real-time processing.

• Delivered as Storm compatible JAR (however, source code changes may be needed).


DyCEP - Scope

• One of the main challenges for modern manufacturing is the possibility to react on particular situations in order to improve the efficiency of manufacturing or avoid some problems.

• Current systems for real-time processing in manufacturing are mainly focused on the simple reactivity, i.e. the possibility to define “simple” situations and react in “one step” on them.

• It is usually implemented through a CEP (Complex Event Processing) engine. However, more demanding manufacturing of the future will require “complex” reactivity that is essentially based on the discovery of complex situations of interest in real-time (zero delay, or even ahead of time).

• Moreover, in order to cope with very changeable environment (internal, external) in the manufacturing, the provided processing infrastructure must support dynamic changes, which is one of the main challenges for the current data processing infrastructure.


DyCEP vs. CEP

CEP Engine as a central verypowerful processor

CEP as a Network

CEP Engine

input stream

input streams

ComplexReal-time Processing Network

Condition

Processing 1

Condition

Processing 2


DyCEP – the structure

• For the case of envisioned dynamic monitoring new processing capabilities are requested, esp. in the context of processing pipelines, i.e. enabling complex processing networks.

• DyCEP assumes several processing elements which are connected in the way that ensures maximal efficiency of the processing task.

• This infrastructure is based on the Storm, the architecture for real-time (streaming) data processing. It uses custom created "spouts" and"bolts" to define information sources andmanipulations to allow real-time distributedprocessing of streaming data.

FITMAN Webinars - June 201516/06/2015

DyCEP – the goal

• Ultimate goal: Discover a complex real-time situation of interest timely (zero delay, oreven ahead of time)

• Subgoals:

1. Define the situation of interest properly

2. Organize data-driven detection process efficiently


DyCEP – the characteristics

• Main advantage of the approach is the possibility to change the patterns deployed in the CEP engines (embedded in bolts).

• In that way we ensure that the processing infrastructure will be able to follow the dynamics of input data.

• In addition, such an approach will enable continuous improvement of the detection process that is very important for decreasing the number of false positive, which is one of the biggest problems for real-time processing approaches.


DyCEP - Architecture

16/06/2015FITMAN Webinars - June 2015

DyCEP – components (1/2)

• Broker is the mechanism for enabling subscriptions for relevant events by DyCEP (pub-sub functionality).

• Web API is responsible for getting input data in the DyCEP. It satisfies NGSI10 interface, as requested by FI PPP infrastructure. The format of data follows the specification provided by use case partners

• Simple event handling spout enables receiving events of interest and forwarding them to the rest of the processing network


DyCEP – components (2/2)

• Esper4FastData is FIWARE GE responsible for complex event processing (realized as external service, offered from a bolt). Outcome is realized through a web service.– responsible for static processing that requires simple complex

event patterns and shouldn’t scale (because of the limited data input)

• ESPER-1 is the CEP engine hosted in a bolt (in contrast to Esper4FastData – realized as external service). Outcome is realized through a web service. In addition notifications can be pushed to mobile devices.– responsible for complex situations and prepared for horizontal

scaling (by being hosted in a bolt, so that several bolt instances can be generated on demand

• GCM Push Bolt enables usage of the Google service for pushing notification to mobile devices


DyCEP – patterns

• The creation of the CEP patterns can be done in the manual way, or using a specialized mobile-based pattern editor provided by Nissatech.


DyCEP – input


TRW use case: Processing pipeline

1. Worker

2. Kinect motion sensing unit

3. Workstation controlling the Kinect sensor

4. Web service interface to the CEP topology

5. Storm topology with CEP

6. Some system called on a notification from the CEP


Provisioning

• As a Virtual Machine image– VirtualBox Sandbox version: deploy on any Linux,

Windows, MacOS system for experiments -ftp://repo.nimbus-ware.com/fitman-dycep.zip

• As source code– SourceForge:

http://catalogue.fitman.atosresearch.eu/sites/default/files/storage/enablers/StormCEP-src-0.1.zip

• Source compiled to a JAR and deployed to a Storm cluster is the preferred provisioning way

• Note: Esper4Fast data distribution not included


http://catalogue.fitman.atosresearch.eu/sites/default/files/storage/enablers/StormCEP-src-0.1.zip

Terms and conditions

• Within the scope of the FI-PPP Programme

– Free use of the software for members till the end of the Programme

– FITMAN provides free support to members till the end of the Programme

• Out of FI-PPP Programme scope

– All software assets are free and Open Source (GPLv2 license applies globally)


Q & A


Thank you!


Science

Fitman webinar 2015 06 Dynamic CEP