Event Processing Course ( Advanced topics in CS 6 - 236606) Lecture one – Introduction (relates to chapters 1 and 2)

Event Processing Course(Advanced topics in CS 6 -236606)

Lecture one –Introduction (relates to chapters 1 and 2)

2 Copyright ©2009 Opher EtzionCopyright ©2009 Opher Etzion

The motto of this course

Focus on emerging technology

Dive deep inside

Internalize and retain over time


Emerging technologies in enterprise computing(Gartner Hype Cycle, Summer 2009)


Dive deep


Internalize by: Experiment

Product

Developer

View

Application

Developer

View


Entry point

Architecturaland programmingPrinciples

Chapter1

PART 1:

ENTRY POINT

Event Processing Agent

Context

EventChannel

EventConsumer

EventType

EventProducer

GlobalState

chapter

3

chapter

6

chapter

6

chapter

6

chapter

5

chapter

4

chapter

7

Filteringand transformation

PatternMatching

chapter

9chapter

8

PART 2:

THE BUILDING

BLOCKS

Engineering andimplementationconsiderations

Challenging topics Emerging directions

Chapter 10

Chapter2

PART 3:

ADDITIONAL TOPICS

Chapter 11

Chapter 12


Lecture outline

• The notion of event

• Event-driven computing

• Event processing and its positioning in IT enterprise computing

• Event-driven architecture

• Model driven approach and major building blocks

• The FFD example


Event and situation

An event is an occurrence within a particular system or domain; it is something that has happened, or is contemplated as having happened in that domain. The word event is also used to mean a programming entity that represents such an occurrence in a computing system.

A situation is an event occurrence that might require a reaction


The fast food alternatives


Example 1:Smart healthcare – Preterm Infants

Patient is hooked up to multiple monitors (in hospital or at home) - the physician can set up event- based rules on multiple measurements and patient’s history when to send an alert and to whom:

DefinedPattern

http://images.google.com/imgres?imgurl=http://www.accd.edu/sac/nursing/nurserec/images/NURSE%25201.gif&imgrefurl=http://www.accd.edu/sac/nursing/nurserec/attributes.htm&h=588&w=380&sz=10&hl=en&start=14&um=1&tbnid=tcp-i7A4her9kM:&tbnh=135&tbnw=87&prev=/images%3Fq%3Dnurse%26svnum%3D10%26um%3D1%26hl%3Den%26rls%3DGGLD,GGLD:2005-07,GGLD:en%26sa%3DG

http://images.google.com/imgres?imgurl=http://www.humec.k-state.edu/images/careers/physician.jpg&imgrefurl=http://www.humec.k-state.edu/future-students/careers/physician.php&h=338&w=225&sz=41&hl=en&start=8&um=1&tbnid=CNLSU2FVXdzlCM:&tbnh=119&tbnw=79&prev=/images%3Fq%3Dphysician%26svnum%3D10%26um%3D1%26hl%3Den%26rls%3DGGLD,GGLD:2005-07,GGLD:en%26sa%3DN


Example 2: RFID on luggage

Orchestrate: Passenger has been re-routed to another destination – send the luggage

Orchestrate:

Bag has reached to the wrong aircraft

Notify:

Bag has been checked but did not reach the ULD within 20 minutes

Notify: Bag has been checked but did not reach the connecting flight

THE LUGGAGE PERSPECTIVE:

Across the 24 largest airlines more than 5.6 million bags went missing in 2006, this is an average of 15.7 bags per 1,000 travelers. 15% of the bags are never found.

BBC News, April 4, 2007

http://images.google.com/imgres?imgurl=http://www.ibizaholidays.com/luggage-service/Luggae%2520service%2520-%2520Ibiza_files/luggage-store.jpg&imgrefurl=http://www.ibizaholidays.com/luggage-service/default.html&h=214&w=300&sz=12&hl=en&sig2=L3QueS-9OKEOyp-bRZNMBA&start=3&um=1&tbnid=kgT-vc7iZBfFTM:&tbnh=83&tbnw=116&ei=I38bRo2PD5O80QSR_9iNCg&prev=/images%3Fq%3Dluggage%26svnum%3D10%26um%3D1%26hl%3Den%26rls%3DGGLD,GGLD:2005-07,GGLD:en%26sa%3DN


Example 3: Safety zone management


Example 4: Alerts in banking

Bank Account alerts; Weekly withdrawals are more than X Balance is less than y My investment portfolio is up z% since the start of the week


Example 5: Fraud detection

http://images.google.co.il/imgres?imgurl=http://1.bp.blogspot.com/_ke_dp6bch8U/Sal1_7Uul6I/AAAAAAAAAAw/EywHULg88fw/s320/fraud.gif&imgrefurl=http://drboyceinsurance.blogspot.com/2009/02/car-insurance-fraud-are-you-next.html&usg=__LCo8ksQ19UaUHepkfasNnNd6dvQ=&h=300&w=300&sz=6&hl=en&start=2&sig2=FEB_1PF1wyUp15l3XOcacA&um=1&tbnid=xBMeb2OwjNw0EM:&tbnh=116&tbnw=116&prev=/images%3Fq%3Dfraud%26hl%3Den%26rlz%3D1T4SPDB_enIL309IL310%26sa%3DN%26um%3D1&ei=Z52_Su6UIMnhsAaj0MzZAg

http://ecommercesite.files.wordpress.com/2008/06/fraud2.jpg

http://images.google.co.il/imgres?imgurl=http://www.anatoa.com/images/diagramsmall.jpg&imgrefurl=http://www.modmysite.com/spam-reduction/9544-new-fraud-scam-detection-plugin-dolphin.html&usg=__DPbRIgFSZwR-ic041BPGPBUyidk=&h=351&w=450&sz=42&hl=en&start=24&sig2=DPmhS_8XUWmoJh3EG3pD4Q&um=1&tbnid=7Uw2Kd3QBbkCDM:&tbnh=99&tbnw=127&prev=/images%3Fq%3Dfraud%2Bdetection%26ndsp%3D18%26hl%3Den%26rlz%3D1T4SPDB_enIL309IL310%26sa%3DN%26start%3D18%26um%3D1&ei=EZ6_StaYN4Gfsgary9Ve


Example 6 – Emergency first responders(presented by Dieter Gawlick from Oracle in the 4th event processing symposium)

Fire StationFire Station

County Office

Hospital

Notification Message

ABC Chemicals


Example 7 – fairness in trading

http://images.google.co.il/imgres?imgurl=http://www.smhshop.com.au/userimages/online_trad_cov_smh_large.jpg&imgrefurl=http://www.smhshop.com.au/details.php%3Fid%3D1149&usg=__xAzC4_zpCWKBvl0CMBXRsdpn4tA=&h=600&w=400&sz=42&hl=en&start=14&sig2=OAfzjKIQZClSlvaQkGHpEQ&um=1&tbnid=xEegUwkno9_P7M:&tbnh=135&tbnw=90&prev=/images%3Fq%3Donline%2Btrading%26hl%3Den%26rlz%3D1T4SPDB_enIL309IL310%26sa%3DG%26um%3D1&ei=76G_StLzBsa0sgba_cRS


Example 8 - Diagnostics: root cause analysis of failure


Example 9 – the cameras in London

CarDB

Billing

Car Theft

http://images.google.com/imgres?imgurl=http://www.indigosystems.com/images/products/micron/micron_corner.jpg&imgrefurl=http://www.indigosystems.com/product/products.html&h=180&w=224&sz=7&tbnid=ZmXAi8N4y94J:&tbnh=81&tbnw=102&hl=en&start=51&prev=/images%3Fq%3Dsensors%2B%252B%2Bcameras%26start%3D40%26hl%3Den%26lr%3D%26rls%3DGGLD,GGLD:2005-07,GGLD:en%26sa%3DN



http://images.google.com/imgres?imgurl=http://www.idmu.co.uk/images/pound.gif&imgrefurl=http://www.idmu.co.uk/&h=374&w=544&sz=4&tbnid=pw6XBT4meT8J:&tbnh=89&tbnw=130&hl=en&start=8&prev=/images%3Fq%3DUK%2BPound%26hl%3Den%26lr%3D%26rls%3DGGLD,GGLD:2005-07,GGLD:en%26sa%3DG

http://images.google.com/imgres?imgurl=http://www.sherlockinvesting.com/Images/magglbig.gif&imgrefurl=http://www.sherlockinvesting.com/&h=233&w=214&sz=11&tbnid=NIuhw84HkZ4J:&tbnh=104&tbnw=95&hl=en&start=14&prev=/images%3Fq%3Dsherlock%2Bholmes%26hl%3Den%26lr%3D%26rls%3DGGLD,GGLD:2005-07,GGLD:en


Example 10: Presence in social network

5 members are online


Event processing

Event Processing is computing that performs operations on events. Common event processing

operations include reading, creating, transforming and deleting events.

Specific properties of event processing

Abstraction—The operations that form the event processing logic can be separated out from the application logic, allowing them to be modified without having to change the producing and consuming applications.

Decoupling—the events detected and produced by one particular application can be consumed and acted on by completely different applications. There’s no need for producing and consuming applications to be aware of each others’ existence, and they may be distributed anywhere in the world. An event emitted by a single producing application can be acted on by many consuming applications. Conversely you can arrange for an application to consume events produced by many different producing applications.

Real-world focus—event processing frequently deals with events that occur, or could occur, in the real world.


Deterministic and approximate event processing

TOLL VILOATOR FRUSTRATED

CUSTOMER


Different types of event processing IBM Haifa Research Lab – Event Processing

IBM Haifa Research Lab – Event Processing © 2008 IBM Corporation

DetectDecide

Respond

EP Solution Segments – Business Value

Real-Time Operational

Information Dissemination

Observation

Predictive ProcessingActive Diagnostics

Reactions to events are done as part of business transactions – achieving low latency decisions, and quick reaction to threats and opportunities

Getting the

right

information

in the right

granularity

to the right

person at

the right

time

Diagnose

problems

based on

symptoms and

resolve them

Quick observation into exceptional business behavior and notification to the appropriate people.

Mitigate or

eliminate

predicted events


Main parts of event processing system


Event processing positioning BPM:

Can serve as consumer and producer.

Some BPM products have ad-hoc EP capabilities.

BAM:

Typically has built-in limited EP capabilities (mostly aggregations for KPI tracking).

Two BAM trends:

Moving from KPI to more observation types;

Moving from batch observations to some on-line

BI:

Event-driven BI is still relatively minor, but the area of “operational intelligence” is event-driven and on-line

BRMS:

Often confusion between the two.

Differences: state processing vs. transition history processing.

Possible synergies:

1. BRMS as consumer: An action for an event can be rule flow.

2. Rules are used for some EP functionality, e.g. routing

3. BRMS systems can be producers of events

4. Event-driven contexts can serve BRMS

5. Patterns from event processing can be used for cross-state reasoning.


Event processing positioning (cont).

Network and Systems Management:

Event-driven but concentrated in limited aspects:

Event correlation to find the root cause of a problem.

Message oriented middleware:

Event may be expressed as message – but not every message has event semantics

Partial overlap in functionality – especially ESB deals with: validation, enrichment, transformation and routing.

Stream computing:

Streams are types of data that are not necessarily express events – audio, signal, video… Some overlap on “event stream” --- various positioning throughout the industry.


Request-Response request

response

Web browser Internet server

Taken from http://www.eaipatterns.comBased on the Enterprise Integration patterns book byHohpe and Woolf

Synchronous interactionsIn a synchronous interaction the provider is expected to send a response back fairly promptly. This usually means a few tenths of a second at the longest, though response times of several seconds are sometimes encountered in web applications. The requestor's thread of execution blocks waiting for the response to arrive without doing anything else in the interim/

An event is an indication of something that has already happened whereas a request,

as its name implies, expresses the requestor’s wish that something specific should happen in the future.

As a result we use slightly different terminology for the participants in an interaction involving events. Instead of talking about a service requestor and a service provider, we refer to an event producer and an event consumer.


Point to Point



Publish-Subscribe



Principle of decoupling

In a decoupled Event Processing System an event producer does not depend on any particular processing or course of action being taken by any event consumer.

Moreover an event consumer does not depend on any processing performed by the producer other than the production of the event itself.


Push vs. Pull


Event vs. Message

There’s one area of possible confusion that it is worth clearing up at this stage, and that is the distinction between an event and a message.

Message passing systems are often used to distribute events between event producers and event consumers, and it’s tempting to equate events with the messages used to transmit them.

The difference here is that the event is the entity that records information about the event occurrence, whereas the message is the mechanism used to exchange that information.

There are messages that don’t contain events, and conversely you can have events that aren’t in the form of messages (for example event records held in an event log).


What Makes EDA Different from Traditional SOA?

In Event-Driven Architecture (EDA):1. Events are always pushed, not pulled.2. The timing of the event message is determined by the source (event server) not by the

event consumers. 3. The event source and the event object never specify what action the consumer will

perform — the consumer determines the operation.

Not all Event Processing is EDA: 1. Event objects may be transmitted in a request/reply message pair (for example, RPC). 2. Or event objects may be at rest in a database or file (used, for example, in a BI

database query).

Event-based programming, also called event-driven architecture (EDA) is an architectural style in which one or more components

in a software system execute in response to receiving one or more event notifications.


Event processing network

Event Producer 1

Event Consumer 1

Event Consumer 2

Event Producer 2

Event Consumer 3

Agent 2

Channel

Agent 1

State

Agent 3


Event Stream

An event stream is a set of associated events .It is often a temporally totally ordered set (that is to say that there is a well-defined timestamp-based order to the events in the stream).

A stream in which all the events must be of the same type is called a homogeneous event stream;

a stream in which the events may be of different types is referred to as a heterogeneous event stream.


PlatformSpecific

PlatformIndependent

Specific DefinitionElements Instances

Platform Specific Definition Elements

Run-time Instance

Platform Independent Definition Elements

Event TypeBuilding Block

Creator for

Translated to


PlatformSpecific

PlatformIndependent

Delivery Request 3329From the Great Flower Shop

Delivery Request Event Type)an event schema of product X(

Run-time Instance

Delivery Request Event Type

Event TypeBuilding Block

Creator for

Translated to



Context

EventChannel

EventConsumer

EventType

EventProducer

GlobalState

The seven

Building blocks



Filter Transform Detect Pattern

Translate Aggregate Split Compose

Enrich Project

Event Processing Agents


Flower StoreVan Driver

Ranking and Reporting

System

Bid Request

Delivery Bid

Assignments,

Bid alerts, Assign Alerts

Control System

GPS Location

Location Service

Location

Driver’s Guild

Ranking and reports

Delivery

confirmation

Pick Up confirmation

Ranked drivers / automatic assignment

Bid System

StorePreferences

Delivery Request

Assignment System

Manual Assignment Assignment

Assignments,

Pick Up Alert

Delivery Alert

Fast Flower Delivery


Lecture Summary

• In this lecture we have discussed the following topics:

– The notion of event

– Event-driven computing

– Event processing and its positioning in IT enterprise computing

– Event-driven architecture

– Model driven approach and major building blocks

Documents

Event Processing Course ( Advanced topics in CS 6 - 236606) Lecture one – Introduction (relates to chapters 1 and 2)